Agile vs DevOps: Which Should You Choose And Why
Whenever you think of a high performance organization, have you ever wondered how or what best practices they follow that enables them to achieve that stature? Do they lean towards agile or devops approaches?
The objective of a development and operations process is to help the company fulfill its requirements, help the team members collaborate and break silos to unblock issues and finally complete the projects/features they are working on, so they can increase their DAU or MAU or in the case of enterprise applications - increase the functionalities of their applications.
When teams work in silos, it results in gaps in communication which in turn results in chaos. Whereas when teams work in tandem, they are more effective.
For the scope of this article, we will cover agile and devops approaches out of the eclectic bunch that are being adopted by organizations and examples surrounding when to adopt specific methodologies and how testing gets impacted in each of these scenarios.
Agile Software Development Practices
When compared to the waterfall development process, agile focuses on one of the core principles which involves: satisfying customers early on in the process and through continuous delivery. Continuous delivery is possible only when we involve the quality team early on in the process and collaborate with them frequently.
Currently in the industry, some of the smaller to mid-size companies are in a state where they are following neither agile approach nor waterfall entirely but a mix of both. These so-called agile team members focus so much of their energy on granular processes like 30-minute standups, 60-minute retrospective meetings, forgetting about the return on investment that they bring to the table, which ends up in a situation where a lot of bugs are found in production.
Has the industry really transformed from a waterfall approach?
Have you ever been part of an organization (which you thought followed agile software development processes) only to realize that the quality team is involved for testing only after development is completed?
In this example, they still called it agile because the development team members worked on multiple branches (or features) at a given time in an iterative manner. But when they had to get their features tested, they waited until all development was complete—which is the core of waterfall methodology. But since companies want to move faster so they can expand their customer base, most of the time, teams end up accumulating technical debt.
The best way to solve this problem is to convert their organization to become completely agile and work hand-in-hand with the quality team iteratively. This involves working with all stakeholders to ensure they understand the consequences of not making the switch and how it may affect the end-user experience.
Agile Frameworks And Its Variants
For the scope of this article we will be discussing two of the most popular agile frameworks:
- Scrum
- Kanban
Scrum is an agile methodology used in software development based on iterative and incremental processes. It consists of adhering to time-boxed development called sprints. The duration of sprints varies according to each organization ranging from weekly to monthly to quarterly.
There’s a sprint planning session, followed by the sprint where implementation and testing happen including daily standups and finally ends with a retrospective session.
Scrum is usually implemented in product development teams where the team members have to timebox their work to meet the customer deadlines. It mostly applies to B2C applications because if you wait too long, your feature is no longer new as someone could have implemented it already!
In B2B applications, which are mostly enterprise companies, a modified version of agile methodology called SaFe - Scaled Agile framework is commonly used.
Most of the projects at enterprise companies fall into one of the following categories:
- Team level
- Program level
- Portfolio level
Team-level projects are feature-based development that involves each team member to own their project and processes. These teams usually use Scrum or Kanban depending on their nature of work.
If the teams fall under research & development organizations or test automation, then it is imperative that they adhere to Kanban, which is less rigid and the goal is more towards completing their current task than jumping onto the next shiny object at hand! If the teams are part of the product development organization, then they tend to use scrum processes.
Program-level projects are those that involve multiple teams working towards a specific goal—an example would be AWS migration. Even though this project can fall into the portfolio level-based project, I would argue that it wouldn’t necessarily affect HR or accounting teams.
In this case, the project AWS migration could last months due to unforeseen issues, so the focus is on completing the AWS migration successfully while breaking it into justifiable sprints.
Portfolio-level projects are those that involve different organizations within the company—an example would be JIRA implementation. Each organization would require JIRA workflows to be implemented differently based on their needs. Human resources teams don’t require testing, the focus is mainly on serving specific requests to employees and once they help them, the task is marked as “complete”.
Agile methodologies should be adopted by organizations that are known to undergo changes constantly. But these changes still have to go through a round of testing and bug fixing and testing isn’t necessarily automated, leading to increased process times.
Finally, when the features are ready to be deployed, it is handed off to the build/ops team. So, testing happens iteratively when compared to the waterfall approach but doesn’t shift far left as the focus isn’t on continuous testing and delivery. Hence the need for the DevOps approach!
DevOps Approach
The DevOps approach focuses on aspects beyond development and testing and evangelizes a completely automated CI/CD pipeline along with monitoring capabilities. While Agile embraces changes, DevOps approach focuses on continuous testing and delivery to ensure frequent releases to the end-users are successful.
The goal of the IT Operations team is to scale the software development process to accelerate the writing and updating of the code responsible for creating new applications and services and updating features within the IT team.
Even in terms of team setup, these days, the quality and IT operations teams are part of the same organization so they can work hand in hand. In fact, a new role called TestOps has been introduced to focus specifically on setting up the CI/CD pipelines where automated tests are run against pull requests, giving immediate feedback to the development teams.
Automated testing is essential to scale testing efforts and its value is lost if it is not part of continuous testing and continuous deployment release cycle. So, the TestOps personnel always have their hands full with automated test infrastructure maintenance.
This ensures that the operations team members are focused on delivering a world-class infrastructure for software development without getting distracted about testing infrastructure.
There are a lot of tools and devops practices in place these days which streamlines the overall process.
- A Terraform state file refers to a set of infrastructures that are defined and managed as a single unit—this is how various testing and development environments are defined and managed. Terraform helps configure servers, but we need an infrastructure to run these servers.
- AWS provides the infrastructure via EC2 instances that are currently the most cost effective way to configure and run these servers. To take a step further, if your tech stack has a lot of microservices, with docker compose, you can define and run multiple container docker environments.
- Ansible automates the process of configuring machines to run any processes or servers.
- Kubernetes helps to manage a cluster of these EC2 instances as a pod, and schedules containers to run on this cluster based on the available compute resources.
Should You Use Agile Or DevOps?
While Agile software development vs Devops approach is a constant debate within organizations, the best way to look at it is based on asking ourselves the following questions:
- What is the adaptability of our organization when it comes to new technologies?
- Are we able to oppose our competitors in terms of the quality of our products?
- Are customers likely to use our products as it meets their expectations?
If the answers to the above questions involve some form of negation, then it is time to focus more on a combination of both approaches and customize it to suit our needs.
The Ideal Software Development Methodology
The key difference between agile and devops based software development processes is that the former focuses more on accommodating constant changes whereas the latter focuses on continuous testing, delivery and deployment. Collaboration between the development and operations teams is crucial so that the dev team is not blocked.
So, in my opinion, an ideal software development process should entail the following:
- Handling customer/user personas and incorporating customer feedback
- Focusing on best practices to prevent technical debt from accumulating
- Continuous improvement, continuous testing, continuous integration, continuous delivery, continuous deployment and monitoring
Here’s what the process would look like:
Handling different customer personas is not restricted to product management teams. At the end of the day, why are we developing these products? What is the purpose of these products if they aren’t being used by customers?
Take the classic example of Nokia—when they kept accumulating technical debt for not keeping up with the market trends, namely their competitor Apple’s innovation. They focused on following rigorous sprint cycles only to eventually be forgotten!
All companies should understand why their features are being developed and what kind of customers it caters to. This will ensure that we are developing user-centric features.
When a new product is released, be it a mobile application or a web application, always ensure there’s a way for customers to leave their feedback. Not all companies follow a strict process of working closely with customer support.
Finally, advocate for AI/ML-based continuous integration and continuous delivery. When a project fails, it doesn’t necessarily connote the lack of skill set of the team members, it is rather the lack of sufficient testing and failure of finding issues earlier during development.
But sometimes even with adequate testing and automation in place, things go wrong—so if there was a recommendation system in place to identify when to release vs when not to, maybe such catastrophic failures can be averted.
Frequent iterations help the development team understand what’s working, what’s not, and aid in thinking about scalability. The caveat being—it is time-consuming! If there was another recommendation/prediction system that could potentially gather data about customers and predict if a specific feature would help gain traction vs fail even before development begins - it would help streamline the process of building a high-quality product from day 1!
The overall objective is to ensure business impacts multiply faster when these best practices are adopted as part of the development process.
For more best practices and tips to shape your development process, subscribe to The QA Lead newsletter.
Also Worth Checking Out: