Agile Methodologies and Frameworks- Kanban and Lean Management Tutorial

What is Agile Methodology?

According to the dictionary, it is an adjective that means to move quickly & easily. Its application is no different from when organizations use it.

Agile methodology is a set of values and principles that ought to be followed to become Agile.

It is an approach to software development where team interaction, customer collaboration, and responding to change are the key themes. In addition to that, Agile methodology provides us with a framework where continuous improvements happen at different stages of the software development life cycle.

Let’s trace back the roots from where it popped up.

Principles of Agile Methodology

Following are the 12 main principles in Agile methodology: –

1. 1. Customer satisfaction through early and continuous delivery of useful software – It means letting the customer know the progress by the distribution of values to the customers. Which, in turn, is achieved by fulfilling the highest priority requirements first.
2. 2. Welcome changing requirements, even late in development – This implies flexible adaptation to the changing needs of the customer. In addition to this, providing them a competitive advantage to external changes.
3. 3.Frequently Delivered Software (weeks rather than months) – This provides immediate value to the customer by delivering working features timely.
4. 4. Work together – i.e., Close and daily cooperation between Business people and Developers by keeping documentation and requirements lightweight.
5. 5. Trust and Support – Projects are successful around motivated individuals who should be trusted.
6. 6. Face to Face conversation – Oral communication is preferred to rule out any discrepancies and be on the same page.
7. 7. Working Software – It is the primary measure of progress and should be delivered timely.
8. 8. Sustainable development – Agile methodology maintains the work-life balance among the team members and promotes happiness by avoiding exhaustion.
9. 9. Continuous Attention- Means consistent attention to technical excellence and good design enhances the agility of any team.
10. 10. Simplicity – It is the art of maximizing the amount of work not done—is essential. It uses the Pareto principle or the 80/20 rule that says that typically 80% of your results may come from only 20% of your efforts.
11. 11. Self-organizing teams – The scrum team has autonomy and responsibility to meet the goals of the sprint.
12. 12. Reflect & Adjust – At regular intervals the team reflects on how to become more productive and then adjusts accordingly. The retrospective meetings ensure the implementation of the lessons learned during the project into the next iteration.

Types of Agile Methodologies

There are several agile methodologies in practice across the world. We are going to learn more in detail about four of the most popular ones.

1) Scrum

Scrum can easily be considered to be the most popular agile framework. The term ‘scrum’ is much considered synonymously to ‘agile’ by most practitioners. But that is a misconception. Scrum is just one of the frameworks by which you can implement agile.

The word scrum comes from sports rugby. Where the players huddle together in an interlocked position pushing against the opponents. Each player has a defined role in their position and can play both offensive and defensive as per the demand of the situation.

Similarly, the scrum in IT believes in empowered self-managed development teams with three specific and clearly defined roles. These roles include – Product Owner (PO), Scrum Master (SM) and the development team consisting of the programmers and testers. They work together in iterative time boxed durations called sprints.

The first step is the creation of the product backlog by the PO. It’s a to-do list of stuff to be done by the scrum team. Then the scrum team selects the top priority items and tries to finish them within the time box called a sprint.

An easier way to remember all of this is to memorize the 3-3-5 framework. It means that a scrum project has 3 roles, 3 artifacts, and 5 events.

These are –

Roles: PO, Scrum master, and development team.

Artifacts: Product Backlog, Sprint Backlog and Product increment.

Events: Sprint, Sprint planning, Daily Scrum, Sprint review and Sprint retrospective.

We will get to know more in detail about each of these in our subsequent tutorials.

2) Kanban

Kanban is a Japanese term which means a card. These cards contain details of the work to be done on the software. The purpose is visualization. Every team member is aware of the work to be done through these visual aids.

Teams use these Kanban cards for continuous delivery. Just like Scrum, Kanban is also for helping the teams work effectively and promotes self-managed and collaborative teams.

But there are differences between these two as well – like during a scrum sprint, the items being worked upon by a team are fixed and we cannot add items to the sprint whereas, in Kanban, we can add items if there is available capacity. This is particularly useful when the requirements change frequently.

Similarly, another difference is that while the scrum has defined roles of a PO, scrum master, and development teams, there are no such predefined roles in Kanban.

Another difference is that while the scrum suggests a prioritization of product backlogs, Kanban has no such requirement and it is totally optional. Thus Kanban requires less organization and avoids non-value adding activities and is suitable for the processes which require responsiveness towards changes.

3) Lean

Lean is a philosophy that focuses on waste reduction. How does it do that?

In lean, you divide a process into value-adding activities, non-value adding activities and essential non-value adding activities. Any activity which can be classified as a non-value adding activity is a waste and we should try to remove that wastage in the process to make it leaner.

A leaner process means faster delivery and less effort wasted in tasks which don’t help to achieve the team goals. This helps to optimize every step in the software development cycle. That is why the lean principles were adapted from lean manufacturing into software development.

Lean software development can be used in any IT project by applying the seven lean principles which are shown below:

These are quite self-explanatory as their names suggest. Eliminating wastage is the first and most important lean principle and we saw how to do that, we just classify activities as value and non-value add.

A non-value add activity can be any part of the code which might make it less robust, increase the effort involved and take up a lot of time while not adding justifiable business value. It can also be vague user stories or poor testing or adding features that are not required in the big picture.

The second principle amplifying learning is again easy to understand as a team needs various skills to deliver products in a rapidly changing environment with new technologies cropping up in quick durations.

Making late decisions can be rewarding in circumstances where it reduces rework like if there are any changes expected then better delay it so that the team does not have to redo the work as the business needs change.

But there is always a trade-off here as the teams need to balance this with the fourth principle of delivering faster. Delaying of decisions should not impact the overall delivery and must not reduce the pace of work. One eye should always be on the complete picture.

Having empowered teams is also very common nowadays and this is something that even agile suggests. Empowered teams are more responsible and can take decisions faster. The sense of ownership in an empowered team leads to better results. To empower a team, they should be allowed to organize themselves and take decisions on their own.

Thus we see that lean and agile have a lot in common with one stark difference – while lean teams can help to refine a product, agile teams are the ones who actually build the product.

4) Extreme Programming (XP)

Extreme programming is another most popular agile technique. As per extreme programming.org, the first XP project was started on March 6, 1996. They also mention that XP impacts software project development in 5 different ways – communication, simplicity, feedback, respect, and courage. These are called the values of XP.

Out of these, it all starts with communication. XP teams collaborate with business teams and fellow programmers on a regular basis and start building code from the very first day. The focus here is on face to face communication as far as possible with the help of the other visual aids.

Extreme programmers also build a simple code and start getting feedback on it from the first day itself. The focus is on not going overboard or predicting requirements which have not been shared. This keeps the design simple and produces just the minimum product which will serve the requirements.

Feedback helps the team to improve and produce a better quality of work. This helps them build respect for each other as they learn from each other and learn how to share their views.

This also gives them the courage as they know that they have gathered everyone’s best ideas and produced a good product with feedback from others. Thus they are also not afraid to include changes or receive further feedback on their work.

This is particularly useful in projects where the requirements are going to change often. Constant feedback will help the teams in including these changes with courage.

Thus we have seen the different agile methodologies like Scrum, XP, Kanban and Lean along with their respective advantages and disadvantages.

Now, we can easily differentiate between them and also appreciate the subtler differences amongst them. We also learned the fundamentals of each of these methodologies and saw how to apply them in our projects as and when required.

5) Feature-Driven Development (FDD)

Feature-Driven Development (FDD) was introduced in 1997 by Jeff De Luca when he was working in a software development project for a large Singapore bank. FDD was also built around software engineering best practices such as domain object modeling, developing by feature and code ownership. The blending of these practices that resulted in a cohesive whole is the best characteristic of FDD. It consists of five basic activities, namely, the development of an overall model, the building of a feature list, the planning by feature, the designing by feature, and the building by feature. Every project will have its own unique model, which will result in a feature list. The last three activities are short iterative processes, with a feature not taking longer than two weeks to build. If it will take more than two weeks, then it will have to be broken down into smaller features.

FDD’s main purpose is to deliver tangible, working software repeatedly in a timely manner. The advantage of using FDD is that it is scalable even to large teams due to the concept of ‘just enough design initially’ (JEDI). It is a great solution to maintain control over agile, incremental and inherently complex projects because of its feature-centric process. It is used in enterprise projects as well as web projects such as the Mousebraker.com online gaming site.

6)Dynamic Systems Development Method (DSDM)

Dynamic Systems Development Method (DSDM) was published in 1995 by the DSDM Consortium, an association formed by vendors and experts in software engineering to provide a structure for Rapid Application Development techniques brought on by object oriented programming. The Consortium jointly developed and promoted a tool- and technique-independent development framework from best practice experiences of people working in big companies such as British Airways, American Express, Oracle and Logica. It has now evolved into a project delivery framework that is fully compatible with ISO 9000 and PRINCE2. In 2007, it was rebranded Atern after the bird Arctic Tern. However, since 2014, it has reverted back to its original name as DSDM Agile Project Framework. Also, in 2016, the DSDM Consortium rebranded as the Agile Business Consortium.

DSDM consists of eight principles that will direct the team and create a mindset to deliver on time and within budget. Principles include focusing on the business need, delivering on time by timeboxing work, and emphasizing collaboration with end users, team members, business representatives and other stakeholders. As a framework and not just a software development method, it can also be used in non-IT projects. It is often used in government projects where it is paired with project management standards, such as PRINCE2.

What is the Crystal Method?

Crystal method is an agile software development approach that focuses primarily on people and their interactions when working on a project rather than on processes and tools. Alistair believed that people’s skills and talents as well as the way they communicate has the biggest impact on the outcome of the project.

Crystal Method is based on two fundamental assumptions:

1. Teams can streamline their processes as their work and become a more optimized team

2. Projects are unique and dynamic and require specific methods

According to Cockburn, we should view product development as a game that should stimulate everyone to interact, become creative, and produce brilliant ideas. He says that instead of focusing on questions like “is our model accurate?” we should be looking for answers to questions like “Is our product meeting the customer’s needs? Or “Do we have our goals aligned as a team?”

Crystal method family members

One of the things that Cockburn discovered is that the project properties changed depending on the number of the people involved in the project and the level of criticality of the project at hand.

While the smaller team can handle and build the product without a lot of status reporting and paperwork, the number of “communication artifacts” rises with bigger teams who are working on large-scale projects.

In other words, the more people you have on the team, the more critical the project is and the more complex the approach needs to be. Therefore, there is no one single Crystal method; there are different Crystal methodologies for different types of projects.

To make this categorization easy to understand, Cockburn named the methodology Crystal and categorized it along the two dimensions – size and criticality – that matching those of minerals – color and hardness.

Essentially, Cockburn developed these families to point out that each project may require a particular set of policies, practices, and processes in order to meet the project’s unique characteristics. Cockburn tried to explain this by calling Crystal “a set of samples that you adjust to your circumstances”.

Which approach will be most suitable for your projects depends on three dimensions:

1. Team size

2. Criticality

What the priority of the project is

Generally, they are characterized by colors, according to the number of people involved in the project:

• Clear – for teams of 8 or fewer people
• Yellow – for teams of 10-20 people
• Orange – for teams of 20-50 people
• Red – for teams of 50-100 people

Crystal method characteristics

Human-powered

This means that people involved in the project are vital and that the processes should be adapted to meet people’s needs. It also emphasizes that people are capable of organizing themselves and that they can become more organized and competent as the processes develop.

Adaptive

Crystal is a stretch to fit methodology meaning that processes and tools are not fixed, but have to be adjusted to meet the requirements of the team and the project at hand.

Ultra-light

Crystal doesn’t involve too much documentation, management and reporting. It keeps things light by focusing on transparent workflow between the team and the client and by practicing open communication between team members.

7 properties of Crystal method

1. 1. Frequent delivery – it allows you to frequently deliver working, tested code to real users. In this way, you don’t have to face the fact that you have invested your energy and time into the product that nobody wants to buy.
2. 2. Reflective improvement –  no matter how bad or good the product is, there are always areas where the product can be improved. Also, there are always new techniques and methods your team can implement to improve their future practices
3. 3. Osmotic communication – with the team who works co-located, information flows around the team. That allows them to pick up valuable information without even being directly involved in the discussion of a certain matter. This gradual absorption of ideas is called osmotic communication. Cockburn believes that this kind of work atmosphere can operate with very little structure.
4. 4. Personal Safety – the only way to build a healthy working atmosphere and a true team culture is by practicing open and honest communication. Team members should be able to speak without fear, no matter whether they are presenting a new idea or talking about a potential problem.
5. 5. Focus – each team member knows exactly what to work on which enables them to focus their attention and avoid switching from one task to another. Also, this boosts team communication and helps the team prioritize and work towards the same goals.
6. 6. Easy access to expert users – Crystal enables your team to maintain communication and get regular feedback from real users.
7. 7. A technical environment with automated tests, configuration management, and frequent integration – very specific tools for software teams where the emphasis is on continuous integration so that the errors could be caught within minutes.

Why is the Crystal method useful?

The fact that Crystal uses a focus on people and communication as its organizing principle is what distinguishes it from other software development methods.

Unlike other agile methodologies, Crystal focuses on adjusting the techniques used in a project with the aim to strengthen the process of team communication. Besides that, Crystal allows:

• Continuous integration
• Flexible and configurable processes
• Active user involvement

Keep in mind that Crystal is primarily created to remind you how to stay centered and focused on your work during the project development.

Agile Framework(APM Framework)

The Agile Project Management typically follows five phases. They are as follows:

1. 1. Envision : Determine the product vision, project scope, project community, and how the team will work together.
2. 2. Speculate : Develop a feature-based release, milestone, and iteration plan to deliver the vision.
3. 3. Explore : Deliver tested features in a short timeframe, constantly seeking to reduce the risk and uncertainty of the project.
4. 4. Adapt : Review the delivered results, current situation, and the team’s performance, and adapt as necessary.
5. 5. Close : Conclude the project, pass along key learning, and celebrate.

APM and PMBOK

Michele Sliger aligned the Agile Project Management Framework to the PMI Project Management Book of Knowledge or PMBOK. The given image shows the correlation between the two.

Here, the Initiation phase is parallel to Envisioning, Planning phase is parallel to Speculating, Executing phase is parallel to Exploring, Controlling phase is parallel to Adapting, and Closing phase is parallel to Closing.

Lean Software Development

Let us go through Lean Software Development below.

What is Lean Software Development?

Tom and Mary Poppendieck introduced Lean software development to the Agile community. Lean software development adopts the principles and practices of Toyota Production System or TPS.

TPS was developed to address issues that affect manufacturing processes, like Muri (Overuse), Mura (irregularities), and Muda (waste). Muri is to cause overburden, such as unnecessary stress on the employees and processes. This is caused by Mura and a host of other failures in the system, such as lack of training, unclear ways of working, and wrong tools. Mura is the waste caused by unevenness or irregularity.

Unrealistic demand results in unevenness in the processes, which leads to waste creation. Mura drives Muda. Muda is any activity or process that does not add value. This can refer to waste of time, resources, and money.

Lean Principles:

The seven lean principles are as listed below:

1. 1. Eliminate Waste : Anything that does not add value to the customer is considered waste (Muda). Unnecessary code and functionality, delay in the software development process, unclear requirements, insufficient testing, bureaucracy, and slow internal communication are a few examples of Muda.
2. 2. Amplify Learning : Learning is amplified through short iteration cycles, refactoring and integration testing, and frequent customer feedback sessions. Each of these techniques shed more light on the problem domain and identify opportunities to improve.
3. 3. Decide as late as possible : The best way to manage uncertainty is by gaining information, minimizing assumptions, deferring commitment to the last responsible moment, and breaking dependencies between components.
4. 4. Deliver as fast as possible : Short iterations or small batches provide valuable feedback opportunities and allow effective decision making.
5. 5. Empower the team : Lean focuses on the team as the source of decision making and management turns to the team to understand the best options and their costs.
6. 6. Build Integrity in : Ensuring that quality is embedded throughout the system requires automation of testing through builds, installs, and continuous integration.
7. 7. Finally, think big, act small, fail-fast, and learn rapidly.

Kanban

Let’s understand about Kanban and Kanban Methods in detail below.

What is Kanban?

Kanban is a Japanese term for a signal board. It was also developed in the Toyota Production System or TPS. Agile has adopted Kanban technique to reflect the throughput of a Sprint or iteration. It showcases the status of each user story within the sprint and helps gauge the cycle time and the throughput of the team.

Kanban boards are also used in most Agile software products, which are useful for Agile distributed teams. Most Kanban boards are located in the team room and have user story cards or post-it notes distributed across different categories. Use of Story cards or post-it notes helps everyone in the team to understand the complete scope of work to be done.

Kanban helps manage the throughput of a process by identifying bottlenecks, setting ‘Work In Progress’ limits, and displaying the status of the entire production system with one view. The sample Kanban board given below showcases the various teams, which a user story has to pass through, to term a story as Release Ready. Also, notice that the WIP limits are set at the top of the chart, which helps minimize the total amount of work done at any point in time.

Kanban Method – Case Study

Kanban is an evolutionary change for an organization. It helps to increase the visibility of project progress throughout the company. It drastically reduces the time-to-market, development, and engineering time. The results of Kanban influence both the Web and software development. It helps in organizing work.

It also boosts communication among team members helping them solve problems quickly. The graphic shows the benefits of implementing Kanban in a company. Kanban reduced the lead time of a project from 22 days to 14 days, the development time from nine to three days, and the engineering took eight days instead of 11 days. It was found that these results could not be achieved using different methods.

Lean Kanban Method

The Lean Kanban Method was established by David Anderson who saw the potential of both techniques in dramatically improving software development activities. Lean Kanban blends Kanban with Lean and other Agile principles to improve processes. The three principles of Lean Kanban and five core practices of Lean Kanban are listed below.

The three principles of Lean Kanban are as follows:

1. 1. Start with what you know : This method does not prescribe a specific set of standards or procedures; rather it insists on using Kanban along with the existing process or workflows. Following this principle helps reduce major changes and eases Kanban implementation.
2. 2. Agree to pursue incremental, evolutionary change : Develop the solution incrementally and regularly get these changes compliant with the business. This leads to the overall solution being developed slowly, thus avoiding sweeping changes being made with minimal resistance later on.
3. 3. Respect current roles, responsibilities, and job titles : Kanban recognizes that there may be value in the existing process, roles, responsibilities, and titles. If changes have to be made, it needs to be introduced incrementally, as this will avoid the fear of progress getting impacted. Minimal changes are easier to implement than large system-wide changes.

The five core practices of Lean Kanban are as follows:

1. 1. Visualize : Visualizing the workflow through the system helps the entire team to know exactly ‘what happens next’. Such visual representations help to refine and optimize the current workflow and improve efficiency.
2. 2. Limiting work in progress : Limiting the amount of work in progress helps to identify bottlenecks and run the entire process efficiently. It also helps to reduce the sunk cost and wastages that might result from changes later on.
3. 3. Manage Flow : Tracking the flow of work through the system helps the team identify issues that can be used to improve effectiveness.
4. 4. Make Management Policies : Ensure all the team members clearly understand the policies and ground rules. This helps them make informed decisions and bring about improvements.
5. 5. Improve Collaboratively, using “safe to fail” experiments : It is better to fail-fast than fail later in the project, as this helps to prevent wastage of effort and cost. Through collaborative experiments, the team should improve the processes it uses.

OpenUP

OpenUP is an open-source variant that IBM released to the public domain in 2006 and is a variant of the Rational Unified Process or RUP. OpenUP is a Lean unified process that applies iterative and incremental approaches within a structured lifecycle. It embraces the Agile philosophy that focuses on the collaborative nature of software development.

OpenUP is a tools-agnostic, low-ceremony process that can be extended to different project types. It targets small and co-located teams involved in Agile and iterative development. Small projects constitute teams of three to six people, involving three to six months of development effort.

OpenUP – Phases

As depicted in the given image, OpenUP has four phases across the project lifecycle.

Inception Iteration Phase

The initial idea or concepts are generated in this phase. The objectives of this phase are:

• Understand what to build
• Identify key system functionality
• Determine at least one possible solution
• Understand the cost, schedule, and risks associated with the project

Elaboration Iteration Phase

Here ideas are fine-tuned or elaborated. The objectives of this phase are:

• Get a detailed understanding of the requirements
• Design, implement, validate, and baseline an Architecture
• Mitigate essential risks, and produce accurate schedule and cost estimates

Construction Phase Iteration

Here the solution is developed in line with the ideas generated in the earlier phases. The objectives are as follows:

• Iteratively develop a complete product that is ready to transition to its user community
• Minimize development costs and achieve some degree of parallelism

Transition Phase Iteration

Here the solution is moved into production. The objectives of this phase are:

• Conduct beta test to validate that user expectations are met
• Achieve stakeholder concurrence that deployment is complete

Information Radiators

Information radiators are important tools for communicating information on Agile projects. The concept of information radiators was invented by Alistair Cockburn. He defined it as: “An information radiator displays information in a place where passersby can see it.

With information radiators, the passersby don’t need to ask any question; the information simply hits them as they pass.” Information radiators are most effective with co-located teams. Some of their features are as follows: They are intended to enable team members to view the current state of the project and its progress. Information radiators are also helpful to provide information to other non-project team members. Most Agile teams implement them to some degree in their processes.

A few examples of information radiators are boards with swimlanes and status columns showing the progress of work items; Big Visible Charts including Burndown charts, showing team velocity across iterations; and Continuous integration build health indicators, which many teams choose to display using lava lamps and street lights. These information radiators improve project communication in a lightweight and easy to manage fashion.

Effective Information Radiators

Effective information radiators should be:

• Simple : Easy to understand
• Stark : Errors should not be masked, rather they should be used to improve the work and performance
• Current : Information displayed should be current
• Transient : Once the problem has been rectified, it should be taken off from the chart
• Influential : Empowers the team to take decisions
• Highly Visible : Easy to see and understand
• Minimal in Number : Key information must be emphasized Information radiators help in building transparency and a shared understanding of the success criteria, deliverables, and other key aspects of the project to align expectations, build trust among the stakeholders, and make informed decisions.

Examples of Information Radiators

Some examples of information radiators, such as Burndown Chart, Burnup Chart, Combined Burn Chart, and Risk Log are shown here. It is recommended to go through these images for better understanding. These examples are however discussed in the later part of this course.

Conclusion

Many teams and organizations have realized benefits, advantages and improvement by adopting agile methodologies. However, challenges continue to come because of unique requirements and transforming business landscapes, especially difficult in large enterprises. Thus, the problem of scaling agile delivery processes continues to be difficult to solve, and is currently being addressed by solutions such as Scaled Agile Framework (SAFe) and Large Scale Scrum (LeSS). Therefore, the goal of all companies to find that repeatable, predictable process that will improve productivity and quality continues on.