Lean Software Development

"Lean" is the name given to the Toyota Production System, which was based upon the ideas of William Edward Deming. Contrary to the mass production systems, inspired by Taylorism, Lean prioritises the economies of flow above economies of scale. e.g.

Rather than build cars in advance as efficient as possible, it is better to build cars on demand, even if this means the production cost of a car is higher.

The approach was so successful, it lead to the economic recovery of Japan, and dominance of Japanese car manufacturers (other manufactures followed suit). Lean processes are also the reason Dell became the leading PC manufacturer. So why do economies of flow beat economies of scale?

One reason is that to mass produce cars, you need to estimate in advance how many cars you will need to build. This requires marketing effort to assess the competition likely demand. Corporate politics have a tendency to over inflate targets, and invariably the estimates are wrong. If they are too low, you have a lost opportunity. If they are too high, you have surplus stock or "inventory".

Inventory needs to be stored, secured, transported and managed. To shift it the company must run marketing campaigns and sales promotions. Sales promotions can reduce the brand value of your product, anger customers who missed out on the promotion, and cause savvy customers to delay purchase. Instead if you can build cars on demand you never have surplus stock. Providing you can build products quickly and cheaply enough, economies of flow beats economies of scale.

A more subtle reason why economies of scale cause problems is because they result in a "push" based system. You must order all the parts, for all the cars up front. Just like the completed product these need to be stored, secured, transferred and managed. Each step of the assembly line must operate at the same rate. If one step becomes delayed, either the entire line stops, or inventory begins to accumulate.

In all likelihood the assembly line may be parallelised, but at some point the lines will need to be composed. When two upstream lines produce at different rates, inventory again begins to accumulate, and the dependent line becomes starved.

Finally mass production based on Taylorism attempts to optimise their by division of labour. Each person has a narrow and specific job. The way in which that job must be performed has been studied, optimised and standardised. Targets are set and performance is monitored.

This can work well in a purely physical role, but fails for anything creative or cognitive. The is no element of autonomy or mastery, and very little purpose beyond renumeration. If all you do is tighten bolts, the purpose of building cars is a distant vision.

Compare this to an on demand approach approach...

The pipeline will start with a customer order. This exerts "pull" on the production system. Just like a traditional assembly line the production and consumption rates of each step in the assembly line must be tuned, but the difference with a lean approach is that when demand exceeds capacity, you can temporarily stop the upstreams without having to manage huge amounts of inventory, because only a limited number of parts were ordered.

Value and Waste

A key aspect of on demand or just in time (JIT) production systems is that they must be quick. Customers are likely to take their business elsewhere if the order times are too long. It is therefore critical to optimise the production system to deliver value and remove waste. One approach to do this is map the process and categorise each step as

• Non Value Adding
• Necessary but Non Value Adding
• Value adding

The Non Value Adding steps should be eliminated. This approach is known as value stream mapping.

It can often make it easier to recognise and explain waste when it is categories. In production systems common categories are

• Faster-than-necessary pace: creating too much of a good or service that damages production flow, quality, and productivity. Previously referred to as overproduction, and leads to storage and lead time waste.
• Waiting: any time goods are not being transported or worked on.
• Conveyance: the process by which goods are moved around. Previously referred to as transport, and includes double-handling and excessive movement.
• Processing: an overly complex solution for a simple procedure. Previously referred to as inappropriate processing, and includes unsafe production. This typically leads to poor layout and communication, and unnecessary motion.
• Excess Stock: an overabundance of inventory which results in greater lead times, increased difficulty identifying problems, and significant storage costs. Previously referred to as unnecessary inventory. Unnecessary motion: ergonomic waste that requires employees to use excess energy such as picking up objects, bending, or stretching. Previously referred to as unnecessary movements, and usually avoidable.
• Correction of mistakes: any cost associated with defects or the resources required to correct them.

Variation

With the waste removed, work should ideally flow through the production system at a constant rate. In practice there will always be some variation. Variation can be categorised as common cause or special cause. Common causes (e.g. fewer cars are sold in the weeks leading up to a licence plate change), do not indicate a problem with the production system special cause. Special cause variation may indicate a problem, but isolating it can be difficult. Approaches are discussed in Understanding Variation - The Key To Managing Chaos

Any variation will result in either over production or under production of parts of the production system. Therefore some inventory is avoidable, and even necessary to avoid starvation. You can think of this like buffers, or queues. Queue length should be monitored carefully. Too little or too much work in a queue indicates that the production rates of part of the system has changed.

Learning

Yet another important part of a lean process is learning. This may be called "inspect and adapt" or "Plan Do Check Act" or better still "Plan Do Study Act". Loosely speaking this means

1. Plan a change
2. Implement the change
3. Study the impact of the change
4. Act on what you've learned (possibly by planning another change)

Lean Software

So what has this to do with software? It turns out many of the concepts from Lean Manufacturing can be transferred. Think about an agile development practice. Typically we would

1. Write epics
2. Break down epics into stories
3. Break down stories into acceptance criteria
4. Prioritise stories in a ready column
5. Pull stories into in progress when development capacity is available
6. Implement the features defined in the stories
7. Build the software
8. Test the software
9. Release the software

How much of this is wasteful, necessary or valuable? What waste has been missed? When you document a system without actually studying it, you typically document how it should work, not how it actually works. What about

• Writing stories that were discarded
• Managing stories in the backlog
• Rework caused by bugs
• Waiting for answers to product questions
• Waiting for answers to support questions
• Waiting due to dependencies
• Waiting to release features
• Building features that were not used
• Overly burdensome reporting
• Seeking approval to release features

Notice how one form of waste ends up creating another. A bug will generate a support ticket, which often is poorly specified, leading to waiting for clarification. The longer a bug goes before being fixed, the greater the likelihood of a duplicate bug being created. Allow too many bugs to accumulate and you will need to prioritise them, and report on them, and argue why you must limit product work in order to fix them, and explain why your application quality is so low.

Create a software development process that as a low defect rate. When bugs do occur, make them visible and fix them urgently.