Continuous Delivery for Software Architects

2013's hot topic: Continuous Delivery

Summary

Continuous Delivery is the next goal for modern software projects. This means that product can be delivered quickly and at short notice. Effectively it expands the concept of continuous integration (which is focused on making sure that commits are not destabilizing the unit tests), all the way to delivery of those changes to final product.

To achieve this, one must increase the modularity of software such that small changes in the source have a very small ripple effect on the objects that change, or even must be redelivered. State of the art software development practice is well versed at OO, separation of concerns, and reuse at the class and object level, but tends to neglect those same principles at the package, module (.war) and application level. The cost of this is that small changes force the recompilation, retest and redelivery of massive portions of the software. This makes it difficult to ensure that a given change is patch compatible and easy to update.

Techniques are highlighted that help reach toward the continuous delivery goal such as

• considering modularity at the java package level
• monitoring the cyclometric compelxity and afferent / efferent coupling of classes, packages and wars
• use of toggles (Togglz) to hide new features until they are ready, yet still allow delivery (or dark deployment) of incremental changes
• include small integration, QA and release and operations into the sprints (adding to the analysis/design/development/testing/demo cycles being done by the dev teams)
• bring the pain forward. To encourage automation and reduction of friction in the development environment, do unpleasant tasks more often. Try regular timeboxing efforts to improve automation or process improvements.
• reduce coupling between components to reduce big-bang deployments, hard upgrades and resistance to change
• Identify bounded contexts, the areas in which changes can be isolated.
• use consumer driven contracts to ensure product meets its requirements others have on it (these contracts are simply the agreement to run the partners tests as part of your own development cycle).

In addition, the role of an architect is changing in a continuous delivery world. Now architecture is something that is not ivory tower, done up front, but pursued and adjusted during product development. This requires and architect who is very involved in the code base on a regular basis. The evolution of the code is going to influence the architecture, and answer questions that simply were not knowable at the product start. This help reduce the need for an architect to make the hardest descisions about a product at a time when the least is known about how it will work.

Continuous Delivery architecture shoots for a good local maximal design, and then continually pivots toward a better one. Traditional architecture shoots for the global maximal design, and rarely achieves it. To this end, architecture becomes focused on being "pivot" friendly (tolerant of change, deferring complex decisions until late as possible). The cost paid for less design up front, is more design on a continual basis. To support this, all layers (including user experience, must provide constant feedback to the architects).

This feedback is collected through the use of metrics and tools.

• cyclomatic complexity - function of lines of code and number of branches in a method/function
• afferent / efferent coupling - number of components depending upon / number of dependencies on others

Cool technique: Kiviat Graphs "doughnut of perfection"

Engineering practices that support these goals include

• hiding new functionality until it is finished
• making all changes incrementally as a series of small changes, each of which is releasable
• canary releases (multi-variant testing)
• branch by abstraction to make large-scale changes
  • refactor the code to function against an interface
  • refactor the library to implement that library
  • create a new libary that implements that interface
  • bind the code to that new library
• strangler patterns - make someting new that obsolets a small percentage of something old, put them live together, rince, repeat (was a possible strategy for the replacement of IO by Pulsar)
  • Useful to make the new system simply more compelling than the old. The old never goes away, the new one is just far more useful
• use components to decouple parts of your application that change at different rates (components are compile-time dependencies)
  • Structure 101 - handy tool to untangle cyclical dependencies
  • JDepend - a way to write unit tests to enforce dependency limitations (util doesn't include web, but web can include util)

Use the scientific method, using metrics to minimize friction in the system to allow improved collaboration and continous delivery.

Details

• Wanting for tiger team testing earlier during sprints is a desire for continuous delivery, shorter feedback cycles to inform decisions on the architecture
• The machine used for test is a product of source file changes, e.g. source describes the testing environment needed to execute the other code
• We want small changes to have a small ripple effect
• Traditionally architecture shoots for global maximum of architectural perfectness, when standing on a flat plane (aka, what is the tallest building when standing on street level, it is hard to see until you are able to go up in a building to get a better vantage point)
• contracts with partners are really and agreement to run their acceptance tests and keep them passing
• don't collect metrics you don't use
• Everyone integrates their code at least once a day
• Use the strangler pattern to obsolete old software
• Aided by keeping complexity low, use of cyclometric complexity and afferent, efferent complexity.
• Minimize the use of feature branches
• Feature toggles