Kaizen, BPM, and Agile Methodologies

by Reza Shafii
05/07/2008

The Japanese business philosophy of Kaizen, also known as continuous improvement, has had a profound effect on the way organizations improve quality and operational effectiveness. In today's enterprise, the more recent concept of Business Process Management (BPM) can serve as an important enabler of continuous improvement. In this article, I explore the ways in which BPM can be used as an implementation of Kaizen principles and the reasons why the delivery of such a solution is optimized through the use of some of the concepts underlying agile development methodologies.

BPM as an Implementation of Kaizen

In the field of business administration, the term Kaizen (which literally means "good change") has its roots in 1950s Japanese industry. It is now more often translates as continuous improvement, which how I will be using the term. Although a detailed look at the philosophy of continuous improvement is beyond the scope of this article, a brief description is in order: Broadly speaking, continuous improvement brings together a set of ideas that encourage a perpetual cycle of inspect and adapt activities, with the goal of eliminating waste and improving quality. One of the principles of continuous improvement is that the users of the processes, be they counter clerks or central office executives, are the ones who are encouraged to constantly analyze their work and propose improvements. At a high level, continuous improvement is a cycle of the following activities:

  • Standardize an operation.
  • Measure the standardized operation.
  • Gauge measurements against requirements.
  • Innovate to meet requirements and improve productivity.
  • Standardize the new improved operations.

Business Process Management can be thought of as an implementation of the abstract concepts of continuous improvement at the business process level: BPM provides a framework where an organization's business processes can be optimized repeatedly. To quote Bruce Silver, a prominent expert in the BPM field, "BPM is based on the notion of a cycle of continuous process improvement." Although no standard or overwhelmingly popular BPM methodology exists currently, it is generally accepted that, at the high level, the cycle of BPM activities are as follows: Process analysis, process modeling, process implementation, and process monitoring. A comparative look at the activity cycles of continuous improvement and BPM allows us to map them together, as illustrated in Figure 1.

Continuous improvement and BPM activity cycles
Figure 1. Continuous improvement and BPM activity cycles (click the image for a larger version)

These activities are typically performed by business analysts—people who have first-hand knowledge of the processes being modeled and their domain. It is also important to note that the goal of performing BPM activities is to enhance the productivity and experience of business users who are the end users of the modeled processes.

BPM solutions are enabled by software referred to as a BPM Suite (BPMS) such as the BEA AquaLogic BPM Suite. An important attribute of BPMS software is its ability to allow business analysts to independently step through the activities of the BPM cycle. As expected, this is in line with the key principle of continuous improvement, which involves empowering end users to take an active role in optimizing the processes they deal with. In BPMS this principle translates to features that provide business analysts: visibility into the performance of existing processes, tools to analyze the outcome of potential modifications to the processes, and a dynamic environment where they can remodel processes and have them implemented.

The BPMS software products that excel are those that provide these capabilities to business analysts with as little IT involvement as possible. This allows business analysts to seamlessly step through the iterations of the BPM cycle and continuously improve the processes at hand. This always-active, uninterrupted effort to realize process enhancements brings to mind an important question: What should be the scale, in size and effort, of a single iteration of the BPM cycle? As it turns out, agile software development methodologies have something to tell us in this regard.

BPM the Agile Way

The topic of agile software methodologies is a rich one and its details are beyond the scope of this article. However, loosely defined, agile software methodologies are those that follow the principles outlined by the Agile Manifesto. This document was composed in 2001 by a number of software development leaders who were using methods that differed from the traditional all-in-one, heavy up-front design, waterfall model, and who tried to come up with a common ground between their approaches. The main driving force behind the principles of the Agile Manifesto is the idea that it is too difficult (some would say impossible) to accurately define, up-front, all customer requirements and associated software design for a software product with complexity that goes beyond the basic levels. This difficulty can lead to a variety of problems including delays in product delivery, a decrease in software quality, and an inability to respond to changing requirements. To remedy this, agile principles prescribe, among other things, an iterative approach to building software one small increment at a time and recommend the assessment of the work done in each cycle in order to improve the process and better meet customer expectations.

Agile software development iterations are typically very short, usually on the order of a few weeks to a couple of months. Each iteration also begins with a planning session and ends with a review session during which the team introspects their own performance then suggests and adapts improvements for the next iteration. This inspect and adapt approach provides a continuous improvement cycle, and methodologies that follow agile principles can therefore be thought of as an implementation of continuous improvement at the software development level.

There are at least two reasons for deducing that an agile methodology would be well suited for the iterations of a BPM cycle. First, as we have seen, BPM is delivered through BPMS software. Each modeled business process slotted for implementation can be thought of as the configuration of the BPMS software platform. By modeling a process, business analysts are, in fact, also creating a software execution model, just like developers are doing when designing and writing code. It is therefore reasonable to deduce that the same lessons learned from the failures of the waterfall model for software development, namely the difficulty of coming up with an accurate and complete set of requirements and design from the get-go and its associated problems, should also apply to process modeling. By the same token, it can be reasoned that the use of an agile methodology for BPM would help alleviate these problems. Second, given the assertion that BPM is an implementation of continuous improvement—as established in the first section of this article—it would be natural for it to call for a methodology that also embraces this philosophy, and, as we have seen, agile methodologies fit this requirement.

There is no existing standard or widespread BPM methodology today and therefore also no agile one. Instead, many organizations have defined their own. To evaluate an existing methodology, it is important to ask the following questions in order to understand its fit with the principles of the Agile Manifesto:

  • Is the methodology iterative and incremental? Agile methods need to have an inspect and adapt iteration cycle with working models delivered at the end of each iteration that incrementally add to the overall solution.
  • Are the iteration lengths relatively short (few weeks to a couple of months)? The longer the iteration lengths, the more the methodology becomes like a waterfall one and therefore is less adaptive to change and is conducive to new improvements.
  • Does the methodology allow for significant changes to requirements throughout the iteration? An agile methodology should not allow for requirement changes during the iteration itself as this is likely to lead to a chaotic "requirement churn" situation with no stable working model delivered at the end of the iteration.
  • Does the methodology embrace communication and feedback between business analysts and the business users? Agile methodologies encourage frequent face-to-face communication with the end users of the deliverable at hand throughout the iteration cycle. This is to ensure that the understanding of the requirements is not a one-time-only affair and can be fine-tuned throughout the iteration.
  • Does the methodology include large amounts of mandatory documentation? Agile methodologies emphasize simplicity and practicality. A large amount of mandatory documentation defies these principles.

If on the other hand, you are developing your own in-house BPM methodology from scratch, you might want to use an existing agile software development methodology as a baseline. When following that route, however, it is important to pick an existing methodology that is more centered around the project management aspects of software development, such as, for example Scrum, rather than those more focused on principles specific to software development itself, such as Extreme Programming (XP).

The Synergies of Agile BPM and Software Development

Everything discussed so far has concerned only the activities of the business analysts. Although the ultimate goal of BPMS technology is to allow for extensive business user involvement throughout the BPM cycle, current BPMS platforms typically require the need for some software development. This development work is usually needed to properly integrate the appropriate process nodes with their target systems and also, if need be, to customize the user interfaces used by the process for human interaction. Such work is typically needed at least for the first iteration of the implementation phase of a new process and sometimes also in subsequent improvement cycles, depending on the type of changes that were made.

When the software development work involved during the process implementation phase of BPM also follows an agile development methodology, its iterations can be synchronized with those of the BPM cycle in order to create a synergistic rhythm between the two teams. As illustrated by Figure 2, such synchronization could be performed so that the process analysis and modeling work of the BPM iteration is used as the basis of the process implementation work performed in collaboration by the business analyst and development teams.

Synchronization of BPM and software development iterations
Figure 2. Synchronization of BPM and software development iterations (click the image for a larger version)

The transition of the process model between the BPM and development cycles, marked by the blue arrows in Figure 2, should not by any means be the only points in time where the business analyst and software development teams communicate. Instead, and in true agile spirit, the two teams should work in close collaboration to ensure that the details process model are properly implemented. The process transition points instead mark the transfer of primary responsibility for a particular version of the process at hand: A new, modified version of the process is passed from the primary control of business analysts to the developers or vice versa.

Two important elements in the synchronization of the BPM and development cycles are the smoothness of the transfer of responsibility, and the simplicity of collaboration between the business analysts and developers during process implementation. These elements can be greatly enhanced by the capability of the BPMS tool to use a shared process model for both types of users. This capability eliminates the need for transformation of the process between different formats every time a transition between development-process work to business-analysis work is required (for a great analysis of this issue, refer to "Roundtripping Revisited" by Bruce Silver).

AquaLogic BPMS, for example, provides such a unified process model, and, as a result, AquaLogic BPM Studio allows users to switch the perspective of a BPM project from a "Business Analysts" profile to a "Business Architect" or "Developer" one through a simple click and without any need for the conversion of the underlying process. Each of these profiles in turn presents the user with different tools that are appropriate for the modeling and implementation needs of their role.

Finally, it is worth noting that the ability to continually deploy new and improved versions of a business process to production requires the use of a BPMS product with strong versioning capabilities. The BPMS product should be able to run multiple versions of a process at the same time in order to ensure that existing process instances of a particular version can successfully complete. A good BPMS product should also allow for the uninterrupted deployment of new versions of the same process. Naturally, AquaLogic BPM Studio offers both of these important versioning capabilities.

Conclusion

BPM can be a great source of continuous improvement for an organization's business processes. Agile software development methodologies can serve as an effective baseline for the development of a BPM methodology that enables continuous improvement and can also lead to important synergies in the interactions of the two main teams—business analysts and software developers— that participate in the delivery of the BPM cycle.

References

Reza Shafii works for BEA Systems professional services.