Kanban for Auditors

Background

Every time we at AgileSparks get a chance to go out of our comfort zone of technology delivery organizations, it excites us. We believe that the thinking, principles and practices we use for technology delivery are very relevant in our knowledge intensive domains as well, and keep looking for opportunities to test that belief. Our work with technology delivery organizations exposes us to other supporting activities in organizations. I recently wrote about an HR group that we worked with (and I have more updates about that group that I need to write about…). This time I want to talk about another type of activity - Audit. An Audit group within an organization is charged with auditing systems and processes mainly as part of Risk Management. Let’s look at what Agile/Kanban might mean in this context.

The context of Audit

The work in an Audit group can range from scripted audits to verify adherence to standards, to investigative research type activity to explore what is going on and identify risks. When talking to a couple of people from such an organization, I thought of Scripted Testing and Exploratory Testing, and running many many learning loops as part of the investigation being what makes a great auditor.

In addition, Audit groups can be very powerful in the organization. Healthy organizations try to maintain their independence so have them report very highly, sometimes directly to the board. With this power comes great responsibility. You really need to make sure you are doing things the right way. Your findings need to be correct. But more than that, you want to get buy in to your findings from the audited group. If you run a one-sided show, it will be hard to maintain Trust and open lines of communication. All this typically means lots of interactions between senior managers especially as findings are being reviewed and prepared for publication, and a lot of attention to details and quality of delivered Audit.

What are the challenges in Audit?

One of the huge and unique challenges is the funnel shape of the work. Imagine a group of 20 auditors. Each of them is working on their own audit project. The first steps of scoping, research, preparation of report, they can and do quite autonomously. But as they approach milestones such as “First Draft sent to audited group” and “Publication”, they need more and more involvement by the more senior auditors and managers of the group. This is due to the sensitive nature I mentioned earlier. This can create a bottleneck or at least lots of variability when you need a non-instantly-available resource like a VP of the group or it’s GM.

Another challenge is that the auditor needs to work with the audited group. He needs them to share documents, allocate time, provide access to IT systems, and various other aspects. Remember he serves a master that is high above the audited group, but also that the audited group is mainly focused on their day to day activities and typically consider the audit unwanted interference. Even if they ARE those who invited the audit they still have their daily activities to take care of, so the auditor might have to deal with lots of non-instantly-available resources - with having to wait.

Side note: This by the way is very similar to the challenges of the Agile consultant when working with a group. Regardless who invited us in, whether we are serving a very high level directive, or invited by the group itself, it is quite hard to get the group to dedicate attention to us, and it’s daily challenges are consuming it. Actually, if we look at what we talk about with Kanban and Lean, we aim to build organizations that know how to effectively balance the important (Improving Capabilities) and the urgent (Delivering current work). Maybe in some years more organizations will do this effectively, but for now, we shouldn’t be surprised to see the problems we are coming in to help organizations with…

Both the challenges above provide rationale for flooding the system with audit projects, to make sure there is good utilization of the auditors.

Another challenges that we are familiar with from other contexts is changes in priorities - Think events triggering a need for a special audit, Shifting business concerns and priorities, etc. This can wreak havoc for a yearly plan of execution. There’s also lots of variability so it is hard to know when to expect completion, and to know when we have good execution and when we are struggling.

What would a great Audit group look like?

Considering the challenges above:

• The group has a good flow of work. Auditors are generally single-tasking, achieving good flow on their main project, getting fast feedback and support from whoever they need including the audited group, their managers/superiors, and able to publish an actionable audit with minimal wasted time due to interrupts waits and rework.
• There is minimal amount of waste due to rework - While some rework and changes are to be expected if new evidence/facts emerge that drive changes in the audit, the audit group feels that unneeded rework is minimal.
• Everyone feels that from one month to the other, one audit to the next, the process becomes more fluent, there are less problems, and we are able to steadily improve on our main objectives.
• The group is able to deal with changes without losing a stride. Stakeholders know the group can deliver in a pinch as well as on it’s strategic plans. The group can rally it’s forces around the highest-priority work.
• Time is spent on doing the work or improving. Minimal time is wasted on replanning and other non-value-add activities

How can Kanban help?

Well, you can say my vision of a great group is very influenced by Lean/Kanban and you probably will be right. But if you do see the above as positive, let’s try to see how the Kanban Method can help achieve it.

The Kanban Method helps you improve in an evolutionary way using Flow Control as one of it’s key tenets. You start with the way you currently work without making any changes. First thing you do is visualizing the flow of work and integrating this visualization into the way you manage your daily work as well as your planning. When you start to enforce Flow Control using “Work in Progress Limits” you will certainly run into the challenges above and need to do something about them. You will see rework in the form of small tickets making their way back to earlier stages of the work, similar to how we portray defects/bug in software development kanban.

Kanban itself will not tell you how. It will nudge you to start considering cycle time efficiency and not just resource efficiency.

You will make the current policies explicit. Policies such as expectations from audited groups, expectations from managers. How you currently allocate the time of the managers to the various projects - Is it by priority of the audit? By it’s age in the system? combination? Even the discussion around the policies will drive some questions and ideas for improvement. Choose carefully what you want to experiment with.

Most importantly, the Kanban Method will not provide you with a great solution up front. Far from it. It will setup a diagnostic system that will show you where you are currently, and give you a system you can use to evolve.

In order for it to actually serve you, you will need to define where you want to go, what concrete capabilities you want to improve. Then you can use something like the Mike Rother’s Improvement Kata to work in small steps towards those goals.

At some point, maybe even early on, you will say something like “Kanban doesn’t provide solutions”. That’s partially true. You WILL get Flow Control, you WILL get something that gives you opportunities/triggers to think about how the work goes, not just do the work. You WILL get pointed towards areas which are currently the biggest obstacles to further improvement in Flow. But you WON’T get all the solutions you need to deal with those obstacles. This is by design.

The Kanban Method talks about improving collaboratively using models. Improving collaboratively means the people involved in the system being part of designing the improvement experiments, which increases buy-in. The collaboration can also be around Choosing the model to try next. There is no playbook giving you a perfect solution, although if several other groups have tried Kanban for an Audit group or a similar context before you might have some good ideas to start with. But there is no guarantee they will help. One of the important models we use is viewing our contexts and systems as complex. In complex environments you need to try things, see if they help, and then decide whether to reinforce or throw away and try something else.

You might try the Theory of Constraints’s five focusing steps around bottlenecks which might give you ideas like subordinating the auditors to the external groups - What would it take to expedite response time? Clarity of the report? Some other thing? If you subordinate to the GM who’s the ultimate internal bottleneck what would it mean? Trying to minimize rework and passes thru his review? When trying to subordinate you might find useful guidance in the work done in IT around Specification by Example and Test-Driven approaches. If we consider Reviews as steps of inspecting quality into the Audit, maybe approaches that build quality in by specifying expectations and tests up front and involving the reviewers earlier on will be more effective? Yes it might require time of the reviewer earlier, but against your intuition it might be the global optimum.

You might try chunking an audit to smaller pieces also called Small Batches. Maybe you can start considering an Audit like a backlog of areas to work on, and drive from resources and time rather than scope. Perhaps even if the scope is fixed, you will reduce the effort if you get faster feedback on initial findings. Maybe you will improve flow if you need the audited group and the internal reviewers for smaller chunks of time versus long reviews?

Summary

This is not a Case Study of Kanban for Audit groups. Maybe sometime in the future. The important point is that even if it was, it wouldn’t necessarily be something you could copycat even if your environment sounds exactly like what I described.

What you can copy is the approach towards improvement. Identifying challenges. Picturing the Ideal Situation. Using Kanban to visualize where you are, establish Flow Control, and then start to experiment together with your people using small evolutionary steps of your explicit policies guided by models that you choose to try and apply.

You might get lucky and get good results by using the models/experiments outlined above. If so then great. We might find that a certain set of models is a great fit for a certain environment. As we know, in software development we do have sets of models that have had great success (Feature Teams, Extreme Programming Engineering Practices, Test/Behaviour-Driven Approaches , etc).

You will also find that the word Audit can probably be replaced in this article with any other type of Knowledge Work. The Kanban Method has a wide applicability. The models have a wide applicability too. There are models that are domain-specific, but I think careful examination even of those will provide insights to other domains. This is the cool factor about the Kanban Method. The Sky’s the limit to what we can do with it, and we are barely scratching the surface…

Maybe one day we will even apply Kanban to the work of Agile Consultants. We certainly are a Knowledge Work domain with lots of variability. We certainly have flow control challenges. Maybe in 2012 :-)