Why an Agile Process is not Enough
Four Thoughts on why “Programmer Tooling” for the Business Guys is a Necessary Complement to Agile Processes
Agile processes have taken hold in many organisations: more or less everybody uses some version of Scrum, you can see Kanban-boards all over the place, and many development teams use (cherry-picked) subsets of XP, such as test-driven development.
However, for an organisation to become really agile, two more ingredients are required. One is agile management. We all know these stories along the lines of this:
How nice of you devs that you use Scrum, but can you please tell us your backlog until 2020, including the overall total estimated effort?
That doesn’t work. But this post is not about that. Rather it is about the agility of the core business processes/rules/calculations/behaviors. Those need to be evolve quickly, as external requirements change or the “product people” breed new ideas. Examples include insurance products, telecom pricing plans, or medical diagnostic algorithms. As you probably suspect if you know me and read this, I am of the firm opinion that domain specific languages are the way to improve the degree of agility here. We have worked in all of the aforementioned domains, as shown by the IEEE Software article Projecting a Modular Future. Subsequent posts will provide more details. This one is a me on a soapbox, preaching some generalities about how I think business and devs should interact. It has been on my mind for years. It wants out.
Why only tools for technical people?
The technical people use tools all over the place: powerful IDEs with integrated refactorings, test automation, build servers, docker-based semi-automatic deployment. We developers could not imagine working without those tools. They are a key ingredient to productivity and quality. Yet, we somehow accept the idea that business or product people specify the core behaviors in tools such as Word or Doors. Maybe we let them use Cucumber to write acceptance tests. But none of the features that make us productive and help us produce quality are available to them. Why do we accept this? I think we shouldn’t. Let us have/build/provide languages and tools that are suitable to the business guy, but provide the same productivity and automation features, to make them just as agile.
We have Devops. Where is Devbus?
Ok, Devbus is a stupid term, but you get my point. The developer community has learned that, in order to achieve today’s expected time-to-market and reaction times, developers and operations people have to integrate more closely. Both in terms of the process, but also in terms of shared tools. A major part of devops is to do in operations what developers have done for a long time: write programs, use tools and automate. We have to set a similar goal with business people who are responsible for the actual algorithmic core of our applications. A joint process, with tools and automation tailored to the tasks of the business people. It’s the obvious next step on the path to end-to-end agility.
How can you expect somebody to take responsibility when they have no way to assure quality?
Developers, agile or not, are expected to take responsibility for the code they write. And of course, professional developers are willing to take this responsibility because they write tests for their code. This way, they can be reasonably certain that the code works. And even if it has the (unavoidable) bug, they are not criticised overly harshly if they have tested the code to a reasonable level. How do business people test their product specifications in Word? They don’t. Because they can’t. So we cannot expect them to really own the core algorithms. So let’s give them better tools. Let’s enable business people to write/program/model/specify all the core algorithms that go into today’s complex business applications, and let’s make sure these tools support in-IDE execution, so they can simulate the behaviors and write tests. And the execution of those tests should be automated, just like our tests, including on the build server. Test-driven business development FTW.
Complete separation of business logic from implementation technology
What is the core problem with legacy software? IMHO the problem is that, if (or rather, when) we decide that we want to change the implementation technology, we have to somehow “extract” the application logic from the implementation code, a challenge generally known as reverse engineering. This is really tough: as Charles Simonyi once said, it’s like trying to reassemble the eggs after they had been processed into scrambled eggs. It’s maybe not completely impossible, but very hard. More seriously speaking, the problem is that you have to recover domain-level semantics from the lower-level semantics used to represent the domain in a programming language. A fundamentally better approach (to overstretch the metaphor) is to keep the eggs in their original form, and only mash them up temporarily when you want the scrambled eggs. You express the business logic in a pure form with a semantically well-defined language, independent of a particular implementation. You generate the implementation (or feed the models into an interpreter) for execution. You can then grab another implementation technology by generating differently (or reimplementing the interpreter). That is just so much better than anything you can achieve by hiding the business logic in implementation code. And by the way: if you want to change your modeling/programming/specification tool, you can just write a generator that generates the model in a format suitable for the new tool: you know your language, not a problem. And then your run all your tests to re-establish confidence (I do know of large-scale projects where this has been done successfully). This approach gives you a degree of sustainability you cannot achieve any other way, AFAIK. Long-term Agility!
Wrap up
Ok, leaving my soapbox. I hope I was able to convince you that end-to-end agility really requires to give business people the same tools as we developers have, of course adapted to their particular needs and skills. In my next post, I will outline what this means. So long!
