Software through the helicoidal lens

Helicoidal, what a lovely word. I’d never heard it before, but whilst reading about Yam cultivation in Papua New Guinea (what do you read for fun?) I came across it as a description for the process of growing yams.

What it describes is a process that is not cyclical, instead, it is more like an elongated spring, with each turn never quite returning to the starting point. As I pondered this it struck me that software development and agile in particular is very much like this.

However, in all the diagrams you see, things are always denoted by a circle. This is a useful way of thinking about it but not sufficient. It is a flattened view and gives rise to unhelpful mental models, whereby we do an iteration and move onto the next one. The circle implies completion, an ending. As if we are done, ready to start again, but really we are moving forward as well as in a circle.

This is a critical difference, as each turn takes us around and forward, therefore, we are learning but also changing. This slightly different view can be used to unlock some of the mental blocks that the circle model puts in place.

For example, instead of thinking in terms of a sprint or iteration delivering a thing it instead suggests that like a turn of a kaleidoscope it brings us to a new beginning, a new place to start from not a defined end. That is key because we are always moving forward and therefore, each turn distorts the world/thing.

Through that distortion, we learn, but it means we cannot simply repeat the cycle, we are in a new cycle, itself moving forward and distorting. The interesting thing, of course, is the implications of this and what it means for how we approach that transition to the next turn.

It is here that many software development models fall down, as they assume we simply move forward from the past to the future state in a neat line, but instead, we are actually in a new future state.

The backlog is a list of things to do. A linear progression from one item to the next but if things are helicoidal we, in fact, need a new backlog as the turn as moved us forward and around. New things have come into view, new futures are possible.

Now I’m not suggesting that this helicoidal lense is the bee all and end all, but it’s interesting the things it brings into view, how it helps us question practice. So go forth and look for the helicoidal, but more importantly look at things through the helicoidal lense and question what that does to them. How does it distort our understanding of the backlog or the sprint review?

References

Ludovic Coupaye, “Yams as Vernacular Methodology? Approaching Vital Process throughTechnical Processes”, in Des êtres vivants et des artefacts, Paris (“Les actes”), 2016, [Enligne], mis en ligne le 20 janvier 2016, Consulté le 08 février 2016. URL: http://actesbranly.revues.org/673

 

Thinking long thoughts of thinking short.

For those not in the know, I have a cat – a somewhat fancy grey beast that’s not really meant to be outdoors unattended, however, she’s taken to being walked on a lead – so my days are often punctuated with a cat walk in the garden.

Now Lulu, that being her name, is a cat (I may have already mentioned) and therefore, is somewhat prone to standing still and staring for long periods of time. These moments, however, are excellent for pondering the intricacies of the world. In many ways, my cat is my mindful muse.

For a moment I shift into the cat’s temporal frame of reference, not thinking about the past or future. I am still. Mentally and physically, able to attend to the present, to the thoughts in my mind.

This is of course when I’ve forgotten to take my phone out with me, otherwise, it’s a quick check of Twitter and headlong tumble into the vertiginous world of the machine.

Anyway, I digress. Today, in one of these phoneless moments of pause, I began to think about thinking. How I think, how my thinking works, what thinking is and how different modes of attention often produce radically different modes of thinking. Thinking with and without the cat for example.

To start with I thought about the recursive nature of the questions. Thinking about thinking and how infinitely recursive that could be. Thinking about thinking about thinking… ad infinitum.

Now, this type of infinite recursion of thought always has a texture in my mind. It has a sensation, a feeling if you will. It is not abstract but somehow embodied. I can feel the thoughts in my mind. Like chewing on a toffee, it has a sensory pleasure with the added frisson that at any moment a filling could be tugged loose.

These thoughts, in turn, led me to think further about what other types of thinking I do, and after a turning this over in my mind for a time, I considered the notion of short and longhand thinking.

By this, I mean how I think about things. Sometimes thoughts are, to use Heideggerian terminology, ready-to-hand and at other times they are present-at-hand.

What this means is that sometimes my thinking can take a shortcut (ready-at-hand), I don’t need to explore the deeper ramifications, the thought is ready to be used, almost fully formed.

At other times my thoughts are longhand (present-at-hand). They are unready, I have to go through a long a laborious process of going back down to the firm conceptual ground and then working my way up and through the morass to clear air. Sometimes this takes minutes, sometimes years. But once I’ve cut my way through the thickets the thought is transformed, it is ready-at-hand, a new symbol for my shorthand vocabulary.

But where am I going with this, other than a simple exercise in writing and thinking?

It occurred to me that I have a propensity to longhand thinking, which can be frustrating. Frustrating, because of my need to find some solid conceptual footing before I can work my way up. This is particularly noticeable when I come across a new idea. I rarely accept the idea as is, I must carefully reach back down to solid theoretical footing and build the mental scaffolding up until I reach the original idea. Only for that labour to be erased as it transitions to shorthand again (it’s utterly maddening).

However, I’ve worked with other people who are simply able to skip along with their thinking, like a stone across a pond. Hopping from one thing to another, accumulating the same knowledge without ever needing to sink into the pool.

Happily, this can also happen with me, sometimes I too can skip over the water to new knowledge, but this is sadly rare for me and depends on many factors, most of which I can’t always identify. Although I’m almost certain walking and sunlight help.

The fact that my modes of thinking are not fixed leads us to some questions: how often do we take account of how we or others think about things (particularly in technology.) and how often are we mindful to the fact that sometimes our thoughts will need to be longhand and at others short?

It is these thinking factors that can deeply influence how we make things and how our teams and organisations work, but rarely do we attend to them.

Do we ever cultivate system/worlds where we can slip in and out of different modes of thinking depending on who we are or the objects and practices that bring that system/world into being?

Instead, we often look to make the system/world uniform. To assume that variability is a bug and not a feature. We fail to recognise the temporality and textures of our thoughts, or in short, that sometimes we are the human and at other times we are the cat.

The false dichotomy of output vs outcome

Hello, not terribly avid readers, long time no post! Ironically in a blog post about outputs and outcomes I’ve really been awfully lax posting to this blog. Put it down to my in-progress MSc and juggling more than a circus cephalopod. However, a tweet caught my eye today and I felt compelled by the power of the internet to respond.

The tweet in question is similar to many I’ve seen (tweet omitted to protect the innocent) where a product manager extolls the virtues of outcomes vs output. Optimise not for output (velocity) dear friends but for the outcomes (value).

Now, these are words I’ve uttered myself from time to time and felt pretty good about it too. I’ve guided the unenlightened to the righteous path of value delivery. But as you may infer from my tone and the title of this post, that somewhat gives it away, I’ve come to a more nuanced perspective.

This perspective has been solidified by watching several teams struggle to ship anything! So obsessed with the value that nothing gets done or the cycle time to delivery rivals that of the big bang waterfall projects of old.

When challenged about the lack of actually shipping, the old saw of outcome over output is normally invoked. We have to do more research, internal tests, iterations and so on because we’re optimising for an outcome. We have to ensure we get the outcome we envisage.

Part of the problem here is the need to get the “outcome you envisaged” as if the outcome is ultimately knowable. But of course to know something we must have the thing in front of us so that it may be knowable. We cannot know things that don’t exist, we can only infer their existence.

The code is just a text file until it is run, and only in its running do you find the bugs in the code (or its outcomes) or its effectiveness. The code until running is not knowable in a material way. The materiality of the object emerges from the running of the code.

And there we have it, the rub, the outcome is nothing without output. The issue is that they are mutually dependent. You cannot push one without the other. They are in correspondence and more than that they are slippery and complex. Not suited to a binary opposition at all.

So the next time you or someone else invokes this false dichotomy, stop for a moment to think – has the talk of outcome just become a hackneyed old refrain used to justify a lack of output. Because, and here’s the kicker, your outcome is utterly dependant on you actually shipping and you cannot know the outcome until it has been output.

What is the purpose of Agile?

It’s been a while since I posted. This is a reflection not of busyness or malaise, but that I’ve reached a new phase in my Agile journey. Things that felt certain are once again in flux as I question many of my previous assumptions about Agile.

This has led me to one fundamental question:

“What is/was the purpose of Agile?”

In many ways Agile was created as a shared space in which we could explore better ways of delivering software. It defined the boundaries of the space so that constructive dialogue could emerge. However, it would appear that those boundaries have not necessarily framed the right questions.

It has helped us define new methods and techniques for many issues in software development, such as a lack of feedback and learning. However, what it didn’t capture correctly is a focus on outcome and therefore became bogged down in methodology.

The Agile space was and is a fertile breeding ground for methods and tools, but it does not, by its very nature, help us devise ways of improving outcomes or delivery of value. We only need look at the plethora of methodologies and the ecosystems of agile tool vendors to see clear evidence that Agile, although successful in spreading and amplifying many positive behaviours, has also amplified behaviours which actively damage the Agile movement.

Therefore it is clear that it is time to re-frame Agile. To be clear about its purpose as a defined space for cultivating improvements in software development and delivery – for those improvements to be about the outcomes of a piece of software or the value that it delivers. In fact one of the first questions is the definition of value itself. Value to us as developers, value to society, value to business, value to customers?

Outcome based Agile is not new. In fact Tom Gilb wrote a paper 2010 speaking about Value-Driven development. Not only  that, Tom Glib has probably been arguing for this very thing since the 1960’s.

As part of this re-framing, we need to be clear that the framing is for the creation of a shared communication space and language. This space would be for collaboration, not competition, and not for commercial gain. For example, materials and books should be released for the benefit of the community and not for the benefit of a small cabal of vendors. In this way we ensure that the very essence of what it is we are trying to achieve is in the DNA of the movement. That the value is not in the system conditions but the outcomes that the system enables (in short the creation of a system is not the outcome but the value it delivers is).

Through questioning of Agile and by examining my thoughts and beliefs I’ve come to discover more and have taken another step forward on my Agile journey. This is a valuable technique and I suggest you consider taking some time to question the purpose of Agile/Lean/Kanban/Scrum now and in the future. It is by questioning our beliefs and ideas we come to learn so much more about them.

Would you consider exploring these questions further in focussed discussions?

References and further reading:

https://flowchainsensei.wordpress.com/2010/08/15/tom-gilb-laments-the-10-wasted-years-of-agile/

AgileByExample 2013: Tom Gilb – Agility is the Tool, not the Purpose

12 TOUGH QUESTIONS May06.pdf – Tom Gilb & Kai Gilb

Evolving organisational structures

There’s a debate going on at the moment about Zappos and their imposition/adoption of a new management structure based on Holocracy.

Basically Zappos want people to self manage and to speed up the process they’ve mandated it, offering severance to anyone who doesn’t want to work in this way. Broadly there seems to be a couple of responses to this:

  1. It’s a good thing because self organisation is a good idea
  2. It’s a bad thing because self organisation is a bad idea

It’s the classic media/internet response. Binary by nature (as if the very essence, 1s and 0s, of the technology underpinning the internet somehow makes our thinking more digital in nature… Anyway I digress). However, the issue here is not what one company decided to do and thinks is right for themselves, but the fact that one company’s structure or way of working is held up as a good or bad thing.

Organisations evolve. They are complex, the larger they are the more complex they become. The conditions in which they operate are unique to them and a product of the unique individuals that inhabit them.

Therefore a structure cannot be imposed or decided upon. However it can be evolved. It is not the management structures or way of working that needs to be codified and repeated but the evolutionary journey and thinking that codifies improvements into the organisation.

The response to the Zappos story also reveals another interesting facet to how we think about evolving organisations: that evolution is a constant conveyor belt of improvements. That somehow a new form of organisation is better than a previous form just because it was created after the first form. Evolution is the adaptation of an organism to the conditions it finds its self in at the time – not some constant refinement of the inferior to the superior.

By seducing ourselves that we are somehow doing things better than we were in the past, we fall into the trap of looking at the previous incarnation of organisations as somehow inferior and therefore miss the context in which they operated at the time.

Organisations need to stop looking at each other for inspiration or trying to improve on the past. We need to codify what is valuable in our current situation and reduce that which is no longer useful. This is how evolution works. It does not work by imposing a new supposedly superior form of structure borrowed without context from others.

Projects are actively harming your products

There are many things that can cause a software project to fail. I suspect we’ve all experienced a failed project at some time or another. The users didn’t like it, it was over budget, it was the wrong thing, it was late, it was buggy, it got cancelled – the list is often endless. But there is one common factor, the project itself. By project I mean a temporary team formed to work together to complete a batch of work.

What’s so wrong with projects?

Projects by their very nature are the antithesis of software development. Software is an evolving and flexible medium. It doesn’t have a fixed end state. There are no pieces of software (or websites for that matter) I use today that didn’t exist in another form in the past. The software and the sites evolved and changed based on new technologies, shifts in the market and customer feedback. This is the very definition of software development. However, when we come to structuring the work of software development, we usually use a project model. A project infers a start and an end. A point that when we reach it, means the project is complete – we can pack up our tools and walk away. When we undertake a software project we draw an artificial line in the sand (usually based on some inaccurate estimates) and pretend that the project will end.

This artificial end point leads to thinking in terms of big releases. The reason we tend to think in terms of big bang releases with projects is that when the project ends we assume that we will no longer be able to deliver any more value. We therefore need to ensure we have as much jammed into the project as possible, because when the project finishes we won’t have a chance to add anything that’s missing. Couple that with the knowledge that our team is likely to be disbanded at the end of the project and we have a situation where we have to ensure as much as possible is delivered for fear of missing an opportunity, forcing us to into a big bang release just because we used the project model.

The waste of project teams

When we incept a project we’ll build a new team from the remnants of the last project team, or we might co-opt some departments together to form an ad-hoc team. This new team will dutifully plough into the new project, slowly learning how to become more effective.

This is a known phenomena described in Tuckman’s stages of group development:  Forming – Storming – Norming – Performing

Working in projects is counter to this dynamic and means that just as we get to the performing stage we take a pickaxe to the team and break it apart, often loosing much of the social capital and working practices that have been built up.

Build capabilities not projects

So what’s the answer? If projects are actually damaging to how we build software what can we do instead?

I’d suggest that we begin to think in terms of building capability to deliver ongoing software improvements – rather than building pre-defined projects. What I mean by this is evolving an environment where we think in terms of continuous delivery of value rather than working on large chunks of value in a project model.

A project is a great big chunk of value, where the value is not realised until the end. This is very risky as any delays will delay the whole set of value. If we break down the project into a continuous stream of smaller chunks of value we spread that risk out. We also allow for course corrections in case we discover something new or the market changes underneath us.

To do this we’ll need to look at how we form and keep teams together. We’ll also need to look at how the work gets to the team and how this work is broken down so that it delivers value in smaller chunks.

In addition we’ll also need to look at how we release software. As without the ability to release small chunks of software we’re doomed to always work in projects, because how we deliver the value dictates the model for creating the value in the first place.

Less projects less estimation

Another benefit to moving away from projects is an intrinsic reduction in the amount of estimation we do. In fact you might even get to the point where you can eliminate estimation altogether. Let’s imagine a big project to rebuild a website, re-write all the copy and migrate it to a new CMS platform. There’s a ton of risk here. The normal business strategy is to estimate and plan in order to work out what might happen so we can understand and mitigate the risk.

However, we can never know exactly what is going to happen and we’re estimating at the point of least information. Which means the moment we start executing the project we’ll find that our estimates are wrong and we’ll have wasted the time we spent estimating. Also, by the time we realise our estimates are wrong they’ll have become commitments. At this point the whole management and reporting model for the project becomes a fig leaf covering the truth of the situation. Not only that we’ve probably got a newly formed team (usually with external experts) that’s learning how to work with one another – so the chances of the estimates ever being right was likely negligible.

Instead imagine the same scenario where we have a stable team, a process optimised for the continuous flow of value and a mindset whereby we look to break the project down into manageable chunks of value.

Now instead of estimating we can decide on the smallest chunk that will deliver value to start with. That’s our risk exposure – the cost of the team for that short period. Then we can see how much is delivered and use real data to do future forecasts on when the rest of the value will be delivered based on that data. Or we can have a phased release of funds for the work. Deliver the first chunk of value, then based on the value delivered get funding for the next chunk. In the phased funding model we don’t even need to estimate, we just get on with the work at hand. I’ve talked about this kind of approach before in my go/no go blog post.

Time to move beyond projects

This has been a fairly quick overview of why I think projects are a bad idea and why I think we should be looking for solutions outside of the traditional project model. This non-project future is not dependant on any particular methodology and you can start thinking about how this might work for you even if you’re not doing Agile. This is just a call to start the debate within your organisation. Are projects the best way we can organise the work? Are projects the best way for us to deliver value to our customers? It may be in some instances they are but I ask you to consider the alternatives and to run some experiments.

Comments welcome as always.

Inspiration, references and further reading:

A recommended Agile reading list

The following books are the ones I have found most influential and useful in my Agile journey. I recommend them as essential reading for anyone working in or with Agile teams.

These books will help both managers and teams shift to a more Agile mindset. They also help frame why focusing on things like “Flow” over “Resource efficiency” will get you better results than just following practices blindly. Lots of them have practical advice for how to implement Agile or improving Agile practices and give you some of the theory behind the practice.

My Agile library

What books have you found useful/influential? Let me know in the comments!

P.S. Here’s the list of the books in the pic (with Amazon links, should you want to buy them):

  1. This is Lean: Resolving the Efficiency Paradox
  2. Implementing Beyond Budgeting: Unlocking the Performance Potential
  3. Kanban
  4. User Story Mapping: Discover the Whole Story, Build the Right Product
  5. Kanban from the Inside
  6. Scrum and XP from the Trenches (Enterprise Software Development)
  7. Succeeding with Agile: Software Development Using Scrum (Addison-Wesley Signature)
  8. Lean from the Trenches: Managing Large-Scale Projects with Kanban
  9. Agile Retrospectives: Making Good Teams Great (Pragmatic Programmers)

Are your clients eroding your culture?

I’ve been reading quite a bit about encouraging or growing certain types of culture in organisations. Cultures which promote continuous improvement or fostering greater empowerment with the organisation. However it seems that one vital facet of the culture of your organisation is always omitted. That of your clients (I’m speaking from an agency perspective here, although this may also be applicable in other contexts).

Much of what is written about changing working cultures is about the internal world of the organisation, but this is only one facet of what makes an organisation. An organisation is made up of many complex interactions and at these points of interaction is where culture appears. For example: how we behave to one another and the shorthands we use. These unthinking actions are the cultural life of the organisation.

Our clients are another cultural interaction point. The way they interact with us can have amplifying or cooling effects on the cultures we are trying to grow. We should pay attention to how a client behaviour may permeate down into our organisation as this behaviour may erode the work we are doing to embed certain values. For instance trust and transparency. Perhaps the client embraces our new way of thinking and reinforces our work culture. Or perhaps more likely they are mystified as we begin to talk in a language alien to them and reject our new behaviours.

So when we look to our organisation in terms of a culture change, or new values we want to adopt, perhaps we should spend some time thinking about how this will effect our clients. Work out strategies for bringing them on the journey with us, or even no longer working with clients that hold us back from our wider goals.

Our organisational growth (wether we like it or not) is subject to factors both within and outside of our control. We should therefore attempt to ensure that the direction of growth is towards the light and is not stunted by unwelcome or unperceived factors.

Are you in denial about value?

I’d like to talk to you about something, lean closer dear reader as I’m going to have to whisper this. The intricate ways you are accounting for value on your Agile projects is probably wrong.

Maybe you don’t believe me, maybe you’ve always had doubts, but a lot of the time I see teams who can confidently proclaim they are delivering value in their iterations. They can show me burndown charts, burnup charts, velocity and a whole raft of metrics. But the real question is: who did they deliver the value to?

This is in fact the key question when considering value. If you are delivering user stories just to say you’ve delivered them and to notch off another mark on the burn down chart, you are very much in denial. There is no value here because more often than not you’re not actually delivering to anyone (your Product Owner doesn’t count, nor does your client if you’re doing contract development).

Value is only realised when the code or solution is in production. That means it’s thrumming away on a live server somewhere, not sitting in your source code repository waiting for the “big” release at the end. Now it should be argued that you still don’t know that it has value. As more often than not there are no metrics gathered showing actual customers using and finding value in your new feature. However for the purposes of this and as a first step to thinking about value in a more realistic way we’ll imagine an ideal world where any feature developed has some value to your customers (no matter how rare that really is).

If value is not in delivering user stories or tasks, then where is it? Well, as you’ll probably already be thinking after reading the above, the value is in the release. Everything else is just work to get you to the release.

That prototype you built, it has no value, it’s just a step to release. Something to steer you towards a more valuable release. That feature you built that’s waiting in source control for deployment. Again it has no value.

This normally means that your Scrum board, task board or Kanban board is a lie. Take a look at it now. Does it represent the steps to realise value? Does it only have cards on it that are representation of units of value (things that can actually be released when they get to the end of the board)? There’s a good chance that they are not. They’re probably just tasks waiting to get batched into a release. Your board is probably just a fancy to do list. Useful but not really representative of the actual process. Also not terribly useful in tracking if value is being delivered in a timely and consistent fashion.

I’d suggest that you try and visualise your actual unit of value on a board. So in most cases this will be a project or a release. It might not feel as nice as having a lovely busy board but at least it will be truthful about the situation. A situation where huge chunks of work are being shoved through the system and blocking up the whole process (all whilst your previous board was showing a lovely stream of falsehoods). This will also go some way to address the feeling that you are working incrementally when in reality you have a micro-optimised development process that is thrashing through large projects. Finally puncturing the illusion that you are agile/incremental/iterative when really you are just blind to the whole value stream.

Don’t worry you can keep your old board, but now it will be a representation of reality – in that it’s just a way to help you organise the work within the large batch of value that you’re trying to deliver.

Once you can actually see the problem then you can start to address it. Slim down your projects, do smaller releases. Work towards single piece flow. But that process can’t start whilst you’re still in denial about the value that you are delivering.

Photo Credit:  ShnapThat!  – https://www.flickr.com/photos/shnapthat/6464681311/

Agile is not a methodology, it’s an Albatross…

One of the hardest things to understand when starting out on the Agile journey, is that although most often sold as a methodology, this is looking at it at the wrong way. Scrum is a methodology, as is Kanban or DSDM or XP. But none of these are Agile itself.

The problem is that often the first experience of anything termed Agile are the methods and tools of a particular methodology. It’s how I was first introduced to it. The methods and tools however are of no benefit in and of themselves. Yes, they can help uncover issues in your current process, Yes they may have some use in helping frame agile in your mind. But this is really of limited value, as a new set of tools is not the answer to a dysfunctional organisation or industry. Frankly that’s fiddling whilst Rome burns. A methodology is not meaningful unless accompanied by a change in mindset, without that you’re just going through the motions.

The problem with Agile today is the word Agile has baggage. So much so that it is pretty much meaningless. If I asked a room full of people, who thought they knew what Agile was, what the term Agile meant, a large portion would describe a methodology (which it isn’t) and the others would probably all give me conflicting answers.

If Agile is not a methodology and it’s not really got a definition – what use is it as term?

The language and the terms we use, help define and bind us as a group. Shared terms, shared understanding, shared beliefs.

So the real meaning of Agile (if we can reclaim the term from the methodologists) is that it is the expression of a collective mindset. A shared set of values if you will. Are those values the ones defined in the Agile manifesto? I don’t think so. I think the mindset and values are evolving and we’re yet to see anything close to a definitive set of values. That’s why we need a collective term to represent and be the vessel for the inchoate and evolving values.

The language we use and the definitions that they collectively hold are important. Shared language is what defines the groups that we belong to. That the word Agile is now so laden with baggage that it has lost its meaning is perhaps a symptom of a deeper underlying issue. An expression of a community that no longer has a shared vision or language.

A community that is becoming so myopic and fixated on the methodology and tools that it is losing its collective vision. In its place a morass of commercial vendors competing to reduce Agile to a packaged and marketable commodity.

Like the term Free software morphing to open source isn’t it time to define a new shared language and reclaim the essence of Agile without the methodology?