Sunday, July 27, 2014

The fallacy of "low hanging fruits"

Why does it take so much time to make small improvements in our software products?

I am noticing how the pace of product improvement slows down as the product becomes larger and more complex over time. What seem to anyone as a small change, perhaps a matter of modifying a few lines of codes, for some inexplicable reason drags on and on and often not gets done at all. I set out to understand why this is happening, particularly when Agile culture seems to be so firmly embraced by the product team.

I found several contributing factors, and perhaps there are more.

First, the complexity and sheer size of the software product plays a role. Put simply, making the same change in the product that is 10,000 lines of code is not the same as touching the product that is comprised of over 1 Million lines of code. While a lot depends on what exactly is touched, in general the number of dependencies is higher in larger products, and hence the likelihood of unforeseen impact is also higher.

Second, the number of people who may touch the same area of code plays a role. The more developers involved in the same or adjacent parts of the source code that need to be modified, the harder it is to avoid misunderstanding about the intentions of individual software developers. This in turn makes it harder to foresee the impact of even a seemingly straightforward change. Those of us who have ever coded can surely recall encountering some questionable coding styles, cryptic naming conventions, commented out blocks of code and lack of comments explaining the intent.

Third, and somewhat related to the previous factor, is the age of the code. Over time layers of code changes are made by different people. Each change is tackling a particular need, often a special case favoring unrelated requirements or a subset of customers. Again, a developer supposedly making a minor improvement may not necessarily know everything about the code he is touching.

Fourth factor is the number and variety of customers using the product. With just one customer, usage of the software product is specific and generally more predictable. On the other hand, with more customers the number of different scenarios, integrations and workflows raises, increasing in turn the cost of adequately designing and testing even the minor change and considering the full impact on all customers. After all what might improve the product for one set of customers may break something for other customers.

Perhaps after all not all of those "low hanging fruits" are hanging low enough or maybe they are not even fruits at all.