I’d like to talk a bit about Organic Growth in IT.
Lets say we have a Client that is using a nightly-batch flow to handle the data-processing. Running everything on a single server was fine earlier but the number of transactions is rising, due to new business and take-overs. And now they’ve run out of “night-hours” – the batch-flow into the early working hours of the next day. The disks are also running full since the processing is replicating data between IT systems.
What do we do and why did this happen?
In opinion there are the following options:
- Do nothing
- Get a bigger box
- Get more boxes
- Reduce the problem
- A combination of the other options
Let’s look at these options in order.
Any strategy starts with answering the question “Why should we do anything?”.
Maybe the results of the nightly runs are available at 10 AM and not 8 AM. But does it really matter?
Only the business can tell you if it is.
If it really is a problem … then we have to do something about it.
Get a bigger box
A simple solution is: if the box is too small get a bigger box. Some might argue that doing that is just postponing other measures. Which might be correct. But getting a bigger box may well postpone those measures long enough for the cost to pay off.
Get more boxes
In some cases it’s possible to move parts of an application to another machine. This will allow a parallelization of the work performed. If it’s possible then this might be the cheapest option – provided the new boxes don’t need to be full size.
Reduce the problem
When IT-systems experience organic growth (see below) an analysis will usually find cases where two systems perform the same calculations and even cases the data are duplicated. Refactoring the systems for reuse will reduce the workload to be performed. In serious cases this may require a complete rewrite of the systems.
It will require a thorough analysis to determine the base way to approach the problem. And in some cases a combination of hardware-upgrades, refactoring and parallelization will do the trick.
The why : Organic growth in IT
How does the situation above occur? In the best scenario the workload increased – just because the company is doing more work. However this situation might also occur by organic growth as previously mentioned.
Organic growth is a simple result of sub-optimization (a well-known topic in organizational theory – basically a process yielding less than optimal results due to lack of coordination).
The IT department serves Business, but typically a sub-department in IT will service some part of Business, whereas other parts of IT service other parts of business. Where the needs of several part of business are the same – is a place ripe for reuse. However without top-level coordination, the IT departments will produce the same functionality for each business function (this might be simple things such as automatic emailing but could also include complex CPU- and data-heavy reporting).
IT: Between a rock and a hard place
After going though the analysis above one might be tempted to think: “Now that we know what to do. Let’s just do it!”. There’s just one problem: If the above has been an exercise solely performed in the IT Department, then starting a project this size – even with funding in place – will tax the IT Departments ability to deliver new functionality on the current system due to people/knowledge constraints. If Business has not been sold on the need of the project friction between IT and Business is sure to increase.
Enterprise Architects and Business mandate
Avoiding this friction and getting Business on-board with refactoring-projects is why a strong connection between the Business needs as a whole and activities in the IT Department (again: as a whole) is critical.
Business representatives tend to focus on the needs of their corner of Business, which is completely natural and understandable. But “someone” needs to look at the Big Picture – this is the job of Enterprise Architects (EA).
Solution Architects are experts on “their” systems – all tasks on the system pass through their hands. However in order to avoid organic growth each Solution Architect needs to coordinate with Enterprise Architecture. This “extra step” in handling requests from Business is time consuming (And Business needs that new report now). A natural reaction would be to bypass EA and “just do it”.
This is why EA needs a strong mandate from Business. The inclusion of EA in any and all IT decisions should be a requirement coming from the top branches of the Business.
If EA is merely seen as slowing down the process they will be bypassed and ignored. And then we’re back to Organic Growth.