Blog Post The Need for a ‘Magic Engine’

Is it really valid to use comparisons with other industries when it comes to IT?

When looking at IT, one common comparison used is the automobile industry. Vehicles have evolved from being hand built to mass production on assembly lines: IT has moved from hand coded applications on proprietary platforms toward the adoption of continuous development and delivery on more automated hybrid cloud platforms. Seems to hold together as a comparison, doesn’t it?

However, the comparison can fail as well – the often-used term of a ‘common engine’ in areas such as databases and analytics may result in too many exceptions being brought through that stretch the idea too far. The main issue is that there is not one type of vehicle on the market, and their capabilities cover a broad spectrum. For example, a 40-tonne truck’s engine would not be that competitive in a race against a Formula 1 or Indycar. The flip side of this is that the F1 or Indycar engine would be useless at hauling freight, even with the F1 engine having 50% greater power than the truck.

So, in the light of what I have just said, let me say that IT needs to be looking at a single engine to manage its workload automation needs.

What? After several paragraphs saying this is stupid, I then say it’s a good idea?

I believe so. This can’t be some ‘average’ engine that attempts to be the ultimate answer and ends up everything badly – it must be something special.

Maybe we can come up with a ‘magic’ engine that adapts itself to the job in hand? One that can take a whole set of variables that change how the engine is brought to bear and operates. Then it wraps the right transmission, brakes, electronics and body around the underlying system to make things happen as they should.

Overall, there are four main classes of what I'll call an IT Automation engine that can then be modified as necessary to create the different type of vehicles that we see today:

• The powerful engine – as seen in trucks, lorries and so on. Designed for moving large loads long distances in the most effective and economical manner. Within the IT world, this is comparable to where we have the need to move large amounts of data from one place to another without there being really pressing time constraints. Consider data archiving – there is already a master copy of this on the platform; there are probably multiple copies in snapshots and backups in other places. Therefore, speed is not of the essence – the data must be moved from one place to another without major impact on other workloads that require the underlying resources. Therefore, like lorries that can utilize crawler lanes on major highways, such activity on the IT platform can be carried out using spare resource capacity as and when necessary.
• The efficient mid-range engine – needed to take reasonable sized loads on relatively short journeys, such as parcel deliveries, often where timeliness is a requirement. In IT, this is equivalent to the need for taking workloads such as containerized microservices and provisioning them out to the operational environment. There is an expectation on timeliness in how rapidly the workload is delivered and activated: the engine must run in a manner that meets these expectations.
• The high efficiency eco engine – electric cars remove a direct dependency on fossil fuels, instead utilizing lower impact electrical power. Generally, not suitable for very long-distance travel, these engines are excellent for low-load, flexible local travel. Many such electric vehicles have multiple operational modes, such as the high-speed, high-impact “Ludicrous” mode in Tesla vehicles. In IT terms, this can be compared with the need for existing operational workloads to be patched and upgraded: the activity must have low impact on the capability for other operational workloads to meet their required response times and outputs. However, certain patches or upgrades must be delivered within short timescales dictated by the importance of the change – as acceleration is delivered in Teslas via the Ludicrous mode.
• The general-purpose family vehicle – As individuals, we try to buy the vehicle that will cover as many of our needs as possible. The vehicle needs to be comfortable enough, be economical to own, carry the family and its luggage over long and short distances, be suitable for going shopping, and so on. We may argue over whether this requires a small or large, fast or economic engine, but overall, the cars that we see on our highways are there to achieve the same general outcome – to get the loads of the people within them along with any additional loads they have with them, from point A to point B.

In reality, the majority of workloads across an IT platform will require this type of general-purpose engine – for example, when considering process automation. Within a business, processes are made up of sets of interlinked tasks, which need to be facilitated by the underlying platform. Such tasks maybe large or small and are wholly dependent on receiving certain data feeds from upstream tasks and in outputting data feeds to the next downstream task. Ensuring that such activity is carried out in an effective, secure and timely manner defines how successful an organisation will be in its markets – workload automation is a necessity here.

It is obvious that each of these engines has distinct needs and capabilities. However, they are all targeted at the same outcome: data must be moved from one point to another and actions taken around that data before and after the move. These actions can be codified and used as rules to define exactly how a ‘magic’ engine works.

IT must look to the automobile industry to see how it continues to embrace standards while still bringing through rapid innovation. Many engines are now built up from smaller discrete items – 4- and 6-cylinder engines using the same core single cylinders bolted together; electric vehicles are streamlining the number of components needed in the drive-train and are bringing in far more dynamic means of introducing new capabilities to vehicles. Electric vehicles remove the need for old-style gear boxes: power is transferred directly to the wheels in a more effective manner. A single design of electric motor can be more easily resized as necessary to meet the different needs of the different vehicles required. Electric vehicles are showing the way for the automobile industry, and every manufacturer is actively entering the market.

The same streamlined and innovative approach should be the case for IT – teams should be looking for new generation workload automation systems that reduce the number of steps required, using the power of big data, cloud computing, artificial intelligence and digital transformation to apply the power of the workload automation engine to the platform. Old-style workload automation engines should be reviewed and replaced where necessary; ones which are the equivalent of electric vehicles as compared to the old petrol and diesel ones should become the norm.

The key is for IT to be able to strip away what isn’t required and embrace new concepts and approaches to workload automation. The ‘smarts’ need to be put in place to be able to configure the basic engine successfully as it is needed and then add the required additional capabilities (in vehicle terms, the chassis, upholstery, electrics, etc) and to resource it sufficiently (again, in electric vehicle terms, by providing the battery) around it to fulfill the exact needs of the workload to which it is being applied.

Through such means, there comes a greater capability to measure the value of the new platform – in both automobile and IT terms, we could see this as the ‘return on automation’. Just how much value has been added by stripping away all the superfluous steps while adding much greater automation and intelligence to the process becomes a simpler metric that can be calculated and presented to the business.

This is true for IT platforms as well: the vast majority of problems on an IT platform are caused by human error. Automation can help to remove unnecessary steps carried out by people. By introducing more best practice rules and policies, and through increasing use of machine learning (ML) and artificial intelligence (AI), workload automation systems can bypass human intervention in more and more areas. DevOps can be more effectively streamlined, with the required feedback loops being driven by how the platform monitors and measures itself, with workload automation engines being called to remediate problems without involving human interaction except where such intervention is truly required. Continuous development and delivery can be better embraced, with workload automation enabling the workflows, checks and balances that such an approach demands.

We should also look at what else has been happening within the vehicle manufacturing environment. There is an increasing move toward autonomous vehicles, moving beyond the standard drive-by-wire electronics that we see in today’s vehicles to one where humans are increasingly removed from the whole equation.

Such an approach when transferred through to an IT platform not only leads to intelligent systems dealing with problems faster and more effectively, but also helping to prevent any issues from happening in the first place.

The ‘magic’ engine for workload automation is finally a real possibility.