Context of technology strategy
The investment in organisations and businesses in technology can be a significant part of the costs of running an organisation. Such decisions can also represent large ‘bets’ on the value to the organisation of new technology. However, the value of technology invested in by any organisation or business tends to vary over time due to how well it fits with changing business priorities. Typically such technology is also subject to a lifecycle. The cost of that technology is also subject to change.
Technology Life Cycle. Source: Wikipedia
The same is true of the 'opportunity cost' of the current technology stack. The value and cost of the other options we could have chosen is also changing. All this means is that, implicitly, when we make a decision to invest in any technology, we need to be option farming, because we don’t want to be stuck with that technology if it ceases to serve our needs. Conversely, technology companies don't want to 'bake in' customised support for certain clients that make it harder to continue to develop their core products. Option farming is a way of analysing all your options, keeping as many open for as long as possible, even as we make decisions and investments.
Economic Rent & One-sided Arms Races
Technology companies may be intent on getting customers into exactly that position of being ‘stuck’ once they invest, while customers of technology companies may inadvertantly lead technology companies into 'dead ends'.
Economists refer to the behaviour of technology companies as extracting ‘economic rent', whereby a company can charge for a service without actually providing good value to the customers worth the price being charged. They no longer need to stay ‘honest’ in their service offering because they know by virtue of the relationship that the customer is stuck with the product and can't switch even if they want to, it’s just too expensive to do so.
Some technology companies build a business model based on extracting economic rents, and don't make any bones about that to investors. It is like an arms race, where only one side (the technology companies) know that they are in the race, because, all too often, businesses and organisations don’t have the ability to strategise in this contest. It's a very one sided chess match. Conversely, key customers may use their leverage to lead nascent technology companies into blind alleys from which they can no longer develop their products.
More organic approaches to managing this problem on both sides are available to organisations. Typically however, technology strategy is managed in a more inorganic way.
Biology & Economics shows that there are two ways to do almost anything
Relate more inorganically (or synthetically) to other work by ignoring or replacing other work or starting afresh. This often means controlling the results to a high degree. Like building a plane.
Relate more organically to other work by leveraging or utilising other work in some way that is happening anyway or has already happened. This often means less control over the exact results. Like evolving a bird.
The insight of Turing Meta is that almost anything can be done in a more organic way and very often there are large advantages to this. The important thing for businesses and organisation is to understand the implications of working in a more or less organic way, and to understand the context where one or the other is likely to lead to the biggest advantages for the organisation. In the context of the arms race between technology companies to snare business and organisations to avoid being snared, the appropriate weapon for the business is organic metrics.
Option farming and constraint satisfaction are more organic metrics, as they don’t dictate exactly what is to be done, only try to shape decisions to ensure that whatever decisions taken satisfy some constraints on whatever is done. Hence, systems can evolve and adapt within the constraints that are defined by the system. This is as opposed to setting objectives where a plan is then executed. The more organic approach is suitable where you have less control over the cost and value of some of the work being done, typically because you are leveraging work by other agents. A technology stack is an example of that, as organisations and businesses typically have limited control of the technology that they are leveraging and the value and cost of that is partly out of their direct control. Therefore, as the cost and value of technology changes, we want to ensure we are always going to be in a good position to switch from one technology to another, if and when we need to.
Not just hedging
Hedging is an inorganic or ‘synthetic’ way of modelling a strategy for managing this risk. Large companies can afford to leverage their vast resources to exert control this way and can simply buy both of two offerings from rival companies, such as rival cloud providers, or rival relational database providers. Smaller companies can't afford to do too much of this. However, option farms are a different way to achieve similar results to hedging without the huge resources that hedging typically requires.
"An option farm is a way of thinking about how to reliably manage work that is of variable cost and value and where control of cost and value is incomplete".
“An option farm is also a way to act intelligently to avoid getting yourself ‘boxed-in’”.
In business and technology these ideas have also been explored by some of the founders of the Agile manifesto, notably, Dave ‘Pragmatic’ Thomas, who spoke on this same subject at the GOTO conference in 2015. He argued that one of only two guiding principles behind agility is to act in such a way so as to always keep your options open. Given two options of roughly the same value, Dave argues you should choose the option with which leaves room for future change. Dave explicitly links his ideas to a more mathematical, scientific principle. In his talk he linked it to the notion of a PID Controller, which can be thought of as a self-righting mechanism in response to uncertain feedback. However, while linking to his first principle of make small changes or adjustments as you move towards your goal, the PID doesn’t contain a concept of ‘keeping your options open’.
However, Turing Meta recognises that there is this same requirement for biological systems to avoid getting ‘boxed-in’ for their own continued viability and better adaptive capacity. Therefore, researchers in computational biology are likely to ‘discover’ the deeper principle behind this which motivates and underlies biological adaptation. This research can then be leveraged in business in the way that Dave Thomas envisaged, but with more flexibility and precision, by leveraging a large amount of cutting edge research in biology which suggests that ‘keeping options open’ may be more important for adaptive capacity than making small steps towards your goal in relation to feedback.
And, indeed, this type of behaviour has been researched by physicists and computer scientists as a way to understand the vast complexity of biological organisms and forms, and the underlying principles by which they may have evolved. An example of what I term an option farm is explored in a biological context, based on the principle 'entropy maximisation', and is described by this reporting article on the cutting-edge computational biology research by Wissner-Gross and Freer.
What the paper shows is that when we think about what intelligent behaviour looks like, one way to think about it, is that it is about always keeping your options at any one time open. Entropy maximisation is a principle that Wissner-Gross has adapted to model this concept with a physics-like concept (entropy is derived here from the physical concept but these details don’t matter). This is a great way to deal with uncertainty, when you are leveraging the work of other people or agents whose contribution is unreliable or uncertain. This principle is effectively what the paper above explores, and it is probably best thought of as a constraint on how we act, rather than an objective.
In one example, the paper shows that complex behaviour reminiscent of walking could emerge from this deeper principle of keeping one’s options open. There is a similarity between an ‘upright’ unstable pendulum which as a result is maximising (‘future’) entropy in the sense used by Wissner-Gross, and the similar biological behaviour of walking which is also an ‘unstable’ configuration which also increases entropy in the sense used by Wissner-Gross.
A modified version of thermodynamics (entropy) causes a pendulum (green) swinging from a sliding pivot (red) to stabilize in an inverted, and normally unstable, configuration, from which it has a greater variety of options for its future motion.
Credit: Physics Focus / A. Wissner-Gross/Harvard Univ. & MIT
The paper also explains how, similarly, given simple agents in a ‘box’, using this modified entropy maximising principle, the agent tends to move under this ‘entropy maximising force’ to a ‘strategic’ position in the centre of the box, as this gives the most options to move in different directions from that point, analogous to the unstable upright pendulum which can also easily move in multiple directions from the upright position. This same principle will be familiar to squash players who try to dominate a game by occupying a position in the centre of the court from where they have the most options to move and to place the ball in a way that makes the opponent run around the court.
Turing Meta’s Translational Research
Although this physics/biology paper offers deep insight, the essential work of Turing Meta is to translate this into practical insights to businesses and organisations. The translation work involves understanding the basic principles and constructing metrics that mimic the same underlying behaviour of these ‘entropy maximising’ agents. The approach that is taken by Turing Meta is to translate entropy maximisation into ‘constraint satisfaction’ and to use the insight of the value of such constraint satisfaction to an organism or organisation. This means we can then act on Dave Thomas’s ideas about a more mathematical approach to agility. This is a way to rescue the agility concept. We define ‘organic metrics’ based on constraint satisfaction to generate metrics about the ‘agility’ of our organisation which directly relates to these deeper scientific principles. This relation to science allows us to be more confident that we can improve and adjust our metrics as we learn more about the principle, and we can leverage more and more hard science from computational biology to help us. All this results from the deeper insight that the problems of organisms and organisations are not that different. They both need to respond to and manage uncertainty in their environment and retain the ability to adapt.
One very useful advantage of the biology approach is that we can then apply these metrics to track our agility at any level of an organisation. This has long been a desire of some agility practitioners, but concepts such as SaFE, the Scaled Agile Framework, intended to provide agility at scale in larger organisations, arguably lack a clear theory as to what agility really means at an organisational level. These models treat ‘agility’ as a specific process or organisational template, i.e. treating is as a noun something specific you ‘do’ when it is really an adjective, i.e. a way of behaving. The translation of biological research helps to solve this problem.
So, rather than modelling agility as a specific ‘organisational template/workflow process’ as SaFE would, we can use constraint satisfaction to generate metrics which accurately characterise the ‘agility’ of an organisation as a way of behaving without any specific organisational structure. One application of this is to model and measure our ability to retain agility in a technology infrastructure.
Organic Metrics for a Technology Strategy ‘Option Farm’
Given a technology infrastructure for a company, there will typically be a large number of interconnected components supplied by various vendors at each part of the technology stack, working from the underlying hardware, to various pieces of software and database technology.
Using the principles from biology of ‘entropy maximisation’ we can translate the basic idea into constraint satisfaction, to give us a way of managing and retaining our ‘options’ for switching from any incumbent technology to another technology if we need to.
So for each technology investment x, we generate a set of options which we review quarterly to understand the ‘agility’ of our technology strategy at that point in the year. We do this by reviewing whether we satisfy the constraints for adaptive capacity that we have set:
For technology x either the set of options to switch to are sufficient in number and known limit of cost c of switch that we are confident we can switch to one if we need;
Or for technology x a smaller option set is invested in and known about sufficiently that we are confident we can switch to that option set if we need as it is also of limited switch cost c;
Or for technology x we are confident that the current option is a safe and useful option for sufficient time into the future.
This defines the options for satisfying our constraints that indicate a healthy state of our option farm as a technology strategy and by following a path which continues to satisfy these constraints at each point in time, we mimic the ‘entropy maximisation’ of the Wissner-Gross paper which seems so useful in biological systems.
To manage these metrics, and to add other various metrics, it makes sense to represent the technology infrastructure and options for switching in a knowledge graph. A suite of graph queries can then be fired against the knowledge graph to characterise the adaptive capacity or ‘agility’ of that technology stack. Graph queries can naturally define the number of viable paths for switching from or to any given technology should the need arise.
Example of a small part of a technology knowledge graph ‘option farm’. Some technologies are currently used, others are options. Credit: Turing Meta.
We can then take this information to motivate further research if we need to for technology x if it doesn’t satisfy our option farm constraints. If we are unable to identify good viable options for switching, we can then decide to take action before it gets too expensive to avoid being boxed-in.
Elementary Query and Results for Technology Knowledge Graph Example in Neo4j. In this example we check the options if we need to switch the commentary tool to another type of commentary tool. This returns two paths but with very different switch costs (‘RES’ column). In this example, despite the name ‘Non-Tableau’ we model so there is a path whereby the new commentary tool is also compatible with Tableau, the current technology, giving a much lower switch cost (top row of table). If we have to switch to Thoughtspot, the cost of switching the commentary tool plugin to another technology is much higher (bottom row of table).
Wider Contexts of Application
Technology lifecycles are such that these kinds of analysis are essential to avoid getting stuck or without a plan to move forwards from legacy technologies that are no longer giving the value that they were. However, it should be recognised that maintaining and feeding into a technology strategy is just one application for modelling organisational agility and adaptive capacity. Because these metrics are built on principles learned from computational biology research, they can be further developed and adapted for use in a wide variety of contexts, just as the Wissner-Gross biology/physics paper above can be applied to study the characteristics of a huge range of biological phenomena.
To find out more about how technology option farm knowledge graphs can help you to maintain organisational agility in your technology strategy, or to explore other ways organic metrics can help with your business strategy, contact adam@turingmeta.org.
This article can also be downloaded here: