Optimisation’s Shadow is Cryptographic
- Adam Timlett
- Nov 15, 2025
- 10 min read
Updated: Dec 27, 2025
Timestamped PDF for Download and citation
This second article will be more of a focus on the underpinnings of the idea of natural cryptography. It follows on from my first article where I introduced the idea, The 2nd Invisible Hand. Learning from Nature and the incompleteness of economics. There I argued that natural systems may have evolved to regulate use of resources by selecting for higher adaptive gradients to reduce the probability of mutations that would result in pollution or excessive use of resources.
In this article I want to argue that, rather than a kind of ‘natural’ thing, it can be shown instead that, mathematically, it is inevitable that any system that is organising to lower a gradient somewhere to access resources more easily is also producing a form of gradient raising in some other dimension. Nature is likely to exploit this fact.
Order / Disorder as a duality
The most common way that we think about this kind of relation is the duality of order and disorder and I will seek to contrast the idea of natural cryptography with this older concept.
Pollution can be thought of as the ‘export’ of entropy for an organising system and this is the order/disorder duality that has been explored previously in physics and information theory.
When Erwin Schrodinger introduced this idea informally in his book, he implied that the ‘shadow’ of organisation is disorder which is created to maintain it, and may better define the organisation we see in biology. A little more formally, the very idea of organisation as ‘negentropy’ implies a duality between the increase of entropy and the organisation that produces it efficiently.
With the development of the maximum entropy principle and its applications to biology the same general idea was built on, to suggest that lots of biological organisation involves essentially maximising the efficient throughput and dissipation of energy and resources.
The idea that I want to explore here, and demonstrate, is that we now have an alternative way to link information to dualities that involve, roughly speaking, changes in order rather than order and disorder. Instead of thinking of the link between order and disorder, we can instead consider the link between different gradients in information terms to process information.
When we decrease a gradient, making it easier or more efficient to act in one dimension A, we may, due to the duality I will describe, make it less efficient to act with another objective in another dimension, B. We can think of the raising of the gradient in dimension B as a natural encryption of the resources that would be more efficiently accessed by lowering that gradient in dimension B.
This is a very simple duality that we can see expressed in something as basic as sorting a table of data.
If we want to lower the gradient to access data items with a high index number we can sort the table so that the highest index items are at the top. At the same time, of course, we have increased the gradient to access the lowest index items, which is just the reverse sort order. We can think of this simple duality of gradients as a duality of optimisation which lowers the gradient for access of one function. This example is a function that takes the first correct item in a queue ordered by the sorting and where that first correct item is likely to be a high index number item. It must then obviously do this at the expense of the function that takes the first correct item in the same queue where that correct item is likely to have a low index value.
Note: Extra indexes in databases are usually added for orthogonal sorting not reverse sorting: The example of a reverse order just simplifies the explanation. However the simple example of the reverse order being a higher gradient applies to the user experience of interfaces with tables. Re-sorting to display from top N rows in one order of a large table A-Z will create a significant gradient in the form of the wait time to re-sort in the reverse order and take the top N rows from the reverse ordering, Z-A.
This principle is so obvious it might seem impossible that anything non-obvious results from it, but there are other very obvious theorems such as the pigeonhole principle that also have probably non-obvious consequences.
The first obvious thing
The first obvious thing is thinking of these as ‘opposing’ functions, such as those I examples above, where we want to identify our priority, and do so at the expense of the function that is the converse of our priority. Obviously there are lots of technologies that can deal with these priorities more effectively. We can toggle functions, and pivot from one sort ordering to another, or add multiple indexes to a table, and switch between them. Or we can program things in a dedicated way to reduce the trade-off between two different functions. For example, the cost of adding lots of indexes to large databases tends to slow things down in other ways such as writing data to the table, but this is often not a problem if querying data is the bottleneck, rather than writing data.
The first non-obvious thing
The first non-obvious thing is that we might want both functionality, i.e., both the low gradient in one dimension and the higher gradient in the other dimension.
That is, instead of just focusing on use cases where we lower gradients, we can consider that all optimisation functions can be considered dual functions that have both gradient lowering and gradient raising functions, and we might want both at the same time in their respective dimensions. If so that would be lucky, because that is what happens anyway.
Here is an example of research exploring just that idea:
Compression is also cryptography
A paper that effectively explores this type of utility is the following by Irvine which identifies that compression has cryptographic effects.
Irvine, Sean A. Compression and cryptology. Diss. The University of Waikato, 1997
Compression is a type of optimisation, and, as we compress, we end up with less redundancy, so that more efficient compression effectively means that the information has higher entropy and so appears to be more random. This raises the gradient for decoding it if you don’t have the right compression algorithm to ‘decode’ it. So it has a cryptographic effect.
Irvine points out that for file transfer, we often compress and potentially also cryptographically encrypt data before we send it. But as compression also has a cryptographic effect, since it’s just a duality of optimisation, we could design compression algorithms that also optimise for a more desirable cryptographic effect at the same time.
Some other research has more recently been done showing that some standard compression algorithms would also have the effect of automatically decoding some classic ciphers, such as substitution ciphers, again exploiting the duality of optimisation and cryptography.
Al-Kazaz, Noor R. Compression-based Methods for the Automatic Cryptanalysis of Classical Ciphers. Diss. Bangor University (United Kingdom), 2019.
The proposed research work and subject of study
The work that I propose that is undertaken now is to look at the general cryptographic value of organisation and functions both in economics and biology.
Essentially, all functions that are optimising to minimise some gradient, either for learning or for easier access to resources for other agents, are also raising some other gradient. As I argued in my previous article on Substack (and published my own website www.turingmeta.org) inefficiencies are just as emergent and potentially useful as efficiencies. That is because such inefficiencies in specific dimensions are ways we can manage agent’s access to resources and resource allocation. This is something we urgently need to understand better in economics, and probably in biology, too.
To illustrate the idea, notice that any idea about the control of access of resources, such as a function to ‘buy local’ more, or to do less international travel, or to throw away less products, can be framed as a desire for an inefficiency in some direction. It should be harder to fly more, or to buy non-local fresh produce, or to throw away products that were used only once. These are inefficiencies that are desirable in certain dimensions often related to non-locality of production versus consumption. We are trying to control the access to resources as part of the rational allocation of resources. We do this by both making lower gradients for some agents for some services and products, so that they can more easily buy long lasting products that are reusable, and goods that were sourced locally, etc. At the same time we try to make it harder to do the converse. As I said in the previous article, in Nature, it may be the case that many species have evolved to manage their resource via precisely higher gradients for specific types of relation, such that the stability and control of population growth, pollution and resource use is maintained.
What I argue now, is that it should really be obvious that every organisational function that increases an efficiency in one objective, raises a gradient in another, and this ‘cryptographic shadow’ of all organisational processes has tremendous potential utility.
It is really rather foolish not to explore this duality because potentially very sophisticated functions which blend lowering and raising gradients are possible, as the ‘compression as cryptography’ research shows.
For human state-of-the-art cryptography it is unlikely that you would find a very safe cipher to use that is also an efficient compression algorithm. In the rest of the world, however, gradients only need to be raised enough to change the probability of access events. Agents need to have their preferences affected through deterrence. It is not necessary that very high effort agents still find it impossible to climb them. Indeed, that is how police forces understand citizen’s domestic security arrangements. It is not that your car, your house and your possessions are impregnable when they are ‘secure’; they are not impregnable to a determined criminal. They just need to represent a high enough gradient to make for an effective deterrent, such that easier pickings are always available.
At the moment governance involves trying to improve efficiencies for things we encourage more of, and, in a separate effort, creating laws regulations and taxes to raise gradients to prevent behaviour we want to discourage. The duality means we can instead view these activities as two sides of the same organisational coin. This is likely to be far more effective, because every organising function has a cryptographic effect anyway, and so it is about the opportunity for understanding and intervening in existing organisational behaviours, rather than only restricting ourselves to adding more and more independent rules and regulations.
A demonstration of the depth of optimisation / cryptographic duality is as a method of function discovery
In my book On the Origin of Risk I explored the cryptographic effects of certain systems that I defined as ‘productive prediction machines’ and I noted that certain kinds of classic ciphers correspond to certain kinds of interesting function to manage risk. In other words, certain types of classic ciphers have a duality as an interesting type of gradient reducing function.
An example I give in the book is skipping ciphers, which is where you name a large book and provide a code to read certain words in the book on certain pages to make an encoded message. This type of classic cipher has a duality as a type of optionality, where you screen a large number of options (which is like the words in the book), and only choose a very few, but you are not in control of the production of the options. As I explain in On the Origin of Risk, this is actually likely to be an important strategy for risk whereby you can add ‘searches’ to existing functions which I call ‘parasitic search’. The search itself is not efficient because there is a primary function driving it which has another purpose, but that also means the search is ‘parasitic’ and therefore doesn’t need to be funded by its own work. (My own research has been very much like this, and is very efficient in its inefficiency). An example of this was the use of the technique by SETI reported in this paper:
Bowyer, Stuart, et al. “The Berkeley parasitic SETI program.” Icarus 53.1 (1983): 147-155.
It has not escaped my attention that a large minority of life on the planet is parasitic. Therefore, I propose that looking at classic ciphers and other types of gradient that are used in IT for security such as gateway server design, etc., will correspond to valuable and overlooked types of risk management function which they are dual to. What I explore here and in my book is probably just the tip of the iceberg/biome.
Therefore, the power of the optimsation / cryptography duality is that it can also be used as a kind of mathematical ‘x-ray lens’ to discover/view different kinds of risk function and better define their unique properties.
One more notable example is that some of the most interesting functions I describe in On the Origin of Risk involve pivoting from one model to another. The type of function is dual to channel switching or channel-hopping ciphers, such as the type originally patented by Hedy Lammar in the 1930s. This type of function is likely linked to very high levels of organisational plasticity, and is naturally strongly cryptographic. It is hard to predict the next ‘channel hopped to’ when we switch from one model to another with the same data. I have a lot of unpublished material on high plasticity systems which relate to this type of cryptography which I will start writing about soon.
Summary
1. I believe that biological systems are exploiting types of risk function utility we are not very familiar with as economists.
2. There is a duality at the heart of optimisation we have overlooked.
3. Rather than order / disorder, there is also a duality of ordering that both lowers gradients and is also raising gradients in other dimensions (cryptographic effects).
4. Because every organisational measure has a cryptographic effect which also affects resource allocation, we can leverage existing systems and modify them to better control access to resources instead of focusing only on standalone ‘negative’ measures such as taxation, regulations etc. (Although they are also essential)
5. A new area of research is to pick up on these functions I describe in On the Origin, using the analysis of their duality of ciphers to better formalise them and to organise the analysis of these overlooked functions.
6. This duality therefore also provides a strategy to potentially discover a whole new ‘menagerie’ of interesting functions that can be used to manage risk, some of which I’ve already outlined.
References
Irvine, Sean A. Compression and cryptology. Diss. The University of Waikato, 1997
Al-Kazaz, Noor R. Compression-based Methods for the Automatic Cryptanalysis of Classical Ciphers. Diss. Bangor University (United Kingdom), 2019.
Bowyer, Stuart, et al. “The Berkeley parasitic SETI program.” Icarus 53.1 (1983): 147-155.
Busch-Vishniac, Ilene, Lauren Busch, and Jill S. Tietjen. “Hedy Lamarr.” Women in the National Inventors Hall of Fame: The First 50 Years. Cham: Springer Nature Switzerland, 2024. 123-132.
