We have all hidden devastating messages from parents and friends or passed chits around in class past the watchful (sometimes sleepy) eyes of invigilators. The art of hiding messages and their true meaning has evolved for centuries and in its heart is lot darker, more mathematical and and sometimes straight up silly. So, let’s take a quick dive into the evolution of cryptography.

Article by Prabhat. Prabhat claims to have been encrypting messages (including notes) since childhood - as to whether these were just spelling errors or signs of a mathematical genius, none can say.

Like most of humanity's inventions, cryptography was borne out of war. It was the result of a simple problem — how to get a message from party A, to party B, without party C getting to know about it. (Usually these situations involved less partying, more…war). So, A set out on a journey to try and stop C from getting to know about any of their plans.

To do this, over time, A figured a few techniques — first hiding the message itself, then changing the message to make no sense whatsoever, but hiding the technique used. Few centuries later, A was fine with everyone knowing the technique used, but still had a trick up their sleeve to hide the message’s true meaning, and finally to the modern era — where everything is known to everyone, but still, a message can be hidden, in apparently plain sight.

The Hiding of the Message

The natural step to stopping a third party to get to know of your evil/good intentions is to stop them from ever intercepting the message. These techniques involved changing courier routes and decoys. But if a courier was caught with the right message, you were in a world of trouble. To sort this out, people came up with innovative ways to hide the message scrolls, most common of which were hidden compartments in shoes and staffs. These were followed even into the Cold War, with hidden comapartments in cameras, lapel pins, loose bricks in walls and many many others.

My personal favourites though, were the use of “unconventional” means of writing the message. There’s the classic invisible ink technique, using lemon juice, or actual engineered invisible ink. Alternatively, you can change what you’re writing on. There’s a famous example of a ruler shaving the head of his servant, writing the message on his head, waiting for a month for the hair to regrow and sending the servant off to “deliver” the message to his allies, where the servant received yet another free haircut. You could write messages on the hides of the horses, and cover them with a saddle, or finally use pigeons — which are obviously extremely trustworthy and not at all easy to shoot down.

Needless to say, these technqiues were cumbersome. It would take way too much time, and wasn’t always successful. A single interception could spell doom for the sender and receiver. The evolution of technology with the radio, meant that almost anyone could tap into a message being transmitted. People realized that instead of hiding the message, they could hide the meaning of the message — this lead to the start of modern day cryptography.

The Gibberish-ification of the Message

The next step in the evolution of hiding messages was, the gibberish-ification of them. Essentially contorting the message in such a way that it would make no sense to anyone except the intended receiver, who could reverse engineer the original message, using some pre-determined techniques.

A unique technique was that of the Scytale - a thin stip of paper was wrapped around a cylinder, and the message then written on that. The paper was unravelled and sent to the receiver. A receiver could only figure out the message if they had a cylinder (a Scytale) of the exact same diameter. But, through trial and error, one could figure out the dimensions required. A new technique involving changing the letters themselves was required.

One of the original techniques used to achieve this was the substitution cipher — here, every letter of the alphabet was replaced with another letter, or a symbol. While writing the message, one would write the original text (a plaintext) and check some kind of reference to see which symbols would have to be replaced and finally generate the encrypted message (ciphertext) which they could send/transmit.

This was pretty ingenious, except for one issue — linguistics. If a message was being sent in English, the most common letter used would be ‘e’. If ‘e’ showed up at the end of a 3-letter word, it was most likely ‘the’ and soon one could break the rest of the cipher. A common example of the breaking of the ciphers in pop culture is Sherlock Holmes decoding the cipher of the dancing men, or in more bloody history, the cipher of Mary Queen of Scotts. Similarly, most languages had extremely common letters or sounds, and these would very often come together in a similar fashion. Reverse engineering the substiution cipher was a bit too easy.

There were solutions around this of course — write the message in the wrong language, or use codewords, use terrible grammar or spelling. But codebreakers were still pretty good at breaking these systems. So the codemakers had to up their game.

The substitution cipher was a good template, but instead of following the same set of substitutions every time, codemakers figured out ways to change these substitutions with every letter or with every message. The most common of these techniques included the Vigenere cipher. During the Civil War, generals used cipher discs — these had two discs with letters, and everyday had a specific keyword. To figure out the substitution, you would rotate the disc by a certain amount based on the keyword for every letter. Soon, codemakers came up with even more complex encryption algorithms, capable of taking small inputs (keywords or keys) and being able to jumble up the output so much that it would take months for codebreakers to solve them by hand.

These complex algorithms were quite a pain to execute by hand, so codebreakers took the next step in cryptography — automation. Most notable among the automated encrypters was The Enigma Machine.

Unleashing the Gibberish — ificators

The Enigma Machine was a beautiful piece of engineering. Employed by the Germans during the Second World War, it provided secrecy to thousands of messages between German lines, and its breaking was probably what ended that war.

Mainly mechanical, it relied on a complex series of gears and switches whose specific combination would lead to an entirely different permutation of plain and cipher letters. The Machine would change this permutation every time a letter was encrypted, therefore giving a completely different output each time. It ensured this through a key — settings of plugs, gears and switches that those in charge of the machine would have to set at the start of everyday. Those using the machine would have a list of these codes (settings) on a sheet of paper, and would do anything to keep it out of enemy hands. The ink was water soluble, so they could easily throw it in water to erase any evidence of the key.

What the Germans were able to achieve, was a system where the enemy could know exactly how they were encrypting their messages, but still be unable to uncover the message’s meaning without the secret key. Although the British had access to a German standard Enigma Machine, they were unable to crack any of the messages passing through, until they figured out a pattern in messaging (a weather report every morning) from which they could reverse engineer the starting configurations (watch the Imitation Game for the dramatization of this).

The breaking of the Enigma showcased a very important rule of modern cryptography — you can reveal the method used to encrypt a message, as long as there is no way to reverse engineer a key. It also highlighted another important issue — current systems only work if the parties have a determined key shared beforehand, just like the Germans did. However, if Allied forces got hold of even one of the key lists, all message for the month would be easily breakable.

Revealing everything

Now with modern communications, having secret keys with everyone is quite cumbersome. Every time you chose to message a new person, you have to contact them in person beforehand, set up a secret key for communication between the two of you (this has to different from the other keys you are using), and then proceed to use the internet for communication — a pretty redundant system.

Instead what is done, is a part of the key is individually generated by both parties. This part key is sent across the communication channel where the other party can piece together the entire key. However, these parts of the keys have to be sent in such a way that the third party cant actually piece it together.

This is achieved through group theory. A & B (our two communicators) chose a an element each from a group, modify it a particular way, and send the modified version across. Combining A’s element with B’s modified element gives the key, while combining A’ modified element, with B’s original, gives the exact same key. However, combining the two modified elements (which is what the third part C has access too) does not give the same key. However, the modification should be in such a way that C cannot figure out the original element from the modified versions.

A slightly easier way of understanding this is as follows. Say, we have a collection of chemical powders. A & B are trying to develop a particular compound but they don’t want their evil colleague C to figure it out, but C can take samples of whatever they send across the lab. Now all of them know that atleast one of the powders should be in powdered form, and the other can be in a liquid, but both should not be liquid (owing to dilution reasons?). They also have access to water for making a solution, but do not have access to a dehydrator (or a way to obtain the powder back).So A selects a powder, B selects a powder, they mix it in water and send it across the lab. A can take the solution from B and mix it in with their powder to get the compound, and B can do the same. However, C is stuck with two solutions, and no way to figure out the original powders (yes, the solutions are colorless and odorless). And so they have made the same compound, even though C had access to basically all their information.

So this has been a quick dive into how cryptography has evolved — from hiding messages, to displaying them proudly and from hiding the techniques used to revealing them as well.