You wouldn’t think this book would be such a page turner, but then perhaps you need to have a certain mindset to appreciate the topic.
I obtained the The Codebreakers through a book club over 40 years ago, and eventually plowed through all 965 pages and learned some valuable lessons about secret codes. The most valuable lesson people can learn about secret codes is not so much how to construct them but the consequences of relying on an unsecure encryption mechanism.
Previously I wrote about the 1942 Battle of Midway that was a stunning victory for the American forces. As mentioned, the American success was due to the drive and determination of our forces, some very disastrous tactical moves by the Japanese commander and not least the Japanese reliance on what they thought was a secure code. As author David Kahn explained, Navy cryptographers in Hawaii started working on Japanese navy codes well before the attack on Pearl Harbor and had figured out that Japan was planning something nefarious. For one thing, they learned that specific instructions were being sent to the Japanese embassy in Washington, D.C. in multiple parts, with the last part to be sent on 7 December 1941. Without knowing the details, America put its remote outposts on notice to watch out for something from the Japanese, especially in the Philippines. Pearl Harbor was not considered to be the prime object of interest. The following year the Navy codebreakers had figured out enough turn the planned Japanese ambush at Midway into a disaster for the Japanese navy. So much for trusting an “unbreakable” code.
I first got interested in codes and code breaking when a Boy Scout project inspired me to invent a code of my own. What a remarkable and brilliant invention it was, too. What I was going to do was to substitute a number for every letter in the alphabet and then code my secret messages by writing the numbers instead of the letters. To decode my secret messages, all the receiver had to do was to reverse the process. All the receiver needed was a copy of my encoding table, which no outsider could ever guess at in a million years.
Not so smart, really. I soon learned that this simple substitution cipher was the oldest, the simplest and the easiest encryption system to break. Puzzles appearing in magazines for retired people routinely ask the reader to crack this simple cipher and produce the hidden message.
Obviously, modern encryption systems are much more sophisticated than this, and in real life they present a challenge that requires much time of the world’s fastest computers to unravel. As a confirmation, you will notice that one of the largest, if not the largest, purchasers of the world’s fastest computers is our own NSA, National Security Agency. Also, if you are a top mathematician and need a job, you might give the NSA a look.
Anyhow, avoiding a lot of the mathematical terminology I picked up in college, I will attempt to explain the general idea behind all modern codes. Rather than using words like “mapping” and “inverse mapping,” I have a real-life analogy that should provide the flavor of the problem.
Suppose everybody lived in one city, and it was well-mapped. If I gave you my address you would have no problem finding me. So, I want to send somebody my address, but I do not want other people to intercept my message, read my address and come knocking.
Now, suppose this city is located on a vast plane where nobody lives. Further, suppose this vast plane is the size of the Sahara Desert or even the size of the ecliptic plane of the planet Earth. Get the picture: City small, empty plane much larger. So here is what you do.
You develop a scheme that, given an address in the city, will compute a corresponding location in the vast and empty plane. What you do next is perform this computation and put the result into your message instead of your real address. Then you send your message. The recipient of your message has a corresponding scheme to compute your address from the location given in the message. The idea is that outsiders will not have a copy of the scheme that translates these spurious locations into addresses in the city.
To further elaborate on the analogy, the addresses in the city represent the set of messages that make sense in, for example, English. The vast and empty plane represents the set of all messages, those that make sense and those that do not. To crack this code, an outsider will need to obtain a copy of an encrypted message and re-invent the scheme that maps points in the set of all possible messages back to the set of messages that make sense. An important point is that the inverse mapping scheme needs to be simple and universally correct. Here’s an example of what I am talking about.
When Orleans Parish, Louisiana, District Attorney Jim Garrison sought to make a name for himself by implicating various people in the assassination of President Kennedy, one of the claims he made was the translation of “P. O. 19106” from Lee Oswald’s notebook into Jack Ruby’s unlisted Dallas phone number. The problem was, the scheme was too elaborately tailored to this specific translation. It was even more obtuse than the Bible code schemes that ran their course about 15 years ago.
Anyhow, that’s a very brief summary, in a very abstract manner, of the basis of most modern encryption schemes. You take a message that makes sense (“Saddam Hussein is hiding out at 209 Ali Street.”) and you hide it in a mountain of gibberish so that outsiders will find it nearly impossible to back out the original text. The core of this idea hit me one day when I was working on a project for a government contractor.
Our job was to develop a system for managing messages on a tactical radio network. Messages that went out over the radio were securely encrypted, but we also needed to make efficient use of the available transmission capacity. It would be best to compress the messages before they were transmitted. So, the question came up, “Should we encrypt the messages before we compress them?”
I thought about this for about two seconds, and the answer hit me. “No.” It was obvious that compressing an encrypted message was tantamount to decoding it. If you had a supposedly encrypted message, and you have successfully compressed it, you have successfully mapped it from the large plane of all possible messages (sense and nonsense) into a much smaller region that must still contain the message of interest. I did not have to, but I did run an experiment to verify that a properly encrypted message cannot be compressed.
Modern cryptography hit the mainstream in 1976 when Whitfield Diffie and Martin Hellman published their paper on public key methods. Hellman wrote up the process in the August 1979 issue of Scientific American and offered a copy of their paper to readers who asked. Naturally I obtained a copy, even though I was never able to work through the math.
Anyhow, cryptography is an interesting topic, and there is a lot of math involved and politics, besides. If you have an effective encryption scheme, you are not allowed to export it to another country, meaning you cannot carry or send a copy outside the country without your government’s permission. I never figured out how this restriction could be enforced, but it does show your government is looking out for you and is doing something.
The fun part for me came down to a moment in one of my college classes. I was trying to get a degree in computer science, and one of my courses was database design. The professor thought it would be a good idea to touch on encryption (for database security). He wanted to show us what encryption was all about, so he wrote two lines on the board. I cannot reproduce the text exactly, because I was not taking good notes, but what he wrote was essentially this:
“What is this?” he asked.
I stared at it. Obviously a code of some sort. If it’s a code, I wondered, what is it? I tried the two lines and thought about applying a frequency analysis to see if this was a substitution cipher. Then I had a better idea. What course were we taking? DATABASE design. The first line was just DATABASE with the letters scrambled. Making the proper letter for letter substitution, the second line was also DATABASE. So I decided to answer the professor’s question.
“The first line is a transmutation cipher, and the second line is a substitution cipher.” And I said no more.
The professor was a little taken aback at first, and he pressed for more. “But what does it say?”
I told him it was DATABASE. And I said no more.
Now the professor was dumfounded. Maybe not so dumbfounded as the Japanese had been years earlier when our navy cracked their tactical codes, but he was still interested. “How did you do that?” he asked.
I remember many years ago reading about this guy who was a great prankster, and he decided to perpetrate a hoax on some magicians. The magicians were having one of their frequent conferences where they all get to together and discuss new tricks, and this guy said he wanted to come and show off a card trick that none of them had ever seen before. At first he was rebuffed. Professional magicians are well versed on all the tricks, and they didn’t want to be bothered by some amateur with some old trick. Anyhow he finally talked his way in, and when his time came he proceeded to demonstrate his trick. The trick involved naming the card that somebody else picked from a deck. There are probably about a thousand tricks that involve this theme, and professional magicians are acquainted with all of them. However, this one was different. I do not recall the details; I don’t even recall the name of the hoaxer, though he was well-known in his time. His trick involved a number of mechanizations to make the act interesting. Then at the critical moment the hoaxer announced the card. It was, he said, “The three of diamonds.”
Of course it was not the three of diamonds. The hoaxer was just hoping to get lucky. “If I had gotten it correct,” he later recounted, “they would be scratching their heads to this day.”
So, this episode came to mind sometime after that when I was in a motorcycle shop, and some teenage boys were doing card tricks. So I thought of the hoax, and decided to give it a try. I told one of the kids to pick a card from the deck and put it in his pocket. That he did, and when he was putting it in his pocket I saw the card. The only thing left for me to do after that was to make the whole thing interesting. They are scratching their heads to this day.
So, years later, in my database design course the professor was asking me how I was able to decode his message. I just told him that I had some previous experience with encryption, and this was one of the things I could do. And I said nothing more. He may be scratching his head to this day.
You can read more and amaze (bore) your friends with neat stuff about cryptography. For a high-level treatment of cryptographic applications in real life you should to read Applied Cryptography by Bruce Schneier.