Rebooting the Eagle
As the world celebrates the 50th anniversary of Apollo 11’s history-making lunar landing, it’s interesting to note that the stories and tributes of the event are reaching hundreds of millions of people on their iPhones and Android cells.
And that’s ironic, for an unsurprising but fascinating reason: those slim little devices in our pockets, those lifelines to the human enterprise, those friendly little couriers of voices and faces and texts and podcasts and song libraries and Internet sites and fake news – have more than 100,000 times the power of the computer that landed the Apollo 11 astronauts on the moon.
If only Armstrong and Aldrin had taken one along!
The US – the world! - is incredibly proud of what Apollo 11 and NASA accomplished on this day 50 years ago. And the focus has rightly been on the mission’s success – doggedly, relentlessly pursued by tens of thousands, through setbacks and dead ends and conflict and tragedy. But lesser known, and almost never discussed, are the failures that occurred – or how close they came to ending the mission in the biggest tragedy of all.
It’s all to do with a seldom-considered fact: spacecraft don’t fly themselves; they are flown by computer. Both the Apollo command module (Columbia) and the lunar landing module (Eagle) possessed a digital guidance computer – and each of these computers did all the heavy lifting, when it came to flying their respective craft.
There are several other facts to consider. When we say “computer” today, we might be referring to almost anything. The phones we carry around are computers. Every desk has a computer. Your watch is probably a computer. Even your jewelry and shoes can contain a computer these days.
But in 1969, a “computer” was a cluster of boxes that was bigger in volume than a meat locker, despite having nowhere near the power of a modern laptop. Even a 1969 “mini-computer” was the size of a refrigerator. And nothing the size of a refrigerator could have been flown to the moon. Instead, the state of the art in computer design took a giant leap – and custom computers specifically designed for the lunar program were designed, weighing in at a mere 70 pounds. These devices were, at the time, the most advanced computing devices ever created.
It’s also important to consider exactly what these things were made of. Remember that the first half of the 20th century was populated by technology based on vacuum tubes – fragile and unreliable. At the time of Apollo 11, it had only been 20 years since the vacuum tube’s replacement – the lighter, more reliable and less expensive transistor – had been perfected and introduced commercially. It been even less time since transistors – logic gates that turn on and off, in a fashion abstractly similar to human neurons – had been etched into silicon, resulting in the integrated circuit. And it had barely been a year since the first microprocessor (a computer built out of integrated circuits) had been deployed – and Intel wouldn’t introduce the first commercial one until after the Apollo 14 mission had flown.
Put another way, digital technology was still in its infancy, and very experimental. Everything going to the moon had to be invented – especially the computers. The 70-pound wonders in the Columbia and Eagle conglomerated more than 5,000 integrated circuits into a hard-wired system that received numerous inputs from the spacecraft in which they were installed, reported a select set of essential outputs and accepted a limited set of commands – all brief digital codes that Buzz Aldrin had memorized. As precarious as it sounds today, it was an astonishing piece of hardware at the time – very much a technology first.
Off they went to the moon. And another technology first abruptly emerged: the reboot.
Stephen Witt tells the story in the July 2019 Wired, based on the memoir of Don Eyles, at the time a young NASA engineer who was one of the team that had to handle the problem from the ground.
The problem: the Eagle was in the process of landing in the Sea of Tranquility – and the guidance computer crashed, right out of the blue.
The Eagle was in the first stage of its descent, slightly less than 50,000 feet from the lunar surface. Because of the moon’s irregular gravity, Aldrin was required to ping the radar to confirm the lander’s calculated position. He punched in the code requesting the comparison – and down went the computer.
We’ve all been there, and we’ve all been there often. We’ve all been forced to do a reboot when a device locks up. We’ve all seen the Windows Blue Screen of Death.
Imagine seeing it as your spacecraft is falling into a crater on the surface of the moon.
Though such a thing had not been seriously anticipated, it had nonetheless been planned for. MIT’s Instrumentation Laboratory at Cambridge had been commissioned to create an operating system specially designed for the lunar mission, one that would recover from all possible errors. Part of the design was a carefully-constructed prioritization scheme that would keep high-priority jobs high in the computer’s queue, with data requests from the crew at the top.
Armstrong and Aldrin had the option – at least for the moment – of aborting the landing, blasting their capsule away from the surface and back to the orbiting command module. But the abort was Houston’s call. They reported what had happened and waited for a reply.
Flight Director Gene Kranz had a protocol in place. Upon learning of the crash, he called for expert opinions on what the astronauts had seen. The error code reported by the computer – 1202 – confirmed that the computer had stored the latest navigation data before crashing, so there was no need to abort. Yet.
The recommendation: reboot. Aldrin complied.
A minute later, Aldrin queried the computer for altitude data. And once again, it crashed.
As thousands of feet of altitude fell away, it happened yet again. With only 2,000 feet to go, the worst crash of all occurred – the computer appeared to die completely. But it came back online, only to go out a fifth time at 800 feet. There was, at this point, no turning back.
Barely four minutes had passed since the first crash. The computer had crashed five times. Armstrong was going to have to fly by wire.
We know, of course, that he managed, and the Eagle touched down safely. But the danger wasn’t over; without the computer, they’d never dock with Columbia, once they left the lunar surface.
That meant the ground team had 13 hours to figure out what was going wrong and fix it – from 250,000 miles away. And, of course, we know that they also managed. Eagle made it back to Columbia, and Columbia made it back to Earth.
The problem, it turned out, had been Aldrin himself – unwittingly. He had turned on the rendezvous radar and intentionally left it on, in anticipation of the possibility of an abort. But it had been on the wrong setting – and was systematically hammering the computer with requests. That meant the computer’s resources were being drained catastrophically, for no good reason, at the worst possible moment – triggering the crashes.
The fix was easy: correct the rendezvous radar setting. Once this was done, Eagle was good to go for its post-moonwalk ascent.
Armstrong and Aldrin, of course, never spoke in any detail of this harrowing few minutes, and it was many years before any of the engineers involved wrote of those tense events. The world waited decades to learn how close the Apollo 11 mission had come to failure – and for the most common and innocuous of reasons, now known to every man, woman, child and grandparent: computer designers can’t possibly think of everything, and turn-it-off-and-back-on just flat-out works.
Comments