Last week’s blog detailed how a group of four men who restore historically significant, vintage computers – Carl Claunch, Ken Shirriff, Mike Stewart, and Marc Verdiell — connected with Jimmie Loocke. Loocke, a former technician at the NASA Manned Spacecraft Center (now Johnson Space Center), had purchased two tons of scrap Apollo hardware at a recycler’s warehouse in 1975. Included in this purchase was a rare Lunar Module Apollo Guidance Computer, or AGC.
The AGC is the computer that navigated Apollo spacecraft to the moon and back, including the landing and ascent of the Lunar Module’s to the moon’s surface. Part 1 ended with Loocke and Steward realizing the AGC was in good-enough condition to possibly be restored. That’s when they contacted Claunch, Shirriff, and Verdiell. Click here to read last week’s blog and catch-up.
Heading To Houston
The restoration team needed to get their hands on the AGC, but Loocke did not dare ship his precious computer to California. So the team traveled to Houston and sequestered themselves, a bunch of expensive and sensitive electronic diagnostic equipment, the AGC, and Jimmie Loocke in a hotel suite for two weeks.
Fortunately, Mike Stewart already had the complete schematics of the AGC because he had spent several years researching and building a faithful replica of the computer. The Apollo Guidance Computer is divided into two trays, A and B. Tray A contains mostly logic modules while Tray B is mostly the memory modules and associated support electronics.
The AGC had a modular design because NASA didn’t trust the reliability of computers at the beginning of the program. This led to the lunar module being equipped with a full set of extra modules that the astronauts could exchange in flight. Although the in-flight repair requirement was later dropped and the computer never failed, the original modular design remained.
The team first removed the A modules and tested each one. As you can imagine, this was incredibly detailed, painstakingly slow work. They connected each module to their “mobile test bench” and tested every gate to make sure it worked. After one week of testing, they were surprised to determine that all A modules appeared to work and were safe to power up.
When powered, the logic modules worked well enough that the computer went through the first instructions of its boot sequence. It did not go very far into the sequence since it still lacked Tray B memory, but the fact that it did so was greatly encouraging.
AGC Quality Design, Workmanship
It is extremely unusual that a computer of this age would try to boot on its first attempt without a month of debug and countless repairs. For example, the 1960’s IBM 1401, which the team had previously restored, had 200 faults. Similarly, the 1970’s era Xerox Alto took nine months of effort to boot. The team was flabbergasted that they could have literally skipped all of the testing and could have just powered up the AGC Tray A, and it would have worked. This was a testament to the quality of its design and workmanship.
For example, the system voltage reference source was specified at 5 volts. When powered, the system came to 4.99 +/- 0.01 volts. Think about that: this is 50 plus year old equipment which had been in storage for decades, and it was running within 0.20% of the original specification! For that matter the system had two power supplies and they both came online perfectly regulated to full load.
On the final day of their two-week exploratory visit to Houston, about 20 minutes before they were to return to the airport, Mike succeeded in connecting an FPGA memory simulator that he had engineered on the spot into the computer backplane (did I mention these guys are smart?). The original Apollo 11 program was loaded and the AGC ran 2,000 instructions. This was another huge success which greatly encouraged the team. Marc said later this was one event that convinced them they would be able to fully restore the AGC.
Memory Problems
Next up was the memory modules, which proved more challenging. Verdiell took the core memory module with him for more analysis. They had already discovered a wire break in the module, but locating and repairing the break was extremely difficult for several reasons.
First, this was one of three potted modules in the AGC, so they could not see or reach inside it for testing. Furthermore, this core memory module contained over 32,000 microscopic ferrite core rings suspended in thousands of fragile 4 mil wires, probably the densest core memory module ever built at the time.
The module was brought to Samtec to be inspected with their sophisticated X-Ray/CT equipment. Scans revealed that the break was not close to the module pins, which unfortunately would be the only location where there would be a chance for repair. Time Domain Reflectometry and Frequency Domain Analysis later performed in Marc’s own lab confirmed that the break was located deep inside the core stack and buried in the potting, in an unreachable location.
Plans for a direct repair were abandoned. Fortunately, for extra reliability, the memory was designed with an extra bit in the form of a parity bit, used solely for error detection. It was decided to reuse the parity bit to replace the faulty bit, and disable parity checking. This would require (reversible) changes in the AGC wire-wrapped backplane, a delicate operation for sure, but not impossible. Also, the analog modules supporting the memory sections were much more difficult to test. The next steps had to be performed in a properly equipped lab.
AGC Road Trip To Silicon Valley
The team began exploring with Loocke the possibility of bringing the entire AGC to Marc’s lab in California to complete its restoration. By this time Loocke had grown comfortable with the restoration team. Special arrangements were made to fly the AGC, in a seat next to Jimmie, from Houston to San Francisco. However, when Jimmie arrived at the Houston airport, the crew would not allow the AGC on the plane, despite Jimmie showing the approval documents and special tickets issued by the airline.
Jimmie reluctantly luggage-checked his rare, irreplaceable piece of history and entrusted it to the baggage handlers of a major airline. Sure enough, when his flight arrived in San Francisco the AGC case was nowhere to be found. Panic and much arguing with the airline personnel ensued, until it magically reappeared hours later; it had been mistakenly loaded onto another flight.
At Marc’s lab, the team discovered three more faults in another potted memory system module. Fortunately, they were able to repair them all. They were also able to repair the read-only rope memory system that contains the programs. The rope memory system on this AGC was a ground test model, also built by Raytheon, but not to the same standards as the space-rated AGC. It behaved much closer to what you’d expect from a computer of this age: it had dozens of faults, many of them intermittent, sprinkled throughout and kept the team entertained for days. Eventually, after all of the modules were repaired and installed in the AGC, it passed all logic and memory tests.
Finally, they began running the original computer programs, the most interesting being P63, the iconic program that controlled the Apollo 11 lunar module (“Eagle”) descent for the actual moon landing. The P63 computerized flight control was essential to a successful lunar landing. NASA had deemed impossible for humans to fly the landing given the barely sufficient amount of fuel allocated. Although Neil Armstrong famously assumed manual control of the ship for the few last minutes of the landing to avoid a rock field, Eagle successfully landed with 24 seconds of fuel left in the tank, mostly thanks to the AGC.
The AGC’s data output and monitors were nothing like the video games of today. The restoration team, just like the original Apollo astronauts, were looking at a small screen with a black background and green block letters, known as the Display-Keyboard interface, DSKY for short. As they couldn’t get any of the major museums to lend them a real one to restore, the team built their own DSKY replica. As Marc later said, “They also had two stick controls. This and the obtuse DSKY makes for an extraordinarily entertaining video game!”
New Connectors
Verdiell stresses that Samtec’s sponsoring of the connectors for the project was essential to the restoration. The team needed replacement connectors throughout the AGC. For you connector historians, the AGC used the old Malco Mini-Wasp systems. All terminal pins are wire-wrap .025” (0.64 mm) square tails with coined posts, and the receptacle contacts are dual-wipe forked. Samtec’s Custom Connector Group tooled these pins and plastic bodies based on original 1962 Manned Spacecraft Center (now Johnson Space Center) prints.
All connectors used in the AGC incorporated this terminal and contact system in a .125” grid array. The restoration team placed these individual socket and terminal pin assemblies in arrays that matched the same size and pin count of the original connectors. These connectors were used for board-to-board, module, and backplane connections. They were in the critical interface between the restored computer and monitor, which was necessary for all system checks, communications, and simulations.
Samtec donated the molds, stamping dies, base metals, plating, assembly equipment, and labor to provide thousands of socket and terminal pin assemblies.
The group is taking the AGC on tour this week, the week leading up to the July 20th anniversary of the first lunar landing. First they are going to a private event with Eldon Hall in Tampa, Florida. Hall is the original NASA chief architect of the AGC. They will demonstrate the AGC by running moon landing simulations on the computer.
The next stop is at the Cradle of Aviation Museum in Garden City, New York on July 18. They will again run demonstrations and simulations.
The last stop is at MIT in Cambridge, Massachusetts, on July 20th at the 50th Apollo Landing Anniversary event sponsored by the MIT Museum. The AGC was developed at MIT.
Why Did You Do This?
At the end of our last conversation, I asked Marc why he did all of this. I assumed his answer would be something like he did it because the lunar landing is arguably one of the greatest human achievements of all mankind, or because it is living history.
But his first response was “because it was fun!” He later acknowledged the historical significance, but then he added, “You can’t top this one. This is where computers went from the size of an entire room to their current size. It’s the first computer used to fly a machine, let alone to fly one into space. It’s obvious it was made by an A+ team. When you restore it, you feel like you’re there in the 1960’s again. You get to relive Apollo; that’s priceless.”
Other Links That May Be Of Interest
- Marc Verdiell’s YouTube Channel
- Jimmie Loocke / Preserving the Apollo Guidance Computer
- Everything, and I Mean EVERYTHING You Ever Wanted to Know About AGC
- Video of Eldon Hall Disassembling the AGC
- Debugging the AGC
- Restoration of the IBM 1401 Mainframe
- Restoration of the Xerox Alto
- You Tube Chronicle of Xerox Alto Restoration
