Day Five: Software, Bill Tindall, and the Little Old Ladies

Bill Tindall, a noted NASA engineer and manager, was responsible for herding an MIT department of high tech cats whose software development activities were poised to make or break the Apollo program.

The Apollo navigation and guidance computer (AGC), designed by MIT and built by Raytheon, was a marvel of technology. It didn’t have the impressive display we have come to associate with today’s computers — but then, neither did HAL, the rogue computer in Stanley Kubrick’s science fiction classic, “2001 – A Space Odyssey,” which hit big screens across America the same year as the Apollo Moon landing.

The AGC was actually more sophisticated in its routine operations that today’s laptop computers. It knew how to do things on its own and was connected to the equipment to do those things: rocket engines, radar antennas, and gyroscopes. The AGC could ask for information, wait for it, and then use that information to handle a range of tasks that included navigating from lunar orbit to the Moon’s surface. It also had a user interface that could accept requests and answer them with real-time data.

The Software Imperative

Quite simply, without the AGC there could have been no Apollo program. Journeying to the Moon required data transmission at the speed of light, with complex instructions executed at the speed of electronic circuitry. You had to use computers…and those computers had to be programmed. Without software, the hardware was just that. Hardware. Creating the programs for NASA’s AGC required people with an entirely new skill set; it required programmers — and while NASA had years of rocket design and development under its belt, software development was a new thing altogether. It ended up nearly being the Achilles heel of the Apollo missions.

At the time of the Apollo program, software \was a new concept, synonymous with programming. It was such a foreign concept for most people that the word itself was typically used in quotation marks, and sometimes spelled in media accounts as “softwear.” The challenges facing MIT’s software development team were many: they had to create programs that would guide the on-board computers through every conceivable navigational situation, and do so with very little in the way of well-established development processes and protocols. The Apollo software developers also had to create their programs with a stringent economy of code, and meet a deadline that was becoming increasingly problematic.  

The Opposite of the “Peter Principle”

Chris Kraft, the head of Mission Control in Houston, was concerned about the lack of progress at MIT in getting the software finished in time for the Apollo program. To rectify the situation, he chose Bill Tindall to go to MIT’s Instrumentation Lab, which was headed by the legendary Doc Draper. Tindall was well versed in the theory, the math, and the challenges of space navigation. He had joined the Navy straight out of high school and got his first exposure to electronics in the radar room of a destroyer operating in the Pacific during World War II. Here he developed a fascination with math and engineering. Following the war, he got an engineering degree from Brown University, and one month after graduation in 1948 he went to work for NASA’s predecessor organization. 

Tindall’s work at NASA included the creation of the trajectory and orbital calculations for much of the Mercury and Gemini programs, including the math that was required for the space rendezvous missions of Gemini 6 and 7 (a worldwide first in 1965). His wife once described him as “the opposite of the Peter Principle.” His ability and experience to absorb, understand, and sort out serious technical problems earned him the respect of his colleagues, even when they didn’t get the decisions they wanted from him. Rather than move up the management ladder, Tindall preferred functioning as a deputy in the divisions in which he worked because it gave him greater ability to get things done with less bureaucratic interference. 

Tindallgrams

Bill Tindall’s NASA memos detailing and commenting on the status of software projects, as well as his recommendations, came to be known as “Tindallgrams,” and were the stuff of NASA legend. It is estimated that over a six year period he produced 1,100 of these, averaging three a week during the frenzied year of 1968.  In one of the 184 remaining Tindallgrams, which could be highly entertaining as well as elucidating, Tindall reported that, “I just got back from MIT with my weekly quota of new ulcers, which I thought might interest you…The first estimate was that the program tapes could be released for November 15, which is exactly three months too late. Rather an interesting proposal, I thought, since it is so obviously unacceptable.”

Like all modern computer memory, the Apollo AGC contained memory composed of 1s and 0s, The AGC had the capacity for exactly 589,824 zeros and ones, which was all that would fit in the memory portion of its 1 cubic foot box. This was before the IC industry began manufacturing memory chips (most notably DRAM and ROM chips). Instead, each zero or one was a wire that was either threaded through a tiny ring magnet (a “1”), or just outside of it (a “0”). These wires were threaded by hand in a factory in Waltham Massachusetts in a process that looked like weaving — which is exactly what it was. Each of the 589,824 wires were threaded with a needle to literally weave the software code that the programmers at the Instrumentation Lab had written.

A “Stitch in Time” Saves Zeroes and Ones

At the time of the Apollo missions, computer programs were stored and moved around using stacks of punch cards and reels of magnetic tape. Once a mission’s programs were finished, they were loaded onto tapes and cards and delivered to Raytheon’s Waltham facility where the instructions on how to fly to the Moon were “stitched” into the computer’s memory by dozens of women (presumably with textile industry backgrounds) wearing blue smocks and sitting at specialized looms. They became known, rather paternalistically, as the “little old ladies.” 

It took 8 weeks to create the “rope cores,” as they were called, for a single computer, and each 12-inch-long memory module contained half a mile of wire. In the 1960s, this was the densest computer memory available, but the limitations it imposed on the software developers meant an on going set of trade-off decisions on what was mission critical and what, while desirable in a perfect world, had to be eliminated in the interest of available space.

When Tindall first starting visiting MIT, the number of people working on flight software was about 130. The months later, when a Tindallgram reported that software development was finally taking place in an “orderly, professional, and unmarried manner,” that number doubled. During most of 1968, the software department staff would exceed 350.

Software Innovations

Whatever its difficulties In meeting NASA’s deadlines, the quality of the software produced by MIT, with the assistance of Bill Tindall, was never in doubt. During the course of 11 Apollo flights with astronauts, there was not a single hardware or software failure — 2,502 hours of spaceflight (104 days and 6 hours) without a single “glitch.” As one historian put it, the AGC performed better than “any other computer designed then or since for aerospace application. Such near perfect reliability was achieved at considerable effort (and) attention to design.”

Two key software innovations in the AGC were its ability to make decisions about what work to do (which may likely have made a critical difference in the decision to continue the lunar landing when alarms were going off in the final seconds before Armstrong set it down on the Moon’s surface), and the ability to quickly and effectively recover from being overloaded or from failure or “bugs”. 

Apollo 11 flew to the Moon as much on MIT’s rope core memories and computer chips as it did on the Saturn V’s engines — and those rope core memories and the programming they contained would not have been possible without Bill Tindall and a cadre of blue-smocked women. Upon Tindall’s death in 1995, Chris Kraft, the head of the Manned Spacecraft Center and the man who sent Tindall to MIT to sort things out, opined, “It would be difficult for me to find anyone who contributed more individually to the success of Apollo than Bill Tindall.”