Exclusive excerpt from upcoming book by ex-Apple engineer explores first iPhone software keyboard design process

Ken Kocienda worked at Apple from 2001 through 2016. On September 4, Kocienda is releasing a new book ‘Creative Selection’ (preorder: Amazon, iBooks), that details Apple’s design process from the inside, retelling his own anecdotes of working on the iPhone and iPad, demoing to Steve Jobs and more.

In this exclusive excerpt, you get a sense of how designing the iPhone software keyboard was anything but obvious. In late 2005, Apple paused all internal development on iPhone and told all engineers to invent keyboard concepts. This excerpt details how Kocienda’s winning design was shot down by Phil Schiller and Tony Fadell, which sent him back to the drawing board …

To give some context to the timeline of events, this anecdote is dated circa October 2005. Much of the software design team did not know what the final industrial design of the phone would look like, but they were tasked with building a multi-touch UI for a phone that would be ‘all screen’.

The UI design and engineering teams used rough hardware prototypes, like the ones pictured here. This is the ‘Wallaby’ referred to in the piece.

Early on, Apple designers established that buttons needed to be at least 44px wide to be easily tappable by a human finger. This posed a problem for the keyboard, in which a native design replicating a standard keyboard would squish so many keys on the same row that each key would be unsuitably small to press.

Whilst designers and engineers beavered away on other parts of the OS for many months, the primary input method was unsatisfactory. In late 2005, head of iPhone software Scott Forstall put everything else on hold and tasked the entire team to come up with possible keyboard solutions. Ken was part of this team and set to work, exploring many different designs.

You can see from these sketches how anything was up for grabs. On the left, Ken imagined a ribbon of little letters spanning from A – Z in a small line (only the letters A and Z are included in the sketch). Tapping on this row would magnify the nearby keys in the larger section above, which could then be pressed to add that letter to the text string.

On the right, you can see early ideation of Kocienda’s plan to put multiple letters on each key. The system would try to guess which word the user actually meant to type, and a suggestion bar (ala QuickType) would let the user specifically select a completion.

Almost incredulously, a design very close to this idea is what Forstall selected as the winner of the derby contest. Kocienda has mocked up what his keyboard prototype looked like:

Kocienda says this is an accurate recreation of what he had built at the time; ignore the status bar details as they are not accurate to what was actually in the builds at the time. (As a reminder of how early this is, the prototype hardware was not hooked up to the cell network at all). You can see how keys are grouped together in twos and threes, with a static suggestion bar above the letters, and a button to switch to an alternate key layout of punctuation.

You can read in the extract what happened when this design was shown to Phil Schiller and Tony Fadell — and was promptly dismissed. This sent Kocienda back to the drawing board where he ultimately invented the idea of a plain-looking keyboard that intelligently adjusted the invisible touch targets on-the-fly, which is what the iPhone ended up debuting with.

Below is the excerpt in full, as written by Ken Kocienda. The full book will be released on September 4.

Within a week of picking my keyboard, Scott scheduled a private demo with Phil Schiller, Apple’s top marketing executive, the man who, after Steve, was most responsible for communicating to prospective customers exactly why we thought our products were great and why they should go out and buy one.

Scott didn’t clue me in on the politics in play between him and Phil or why he had scheduled the demo. I imagined that Scott was eager to show off the results of the keyboard derby, which must have been a topic for discussion up at the executive level. In any case, my job was to prepare my demo so it worked as it did for the demo derby, so that’s what I did.

When Scott brought Phil to the conference room, I was waiting. This was the first time I ever met Phil, and I was nervous. I set everything up as I had a few days earlier, but I had already made a couple of changes to the keyboard user interface. Scott introduced me. Phil greeted me with a quick courtesy that showed he wanted to get right down to business.

He picked up the Wallaby and tapped a few times. I didn’t see what he typed. Phil asked me why I’d put more than one letter on every key. He was pleasant but direct. He seemed to think that my keyboard looked odd, that it required an explanation.

I tried to give him one. I told him about our decisions to make big keys that were easy to target and couple them with suggestions from a dictionary.

Phil wasn’t satisfied, and he said so. Then that was it. I was surprised we were done so fast. The demo was over in about two minutes.

It was sobering to hear Phil’s point of view. Obviously, he had none of the emotional connection I had to my keyboard. While I had been working hard on it, for Phil it was brand new, and he was indifferent to it. He expected the software to win him over, and apparently, it didn’t. This mattered for two reasons. First, as I said, Phil would be playing a pivotal role in pitching the Purple phone to people in the outside world once we were done developing it. Second, and perhaps more important, his reaction was just like a prospective customer evaluating a product from scratch. My keyboard would be a part of the overall impression, and Phil was confused rather than convinced.

A couple days later, Scott and I repeated the private demo performance for Tony Fadell, the executive in charge of the iPod division. I had never met Tony before either, but I didn’t have to know him to see how preoccupied he was. When he walked over to the conference room table with my demo on it, he barely glanced at my keyboard. He didn’t ask any questions. Then he tried my software, but he couldn’t have typed more than a word or two. The demo with him was even shorter than the one for Phil, and within a minute, he and Scott went off together for a private meeting, leaving me alone in the conference room to clean up the Mac, the Wallaby, and the wires connecting them.

Two demos with less-than-positive responses. Add that to my fellow derby entrants’ lack of excitement, and I could tell we didn’t yet have exactly the right solution. I didn’t get to demo the software for Steve. Maybe Scott concluded that we weren’t ready for the big time, but he never said anything specific to me about these executive demos, good or bad.

I didn’t feel like I had let Scott down. My code was the same as it was on derby day. There were no bad bugs during these executive demos. As I tried to interpret the feedback and decide what to do next, I thought back to the Black Slab Encounter with Safari. That breakthrough didn’t represent an end; it signaled a beginning. As exciting as it was to see our web browser render the first sliver of a web page, we realized what the milestone meant. I began to look at my derby-winning design in a similar way, as if it were a successful audition rather than a sold-out performance.

I started to think about improvements, and to help me keep my keyboard goal literally in sight as I sat in my office, I measured and cut out a small piece of paper, about 2 inches wide by 1.3 inches tall, a little smaller than half the size of a credit card turned on end. I pinned up this little slip of paper on the bulletin board next to my desk. I looked at it often. This was all the screen real estate I had available for my keyboard.

This was my touchscreen typing canvas. People would have to tap-tap-tap in that tiny rectangle to type, and I had to figure out how to make that happen. As I pondered that small shape and took stock of my software, I got accustomed to the idea that I might need to rethink some of the decisions that led to the derby-winning design, perhaps all of them.

Excerpted from CREATIVE SELECTION: Inside Apple’s Design Process During the Golden Age of Steve Jobs by Ken Kocienda. Copyright © 2018 by the author and reprinted by permission of St. Martin’s Press.

You can follow iPhoneFirmware.com on Twitter, add us to your circle on Google+ or like our Facebook page to keep yourself updated on all the latest from Apple and the Web.