Wild West to Agile

We've all heard the terms "agile" and "waterfall", but what is the real history behind each? and how did we migrate from one to the other? Here is a firsthand account.

A book cover. "Wild West to Agile" by Jim Highsmith. It shows a transition from a typewrite to a laptop.

I ran into Jim Highsmith recently at a Denver Agile Meetup. Everyone was very star struck to meet him. I had trouble placing the name at first (Sorry Jim). Someone was like "Oh he was a signer of the Agile Manifesto." Well that got my attention. I had a really good chat with Jim and he mentioned that he recently wrote a book (not the first book he's written - although at the time I didn't really know about those either). Everybody else who had gathered around seemed to be big fans of his latest book, so I thought I should probably read it. I was glad I did.

This book is a tour of software development ideology over the past 60 years. Jim breaks those 60 years up into 4 eras. The Wild West (1966-1979) is the period when software development was in its infancy. No one really knew what they were doing. Most were just happy that it all somehow worked. The Stuctured Era (1980-1989) is the period of heavy-handed methodologies, of corporations trying to gain control by imposing a bunch of top-down restrictions. There was a focus on documentation, requirements gathering and Computer Aided Software Engineering (CASE) tools. The Roots of Agile (1990-200) marked the beginning of the coming revolt. Companies were realizing that even with their top-down attempts to gain control, projects were still coming late and over budget. The software developers were feeling stifled and drowning in documentation and process. They began experimenting with new techniques. The Agile Era (2001-2021) starts when a bunch of these software engineers get together in Snowbird and create the Agile Manifesto. This leads to our modern software development landscape.

Jim does a great job of explaining the differences between the various eras. One thing I hadn't appreciated was the difference in the technology between the eras and how that contributed. I obviously knew the technology had changed over time and I under-estimated its impact. He takes note early on of the shift from optimizing for the computer (due to its limited resources) to later optimizing for the human. He also talks how the way in which humans interact with and view computers and their ubiquity has changed the way businesses and developers approach software development. It adds some context to some of the older practices - a way of saying "These practices might not make sense to you through today's lens, but at the time they seemed like an appropriate response to the situation." It softens and adds some empathy to the best practices debates.

In honor of the human side of software development, Jim weaves in a bunch of personal stories, both his own and those of other notable figures of the various eras. They add a lot of color and really help to understand the thought process behind many of the practices and their evolution. I particularly like that he weaved in a bunch of climbing stories. I am also an avid climber and always felt like climbing taught a lot of important lessons that were applicable to life in general and more specifically to software development. I was just often not able to articulate them clearly. Jim does a great job of connecting those dots.

What does the future hold? The last chapter talks about the future of software development. I found it quite applicable. Jim talks about how the pace of change has only increased and the key to surviving is to be agile - able to adapt by changing direction with the technology of the day. He lists 3 key personality traits: Adventurous, NonConformist, and Adaptable. I both laughed and smiled when I read that, since I think that easily describes my personality. And that's not just my own delusions, Nancy recently branded me "The Architect of Adventure". Perhaps that is why an agile software development style always appealed to me.