Will there ever be a Flying Shuttle for Software?

[elfs]

Fred Brooks once famously wrote that there was no silver bullet for software development. Once a project and its tooling were agreed upon, software development took the time that it took. Brooks’s insight was that software developers spend their days more or less at the limits of their intellectual capacity, and no amount of tooling or management could expand that limit. Since a developer is usually fully engaged with any given task, adding a second developer to that task will actually slow production down: the two developers now have the overhead of communicating with each other the nature of the task and each may be blocked by the other to accomplish a signifcant subtask.

The term for this “No Silver Bullet,” and the book by that title has gone on to be a perennial best-seller, usually purchased by harried software developers and left anonymously on the desks of clueless project managers.

Treadle-based handlooms for weaving cloth have existed for almost two thousand years, and their current form is unchanged from the one that emerged in Germanic and Italian forms a thousand years ago. A loom has a frame of vertical threads in two alternating sets; each press of the foot peddle pulls one set apart from the other, and the user strings a spool of thread called a “shuttle” across this horizontally, then cycles the foot peddle to close the lifted threads down, then lift the other set up, giving the weaver that alternating over-under sequence that holds cloth together. Skilled users were known to “throw” the shuttle across the loom field, but cloth could only be woven as wide as one person could reasonably reach.

In 1733, a man named John Kay built a narrow wooden track, put the shuttle into the track, and with a piece of string and a handle, “jerked” the shuttle back and forth across the loom field. A few improvements later, he had made a loom three times as wide as existing looms, and one that could be operated twice as fast. Kay’s “flying shuttle” replaced the thrown shuttle almost instantly, and the industrial revolution had its first major component.

The more I write software, the more I have this sensation that something is very wrong with the way we write software. I listen to developers describe their projects and my overwhelming thought is, “Didn’t someone do that already?” I can’t begin to count the number of times I have implemented the same small algorithms over and over. Consider Stack Overflow, in which questions “How do I find a substring?” has several different code snippets that dozens of people will now simply cut and paste into their own code. Programmers get paid like princes these days mostly to know how to know these things, and glue them together in the right order for our corporate masters, and make them work profitably.

If there’s No Silver Bullet, could there be a Flying Shuttle waiting for us? I don’t have an answer, but I fear that there is: there’s an answer that glues everything from the hardware to the UI together into a meaningful whole, that answers the questions people want answered, and does the things people want done. I suspect it’s already here; we just haven’t identified it for what it is yet.

Comments

The silver bullet

[ideaphile.livejournal.com]

I've met Fred Brooks Jr. twice, the second time for an interview about his book The Design of Design. I call myself a computer architect because what I do-- when I'm doing what I'm best at, at least-- meets his definition of that term.

I have a silver bullet. I got the seed of the idea in the mid 1990s and I've been developing and expanding it for 20 years now. It specifically addresses the communication-overhead problem that developers face along with most of the well-known problems in computer programming (e.g. aliasing, and from that Infoworld article you wrote about in another post, multithreading, too-big data, security, encryption, and identity management; alas, I have absolutely no idea what to do about measuring or mitigating hardness).

That original seed of an idea really is no more complicated than the flying shuttle, though computers are many orders of magnitude more complex than looms, and the consequences of this one basic idea change essentially everything. Any language that can be used on existing computers could be used on these new systems, but new languages would work dramatically better. Legacy apps would gain almost none of the benefits, and legacy operating systems would be largely useless except to support those legacy apps.

I've discussed the high-level concepts with a few people here and there; the consensus seems to be that the benefits of vastly improved reliability, predictability, security, etc. aren't worth the cost of essentially obsoleting the entire installed base of hardware and software. I strongly disagree, but of course the only companies that could support this work are the ones that developed all that stuff and need to keep selling it.

I didn't have time to discuss any of this with Brooks during our interview; he was basically on tour with his publisher and had other interviews to get to. I still wonder what he'd think about it, but not enough to try to find out. It might just be too depressing to think that a silver bullet is available, but it's too late to use it.