Best Software Writing I
Joel Spolsky
Apress, 2005
Yet Another Book Review
by
J. L. Sloan
2005-07-13
It is sometimes just
a little too cute. That was my first impression of the collection of recent essays
on software and the software development process edited by Joel Spolsky.
Spolsky, author of the book and popular weblog Joel on Software (http://www.joelonsoftware.com), is obviously well read, but his selections
for this volume occasionally reflect more on his sense of humor, or maybe his
desire to suck up to his friends, than his intellect. Still, among the
twenty-nine essays collected in this book, less than a handful fall into the
clever but maybe not smart category, and even those are at least entertaining.
The remaining essays are all over the map, ranging from introductions to
programming languages to how to sell software.
Remarkably, only one
essay, Mary Poppendieck’s “Team Compensation”, was originally published in a
magazine or journal (“Unjust Desserts” from Better Software, July/August
2004). A handful are from keynote speeches at conferences. The rest are from
the authors’ weblogs. I like to
think that I am pretty well read myself in this field. With just a few
aforementioned exceptions, I found the articles to be uniformly interesting
enough to be worthy of being included in an annual “best of” collection. They
might not have all been my choices, but I can see where they could be someone’s
choices. If these essays really represent the best writing on the topic of
software, it does not bode well for the software magazine industry.
The essays are all
over the map in terms of subject matter. Spolsky did not group them in any way.
But I sensed some recurring themes that I will address below. These are
probably more indicative of the editor’s interests than any major trends in the
software field. Or maybe they say something about my own interests.
Social Software
The idea of software applications as a social tool, or even of a
force in society, is not a new one. Before the World Wide Web existed, even
before the Internet existed in the larval form of ARPAnet, USENET was creating
communities of users for whom the technology of communication was just a means
to an end. Anyone who has received spam – that would be anyone that doesn’t
have the paranoia of the NSA regarding their electronic mail address – has been
the victim of a software tool used for social purposes. Several of the essays
in Spolsky’s book address social aspects of software.
In “Autistic Social Software”, danah boyd argues that most
software interfaces reflect the latent (or not so latent) autism of the typical
software developer. The essay was based on a talk she gave at the 2004 Supernova
Conference, and in it she reflects on the number of people in the audience
using their wireless laptops while she was speaking.
“Many of you are staring at your laptops,
multitasking. Although you will only remember a fragment of this talk, you will
probably tell me that you remembered the important part or that you were
practicing your continuous partial attention. Some of you may already be ninja
masters at this, but the majority of you are probably paying poor attention to
both the computer task and me. But you want to be a continuous partial
attention ninja master because you’ve been told that all of the cool kids are.”
She goes on to link
such multitasking with Attention Deficit Disorder and Asperger’s Syndrome. When
I read this, I initially wondered if it was a little harsh. I’ve heard many
rationalizations for using a laptop during meetings and talks: taking notes and
researching references in real-time; the fear that if one does not keep up with
one’s email in real-time, bad things will happen; that most meetings only
require your partial attention (doodling may become a lost art). But given that
many technologists may have a mild form of autism like Asperger’s (or so it has
been suggested – and it is not hard to believe), boyd may have a point.
A colleague of mine
who, like me, had spent some time in the People’s Republic of China, mused once
that it would be interesting to see the culture that develops from a ruling
class dominated by only-children (that is, children raised without siblings), a
result of the PRC’s birth policies. My spousal unit made a similar remark about
the emerging male dominated society in the PRC. Furthermore, since the PRC is
exporting many of their female infants, particularly to the west, they may find
themselves in the position of either having to import wives from the west, or
worse, having their most productive citizens flee to the west to find wives.
Similarly, I wonder
now about a society in which no one is accustomed to giving anything their full
attention. There are lots of problems so hard that they require all of your
concentration and then some to get your head around them. Is it possible that
one of the reasons for the lack of students in science and engineering, if
true, is due to the fact that other branches of endeavor do not require one’s
full attention in order to be successful?
Clay Shirky’s two
essays, “A Group is Its Own Worst Enemy” and “Group as User: Flaming and the
Design of Social Software” describes the common patterns of social dynamics that
arise from different forms of communication, electronic and otherwise. Different
forms bring rise to different behavior. For example, mailing lists and
discussion forums like USENET are many-to-many and frequently give rise to
flaming (heated and protracted responses). Weblogs are one-to-many, eliminating
both the ability to have a dialog and to flame as a response. Shirky discusses
how flame wars and other patterns inevitably arise in many-to-many
communication media. They emerged in very early electronic forms like the Plato
system, circa 1960. Science fiction fans witnessed these patterns emerge in the
letter columns of mimeographed fan publications or “fanzines” as far back as
the 1940s. They can surely be traced back much farther than that. More recently
they have emerged in multi-player adventure games.
Shirky cites the
work of psycho-therapist W. R. Bion who, in his 1961 paper “Experience in
Groups” described many of these patterns as he observed them in group therapy
sessions. Bion observed that humans are “hopelessly committed” to both
individual identity and to group membership, behaving simultaneously in both
roles. Humans are at once fundamentally individual, and fundamentally social.
Shirky cites as an early example of this “The Tragedy of the Commons”, a term
originally used to refer to the situation in which many farmers shared a common
grazing area (hence, geographic names like “Boston Commons”.) The group as a
whole has an incentive to maintain the long term viability of the commons. Yet
every individual has an incentive to overgraze in order to maximize the value
they can extract from the commons. And so it is even in electronic-mediated
social interaction. (And, I would claim, so it is in a pay-for-performance
culture, where there is incentive to both optimize the performance of the
group, and the performance of the individual at the expense of the group. But
more on that later.)
A common theme in
both boyd’s and Shirky’s essays are that if your intent is to develop useful
mechanisms for social interaction, electronic or otherwise, you must take into
account how people react both as individuals and in groups. Both authors
criticize technologists for seeing this as a technical issue, when in fact it
is both a technical and a social issue. No matter how you design your software,
people will find a way to bend it to their preferred way of doing things,
regardless of the relatively autistic perspective of the system designer. Or,
if that is too difficult, they will abandon its use completely.
Mechanisms for
social interaction including not just recent inventions like electronic mail,
and web-enabled technologies like weblogs and Wiki, but telephony. Shirky
describes a self-made multimedia conference call in which the voice
conversation was mediated by an ad hoc protocol using instant messaging in a
chat room, and the meeting notes were developed collaboratively in real-time
using Wiki. None of these technologies were integrated. They were all separate
tools, used opportunistically because they just happened to all be available to
all the participants.
I recently spent a
week visiting my parents back East at their retirement community. It occurred
to me then that theirs will be the last generation who will find it acceptable
to retire to a place in which broadband Internet access is not a given. If I
can afford a retirement home when the time comes, you can be sure it will have
to provide ubiquitous wireless broadband Internet, just as cable television is
considered a necessity now. Laptops are now the perfect entertainment delivery
system for dorm rooms today, and so shall they be in the efficiency apartments
for retirees in the coming years, for all the same reasons.
Similarly, it
occurred to me that ours may be the last generation that will find a communication
system that provides only voice acceptable. Future users will expect to find it
easy to integrate many modalities, voice just being one of them. Convergence
may simplify this, since it is likely to be economical to carry all modes of
communication over a single network. Whether or not it is the traditional
telephony equipment manufacturers that appreciate the implications of this
integration, and capture these emerging markets, remains to be seen. Some forms
of this of course occur already, in the form of web access to multi-media
messaging and the like. But just as Xerox could not see past its photocopier
roots when pondering the significance of the developments at its Palo Alto
Research Center (PARC), so too, I wonder, if telephony equipment manufacturers
can see past their historical delivery of dial tone in one form or another.
Adam Bosworth, in
his “ICSOC04 Talk” (for the 2nd International Conference on
Service-Oriented Computing), also tackles aspects of social computing. He
enforces the themes presented by boyd and Shirky arguing for simpler, more
flexible interfaces for social software. Rather than anticipating how such
systems will be used, make them simple to understand, and flexible enough for
their user communities to create their own models of use. Bosworth cites one of
my favorite quotes, from Seven Pillars of Wisdom by T. E. Lawrence
(a.k.a. Lawrence of Arabia):
“All
men dream, but not equally. Those who dream by night in the dusty recesses of
their minds wake in the day to find that it was vanity: but the dreamers of the
day are dangerous men, for they may act their dream with open eyes, to make it
possible.”
Shirky develops
specific checklists regarding design of social software, things to look out for
and common patterns to anticipate. His were among my favorites in the book, and
in general the essays on social software were the best parts of the book. In
his class “Market Management for Technologists”, Barry Karafin remarked that
most technical knowledge has a short half-life. I think this is why the older
you get, the less you value technical knowledge. As you age, the duration for
which technical information is useful represents a smaller and smaller fraction
of your life, and so has ever diminishing importance.
Staffing from Both Sides of the Table
In his essay “The
Pitfalls of Outsourcing Programmers”, Micheal Bean says companies have confused
the chocolates with the box they come in. He illustrates this with the story of
how his favorite confectionary store outsourced the manufacture of the fancy
boxes in which they sell their chocolates, their supplier building the boxes to
the store’s design. But the store would never outsource the manufacture of its
chocolate, since that is where all the value is added to the product that makes
it competitive. Every single line of code written for a software product
represents a management decision, made by the developer, where value is either
added or subtracted from the product according to the skill of the developer.
Smart companies realize this, and do not treat their development staff as blue
collar workers in a manufacturing process.
(Bob Lewis echoes
this sentiment in his book Leading IT:
The Toughest Job in the World, IS Survivor Publishing, 2004, which I also
recently finished. Lewis, who writes the popular “IS Survivor” column for InfoWorld
magazine, says that companies that outsource their core competency are moving
the value chain outside of their control. Such companies are not likely to
persist.)
What Paul Graham
means by “Great Hackers”, the title of his essay, are those individuals
identified by Fred Brooks circa 1974: those developers whose productivity are
an order of magnitude or more greater than their peers who were still
considered competent. Graham remarks that
“If
variation in productivity increases with technology, then the contribution of
the most productive individuals will not only be disproportionately large, but
will actually grow with time. When you reach the point where 90% of a group’s
output is created by 1% of its members, you lose big if something (whether
Viking raids, or central planning) drags their productivity down to the
average.”
He argues that
successful companies must understand what motivates these extraordinarily
productive individuals so that their productiveness is not frittered away.
First, money is not
their primary motivator, and that’s good, otherwise you would have to pay them at
least ten times what you are paying their co-workers who are competent but not
great hackers. Great hackers want and need good tools. They want a supportive
environment in which to work (in a world of cubicles, this often translates as
“peace and quiet and a lack of interruptions”). They want interesting projects
on which to work. The company that learns to exploit and reward great hackers
will reap the benefits of their order of magnitude increase in productivity.
How do you identify
a great hacker until you’ve worked with them? According to Graham, that’s
difficult, even for other hackers. Fortunately, because software developers
almost inevitably work collaboratively, over time great hackers identify one
another. (I once worked for a manager whose background was math education. He
had a professor once who said “How do you find all the smart kids? You can’t,
but you really only need to find one smart kid. That kid will know all the
other smart kids.”)
Along these same lines
is the essay “Passion”, by Ron Jeffries, who talks about the people with whom
he prefers to work, who seem to be pretty much the kind of people Graham
describes.
Eric Sink is one of
the most represented authors in the book, three of his essays the business of
running a software company having made the cut. His essay on the “Hazards of
Hiring” actually made me revisit my resume and rewrite a potion of it. (If you
are morbidly curious, you can see the results at http://www.diag.com/jsloan/personal/Resume.doc).
His most interesting
remark was “look for self-awareness” of “the first derivative”. By this he
means in part, look for the person who is driven to constantly learn. “People
who are committed to constant learning are people who know what they don’t
know. They know their own weaknesses, and they’re not insecure about talking about
them.”
He also
distinguishes between “developers” and “programmers”. Programmers write code.
Developers exhibit a breadth of skills among which writing code is one. If you
are a small software company, you cannot afford to hire specialists like programmers.
You have to hire generalists like developers.
In the only essay in
this book to have actually been reprinted from a magazine, Mary Poppendieck
writes about “Team Compensation” and the perils of pay-for-performance. Her
lesson echoes many of my arguments based on the work of Robert Austin on
measurement dysfunction. She lists a checklist of dysfunctional behaviors
driven into an organization by pay-for-performance: competition, the perception
of unfairness, the perception of impossibility, suboptimization, and destroying
intrinsic motivation. (Once again, many of her remarks are echoed in Bob Lewis’
book.)
In “What to Do When
You’re Screwed: 5 Scenarios for High-Velocity Engineering Managers”, Michael
“Rands” Lopp tells you not only “what to do when you’re screwed”, but how to
tell (sometimes) if you’re screwed, or about to be screwed, or if your job or
company “sucks or is about to suck”. This is a precious article that is both
funny and chock full of useful career advice. He cites examples from his own
career where he didn’t follow his own advice, to his own detriment and dismay.
(Another essay in this book – I don’t remember which – remarks that experience
is never the best teacher. Would you rather learn “Here be alligators!” from a
sign, or from personal experience? Lopp’s essay is an example of things it is
better not to learn from
experience.)
The Business of Software
Eric Sink founded
the ISV SourceGear, which develops and sells tools for developers. (Spolsky
tells us that the term ISV, for Independent Software Vendor, was invented by
Microsoft to mean “Any software company except Microsoft.”) In his two-part
essay “Closing the Gap”, he talks about how to market and sell software without
a marketing or sales department. Not particularly applicable to a big company,
but of extreme pertinence to a small software shop. The gap to which he refers
is the gap between your product and your potential customer. To get the
customer to buy your product you must close the gap.
You can do this via
“proactive sales”, that is, actively selling your product to a customer on a
one-to-one basis. This works for a big company which employs at least one
“sales guy” (“guy” being a gender neutral term). This moves the customer closer
to your product, convincing them to buy it. Proactive sales are expensive. You
have to have a sales guy. You have to deal with the sales guy. You have to
compensate the sales guy with commissions.
(Much of what Sink
says about the drastic difference between the sales guy’s personality and the
technologist’s was echoed by Barry Karafin in his class on “Market Management
for Technologists”. The same manager with the math education background I
mentioned earlier once said he really liked the Myers-Briggs Type Indicator
because the MBTI test was the first time many people came to realize that not
everyone else thought the way they did. I suspect Karafin’s single slide on
personality differences could achieve a similar effect.)
Or you can do this
via “responsive sales”. This moves the product closer to the potential
customer. Sink describes ways to do this – mostly web based – without any
marketing or sales department. He warns the reader not to get carried away. For
example, Amazon.com has an amazing web site to support their sales. But Amazon.com
has a huge organization just to support this web presence. A small company can
successfully sell software over the web using much simpler mechanisms. His
methods seem applicable to smaller – even really really small – software
companies. I would consider his essays (and his blog) required reading for
anyone wanting to make a living selling software.
Bruce Eckel’s
“Strong Typing vs. Strong Testing” and Larry Osterman’s “Larry’s Rules of
Software Engineering #2: Measuring Testers by Test Metrics Doesn’t” both deal
with software testing.
Eckel argues that
strongly typed languages have lulled some programmers into a sense of
complacency, into thinking that if their code compiles, it must work. He says
“If it’s not tested, it’s broken.”
(If you want to see
an example of what I personally consider to be adequate unit testing, see the Desperado
open source library of components for embedded applications at http://www.diag.com/ftp/desperado-cascade.tgz, or any firmware written by Tam Noirot.)
Osterman gives some
examples of measurement dysfunction when applied to software testers. When you
reward testers for finding bugs, they will optimize their efforts into finding
the largest number of bugs, and waste developers’ time fixing those bugs,
instead of finding the most important, show stopping bugs. (Once again, if you
have heard my talk on
Eric Lippert
describes why it takes months to make a two line change to the Microsoft
product VBScript in “How Many Microsoft Employees Does It Take to Change a
Lightbulb?” This will come to absolutely no surprise to anyone who has worked
in a large code base in a big company. (This experience is, without question,
the most unique and valuable that I will carry away from my time at Lucent
Technologies/Avaya.) This essay has great utility just to give to read to the next
person who asks “How could this possibly be taking so long?”
Final Thoughts
I’ve discussed less
than half of of the essays contained in Spolsky’s book. The rest are worth
reading too, but I’ve already taken up enough of your bandwidth.
But I have to
mention one final essay. Paul Ford is an editor at Harper’s magazine and
a commentator on National Public Radio. His essay “Processing Processing” was
over my head. I think it was about
the use of technology, particularly web-based technologies, to express abstract
semantic content. Maybe. I’m not sure. I’ve switched back and forth between
management and engineering during my (so far) thirty-four year career, and
during that time I’ve developed a sensitive BS detector. It didn’t go off. I
think Ford talks the talk and walks the walk, and clearly invests a lot of his
own time and money in his personal research project. But the idea that a guy
with an English degree could write something regarding computing technology
that I could not understand is a humbling experience. I learned decades ago
that people are smart in different ways, some of which I have problems
appreciating.
Thank God for
diversity.
(For a bibliography
of references like this book and others that I have found useful, visit
http://www.diag.com/ftp/Bibliography.doc.)