In their CHI 2008 paper, Greenburg and Buxton distinguish design sketches from prototypes, arguing that "early designs illustrate the essence of an idea ... " and help "make vague ideas concrete, reflect on possible problems and uses, discover alternate new ideas and refine current ones." They go on to argue that standard usability testing often only inhibits these early ideas, because at the earliest stages finding the "right design" is more important than "getting the design right." I found that this paper mirrored some of my own frustrations with academic HCI: incrementalism, an over-emphasis on novelty in early stage design, and the dominance of user studies, regardless of their pragmatic utility.
The language of the Greenburg and Buxton piece mirrors somewhat Paul Graham's 2003 essay Hackers and Painters. Graham argues that hacking more closely resembles craft than science, and yet is shoehorned into the awkward "computer science" moniker in the academy. The result is that academic success tends to be a poor predictor of hacking ability. Greenburg and Buxton argue that HCI has a similar bias toward science and therefore inevitably away from open-ended creativity and innovation (although Jono would beg to differ). This necessitates a focus on incrementalism since only the well known can be thoroughly tested. It is not surprising, then, that many, indeed most, truly innovate technologies have percolated outside of the bonds of scientific rigor.
Furthermore, countless times in graduate school I witnessed an advisor or senior student collapse a project or idea only because something similar had once been described in a paper, or, even more astonishingly, that some technique described in a paper could possibly be made to do something similar. This misses the point of early stage sketches. As Greenburg and Buxton write, design sketches are not necessarily "suggestive of the finished product," but instead serve as a means of exploration. Or as Schrage put it in "Serious Play," these early sketches "externalize thought and spark conversation." That is, the sketch is the conversation. Ideas and talk are cheap: speculation about an idea is just that.
This leaves me with a quandary for my work now. After all, I did a methodology thesis. But I am reluctant to apply most methods to my work now because they are so cumbersome that they can drag-down otherwise exciting sketching/iteration (although it is even worse for academics who have to worry about IRB). I think that for researchers "finding the right design" is more-or-less the name of the game. And frankly, the best place to "get the design right" is probably not academia, but a startup where the pressure is much higher to succeed (and the need to publish much lower) and there can be just as much of a user-centered focus as in mainstream HCI. So... I think I have argued myself out of the utility of my thesis (but that was inevitable ;).
