Trevor Owens

In Please Write it Down: Design and Research in the Digital Humanities I suggested that there are some valuable ways of thinking about the connections between building/designing and creating knowledge and scholarship.  In particular, I suggested that those interested in learning through building in the digital humanities might find some value in work in educational research over the last decade which has tried to define what exactly what a design based research methodology might look like.

This is the first post, in what I imagine might be an ongoing line of thought here, to try to put ideas from design based research in conversation with the digital humanities. As a point of entry, I am going to walk through one emerging genre of writing in design based research, the design narrative. Before getting there, however, I would briefly pause to note that the journal this piece appeared in, Educational Technology Research and Development, is itself an interesting note to the digital humanities. I for one, would love to see a journal in the digital humanities similarly situated as a place for sharing and disseminating R&D knowledge.

The Case of Environmental Detectives

In Environmental Detectives: The Development of an Augmented Reality Platform for Environmental Simulations Eric Klopfer and Kurt Squire offer a summative and reflective report on their work developing the augmented reality game Environmental Detectives. The paper makes some valuable suggestions for how we might better design augmented reality games, but I think its primary strength is as an example of a particularly novel and useful genre of design based research report. 

Brenda Bannan-Ritland’s article, The role of design in research: The integrative learning design framework offers a robust framework for thinking through how the design process and the research process can fit together. See her diagram below  (don’t get lost in the details). The intellectual work that diagram and her approach offers os to illustrate what happens if you mush together the steps in an array of design processes and research approaches. The diagram illustrates how the features of product development, research design, and user centered design can leaf together. 

If you look a the top part of the diagram carefully you will notice that practically every step in this process has an arrow that points over to the publish results box. This is a key concept here, the idea behind design based research is not that the design process is itself a research method, but that throughout the design process there are a series of publishable results and lessons learned that emerge which warrant being refined, shared and communicated. Squire and Klopfer’s article is a great example of the kind of piece one would want to write as a summative result of an extended design research process.

Design Narrative as a Genre of Design Based Research Article

Design based research can generate publishable results in any particular research tradition. You can find interviews, ethnographic approaches, micro ethnographic approaches,  case studies, randomized clinical trials, and methods from usability studies like eye tracking used at different points in the design and development process. In short, there are any number of ways to use existing research methods approaches to reflect on and report out results of research in the process of informing design. Part of what is particularly interesting about Klopfer and Squire’s paper is that it represents a somewhat novel mode of research writing, the design narrative.

Drawing from Hoadley’s 2002 piece, Creating context: Design-based research in creating and understanding CSCL, Klopfer and Squire offer a reflective narrative account of their work designing, developing and researching the Environmental Detectives game. Unlike other papers they published, which might report parts of this research in terms of a case study, or the pre-post test scores or the results of a particular evaluative test of the game’s outcomes, this summitive piece serves to reflect on the design process and offer an account of the context and lessons learned in the course of the design process. It is worth reporting on actual structure of the piece.

Review of literature that informed the design: After explaining background on the idea of design narrative Klopfer and Squire offer an account of both the extent literature on augmented reality games and a review of the existing games projects that they looked to which informed their design. This serves to provide the conceptual context that they began from, it sets the reader up to understand exactly where the project started from while also providing information on what theory and knowledge at the time of the projects start looked like.

Retrospective and Reflective Design Narrative: The bulk of the paper then reports out on each phase of their design process. In their particular case they describe six phases of their research, brainstorming, designing the first instantiation, developing a first generation prototype, classroom field trials, classroom implementations, expanding to new contexts, and a sixth phase in which they added customized dynamic events to the game. It is not necessary to go into the details of each section for this review. What matters is to stress that each section begins by explaining how they went about their work in the given phase and reports a bit on what they learned in that phase. What is essential in this approach is that each section explains what worked and didn’t work in any given phase and how exactly Klopfer decided to remedy their approach and design to respond to problems.

As is generally the case with qualitative research, the moments when things don’t go according to plan and exactly how we make sense and work through those moments are generally the most valuable parts of the process. The value in this kind of retrospective account is two-fold. It provides a context for understanding why the game they made does what it does, but more importantly, the design narrative’s primary value is as a guide to other designers on what parts of the design process were particularly valuable. This kind of narrative helps us to refine our ideas not only about this particular design situation, but more broadly about how we can refine our own design practices.

Conclusions and Implications from Reflection: After reporting the design narrative the paper presents a set of technological and pedagogical implications. In much the way that the discussion section and conclusion sections of research reports function, this section attempts to suss out and distill the lessons learned from the work. In their case, they present a range of specific implications for the design of augmented reality games that emerged from their design approach.

The Value of Design Narratives

If you read through their references, you can see that they have published about this work on a few previous occasions. It is not that they are double dipping on publications, instead those other publications report results from subsets of this project, some of the earlier findings, or any of the points in the design process that resulted in interesting findings. This paper is really a summitive report, retracing the design narrative of the entire project.
I see the value of this particular design narrative approach as having two primary values, two values that I think are particularly useful to the still emerging world of the digital humanities. Composing these narratives serves an internal value to designers as part of reflective practice. Sharing these narratives makes the kinds essential tacit knowledge that comes about as part of doing design accessible to others.

Reflective Practice is Best Practice: If you can hold yourself to some sound practices for documenting the stages in your design process (the ideas that you had, how you went about implementing and revising them, and the results), you are in a good position to use that documentation to reflect on your practice. In this sense, the design narrative, the retrospective account of what you did, why you did it, what you learned  is an essential piece of doing reflective design practice. When you go back and think through your own process you are not simply reporting on what you learned you are actually making sense out of your trajectory and coming to understand what it is that you actually learned. Like much of qualitative and hermeneutic research, the process of writing is not a process of transmission of knowledge but of the discovery of knowledge. Writing a design narrative is the process by which we come to know and learn from our work.

Making Tacit Practical Design Knowledge Explicit and Available: It is essential that the knowledge developed in the design process is documented and shared. While the individual studies that come out of a design research process provide evidence of the value, or of particular lessons learned in part of a design project, they leave a considerable amount of the bigger picture knowledge off the table. Quite frankly, much of the most essential parts of design are not about explaining that something works, if someone wants to get into design they need access to the deeply pragmatic, heuristic driven, knowledge that develops on over time in the process of design. The design narrative is an essential medium for capturing and disseminating this kind of tacit knowledge.

In short, I would suggest that this particular piece of scholarship serves as a great example of the value of reporting design narratives and an exemplar for others to use as a model for composing their own design narratives.