/* ---------------------------- HTML STARTS HERE----------------------------------------------------------------------------------------------------- */

Mobile Community Design
Research and design information for mobile community developers.

Tuesday, May 30, 2006

Effective Use Of Participatory Design Methods

This is an excerpt from an unpublished technical report. It may eventually form part of a journal paper, but it is posted here to provoke discussion on the topic.

Participatory design (PD) is a design framework and related methods which advocate user involvement in design, and a political stance advocating worker rights. It originated from Scandinavian software development practices in the 1950s and inherits some of the social democratic intentions of that area.
This essay is not intended to dissuade researchers from using PD, nor to support traditional non-human-centered software development. Rather it seeks to provoke an honest discussion about both the advantages and disadvantages of PD theory and methods. Frameworks such as PD can be inherited without critical review, which hinders positive evolution, application, and adaptation of existing PD and user centered methods.

There are a number of positive aspects to PD. Participatory methods are often used in the natural environment of the user (e.g. a workplace) and thus offer high ecological validity and are heavily user centered. Co-designing with real users in realistic situations and environments helps improve the quality of feedback users provide. Frequent iteration between users and designers reduces misconceptions designers make (in part due to insufficient domain experience). Additionally, the social intent of PD to avoid deskilling of workers and create humane products is admirable.

However, some of my recent research and a review of other PD research has revealed a number of potential pitfalls in the application of participatory design methods. There are a range of studies and methods which many researchers agree are PD, however, what is or is not PD is still the subject of debate. Consequently the pitfalls discussed below do not apply to all PD studies and methods, but would certainly apply to some studies which claim to be PD. The ten pitfalls, and potential solutions are briefly discussed below.

Pitfall 1: Asking participants to design objects themselves

This is different than participants providing feedback or proposed variations on existing designs. Participants are usually not trained designers. Consequently they can produce bad designs or feel uncomfortable doing unfamiliar design activities. A potential solution is to provide participants with simple designs in primitive forms that invite variation and re-appropriation. A related problem is that if users have too much power to control designs, they may advocate poor designs, or designs that avoid automation. Few workers want their jobs replaced, but many automation technologies from traffic lights to ATMs make our lives easier and more effective. Pitfall 2 also relates to another aspect of expecting users to design.

Pitfall 2: Expecting domain experts to be technology experts

New technologies can be extremely complex and their advantages and disadvantages often reveal themselves only after long usage in real situations. In most cases participants won’t be able to accurately tell us what technology offerings they want. However, they can show what they need through other methods. It is possible to have people show us their needs through normal activities; and then present them with fictional devices that address those needs. For example, when developing mobile phones participants are not asked to understand the ramifications of wireless networking standards or future smart phone designs in real usage; however they are still able to demonstrate what phone functionality they would likely use through prototypes.

Pitfall 3: Asking participants to predict theoretical usage

No one can accurately predict their future behavior, particularly in complex mobile and social environments with unfamiliar technologies. Watching actual use of prototypes in-situ and getting feedback on them (preferably in-situ) provides more realistic input from users.

Pitfall 4: Asking participants to start from scratch

Participants work better with some scaffolding to direct design ideas. Tangible objects can be interacted with and used as props for discussion. Consequently it is possible to give people simple prototypes and asking them to discuss their own use of the “future technologies” in an appropriate context. Thus, they can talk around the existing designs instead of intangible possibilities.

Pitfall 5: Expecting participants to want to contribute

Participatory design came from trade union roots where organizations ensured workers understood the impact of technologies and were motivated to help construct new designs. In more contemporary design situations users may not be willing to devote time to help build technologies which other people profit from. A key challenge is determining how to interest or motivate participants to help build a product they may not use, and which may be years from production. More modern methods of incentivizing potential users may need to be adopted.

Pitfall 6: Letting small numbers of users greatly impact design

Representative user feedback is useful, and being aware of non-representative issues affecting design is useful. However, thinking that extreme issues are commonplace when they are not can ruin a design. Using a small sample of participants runs the risk of one user being an outlier with unrepresentative concerns. These participants can provide useful challenges to the design, but they should not drive it. PD approaches that repeatedly use the same group of future users stand a greater chance of this occurring. It is possible to take the approach of using different (small) groups of users in an iterative fashion, to sample a more diverse set of opinions about proposed designs and avoid over-emphasizing outliers. This often requires PD methods to adjust to avoid long-term interactions with the same user group.

Pitfall 7: Focusing on what participants design instead of what they need

Potential users can easily draw an interface on a piece of foam or paper – that does not necessarily make it a workable design or help solve their real problems. Participants can usefully review product ideas if they are presented in realistic situations, and in language the participants can relate to. They can also use technologies in basic forms, which allow for generation of new design ideas based on observed usage. While user-generated solutions should not be ignored, focusing on work practices and observed problems provides a great deal of participatory design input in the form of high-quality requirements.

Pitfall 8: Confusing design education with creating good designs

Iterative processes have a good dialogue between designers and users. This necessarily requires the user to learn a bit about design, and for the designer to learn a bit about the domain being designed for. Sometimes PD attempts to have users do the design work, and design ideas originating from users are presumed to be of high quality – because they originate from users. Some PD projects have even had explicit goals of educating users about design during the design process. While this could be appropriate in some education settings, in many professional software development projects there isn’t a goal of training users in design, nor money to waste on implementing poor design ideas. Professional designers are hired because they have expertise to create good designs, and with the input of users they can efficiently and accurately do so.

Pitfall 9: Attempting to prescribe humane workplaces via designs

Since the beginning of PD, there has been a social goal of creating democratic workplaces where workers are able to design their own future tools and avoid dehumanizing technologies. Modern software development would do well to adopt this awareness of the social impact of the tools they create. However, it should be remembered that the technologies in use are only a small part of the picture of a humane workplace. Any technology can be harnessed into a system designed to subjugate workers. It is the larger system of workplaces policies and law which ultimately decide if workers are respected. Furthermore, many technologies are used in unpredictable ways as the work environment unexpectedly changes. Thus a product designed by workers may not have the social effect that was intended at the time of introduction.

Pitfall 10: Judging what is not participatory design

Traditional PD advocates are quick to note that many new participatory methods are not PD. This is usually because the studies don’t have political or social intent, or don’t engage users in design in an accepted fashion. PD is now having to cope with a variety of new development situations, from mobile devices to rapid non-unionized industry settings. Participation will happen in a variety of new ways, and much of it won’t fit the idealized conceptualizations of traditional PD. PD needs to embrace adaptation of how it is used if it is to be effective and increase in popularity.

The above sections have identified potential weaknesses in PD in the hopes that it will spark conversation about how to improve relevant theory and methods. Many PD methods are already very good, but as they are applied in new environments they will need adaptation. In particular, merging PD concepts into more traditional user centered design (UCD) processes would be beneficial to modern software development and increase the chance of their adoption. However, for this to occur some of the above issues will need to be addressed.

p.s. You are welcome to comment on this in the Tribe discussion group, by clicking the comments link.
- Yoko has responded to this on her blog

Monday, May 29, 2006

Sampling Mobile Opinion Tech Report

We have published a new tech report via ePrints on the topic of gathering requirements from distributed communities.

Sampling Mobile Opinion: A Contextual Postcard Questionnaire Study

Axup, Jeff and Viller, Stephen (2006) Sampling Mobile Opinion: A Contextual Postcard Questionnaire Study , School of Information Technology and Electrical Engineering, The University of Queensland.

Full text available as:


Understanding requirements of mobile communities is challenging because of their geographical distribution and frequent movement. We present a study of backpackers travelling in Australia which utilizes a research method called contextual postcard questionnaires. The method uses brief, open-ended questions to solicit contextual responses from backpackers that are relevant for development of tourism and mobile communication technologies. 800 postcards were distributed via hostels and a travel agent, questioning travellers about their current situation. Questions asked how they had heard about their present location, what kinds of virtual-graffiti they would leave there, and what their greatest worry currently was, among others. Results indicated that backpackers have a great deal of practical and serious concerns to contend with as they travel. They are physically cut off from family and friends and rely on a range of communications media to stay in touch and exchange emotional support. They have a great deal of practical travel experience that would be useful to other travellers, but which is currently only conveyed haphazardly via word-of-mouth. Practical usage of the contextual postcard questionnaires is discussed and design recommendations for mobile group products are offered.