As presented at SIGCHI 2005 Conference [DOI Bookmark]
Preston Smalley and Jeff
Herman
eBay User Experience & Design Group
2145 Hamilton Ave, San Jose, CA, 95125
Abstract
Increasingly, organizations are looking for ways to use technology to document and share information in order to increase effectiveness and foster a sense of community [1, 3]. As one example of a “community of practice” [4, 10], the eBay User Experience & Design group recently created a knowledge management system known as the Playbook to enable the eBay design community to share design best practices and other information.
This paper describes the considerations that went into creating the Playbook and how the working team enabled the rest of the eBay design community throughout the world to contribute their best practices. We also discuss how the group as a whole has benefited from this system as well as lessons learned during this process which may help other design communities outside of eBay.
eBay is an online, worldwide marketplace that enables people to buy and sell goods and services. eBay’s User Experience & Design (UED) group is responsible for researching and designing the user experience for the various eBay sites around the world. Over the last several years, the UED group has experienced dramatic growth and now has over 125 people in ten offices around the world (including Asia, Australia, Europe, and North America) working on the user experience for sites in over 30 countries.
To enable and support this growth rate, over the years the group has evaluated and sought to improve its processes and communication. For example, in early 2004 we heard from UED members in offices outside the US who wanted greater visibly into existing best practices and who wanted to share their best practices with group members in other countries. In general, we found that best practices, including project processes, were not being shared effectively within design offices or between offices.
The process for sharing best practices was inefficient for the person looking for information as well as the person who had the information. Since the best practices were often hard to find, we heard several cases where a designer would document best practices, assuming it had never been done before, only to discover later that in fact someone else had done so previously.
Also, as the group grew it was increasingly likely that multi-disciplinary product design teams would be made up of several people who were new to eBay and/or who did not have experience working together. As a result, teams needed clearer definitions of each role at each stage of the process.
To address these issues, we started the process of creating a knowledge management system [1] which we called the Playbook. Our goals were to:
UED is a cross-functional group that includes five disciplines: Content Management, Creative Design, Prototyping, User Experience Research, and User Interface Design. We identified one person in each discipline to be on the working team. This had the benefit that each discipline’s needs would be addressed. In addition, it ensured that knowledge, expertise, and support for the project would exist across the organization
The team’s objectives were to create the Playbook within three months so that the rest of UED could start benefiting from it and to prove the benefits of knowledge management for the eBay design community.
The team started by applying a user-centered design process and considering user needs for such a system, including understanding each user segment, creating use cases, determining what content people needed, how they wanted to access it, how they would look for it, and how they would add, edit, and delete content. We also interviewed colleagues at eBay who had experience using knowledge management systems at previous companies to gain insight into what worked and what didn’t in these instances.
As a result of our research, we created the following requirements for the Playbook. It had to:
There was one additional requirement we placed upon the initial system: as a practical matter it had to be cost-effective. Since we considered this initial system to be a test, the initial budget we requested was relatively small. We saw the initial version as the way to measure the benefits of the Playbook and to warrant future investment in what we thought would be an expanding enterprise [11].
Next, we investigated software solutions that would enable us to meet these requirements. UED’s Prototyping group, made up of several software engineers, provided the necessary expertise to consider a range of technical solutions, including whether to build the software ourselves, buy it from a third-party, or whether to adapt an existing open-source content management solution.
After considering a number of technical solutions, we found that Plone [12], an open source solution, met our user needs and requirements. It came with a user interface that enabled people to easily search for, add, edit, and delete information. The team also created modules to search for information in a variety of common file formats (e.g. PowerPoint and Word). In addition, the team made the system more dynamic by adding a module to automatically highlight information that had been added since the user’s most recent visit.
The working team also created the high-level information architecture for the Playbook. A recently created binder for new group members was a helpful guide as to what members needed to know. In addition, we interviewed people who had recently joined the group to determine their most common questions, what information they found useful, and what information they had difficulty finding. We also interviewed more experienced members of the group to determine what questions they were most frequently asked and what information they thought would be most useful to others.
As a result of these interviews, we identified the major content sections, including:
It was important to launch the Playbook with basic content so that, at launch, it would already be useful to group members. The working team identified content experts in each area and enabled them to add their content to the Playbook before we launched it to the wider group. Much of the information, such as the design principles, already existed so it was relatively easy to add this to the Playbook so everyone in each design office would have easy access to it.
The team launched the Playbook on schedule to the entire UED group. In order to ensure that the Playbook grew beyond the initial set of content, we raised awareness of the Playbook by giving presentations, sending email announcements to group, and announcing week-to-week growth rates of the number of users and contributions. In addition, the Creative Design group within UED created a series of posters that advertised the benefits of using the Playbook.
Six weeks after launch, the team (led by members of the User Experience Research group within UED) conducted an online survey to gain a better understanding of how the Playbook was serving needs and where further improvements were needed. The results of this survey informed our decisions on what to improve and we plan to repeat the survey at regular intervals.
In addition, the team leveraged a global UED summit, involving members from around the world, to officially launch the Playbook to all eBay design offices. During the Playbook session, members were provided an opportunity to:
To date, we have focused on understanding and addressing user needs, launching the initial version, adding content, and raising awareness of the Playbook to ensure it is used. As the Playbook is gaining acceptance and maturing, our next challenge includes finding a way to recognize not just quantity of contributions but also quality [13].
Based on the number of users, return visitors, content added, survey results, and anecdotal feedback, the launch of the Playbook has achieved the requirements we specified at the beginning of the project. We believe this success is due to several factors, including:
We recognize that the Playbook will continue to be successful only if we ensure that the group works together to keep the information up-to-date and that those who need information are aware of the Playbook and find it easy and effective to use. Finally, we believe the lessons learned and summarized above are not unique to the success of the Playbook at eBay. We believe that other communities who follow a process with similar attributes will increase the chances that their system will be adopted by everyone involved and leveraged to serve the entire community.
The initial UED Playbook working team was composed of the authors as well as Ketan Barbaria, Elaine Fung, John Giang, Frank Gruger, Scott Joaquim, Yathish Sarathy, Steve Tse, and Craig Villamor. The following people also provided significant contributions and support: Kate Aronowitz, Kelly Braun, Larry Cornett, Nancy Dickenson, Andre Haddad, Beth Leber, Ishantha Lokuge, Justin Miller, Suja Raju, Lynn Reedy, John Sanborn, Luke Wroblewski, and everyone in eBay’s User Experience & Design group.
