Methodologies for Evaluation of Collaborative Systems
To be held Nov. 14 in Seattle - just prior to the start of CSCW conference.
Jean Scholtz, Laurie Damianos, Andrew Greenberg, and Robyn Kozierok
This workshop will discuss different approaches used to evaluate CSCW
systems. There are several purposes for evaluation of CSCW systems.
Researchers may be interested in measuring the effectiveness of new
techniques or functionality. Purchasers may be interested in comparing
systems to determine the one that best suits the needs of a particular
group. Our goal is to produce a taxonomy of evaluation methodologies
for CSCW systems, identifying the type of systems for which this
technique is most useful, the stage of development in which this
methodology is appropriate, the resources needed to conduct the
evaluation, and the appropriate measures for the various techniques. We
plan to discuss various methods of data collection for collaborative
work and identify the evaluation methodologies for which various types
of data collection are most appropriate. Other issues we hope to
discuss during the workshop include sharing and comparing collected
data, the usefulness of standardized component tests, and the
organization of evaluation results to make them more accessible to the
Those wishing to attend should submit a position paper outlining:
1. Evaluation experiences including
- a brief description of the CSCW system
- the purpose of the evaluation
- the evaluation methodology used
- data collection techniques/ tools
- advantages/ disadvantages of using this methodology
2. Any issues to add to the discussion list
Summaries of the evaluation methods and a compiled list of issues will
be made available to workshop attendees prior to the workshop.
Participants are expected to review this material prior to the workshop.
Position papers are due no later than Sept. 1. Notification of
acceptance into the workshop will be no later than Oct. 1. Position
papers should be submitted to either via hard copy or e-mail to:
Bldg. 225, A216
Gaithersburg, MD 20899
Fax: 1(301) 975-5287
e-mail: [log in to unmask]
For more information, contact Jean Scholtz. Workshop information will
also be available at http://zing.ncsl.nist.gov/~cscw.