*****************************************************
Call for Papers
Workshop on Empirical Evaluations of Adaptive Systems
in conjunction with UM2001
July 13, 2001
Sonthofen, Germany
*****************************************************
Empirical evaluations of adaptive systems are
hard to find, e.g., only a quarter of the
articles published in UMUAI are reporting
significant empirical evaluations. Several
reasons have been identified to be responsible
for this lack. These include both structural as
well as adaptivity specific reasons. However,
user modeling systems are designed for
human-computer interaction and thus, an
empirical approach is absolutely necessary if we
want to justify the enormous efforts which are
needed to make systems adaptive. The aim of this
full-day workshop is to systematize, coordinate
and encourage research on evaluation issues by
bringing together people who are interested in
comparing research in different domains,
identifying common problems, and discussing
possible solutions.
Workshop Format
Each session focuses on a few exemplary
evaluations in a specific domain: participants
are required to bring along a description of a
finished or planned empirical evaluation. Each
evaluation will be introduced by a talk of 5-10
minutes (depending on the total amount of
participants). Evaluations that have been
completed will be discussed in terms of what
worked well, what went wrong, and how it might
be done better next time, etc. Discussions on
planned evaluations will identify strengths,
weaknesses, pitfalls, and possible methods of
analysis based on the experience of the group
members with similar cases. This workshop format
will help encourage discussion, since the
participants will be discussing specific
empirical evaluations that they have done or
plan to do rather than evaluations in the
abstract.
The topics of interest include, but are not
limited to, the following list:
- evaluation frameworks for user-modeling systems
- introduction, comparison or, discussion of
evaluation criteria
- influence of adaptations on behavioral or
cognitive variables
- evaluation of the accuracy or correctness of
either user models or inference techniques
- effectiveness of user adaptations for task
accuracy, task completion time, task quality
- evaluation methods and techniques (experimental
design vs. field studies; transfer of usability
techniques; questionnaire design; behavior
observation; eye-tracking; etc.)
- qualitative methods applied to user modeling
- design-guidelines or heuristics inferred from
empirical evaluations
- solutions of problems that are specific for the
evaluation of adaptivity
- investigation of evaluation strategies
Submission and Information
Due to the workshop format there are several
submission requirements. Please, refer to the
workshop web page for detailed information and
submission instructions:
http://art2.ph-freiburg.de/um2001/index.html
Important Dates
March 1, 2001: submission of abstract (recommended)
March 8, 2001: submission deadline for workshop papers
April 1, 2001: notification of authors
July 13, 2001: workshop
Workshop Organizers
Stephan Weibelzahl
PH Freiburg, Germany
[log in to unmask]
David Chin
University of Hawaii, USA
[log in to unmask]
Gerhard Weber
PH Freiburg, Germany
[log in to unmask]
________________________________
Stephan Weibelzahl
Institute of Psychology
Pedagogical University Freiburg
[log in to unmask]
++49 761 682-523
________________________________
|