ACM SIGCHI General Interest Announcements (Mailing List)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Max Wilson <[log in to unmask]>
Reply To:
Max Wilson <[log in to unmask]>
Fri, 11 Jan 2013 09:57:06 +0000
text/plain (123 lines)
By popular demand: Extension to the 18th!
(however submission is open, so feel free to submit when ready)

Key Facts
Submission Deadline: now 18th Jan 2013, 11:59pm PST

What to Submit: Experience Reports about Replications*
 - Confirming/challenging research, replicating experiments, or applying research in practice.
 - Whether research papers have been previously published or not*
- Also: Detailed Position Papers

Workshop URL:
Submit URL:

Organisers: Max L. Wilson, Ed Chi, David Coyle and Paul Resnick.

* We believe novel research findings should be published at appropriately sized venues. Authors will retain copyright such that they can later publish papers on studies described in experience reports at RepliCHI.

About the Workshop
RepliCHI 2013 is two-day workshop focusing on the meta-issues, challenges, and benefits of replicating HCI research. The workshop builds on the successful RepliCHI SIG (CHI2012) and Panel (CHI2011). It will provide the next step in an ongoing conversation, aimed at encouraging a culture shift towards embracing and rewarding replication and reproducibility of research in the HCI community.

The RepliCHI venue aims to provide:
a) A venue in which to present experiences of research that focused on replicated (and extended) prior HCI findings.
b) A discussion venue, where we debate and consider when and where replication is important, and when it is less important.
c) A learning venue that helps us to understand how to facilitate replication in HCI research.
d) An opportunity to begin building a library of replication attempts of prior HCI work.
e) A venue to continue ongoing conversations about replication and RepliCHI

The workshop website ( ) provides further details of the workshop aims and schedule. It also has additional details on the benefits of replication, different forms of replication, and the different motivations that might drive researchers, practitioners and also teachers to consider replicating prior research.

We are inviting papers that describe experiences surrounding attempts to replicate HCI research. In general we believe that novel research findings should be published at venues equivalent to the magnitude of the results and their contribution. Experience Reports may be about a) previously published, b) unpublished, c) teaching, d) industry replications. Authors will retain copyright such that they can publish unpublished studies described in RepliCHI experience reports at a later time. The aim of these reports, therefore, is to help document the nature of replicating HCI systems and research, and provide a starting point for further discussions.

Consequently, we primarily invite experience reports (in 3-5 pages of ACM Extended Abstract format) that elaborate on:
 1. The motivations for your replication work.
 2. The challenges you faced.
 3. Your successes and the limitations of your replication.
 4. Whether the results confirmed or challenged the findings of prior studies.
5. The potential causes of differences with prior studies. For example, might variations in experimental protocols have accounted for differences?

We will also accept considered position statements about replication of HCI research that contribute to our community's understanding of the topic.

To Submit:
Deadline: 11th Jan 2013 11:59pm PST.
Queries: [log in to unmask]<mailto:[log in to unmask]><mailto:[log in to unmask]><mailto:[log in to unmask]>

At the workshop - Tandem Presentations
Day 1 will focus on selected case studies from accepted papers. Where possible, the original authors of work that has been replicated will also be invited to participate in the workshop. In building the RepliCHI community, we aim to accept as many papers as possible and additional papers, not presented as case studies, will be presented as posters. Day 1 aims to trial our expected format of future events.

Day 2 will be more reflective and in an open discussion format. The aim, therefore, is to reflect on our trial of the format of future events (day 1) by synthesising lessons learned, discussing issues around replication, and planning future RepliCHI events.

Max L. Wilson - University of Nottingham, UK
Ed H. Chi - Google, USA
David Coyle - University of Bristol, UK
Paul Resnick - University of Michigan, USA

More on Replication
In many domains, the validation of findings - through independent replication other researchers’ work - is a key stage in demonstrating the strength of new findings. In HCI, however, replicating prior work is not “the norm”. In part this is due to the need keep up with rapidly changing technology. It is also driven by strong pressure and incentives to always break novel ground. This trend can be particularly strong is competitive venues like CHI.

Direct Replication - consists of attempting to entirely replicate a study or system, using the same format and with the same tools, and experimental protocol. The aim of direct replications is often to replicate a specific finding. Direct Replication is often driven by the aspirations of strong science to confirm that results are true, are not created by an unseen bias, or that they apply in different contexts (geographic, cultural, topic, task) to the original study [2]. This method is often used as a teaching method for postgraduate students.

Conceptual Replications - are systems and studies that focus on a certain principle or phenomenon and confirm findings using alternative methods. Of the three approaches, this is most common in HCI, in that multiple studies demonstrate the principles of direct manipulation. Many instances, however, are post-hoc reflections of their findings in the context of prior work. Through this approach we surmise heuristics about best practices for design or for evaluation.

Replicate+Extend - is a common research method in which people first reach the level of prior research before investigating it further. This may involve reproducing a phenomenon before specifically investigating it further, or by building on the findings of the study. This form of research is often essential in understanding a form of interaction, after learning about the limitations of an initial approach, for example. However, Replicate+Extend is associated with the high risk of being described as ‘incremental’, and being rejected from prestigious peer-reviewed venues.

Applied in Practice - One common form of replication is application – a special instance of conceptual replication. If HCI research produces a finding, and its application in real world contexts confirms it, then case studies are a form of replication. It is perhaps often one of your core desires that research findings are applied in real world contexts. Case studies of applying research findings, by practitioners in real world settings, are highly desirable, and this is an opportunity to engage with practitioners.

Dr Max L. Wilson
Lecturer in HCI and Information Seeking
Mixed Reality Lab
School of Computer Science
University of Nottingham, UK
[log in to unmask]<mailto:[log in to unmask]>
+44 (0) 115 84 66551

This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it.   Please do not use, copy or disclose the information contained in this message or in any attachment.  Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an attachment

may still contain software viruses which could damage your computer system:

you are advised to perform your own checks. Email communications with the

University of Nottingham may be monitored as permitted by UK legislation.

    For news of CHI books, courses & software, join CHI-RESOURCES
     mailto: [log in to unmask]

    To unsubscribe from CHI-ANNOUNCEMENTS send an email to
     mailto:[log in to unmask]

    For further details of CHI lists see