ACM SIGCHI General Interest Announcements (Mailing List)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Condense Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
text/plain; charset="us-ascii"
Tue, 22 Apr 2014 15:46:45 +0100
"Halvey, Martin" <[log in to unmask]>
"ACM SIGCHI General Interest Announcements (Mailing List)" <[log in to unmask]>
"Halvey, Martin" <[log in to unmask]>
text/plain (78 lines)
** Apologies for cross postings. Please distribute as appropriate **

Workshop on Gathering Efficient Assessments of Relevance (GEAR): Call for papers At SIGIR 2014, July 11th 2014, Gold Coast, Australia

Workshop website:


Evaluation is a fundamental part of Information Retrieval, and in the conventional Cranfield evaluation paradigm, sets of relevance assessments are a fundamental part of test collections. In this workshop, we wish to revisit how relevance assessments can be efficiently created. A discussion and exploration of this issue will be facilitated through the presentation of results based and position papers on the topic. Participants will also be invited to participate in a design task focusing on developing a benchmarking exercise. After all of these activities the workshop will conclude with an open discussion session, part of which will focus on future directions. As well as providing a forum for discussion and exploration, it is hoped that this workshop will lead to future activities and collaborations in this area.

Workshop areas of interest include:
 - How the method of generating assessments, via conventional means or crowdsourcing, affects the judgments gathered, such as issues of assessor expertise, payment, etc.
- The process by which individuals, or groups of individuals, assess documents (text, image, video etc.) for relevance
 - Issues relating to the effort required to generate relevance assessments for different types of topic, and different types of material (text, web, image, video, etc. and multiple languages)
- To revisit the concept of "relevance", from a practical, operational standpoint, for the purposes of IR evaluation.

An aim of the workshop will be to provide a forum where short research papers can be presented, reporting work which may not conventionally be published in papers at formal venues, including "practice and experience" papers concerning relevance assessment gathering, and position papers concerning the concepts and issues. Papers reporting negative results are also welcome.

Important dates
Submission deadline: April 28th, 2014
Notification date: May 9th, 2014
Camera-ready due: May 16th, 2014
Workshop date: July 11th, 2014

Note: all deadlines are on the date specified at midnight in the UTC-12 time zone.

Paper Submissions
The workshop welcomes short papers (4 pages) and position
papers (2 pages)

Two types of paper are solicited:
- Short papers: 4 pages max, which present research work related to the workshop's areas of interest
- Position papers: 2 pages max, which present provocative ideas and challenges related to workshop's areas of interest.

Submissions should be in ACM SIG proceedings format (, and should be submitted through the EasyChair system at

Further information is available on the workshop website:

Workshop Organisers
- Martin Halvey, School of Engineering and Built Environment, Glasgow Caledonian University, UK
- Robert Villa, Information School, University of Sheffield, UK
- Paul Clough, Information School, University of Sheffield, UK

Dr Martin Halvey
Lecturer in HCI
Department of Computer, Communication & Interactive Systems
School of Engineering and Built Environment
Glasgow Caledonian University, UK
[log in to unmask]<mailto:[log in to unmask]>
+44 (0) 141 273 1807

Glasgow Caledonian University is a registered Scottish charity, number SC021474

    For news of CHI books, courses & software, join CHI-RESOURCES
     mailto: [log in to unmask]

    To unsubscribe from CHI-ANNOUNCEMENTS send an email to
     mailto:[log in to unmask]

    For further details of CHI lists see