CrowdScale 2013: Crowdsourcing at Scale
A workshop at HCOMP 2013: The 1st Conference on Human Computation & 
Crowdsourcing
November 9, 2013
https://sites.google.com/site/crowdscale2013/


Overview
--------
Crowdsourcing and human computation at scale raises a variety of open 
challenges vs. crowdsourcing with smaller workloads and labor pools. We 
believe focusing on such issues of scale will be key to taking 
crowdsourcing to the next level – from its uptake by early adopters 
today, to its future as how the world’s work gets done. To advance 
crowdsourcing at scale, CrowdScale will pursue two thrusts:

*Track 1: Position Papers*. We invite submission of 2-page position 
papers which identify and motivate focused, key problems or approaches 
for crowdsourcing at scale.

*Track 2: Shared Task Challenge*. We invite submissions to a shared task 
challenge on computing consensus from crowds: how to generate the best 
possible answer for each question, based on the judgments of five or 
more raters per question.  Participants will submit 4-page papers 
describing their systems and preliminary results, with *$1500* in prize 
money awarded to top performers.

One may participate in either or both tracks. Submitted papers will not 
be peer-reviewed or archived, so work shared in these papers can be 
later submitted to peer-reviewed venues. All papers will be posted on 
the workshop website to promote discussion within and beyond workshop 
participants. Workshop organizers will review all submissions to ensure 
quality, with high acceptance expected.


Position Papers
https://sites.google.com/site/crowdscale2013/cfp
---------------
We invite submission of short (2-page) position papers which identify 
and motivate key problems or potential approaches for crowdsourcing at 
scale.  We encourage submissions identifying and clearly articulating 
problems, even if there aren’t satisfactory solutions proposed. 
Submissions focusing on problems should a clearly describe a problem of 
scale, why it matters, why it is hard, existing approaches, and desired 
properties of effective solutions.  We welcome early work, and 
particularly encourage submission of visionary position papers that are 
forward looking.

Each submitted paper should focus on one problem. We encourage multiple 
submissions per author for articulating distinct problem statements or 
methods.

During the workshop, authors will self-organize into break-out groups, 
with each group further elaborating upon a particular critical area 
meriting further work and study. Each group will summarize and report 
its findings at the workshop’s close. In continuing discussion beyond 
the workshop, organizers and participants will co-author a summary paper 
articulating a road map of important challenges and approaches for our 
community to pursue.

Position paper ideas include (but are not limited to): 
https://sites.google.com/site/crowdscale2013/position-paper-ideas


Shared Task Challenge
https://sites.google.com/site/crowdscale2013/shared-task
---------------------
To help advance research on crowdsourcing at scale, CrowdFlower and 
Google are sharing two new, large challenge datasets for multi-class 
classification. Both datasets are available for immediate download. To 
make it easy to participate, we have provided multiple formats of the 
data, and pointers to open source software online that is available to 
get started.

All participants are expected to submit a paper (up to 4 pages) 
describing one’s method and preliminary results on shared task metrics, 
and to present a poster at the workshop. Final results will be announced 
at the workshop, with prize money awarded to the best performer(s), as 
well as recognition during the workshop and in our workshop report.

Shared task participants are also invited to participate in workshop 
discussion throughout the day.

*Important Dates*
-----------------
October  1: Position papers due
October 20: Shared task runs due
October 27: Shared task papers due
November 9: Workshop

Please see workshop website for additional information on schedule.


*Questions*: Email the organizers at: [log in to unmask]


Workshop Organizers
-------------------
Tatiana Josephy, CrowdFlower
Matthew Lease, University of Texas at Austin
Praveen Paritosh, Google


Advisory Committee
-------------------
Omar Alonso, Microsoft
Ed Chi, Google
Lydia Chilton, University of Washington
Matt Cooper, oDesk
Peng Dai, Google
Benjamin Goldenberg, Yelp
David Huynh, Google
Panos Ipeirotis, Google/NYU
Chris Lintott, Zooniverse/GalaxyZoo
Greg Little, oDesk
Stuart Lynn, Zooniverse/GalaxyZoo
Stefano Mazzocchi, Google
Rajesh Patel, Microsoft
Mike Shwe, Google
Rion Snow, Twitter
Maria Stone, Microsoft
Alexander Sorokin, CrowdFlower
Jamie Taylor, Google
Tamsyn Waterhouse, Google
Patrick Philips, LinkedIn
Sanga Reddy Peerreddy, SetuServ

-- 
Matt Lease
Assistant Professor
School of Information
University of Texas at Austin
Voice: (512) 471-9350 · Fax: (512) 471-3971 · Office: UTA 5.442
http://www.ischool.utexas.edu/~ml

    ---------------------------------------------------------------
    For news of CHI books, courses & software, join CHI-RESOURCES
     mailto: [log in to unmask]

    To unsubscribe from CHI-ANNOUNCEMENTS send an email to
     mailto:[log in to unmask]

    For further details of CHI lists see http://listserv.acm.org
    ---------------------------------------------------------------