cir 2011 - 2nd SIGIR Workshop on Crowdsourcing for Information Retrieval

Crowdsourcing for Information Retrieval: A SIGIR 2011 Workshop
https://sites.google.com/site/cir2011ws/

**Deadline Extension to Thursday June 9th**.

CALL FOR PAPERS

The advent of crowdsourcing is driving a disruptive shift in IR areas
such as evaluation, learning to rank, and development of new hybrid
man+machine systems which blend automation and crowd computation
(potentially in real-time) to deliver innovative functionality and
search experiences. Traditionally manual-labor intensive tasks like
judging relevance and annotating training data can now be accomplished
more quickly and accurately, and at a fraction of traditional
costs.Despite this potential and early successes of crowdsourcing on
IR, many significant challenges remain with regard to theory,
methodology, policy, and best practices that currently limit our
ability to realize this potential in practice.

We invite submissions of papers describing novel, unpublished research
addressing one or more of the following areas:

* General: Theoretical, experimental, and/or methodological

developments advancing state-of-the-art knowledge of crowdsourcing for

IR

* Applications: search blending automation with the crowd, especially

real-time systems which must model dynamic and temporal properties of

crowd behavior

* New functionality: use of crowdsourcing to realize innovative

search features (e.g. using geographic dispersion of the crowd for

local search or to detect geo-specific intents, etc.)

* Machine learning: consensus labeling for vote aggregation, active

learning strategies for efficient labeling, learning to rank with

noisy crowd labels and multi-labeling

* Evaluation: evaluating systems with noisy and multi-labeled

relevance judgments

* Infrastructure: new software packages and tool kits which simplify

or otherwise improve general support for crowdsourcing or particular

tasks (e.g. TurkIt, Get Another Label)

* Human factors and task design: how to design effective interfaces

and interaction mechanisms for the crowd; how to enable effective

crowd performance on tasks traditionally requiring scare and expensive

domain experts; how different forms of crowdsourcing or crowd

motivations (fun, socialization, prestige, economic, etc.), might be

selected or tailored for different IR tasks (e.g. Page Hunt)

* Vision: Reflective or forward-looking position papers on use of

crowdsourcing for IR

IMPORTANT DATES

Workshop Papers

Submissions: June 9, 2011

Notification of acceptance: July 5, 2011

Camera-ready: July 12, 2011

Workshop: July 28, 2011

WORKSHOP ORGANIZERS

Vaughn Hester, CrowdFlower, USA
Matthew Lease, University of Texas at Austin, USA
Alex Sorokin, CrowdFlower, USA
Emine Yilmaz, Microsoft, UK

You can email the organizers at cir2011-org *at* googlegroups *dot* com.

PROGRAM COMMITTEE

Omar Alonso Microsoft Bing
Paul Bennett Microsoft Research
Adam Bradley Amazon.com
Ben Carterette University of Delaware
Charlie Clarke University of Waterloo
Harry Halpin University of Edinburgh
Jaap Kamps University of Amsterdam
Martha Larson Delft University of Technology
Gabriella Kazai Microsoft Research
Mounia Lalmas University of Glasgow
Edith Law Carnegie Mellon University
Don Metzler University of Southern California
Stefano Mizzaro University of Udine
Stefanie Nowak Fraunhofer IDMT
Iadh Ounis University of Glasgow
Mark Sanderson RMIT University
Mark Smucker University of Waterloo
Ian Soboroff National Institute of Standards
Siddharth Suri Yahoo! Research