MLCROWD 2013 - Workshop: Machine Learning Meets Crowdsourcing

ICML '13 Workshop: Machine Learning Meets Crowdsourcing
Date: June 21, 2013
Location: Atlanta, USA
http://www.ics.uci.edu/~qliu1/MLcrowd_ICML_worksho...
Deadline: 15th Apr, 2013
Acceptance notification: 15th May, 2013.
Workshop date: 21st June, 2013.
Our ability to solve challenging scientific and engineering problems relies on a mix of human and machine intelligence. The machine learning (ML) research in the past two decades has created a set of powerful theoretical and empirical tools for exploiting machine intelligence. On the other side, the recent rise of human computation and crowdsourcing approaches enables us to systematically harvest and organize human intelligence, for solving problems that are easy for human but difficult for computers. The past few years have witnessed widespread use of the crowdsourcing paradigm, including task-solving platforms like Amazon Mechanical Turk and CrowdFlower, crowd-powered scientific projects like GalaxyZoo and Foldit game, as well as various successful crowdsourcing business such as crowdfunding and open Innovation, to name a few.
This trend yields both new opportunities and challenges for the machine learning community. On one side, crowdsourcing systems provide machine learning researchers with the ability to gather large amount of valuable data and information, leading advances in challenging problems in areas like computer vision and natural language processing. On the other side, crowdsourcing confronts challenges on increasing its reliability, efficiency and scalability, for which machine learning can provide power computational tools. More importantly, building systems that seamlessly integrate machine learning and crowdsourcing techniques can greatly push the frontier of our ability to solve challenging and large-scale problems.
The goal of this workshop is to bring together experts on fields related to crowdsourcing such as economics, game theory, cognitive science and human-computer interaction with the machine learning community to have a workshop focused on areas where crowdsourcing can contribute to machine learning and vice versa. We are interested in a wide variety of topics, including but not limited to:
---- State of the field. What are the emerging crowdsourcing tasks and new opportunities for machine learning? What are the latest and greatest tasks being tackled by crowdsourcing and human intelligence and how do these tasks highlight the need for new machine learning approaches that aren’t being studied already?
---- Integrating machine and human intelligence. How to build practical systems that seamlessly integrate machine and human intelligence? Machine learning algorithms can help the crowdsourcing component to manage work flows and control workers’ qualities, while the crowds can be used to handle the tasks that are difficult for machines to adaptively boost the performance of machine learning algorithms.
---- Machine learning for crowdsourcing. Many machine learning approaches have been applied to crowdsourcing on problems such as output aggregation, quality control, work flow management and incentive mechanism design. We expect to see more machine learning contribution to crowdsourcing, either by novel ML methods, or on new crowdsourcing problems.
---- Crowdsourcing for machine learning. Machine learning largely relies on big and high quality data, which can be provided by crowdsourcing systems, perhaps in an automatic and adaptive way. Also, most machine learning algorithms have many design choices that require human intelligence, including tuning hyper-parameters, selecting score functions, and designing kernel functions.
How can we systematically "outsource" these typically expert-level design choices to the crowds in order to achieve results that match expert-level human experience?
---- Crowdsourcing complicated tasks. How to design work flows and aggregate answers in crowdsourcing systems that collect structured labels, such as bounding box annotations in computer vision, protein folding structures in biology, or solve complicated tasks such as proof reading, and machine translation? How can machine learning provide help in these cases?
---- Theoretical analysis. There are many open theoretical questions in crowdsourcing that can be addressed by statistics and learning theory. Examples include analyzing label aggregation algorithms such as EM, or budget allocation strategies.
Invited Speakers
~~~~~~~~~~~~~~~~~~
- Jeffrey P. Bigham. University of Rochester
- Yiling Chen. Harvard University
- Panagiotis G. Ipeirotis. NYU Stern School of Business
- Mark Steyvers. UC Irvine
- More ...
Submission Details
~~~~~~~~~~~~~~~~~
Submissions should follow the ICML format (http://icml.cc/2013/wp-content/uploads/2012/12/icm...) and are encouraged to be up to eight pages. Papers submitted for review do not need to be anonymized. There will be no official proceedings, but the accepted papers will be made available on the workshop website. Accepted papers will be either presented as a talk or poster.We welcome submissions both on novel research work as well as extended abstracts on work recently published or under review in another conference or journal (please state the venue of publication in the later case); we particularly encourage submission of visionary position papers on the emerging trends on crowdsourcing and machine learning.
Please submit papers in PDF format at https://cmt.research.microsoft.com/MLCROWD2013/.
Important Dates
~~~~~~~~~~~~~~
Extended abstract submission deadline: 15th Apr, 2013.
Acceptance notification: 15st May, 2013.
Workshop date: 21st June, 2013.
Workshop Organizers:
~~~~~~~~~~~~~~~~~~~~~
Paul Bennett, Dengyong Zhou, John Platt, Microsoft Research Redmond
Qiang Liu, UC Irvine
Xi Chen, Qihang Lin, CMU