2011 - workshops : Combining Strategies for Reducing the Cost of Learning

Over the years, many paradigms have been developed that extend the classic supervised learning setting, often to reduce the cost of acquiring labels or to make use of auxiliary data. These approaches generally have the flavor of “supervised + X learning.” For example,

Active Learning = supervised + oracle queries
Semi-supervised Learning = supervised + unlabeled data
Multitask and Transfer Learning = supervised + data from a related task
Domain Adaptation = supervised + data from a source domain
Multiview Learning = supervised + alternative data views
Learning in a Bandit Setting = supervised + limited feedback
Multiple-Instance Learning = supervised + relaxed label granularity
Learning with Expert Knowledge = supervised + side information
Learning with Weakly Labeled Data = supervised + free (possibly inaccurate) labels
Human Computation and Crowd-sourcing = supervised + free/inexpensive annotators
Learning on a Budget = supervised + misc. cost constraints
There are many naive ways to combine some of these ideas. The goal of our workshop is to go beyond this and understand the assumptions made in these learning settings, and use this understanding to create combined methods that are more than the sum of their parts. At the same time, we also encourage submissions that propose novel methods under one of the above mentioned paradigms, and also explore connections to the other paradigms, which may potentially lead to ways of combining them. This workshop is a success if it leads to non-trivial cross pollination between researchers and ideas in these different fields.

3. Motivation (why a workshop on this topic?)

A variety of extensions to supervised learning have been developed over the years. Many of these approaches are designed to reduce the amount of time, effort, or expense required to obtain quality labeled data for training. For example, active learning aims to minimize labeling costs by allowing the learner to “query” for the most informative data points, while semi-supervised learning aims to also learn from unlabeled data that is often available “for free.” Some other approaches rely on injecting domain knowledge, leveraging external resources, or varying the data (or model) representation.

Although these methods often look different, and may be based on different assumptions, they are unified by a common goal: to improve upon supervised learning for the task at hand. Furthermore, we believe it should be possible to leverage the individual strengths of these approaches in order to arrive at new machine learning paradigms that are even more powerful and adaptable. We envisage that in ten years or so most real-world systems based on machine learning will employ at least three or four of these different extensions, working symbiotically. After all, the learning organisms we observe in nature operate by combining multiple strategies, so why not machines? We see this workshop as an important opportunity to foster thinking and collaboration in this direction.

Some recent work in machine learning has already begun to consider aspects of integration. For example, active learning is traditionally applied to a single learning task, but it has been shown that multi-task learning [7,11], transfer learning [2], and domain adaptation [3] can all benefit from learning with queries. Active learning has also been combined with semi-supervised learning [4], crowd-sourced label acquisition [10], and multiple-instance learning [8] (which in turn can be seen as a kind of semi-supervised learning [9]). Similar opportunities lie in the combination of other extensions from the list above; there is still much to be explored. We also know quite little about how the combined interactions of these approaches can impact learnability from a theoretical perspective, e.g., recent work in multiview active learning has shown promising sample complexity results in the non-realizable case [5]. We anticipate that this workshop will help shed more light on such issues, foster the cross-pollination of ideas among experts in these respective subfields, and encourage researchers to consider the possibilities in developing the next generation of learning systems.

4. Impact and expected outcomes (what will having the workshop do?)

The workshop is likely to foster future collaborations among researchers working on these related topics, with the hope that it would lead to a better understanding of the existing methods, discover unifying themes, and possibly lead to improved algorithms than the current state-of-the-art.

5.Organizers

Hal Daume III (University of Maryland)
Piyush Rai (University of Utah)
Burr Settles (Carnegie Mellon University)
Jerry Zhu (University of Wisconsin)
6. Bio for each organizer (who are you?)

Hal Daume III is an assistant professor of Computer Science at the University of Maryland, College Park. He previously held a position in School of Computing at the University of Utah. His primary research interests are in understanding how to get human knowledge into a machine learning system in the most efficient way possible. In practice, he works primarily in the areas of Bayesian learning (particularly non-parametric methods), structured prediction and domain adaptation (with a focus on problems in language and biology). He associates himself most with conferences like ACL, ICML, NIPS and EMNLP. He earned his PhD at the University of Southern California with a thesis on structured prediction for language (his advisor was Daniel Marcu). He spent the summer of 2003 working with Eric Brill in the machine learning and applied statistics group at Microsoft Research. Prior to that, he studied math (mostly logic) at Carnegie Mellon University.

Piyush Rai is a PhD student (2007-till now) at the School of Computing, University of Utah. His advisor is Hal Daume III. His research interests are into Bayesian methods; in particular, Bayesian nonparametrics (i.e., models that can adapt their complexity with the amount of the data), approximate inference, and learning in scarce labeled data settings (e.g., transfer learning, multiview learning, active learning and semi-supervised learning). Prior to starting grad school, he received his undergraduate degree in Computer Science from the Institute of Technology, BHU, India.

Burr Settles is a Postdoctoral Fellow in the Machine Learning Department at Carnegie Mellon University. His main research project at the moment is NELL ("Never-Ending Language Learner"), a web-scale information extraction system which integrates semi-supervised, multi-view, and multi-task learning to learn to read over the long-term. Dr. Settles's research interests focus on machine learning (particularly active learning) that resembles a "dialogue" between computers and humans, with applications in natural language processing, biology, and social computing. He received his Ph.D. in Computer Sciences from the University of Wisconsin-Madison in 2008 with thesis work on active learning with structured instances, including multiple-instance learning tasks. He also runs the website FAWM.ORG, prefers sandals to shoes, and plays guitar in the Pittsburgh pop band Delicious Pastries.

Xiaojin Zhu is an Assistant Professor at the University of Wisconsin-Madison in the Department of Computer Sciences, with affiliate appointments in Electrical and Computer Engineering and Psychology. His research interests include statistical machine learning (in particular semi-supervised learning) and its applications in cognitive psychology, natural language processing, and computer systems. Dr. Zhu received his Ph.D. from the Language Technologies Institute, School of Computer Science at Carnegie Mellon University in 2005, with his Ph.D. work on semi-supervised learning on graphs. He received his M.S. and B.S. in computer science from Shanghai Jiao Tong University, China in 1996 and 1993, respectively. He was a research staff member at IBM China Research Laboratory in 1996-1998, working on Mandarin speech recognition. Dr. Zhu won the National Science Foundation CAREER award in 2010.

10. Workshop URL (where will interested parties get more information?)

http://icml2011csls.wikidot.com/