ETS 2014 - Novative Assessment of Collaboration 2-day Working Meeting

The rapidly growing literature on teamwork and collaborative problem solving suggests that these skills are becoming increasingly important in today’s organizations. However, significant developments in the evaluation of these skills are occurring within different research communities, often independently of one another.
Educational Testing Service (ETS) is hosting a two-day working meeting organized around five thematic panels that will bring together researchers in organizational teaming, educational collaboration, tutoring, simulation, gaming, and statistical and psychometric process modeling. Practitioners and researchers who approach teamwork and collaborative problem solving from different perspectives will come together to share insights and contribute to the development of reliable and valid collaborative assessment.
Organizers
Alina A. Von Davier
Senior Research Director, Center for Advanced Psychometrics, ETS
Patrick Kyllonen
Senior Research Director, Center for Academic and Workforce Readiness and Success, ETS
Mengxiao Zhu
Associate Research Scientist, Center for Advanced Psychometrics, ETS
Tentative Proposed Panels & Presenters
Panel 1: Evaluation of Team Performance in Organizations
This panel will discuss methods and findings for evaluating collective performance in organizational settings. Presenters include Nancy Cooke (Arizona State University), who specializes in the development, application, and evaluation of methodologies to elicit and assess individual and team cognition and performance; Eduardo Salas and Stephen Fiore (both from University of Central Florida), who are experts on teamwork, team training, advanced training technology, decision-making under stress, learning methodologies and performance assessment; Leslie DeChurch (Georgia Institute of Technology), who studies leadership and teamwork in organizations with a special focus on leadership networks and multi-team systems; and Noshir Contractor (Northwestern University), who investigates factors that lead to the formation, maintenance, and dissolution of dynamically linked social and knowledge networks in various types of teams in both real-world and virtual-world teams.
Panel 2: Evaluation of Students Working Together in Tutoring Environments, Games, and Simulations
This panel will focus on interactive learning environments, such as intelligent tutors, games, and simulations. Presenters include Art Graesser (University of Memphis), who designed, developed, and tested software that integrates psychological sciences with learning, language, and discourse technologies; Vincent Aleven (Carnegie Mellon University), who develops novel technologies for instruction, in particular for creating intelligent tutoring systems, and Carolyn Penstein Rose (Carnegie Mellon University), who studies the approaches from computational discourse analysis and text mining, conversational agents, and computer supported collaborative learning; and Saad Khan (ETS), who has the expertise in developing computer vision and human machine interaction algorithms.
Panel 3: Evaluation of Collaborative Problem Solving Performance in Educational Settings
This panel will discuss students working collaboratively in educational settings, such as problem solving. It includes Patrick Griffin (University of Melbourne), who is an expert on broad topics in assessment and evaluation and has developed dozens of online collaborative problem solving tasks that are used in 10 countries; Alina Von Davier and Patrick Kyllonen (both from ETS), who are both assessment experts and will discuss the collaborative problem solving projects at ETS; and Matthias Von Davier (ETS), who is an expert on psychometric methodologies and will discuss modeling strategies for the PISA 2015 collaborative assessment data.
Panel 4: Statistical Models for Dependent Process Data
In this session, the presenters will discuss several modeling approaches to dependent time series that represent the individuals’ actions during a collaborative problem solving task or other form of team interaction. The presenters also may address the integration of the models for process data and outcome data. Presenters include Peter Halpin (New York University), who together with Alina Von Davier has proposed a model for studying interactions based on the Hawkes process. This panel will also include Yoav Bergner and Jiangang Hao, both from the Center for Advanced Psychometrics (CAP) at ETS, who apply novel techniques, such as cluster analysis and editing distance, in analyzing process data; and Ron Stevens from UCLA and TLC Inc., who is using EEG to model team Neurodynamics in settings as diverse as US Navy Submarine Piloting and Navigation and high school problem solving.
Panel 5: Statistical Models for Collaboration and Group Dynamics
This panel will discuss advanced statistical models used in analyzing collaboration and group dynamics, such as social network models and Bayesian modeling. The presenters include Mengxiao Zhu from CAP at ETS, who studies the impact of social networks on team assembly and performance; Tracy Sweet (University of Maryland), who has expertise in social networks applied to education; Emilio Ferrer (University of California, Davis) and Joel Steele (Portland State University), who studied change with time in married couples (e.g., relationship dynamics) using structural equation modeling, multi-level or mixed effects modeling, state-space modeling, and Bayesian modeling.
Meeting Logistics & Registration information will be available soon.
Sincerely,
Katie Faherty
Educational Testing Service