RRNLP 2015 - Workshop on Replicability and Reusability in Natural Language Processing: From Data to Software Sharing

Even experiments reported in the best journals often present results that cannot be reproduced, either because the original data or software is not available, or because re-running the presented programs leads to different results. Moreover, slight changes in the implementations often (e.g., different parameter settings, dataset size or representation) often lead to significantly different results. At the same time, a number of repositories, web services or processing pipelines for natural language processing (NLP) are being developed, but, so far, without a focus on experiment replicability and comparability.
Therefore there is an urgent need to provide repositories of stable NLP datasets, as well as mechanisms and platforms for running experiments that would enable easy comparison of results. Such platforms, enabling the sharing of workflows, could provide major benefits for the entire community: for replicability of experiments it should be enough for the authors to provide the datasets, the URL of the developed workflow and the parameters used.
We welcome submissions of long and short papers and demonstrations, relating to any aspect of replicability and reusability in Natural Language Processing, including but not limited to:
replicability and comparability studies for NLP experiments
methodologies for publishing and versioning experimental datasets
platforms to enable comparison and evaluation of NLP tools
design and implementation of NLP pipelines, workflows and web services
platforms for NLP workflow construction and execution
collaborative developments of NLP workflows
unity sharing of NLP web services, pipelines and workflows
data standardization and process interoperability, including guidelines for inputs and output format for recomposing new NLP workflows