ACM SIGMM Interest List


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Alejandro Bellogin Kouki <[log in to unmask]>
Reply To:
Alejandro Bellogin Kouki <[log in to unmask]>
Tue, 16 Jul 2013 08:17:25 +0200
text/plain (126 lines)
[Apologies if you receive this more than once]


                             ACM RecSys Workshop on

 Reproducibility and Replication in Recommender Systems Evaluation - RepSys2013

              7th ACM Recommender Systems Conference (RecSys 2013)

                        Hong Kong, China, 12 October 2013



*  Submission deadline: 22 July 2013 *

*  Keynote speaker: Mark Levy, Senior Data Scientist at Mendeley *

== Scope ==

Experiment replication and reproduction are key requirements for empirical 

research methodology, and an important open issue in the field of Recommender 

Systems. When an experiment is repeated by a different researcher and exactly 

the same result is obtained, we can say the experiment has been replicated. When 

the results are not exactly the same but the conclusions are compatible with the 

prior ones, we have a reproduction of the experiment. Reproducibility and 

replication involve recommendation algorithm implementations, experimental 

protocols, and evaluation metrics. While the problem of reproducibility and 

replication has been recognized in the Recommender Systems community, the need 

for a clear solution remains largely unmet, which motivates the present 


== Topics ==

The topics the workshop seeks to address include –though need not be limited to– 

the following:

 * Limitations and challenges of experimental reproducibility and replication

 * Reproducible experimental design

 * Replicability of algorithms

 * Standardization of metrics: definition and computation protocols

 * Evaluation software: frameworks, utilities, services

 * Reproducibility in user-centric studies

 * Datasets and benchmarks

 * Recommender software reuse

 * Replication of already published work

 * Reproducibility within and across domains and organizations

 * Reproducibility and replication guidelines

== Submission == 

Two submission types are accepted: long papers of up to 8 pages, and short 

papers up to 4 pages.

All submissions shall adhere to the standard ACM SIG proceedings format:

Submissions shall be sent as a pdf file through the online submission system now 

open at:

== Important dates ==

 * Paper submission deadline: 22 July

 * Notification: 16 August

 * Camera-ready version due: 30 August

== Organizers ==

 * Alejandro Bellogín, Centrum Wiskunde & Informatica, The Netherlands

 * Pablo Castells, Universidad Autónoma de Madrid, Spain

 * Alan Said, Centrum Wiskunde & Informatica, The Netherlands

 * Domonkos Tikk, Gravity R&D, Hungary


To unsubscribe from the MM-INTEREST list:
write to: mailto:[log in to unmask]
or click the following link: