Artifact Evaluation

We give the opportunity to authors of accepted papers to submit an artifact establishing the reproducibility of the computational results described in their paper.

The objective of Artifact Evaluation (AE) is to reward efforts made by researchers to allow others to replicate their experiments. The AE enables to reproduce results, which helps researchers to build on top of each other's work and to compare results in a fair manner. The AE reviewers do not evaluate the correctness of the experimental results obtained using the artifact, only that these results can be reproduced.

The reviewing process is single-blind. Reviewers should be able to evaluate submitted artifacts using regular computing resources. Please feel free to contact the AE chair if you would like to submit an artifact for which this is not possible.

Submitted artifacts will be awarded badges as described in the ACM policy. See the program or accepted papers for papers with awarded badges.

Important dates

• Artifact submission deadline: August 27, 2018 (23:59 UTC-12)
• Notification of acceptance: September 17, 2018 (23:59 UTC-12)

Instructions to authors

Specify the following fields:

• Authors: same as for the (already accepted) paper,
• Title: same as for the (already accepted) paper,
• Keywords: use this field to specify the url at which we can find the information about the artifact and the artifact code. You should specify at least three keywords. Among these there must be a file containing instructions for the evaluators. If not all three lines are needed, fill three lines of keywords to enable the submission. A typical example:
• instructions: url of instruction file (for instance a .pdf or .txt file);
• code: link to the code (for instance a zip file or a repository);
• vm: link to a virtual machine image (if any).
• Paper: include the PDF version of your paper. The instruction file should specify which results in the paper are reproducible.

A nice HOWTO for preparing an artifact evaluation package is available online at http://bit.ly/HOWTO-AEC.

To submit an artifact, use the easychair submission website.

Additional Information

Here is a list of useful links related to artifact evaluation in other computer science research communities:

• How to package an artifact, http://bit.ly/HOWTO-AEC
• About Reviewing, http://dl.acm.org/citation.cfm?id=2618142
• Examples of accepted artifacts at other conferences, http://ctuning.org/ae/submission-20160509.html#examples.

Artifact Evaluation Chair

• Alessandro Vittorio Papadopoulos, Mälardalen University, Sweden

Artifact Evaluation Committee

• Alessandro Papadopoulos, Mälardalen University, Sweden
• Tomasz Kloda, University of Modena, Italy
• Alessandro Biondi, Scuola Superiore Sant'Anna, Italy
• Matthias Becker, KTH Royal Institute of Technology, Sweden
• Muhammad Ali Awan, CISTER Research Unit, Portugal
• Houssam-Eddine Zahaf, University of Lille, France

Contact

For any questions, concerns or comments, please contact the chair: ae-rtns2018@ensma.fr.