Thank you for your time and effort in reviewing for MMM 2023. Your expertise directly contributes to maintaining the high quality of the MMM conference series and strengthening the multimedia research community.
As a Technical Program Committee (TPC) member, we expect that you are already experienced with writing excellent reviews. However, in practice we find that guidelines can help the review process and the most important of our guidelines are:
You should write a review that you would like to receive yourself. Your reviews should be helpful to the authors, even if the review recommends the rejection of the paper.
The MMM conference deals with multimedia and as such, while single media papers are acceptable, papers dealing with multimedia data are especially welcome.
Papers do not need to be focused on applying AI techniques to multimedia data. Our research community is rich and diverse, and papers addressing many aspects of multimedia analytics and modeling are welcome.
Even though all reviews are anonymous, we hope that you prepare excellent reviews, reviews that you would be proud to associate with your name.
We now present our guideline for best practices when reviewing for MMM 2023.
Best practices for reviewing
1. Check the paper topic:
Confirm that the paper that you are reviewing falls into the topical scope of MMM, as defined by the Call for Papers. Eventually, we rely on your judgement and the collective wisdom with your peers to decide if the paper aligns with multimedia topics.
Remember that the problem addressed by an MMM paper ideally involves multi-modal data, or is expected to be related to the challenge of how people interpret and use multimedia.
Although many submissions to MMM make a technical contribution in the form of a new algorithm or AI application, not all do, nor is it a requirement of the MMM conference. Do not give less value to papers that carry out studies of new multimedia problems because they do not make a novel algorithmic contribution. Instead, judge these papers by the novelty of their insights and the value these insights could have for the community.
2. Support your statements:
Reviews should not just state, “It is well known that…”, but rather, they should include citations. Likewise, reviews should not just state, “Important references are missing…”, but rather, they should include examples.
Reviewers should list their own references only in very rare cases when these are indeed the most relevant references for the authors to refer to.
Reviews should not just state, “Authors should compare to the state of the art…”, but rather, they should cite specific work (i.e., peer-reviewed references) that they feel the authors should have considered and why.
3. Respect the authors:
Reviews should critique “the paper”, and not the authors.
Reviews should try not address the authors directly, esp. not as “you”. (A direct address can be interpreted as an affront by the reader).
During the review process, no attempt should be made to guess the identity of the authors.
4. Please include in your review:
Statement of novelty: What does the paper contribute? Is that contribution valuable for the multimedia research community? Does the paper cover all the relevant related work, and explain how its contribution builds on the related work?
Statement of scientific rigor: Are the experiments well designed? Are the experiments sufficient to support the claims made by the paper? Are they reproducible? Have the authors released a resource, such as a data set or code?
Fixes that the authors should make for the camera ready. We can trust the authors to correct minor errors. Authors generally also will state their commitment to correcting minor errors found during the review process during the rebuttal. However, major flaws must lead to rejection, since it is not possible to confirm that the authors have actually corrected major flaws successfully (i.e., the paper does not go back to the reviewers for checking).
5. Special regulations due to COVID-19:
Reviewers should take into account that certain types of experiments involving people, e.g., user studies, dance experiments, were more difficult to perform since 2020 because of the social distancing measures in many locations. Therefore, if authors propose covid as a limiting factor in their research, that should be taken into account by the reviewers.
Policy on arXiv papers: We consider a “publication” to be a manuscript that has undergone peer review and has been accepted for publication. This means that the following points apply to arXiv papers (and any other papers available online that have not been peer reviewed):
If the paper that you are reviewing is available on arXiv, and has not been published elsewhere, it is an acceptable submission to MMM, since arXiv papers are not peer reviewed and are not considered to be publications;
Please do not insist that the authors cite a paper that is only on arXiv and has not otherwise been published. Since arXiv papers are not all peer-reviewed publications, missing an arXiv paper does *not* count as missing related work;
Likewise, if the authors do not compare their work with an approach described in an arXiv paper, it does *not* count as a weakness in their experimental evaluation;
If you know of an interesting arXiv paper relevant to the paper you are reviewing, you are more than welcome to tell the authors about it, but make sure you mark the reference as FYI “for your information” so that the authors know that you do not regard it as missing related work.
If you have any questions about the guidelines, please contact the Technical Program Chairs at firstname.lastname@example.org