GUIDELINES FOR TPC AND REVIEWS
Presentation slides from the INFOCOM 2020 TPC Informational Meeting, held during INFOCOM 2019.
All Area Chairs, regular and advisory TPC members are strongly encouraged to attend the TPC meeting to be held in Orlando, FL, USA, on November 23, 2019.
Review Quality — Area Chairs — Regular TPC Members — Advisory TPC Members—Ranking Papers in EDAS
Guidelines on the quality of INFOCOM reviews
Reviews with high quality is of utmost importance to IEEE INFOCOM, and the most important contributions that a member of the TPC can make to INFOCOM. Please avoid submitting short, unsubstantiated reviews, and do not cut and paste words and sentences across different categories.
Try not to be overly negative, as any research paper has its advantages and drawbacks. Please write a review with the quality that you would expect for your own papers. Keep a fair and open-minded approach, favoring selection of breakthrough papers that can open new areas and new research directions. Please also follow a distribution curve (based on the definition for each rating on the review form) when rating the papers assigned to you. This is particularly important as each TPC member perceives the quality of a paper differently if the paper is rated in isolation. The Area TPC Chairs and the TPC co-chairs will read the submitted reviews, and monitor their quality.
Responsibilities of the Area Chairs
As an Area Chair, you are a member of a distinguished group of experts who are essential to maintaining the high quality of INFOCOM. You are responsible for overseeing the review process of the papers assigned to you.
Step 1. Suggest “Early Rejects” to TPC Chairs.
The first step is for you to check the papers assigned to you, and identify papers that are (i) clearly out of scope for INFOCOM, or (ii) violate the double-blind submission policy.
Step 2. Oversee the discussion phase and intervene if the TPC Lead is not effective.
During discussion phase, please ensure an active and fair technical discussion is ongoing. Discussions such as “let’s just discuss at the TPC meeting” should be avoided. Add additional reviewers when necessary either from advisory TPC members (preferred solution) or from external reviewers. You can enter your own review and/or ask reviewers to modify their reviews/scores to be consistent with the discussions among TPC members during the online discussion phase, the in-person TPC meeting, and in the final recommendation.
Step 3. Provide your recommendations to the TPC co-chairs.
There are three options: accept, reject, and discuss at the TPC meeting. Make sure the TPC-Lead summary is consistent with TPC reviews and online discussions.
Step 4. Participate in the TPC meeting and lead group discussions.
Responsibilities of the Regular TPC Members
All reviews are automatically assigned by the review assignment system, based on the result of globally optimizing the sum of similarity scores across all review assignments. The similarity scores are computed by comparing, pair-wise, the full text of the submitted paper and TPC authored papers.
A regular TPC member is responsible for providing high quality reviews to the assigned papers (around 11). Each regular TPC member will also serve as the TPC-lead for about 4 papers, and should actively participate in both the online discussions and discussions at the TPC meeting.
For papers for which you are the TPC Lead, in your EDAS screen (after you login, select "My TPCs" and INFOCOM 2020 from the list of conferences), you will see a review link and a Meta review link under the "Type of review" column. For those papers for which you are a regular reviewer, you will only see the review link under the "Type of review" column.
First, you need to provide a review for each of the papers in your list by using the "review" link and filling out the review form by Oct. 8, 2019. You may delegate up to 4 regular reviews to external reviewers (who should not be students) with sufficient expertise. You must follow up with your delegates to finish their reviews, and oversee the review quality.
Starting Oct. 9, 2019, there will be an online discussion phase, during which you should actively discuss with other reviewers/TPC leads the outcome of each paper that has been assigned to you for review. The delegates are also responsible for actively participating in the online discussion phase. The messages exchanged during the online discussion phase will not be made available to the authors.
Once the discussion phase is over, for those papers that you serve as the TPC lead, you will need to provide a summary of the discussions in the form of a Meta review. Please click on the Meta review link under the "Type of review" column for your TPC lead papers, fill out the TPC summary form, and choose one of four categories for each paper accept, discuss, needs additional review, reject. The TPC lead recommendations (meta reviews) should be completed by October 29, 2019. They will not be made available to the authors.
Responsibilities of the Advisory TPC Members
An advisory TPC member will be assigned papers by the TPC co-chairs and the Area Chairs during Oct. 9 -- Oct. 20, 2019. Advisory TPC members play a critical role in the review process, by providing high quality reviews in a very short time span. The advisory TPC members are expected to return the reviews as soon as possible, but no later than October 27, 2019. These reviews are typically requested due to inconsistency in existing reviews. Advisory TPC members should also actively participate in the online discussions after submitting the review and discussions at the TPC meeting.
Instructions on the new ranking tool in EDAS
INFOCOM 2020 will continue to incorporates the new paper ranking system introduced in INFOCOM 2018, along with the traditional numerical scores in the review process. We ask each TPC member and reviewer to rank all the papers s/he reviewed (1: best). The ranking will be used to provide
Please note that in addition to other boxes and options to choose from that we used to have in the past years’ review process, this year there will be an additional drop-down box for ranking the reviewed papers uniquely by each reviewer (1st means the best). A dropdown box in the review form for each paper should look like
Once a paper has an assigned rank, it cannot be changed from the review form. However, you can change it by the following process. You need to click the up-down arrow as shown in the above screenshot or arrow of under the listing of your review assignment section, to see a list of papers ranked from top (i.e., the best) to bottom (i.e., the worst). Then you can change the ranking of a paper by dragging the icon that appears in front of the paper, to the desired place in the list and drop it there. For example, you have three papers ranked as shown in the screenshot below, but you want to swap papers 2 and 3. You need to click and drag the arrowin front of paper 3, and drop it after the 1st paper.
The updated paper ranking will look like the following screenshot.