Conferences generally know from the beginning how many papers they will accept. This is a function not only of having to provide time and space for each paper to be presented, but also a holdover from the days when proceedings* were expensive to publish, and every page counted. So instead of the work being compared to some specific standard of quality set by the publisher, conference papers are in direct competition with each other, as judged by the competitors and their rivals. Acceptance rates are single digits for a few established and prestigious conferences, and are published as a matter of course.
Most of the conferences I target have an acceptance rate of one in five or six, so while it's no fun to get a rejection, neither is it usually a big surprise. It's what I was expecting when I submitted a paper in December for a June conference. The writing was jammed between the end of semester, the holidays, and several other work deadlines, so it was not my best work. The project itself seemed interesting enough, and the paper was good enough that I wasn't embarrassed to submit it, but only just. My plan was to take the feedback and refine it to submit somewhere else.
Every paper is assigned a primary reviewer, who is responsible for rounding up three additional referees, all of whom provide ratings and feedback. The ratings are on a 1 to 5 scale, with 5 signifying "better than I could have written," and 1 meaning "it's burning in my wastebasket as we speak." For better conferences (like this one), some papers are rejected outright, some (those with all fours and fives) are accepted, and the rest are given a chance to write a rebuttal of the reviews. Then a committee meets to decide who's in and who gets the home version as a consolation prize.
|Remember to keep it close to the wall through the Discussion section. Image from here.|
Rebuttaling is a delicate business. Regardless of whether a paper gets accepted, there is a very high chance that the same people will review your work in the future, so the "Reviewer 2 is a poopy pants" approach is not generally recommended. We pretend this is all anonymous, but academic communities are very specialized and surprisingly small these days, so everyone knows who is skewering whom. The safe approach is to thank the reviewers for telling you your writing is awful, assure them that their reviews were better than Cats, and start working on the next publication.
My ratings averaged to "meh," which normally means thanks for playing. Imagine my surprise when comments from my primary reviewer said essentially, "I would like to see this in the conference. Write a rebuttal that will convince us. You have five days and up to 5000 characters."
First, let me say that if you ever receive such a notice, it's important not to read 5000 characters as 5000 words. Just don't do it. But it got written and submitted, and it's arguably better than the paper, so I consider it a good exercise. Hopefully it will be just enough to edge out some poor slob whose reviewer hates his Ph.D. advisor, and I will have some work-funded travel to look forward to. I will know in a few days. In the meantime, I guess I will start on the next one.
Updated: After a successful rebuttal and two rounds of revisions, the paper has been accepted. I will spare you any description of the subject matter. If I can manage to get the camera-ready version uploaded to the website, it looks like I will be heading back across the pond this summer.
* All of the papers presented, as well as descriptions of selected other activities and messages from Powers that Be, are collected into the Proceedings of the 24th Annual Conference for Whatever Arcane Topic and published. They used to be bound as books. Now most are distributed on DVD, or increasingly, online.