A Case for Paper Grading Boards (Or: A Plea for More Proposals to Radically Reform Peer Review)

November 07, 2026

2026

Our peer review system in robotics (and I’m sure in other fields too, but I can only speak to robotics) is quite broken:

  1. There are far too many papers. Each new paper spawns more, like Harry Potter’s Geminio curse (our Gemini curse?).
  2. Qualified reviewers are collectively drowning under the paper flood. “Use LLMs to review!” sing the sirens.
  3. Back on shore, authors wait for months, sometimes years, to learn their paper’s fate.
  4. No paper is ever really dead, but many are undead, rejected and resubmitted to venue after venue.

The time is ripe for radical change. I don’t have a Silver bullet that would fix all of these deep-rooted issues in “one shot”, but I have one concrete-ish proposal, and I want to share it in the hopes that it will spur others to throw out similarly radical ideas.

The proposal is this:

  1. End journals.
  2. Keep conferences, but think of them only as meetings, and make the barrier to entry low.
  3. Institute paper grading boards (PGBs). Like journals, PGBs have expert reviewers, and a paper is submitted to exactly one of them, once. Unlike journals, a PGB never rejects a paper; instead, every submission receives a public report card with grades in categories (novelty, predicted impact, etc.).

The most important part is bolded. As an author, if I’m only getting one chance at a paper grade, I’m spending a lot more time polishing that paper. This would slow the field down in a very good way.

Journals and conferences (“the Old Way”) are trying and failing to do three things at once: (1) improve papers through the revision process; (2) disseminate work through the journal and conference proceedings; and (3) reward authors with a line item on their CV, incentivizing more good work.

The New Way would work like this: (1) The onus would be on the authors to improve their own papers and solicit informal feedback before submission. (Asking low-bandwidth experts to do this as part of review is no longer sustainable.) (2) To start, work would be disseminated the way that it functionally is now: social media, conferences, word of mouth. (There are major issues with this, and we should fix them too, but I’ll leave this out of scope for my PGB proposal.) (3) PGBs would reward only. I know that as a reviewer serving on a PGB, I would appreciate this clarity of mission. But more than that, I would appreciate the absence of premature and undead papers in my pile.

Some problems of the Old Way would carry over to the New Way. How do we incentivize reviewers to do their job well? How do we mitigate noise when, even in good faith, different people view the same paper so differently? How do we resist insidious influence of AI on both sides of the process? How do we agree on what categories matter when evaluating a paper? I don’t know. The most I will claim is: on these points, the New Way would be Not Worse.

And I’m sure this proposal has many flaws. After all, this proposal on peer review was only lightly peer reviewed (see the acknowledgments below). But I’m also confident that we need something radically new, and that we shouldn’t let the perfect be the enemy of the good as we fight against inertia. Let’s keep the radical review reform ideas coming.

Acknowledgments

Thanks to Hannah Blumberg, Yixuan Huang, Matteo Merler, and Andreu Matoses Gimenez for providing feedback on a draft of this post.