ElementReview solicitations (ER-S), if accepted, entail payments by the platform to the submitting reviewer. These payments are funded directly from manuscript submission fees.
SynthesisReviews are uncompensated, but the author does receive benefits in the form of recognition and status.
The challenge system addresses at least two problems relating to the production of high quality reviews.
- There may be very many ElementReviews and/or SynthesisReviews submitted for a particular paper. It is helpful to maintain a featured subset as the ”best” representatives for the purpose of quickly determining the most salient kinds of critique a work has been subject to and how these change over time.
- Reviewers are already incentivized to contribute reviews via compensation. They are disincentivized from submitting obviously poor or misleading reviews due to the fact that all reviews become part of their public record. However, we additionally need incentives for the continued exploration of conflicts between rival critiques.
The platform allows reviewers to initiate either:
- Direct Challenges: Challenge an existing featured ElementReview or SynthesisReview. If the challenger's review wins, their review replaces the losing review as part of the featured subset.
- Petition Challenges: Publicly petition for another reviewer's review replacing an existing featured review of the same type.
Initiating a direct challenge incurs a modest fee. The purposes of this fee are to discourage excessive or frivolous challenges, and to sustain the financial health of the platform.
Petition challenges incur no fees.
How are challenges evaluated?
It is important to note that there may be more than one challenger to a featured ElementReview or SynthesisReview. When a challenge is initiated, the challenger must select an election date. Successive elections are between two reviewers: the current incumbent and the challenger. Suppose a reviewer challenges an incumbent's featured review and sets the election date, D, which is 2 weeks into the future. If the incumbent loses another challenge before D, this election is canceled and the challenger's fee becomes a credit that can be applied to a future challenge.
Votes are counted from petition challenges as well as from any platform users that happen to come across the in-progress challenge and cast a vote. The platform enables users to easily see (e.g., via some sequence of clicks) a list of all challenges associated with manuscripts or reviews they have previously engaged with.
A few rules:
1. All challengers are identified.
2. Challengers must set an election date at least 1 week into the future and on a date that does not overlap with existing challenges for the same featured review.
3. Voters may change their votes as many times as they wish. Votes accumulated by election time determine the winner.
4. Voters can inspect all previous losing challenges submitted for the manuscript.
5. There can be at most M challenges to a given ElementReview or synthesis review within the last 30 days.
6. Each vote is weighted based on alignment between the vote caster's review record and the featured review in terms of demonstrated expertise.
7. Former incumbents cannot re-challenge within 30 days.
8. If the incumbent previously won an election, any votes received during the previous election default towards the incumbent in their next election.