Phase 1: Public Review
Before the platform begins soliciting reviews, a manuscript is publicized on the platform for a short period (e.g., 1-2 weeks). During this time, platform users may petition to review the manuscript, or petition that certain CRWE criteria be applied to the manuscript. This engagement informs later algorithmic review solicitation.
Double Blind Review (Author pays review fee)
The double-blind public review period is specific to each manuscript. Both authors and reviewers remain anonymous. This phase is open to all platform users, though only solicited reviews may receive compensation sourced from manuscript submission fees.
The C-SQD algorithm creates ElementReview solicitations (ER-S) in a stochastic manner. Each ER-S consists of a reviewer ID, manuscript, CRWE criterion, and payment scheme. The probability that a specific reviewer is matched with an ER-S depends on multiple factors including their demonstrated expertise in past reviews, publication record, whether they petitioned to review the manuscript, and status as a platform subscriber.
The payment scheme may be simple (e.g., guaranteed payment for accepting the ER-S and submitting the completed ElementReview) or more complicated.
When accepting an ER-S, reviewers commit to this ElementReview becoming part of their publicly accessible review record. Along with their ElementReview submission, reviewers can submit petitions for other CRWE criteria to be applied to the manuscript. The algorithm uses these petitions when creating new ER-S assignments and constructing the manuscript’s evaluation tuple. This system balances two needs: biasing new ER-S creation towards potential problem areas while maintaining independence among reviewer judgments by not presupposing specific issues.
The public review period continues generating new ER-S assignments until the allocated review funds are depleted. After closure, reviewers can still submit uncompensated reviews and accrue status benefits from quality contributions. Additionally, platform users can fund new review rounds or establish Bug Bounties.
Each reviewer is limited to a maximum number of concurrent active review ElementReviews. Once all 3 review phases are completed, perhaps with some delay, ElementReviews are re-identified and become part of each reviewer’s public record.
Non-anonymous Review (Author does not pay review fee)
Authors that do not pay for review may still publish their work on the platform. These manuscripts also enter a public review period with a maximum duration (e.g., 2 years). All review submissions are uncompensated.
Phase 2: Author Revisions and Response
During and following the public review period, the author may submit a response with revisions up to 2 times. They may accept or argue against aspects of the public review period findings. They may or may not include additional analyses, experiments, etc., however, doing so may warrant a second public review period (and additional review fees). At the end of phase 2, the manuscript’s evaluation tuple will be made available.
Phase 3: Expert Synthesis Review
Any manuscript, whether subject to a combination of solicited and unsolicited public review, may rise to a significance level that warrants integrative, narrative review. This significance level is based in part on the manuscript’s evaluation tuple at any time after the close of the public review period.
Following the public review period, algorithmically solicited ”experts” craft SynthesisReviews that take into account the results of the public review period. During this phase, manuscript authors and expert reviewers remain anonymous to each other if the public review phase was double-blind. Manuscript authors can specify preferences for reviewer communities from which the algorithm selects expert reviewers, though these preferences bias rather than constrain the selection process. This phase is unpaid but carries high status. It is possible that the expert reviewer that accepts solicitation for a manuscript was also a reviewer in the public review phase.
Each expert reviewer is limited to a maximum number of concurrent active ElementReviews. Once all 3 review phases are completed, perhaps with some delay, SynthesisReviews are re-identified and become part of each reviewer’s public record.