Blinding applicants in a first-stage peer-review process of biomedical research grants: An observational study

Abstract

To blind or not researcher's identity has often been a topic of debate in the context of peer-review process for scientific publication and research grant application. This article reports on how knowing the name and experience of researchers/institutions influences the qualification of a proposal. We present our experience of managing the peer-review process of different biomedical research grants. The peer-review process included three evaluation stages: first, blinded assessment; second, unblinded assessment by the same reviewer; and final, assessment of the better qualified proposals by an ad hoc committee. The change between the first (applicants blinded) and the second assessments (unblinded) for each evaluation and reviewer was evaluated. Factors associated with change were analysed, taking into account the characteristics of proposals, reviewers, and researchers. A qualitative content analysis of the reviewers' comments was also carried out to assess the reasons for change. The analysis of 5,002 evaluations indicated that in 18.5% of the evaluations (from 10.5 to 27.7% depending on the year of the edition), the reviewer changed the second assessment: either for better (11.9%) or worse (6.6%). Our findings also suggest that a change in the second assessment was highly correlated with a positive evaluation of the experience of the principal investigator or research team. With a change of 1 in 10 to 1 in 4 depending on the year of the edition, we believe that concealing the identity of researchers/institutions could help to focus exclusively on the proposal and reduce some of the common biases of the peer-review process in grant decisions.