NIFA Peer Review Process for Competitive Grant Applications

Table of Contents

• The Request for Applications
• Selection of a Panel Manager
• Selection of Panelists and Proposal Review
• Confidentiality and Conflicts of Interest
• Prohibited Use of Generative AI Technology in NIFA Peer Review Process
• The Review Panel Meeting

The Request for Applications

The review process begins with the publication of the Request for Applications (RFA) for the NIFA competitive program of interest. The RFA is published on the agency website and is accessible through NIFA’s funding opportunity webpage. The RFA can also be accessed through Grants.gov, the website for Federal government grants. Occasionally, RFAs are also published in the Federal Register. 

The RFA includes all of the pertinent information for the current funding cycle, including:

• program purpose
• legislative mandates
• award types
• eligibility requirements
• evaluation criteria
• submission instructions
• program goals and funding priorities
• application submission deadlines
• application submission instructions.

After reading the RFA, applicants often contact the program staff to discuss the applicability of a topic to the program goals and suitability of a prospective submission as an application. Applicants are encouraged to submit only those applications responsive to the funding priorities outlined in the RFA. Applications that do not respond to priorities in the RFA are not reviewed.

Letters of Intent

Some individual program area priorities within the Agriculture and Food Research Initiative, or AFRI, also require submission of letters of intent prior to application submission. For these program area priorities, applications submitted without prior approval of the letter of intent by the National Program Leader are not reviewed and not considered for funding. The letter of intent contains a descriptive title of the proposed project; names and roles of the project directors and other key personnel, along with their institutions; and a brief statement of approaches and objectives, including the program priority to which the project is responding.

Detailed instructions for preparation of a letter of intent are included in the RFA. NIFA program staff evaluates these letters for the suitability of the project to program goals and priorities and in relation to program scope and needs. Invitations to submit a full application are then issued by the National Program Leader in response to letters describing proposed projects best fitting these criteria.

Selection of a Panel Manager

Many NIFA competitive programs utilize a Panel Manager who is selected by the National Program Leader to assist with administration of the program. The Panel Manager is an active, established scientist possessing broad-based knowledge in the program area and a strong reputation for fairness and impartiality. The Panel Manager will have experience in research, education and/or extension, as is appropriate for the program. The professional stature of the Panel Managers within their respective scientific communities brings additional visibility and recognition to the program. 

Panel Managers become part-time, temporary (1 to 2 years) employees of the USDA. Duties of the Panel Manager include: assisting National Program Leaders in selecting panel members and ad hoc reviewers, assigning applications to reviewers, chairing the panel meeting, and assisting National Program Leaders with funding decisions. Panel Managers (or their family members) cannot submit an application to the panel that they head, as project director (PD), Co-PD or collaborator.

Selection of Panelists and Proposal Review

The National Program Leader and Panel Manager aim to assemble a diverse panel active in research, education, and/or extension, as appropriate for the program, related to the subject matter in question. The goal is to create a balanced panel with the necessary expertise to cover the range of the applications, while also maintaining diversity in geographical location, institution size and type, professional rank, sex, race, and ethnicity. Programs also strive to have continuity on the panel from previous years by inviting at least 30 percent of the panelists to return for a subsequent year. Potential panelists must be dedicated to providing high quality, fair reviews, and be able to devote sufficient time to the review process.

Note: Please keep in mind that volunteering may not always result in your participation in a panel due to recruitment requirements and expertise needed at that given time.

No more than one individual, including the Panel Manager, can serve from a single institution or, with very few exceptions, from a single state. Panelists cannot apply to the panel on which they have agreed to serve. The integrity of competitive programs depends upon the stature and qualifications of the individual panelists and the fairness and scientific skill with which they administer their scientific review responsibilities. All these qualities are necessary for the careful review and evaluation of the submitted applications.

The National Program Leader and Panel Manager study the applications carefully and assign to them for review to panel reviewers and, when additional expertise is needed, to ad hoc reviewers. Typically, three to four panelists review each application. If needed for additional expertise, up to three ad hoc reviewers may also evaluate an application. 

Each panelist is typically assigned 8 to 15 applications, for which they provide written reviews. During the review panel, discussion of each application begins with each panelist providing an oral evaluation, based on their written review, of their assigned application.

Reviewers prepare their written reviews and assign a review score based on the evaluation criteria, published in the RFA and available on the NIFA website, to assess the strengths and weaknesses of each application. For AFRI and most other NIFA programs, review scores include "excellent," "very good," "good," "fair," and "poor." Some programs may first assign points to evaluation criteria and align these to similar scores of "excellent," "very good," "good," "fair," or "poor." These scores may be useful in panelist discernment between the applications they reviewed and valuable to the rest of the panel in preparation for the subsequent in-depth discussion that takes place during the peer review panel meeting. 

Review criteria are specific for the NIFA competitive program. For the AFRI program, applications are evaluated for scientific merit; qualifications of project personnel and adequacy of facilities; and relevance to program priorities including importance of the topic for agriculture.

Confidentiality and Conflicts of Interest

Confidentiality is critical to ensuring the integrity of the peer review process. Names of submitting institutions and individuals, as well as application content and peer evaluations, are kept confidential, except to those involved in the review process, to the extent permitted by law. 

Identities of peer reviewers remain confidential throughout the entire review process, and the names of the reviewers are not released. Reviewer comments and discussion during the review panel also remain confidential. This issue is emphasized repeatedly from the time a panelist is invited to serve on the panel to completion of the panel. 

The Panel Manager, National Program Leader, panelists, and program staff are permitted access to the written reviews immediately before and during the panel meeting. Otherwise, written reviews and evaluations of each application are shared only with the respective applicant.

During the review process, special care is taken to avoid conflicts of interest. Individuals involved in the review process may not participate in any aspect of the proposal evaluation if they have: 

1. served as an adviser or advisee to the applicant(s)
2. collaborated or served as a coauthor with the applicant(s) during the past 3 years
3. are currently affiliated with, were previously employed within the past 12 months by, or are being considered for employment at the institution(s) of the applicant(s)
4. participated in a consulting/financial arrangement with the applicant in the past 3 years
5. are the spouse, child, sibling, parent, partner, or close friend, or otherwise have a relationship that might affect judgment, or could be seen as doing so by a reasonable person familiar with the relationship.

These conflict-of-interest rules apply to everyone involved in the review, including NIFA program staff, the Panel Manager, panelists, and ad hoc reviewers. When a proposal comes up for discussion during panel, any panelist with a conflict of interest leaves the panel room and does not participate in the review, discussion, or ranking of that proposal. Similarly, if the Panel Manager or National Program Leader has a conflict of interest with a proposal, they do not participate in any aspect of the review for the proposal, including assigning reviewers or being present during panel discussion.

Before accessing each proposal for evaluation, all panelists and ad hoc reviewers are required to affirm that they will abide by NIFA’s confidentiality rules and do not have a conflict of interest with that proposal.

Prohibited Use of Generative AI Technology in NIFA Peer Review Process

Artificial Intelligence (AI) is a broad field that has been maturing over the past few decades. The field of “Generative AI” has rapidly evolved with the creation of various tools such as ChatGPT, Dall-E, Google Bard, Bing AI, Apple Intelligence, and others that are able to produce text, images, and audio. While NIFA recognizes the potential of generative AI systems to promote both scientific advancement and the enhancement of Agency processes through streamlining and increased productivity, the Agency must also consider the potential risks posed by its use. To safeguard the integrity of NIFA’s extensive peer review process, NIFA prohibits the use of generative AI tools during the proposal evaluation process.

USDA issued interim guidance to all its Agencies’ staff and contractors prohibiting the use of public third-party Generative AI tools until further governance can be put in place. NIFA is abiding by that guidance to maintain security and confidentiality which is crucial for safeguarding personal and scientific information that is exchanged with NIFA.

NIFA cannot protect non-public information disclosed to a third-party generative AI system from being accessed by undisclosed third parties. If information from the peer review process is disclosed without authorization through generative AI or otherwise, NIFA loses the ability to protect it from further release. This loss of control creates a significant risk to researchers and their ideas. As a result, NIFA cannot adequately preserve the confidentiality of the information included in funding proposals submitted to the Agency. The potential disclosure of proprietary or privileged information may result in legal liability for and erode trust in the Agency. NIFA ensures the public that it safeguards scientific ideas, non-public data, and personal information that stem from proposals, review information, and related records in the merit review process.

The Review Panel Meeting

When a panel meeting is on-site, panelists are seated around a single table to allow the discussion among the various panelists assigned to an application. This arrangement also allows the entire panel to participate in the discussion of any application reviewed in the panel for which they do not have a conflict of interest. 

More recently, NIFA has moved to a panel meeting format that is almost exclusively virtual, which often allows individuals to serve on review panels when they may otherwise be unable to do so due to professional commitments, family obligations, travel restrictions or other reasons that would make travel to an on-site panel meeting impractical. The transition to virtual panels was fortuitous and allowed NIFA program staff to gain valuable experience conducting panels in a virtual environment several years before the subsequent COVID-19 pandemic prevented travel to all review panels.

Prior to the panel meeting, the Panel Manager and National Program Leader read the applications to identify special issues affecting panel discussion. Throughout the meeting, the Panel Manager and National Program Leader enforce rules about conflict of interest. They ensure that panelists leave the virtual meeting room during review and discussion of applications submitted from their own institutions or from individuals with whom they have a conflict of interest. They also emphasize confidentiality regarding all matters concerning submission, review, recommendations, ranking, and panel composition, and that confidentiality must be maintained outside the panel meeting room and after the panel meeting as well.

The Panel Manager and National Program Leader serve as chairs of the panel meeting and are responsible for ensuring that every application receives a thorough and objective review. The Panel Manager and National Program Leader do not provide an opinion or review of the application. The rating and ranking of the applications are based entirely on the consensus opinion of the panel. The Panel Manager and National Program Leader also ensure that different types of applications, such as research, integrated, and strengthening applications, are discussed and ranked separately.

Application Evaluation and Ranking

During the meeting, the panelists discuss each application and arrive at a consensus rating and ranking that reflects the overall merit of each application in consideration of, for example, the program priorities (including evaluation criteria), likelihood of success, and projected impacts and outcomes.

For AFRI, ranking categories are "outstanding," "high priority," "medium priority," "low priority" and "do not fund." Only applications ranked in the first three categories may be considered for funding; those ranked in the latter two are ineligible for an award.

Similar categories are used across NIFA competitive programs. Applicants should refer to the Funding Opportunity webpage for programs of interest to see the previous year’s success rate and gauge the level of competition for that particular program.

Following the evaluation and initial ranking of each application, a "panel summary" document is written by a panel member reflecting the panel consensus. It details the salient points of the panel’s assessment of the strengths and weaknesses of the application. The panel summary also contains a section with synthesis comments, describing areas for improvement and potentially providing suggestions for improvement. The synthesis also provides comments generally indicating the application’s overall merit and the review panel’s level of enthusiasm, or lack thereof. 

On the final day of panel meeting, the panelists reassess the initial rankings of the applications and rerank applications, as needed, to ensure that they are accurately categorized and appropriately ordered. After the completion of the panel, the National Program Leader and Panel Manager use the panel ranking to determine the applications that will be recommended for funding.

The National Program Leader and Panel Manager also review both the budgets of the top applications to ensure the funding request is appropriate, and the project is not already funded by another funding agency. Generally, applications are funded according to the panel ranking until program funds are used up. 

In the AFRI program, lower-ranked applications that fall below this funding line may be supported with "strengthening" funds, a percentage of the AFRI budget set aside to support applications from eligible small to mid-sized institutions, minority-serving institutions, or those in EPSCoR states (see the RFA for definitions and eligibility requirements). AFRI program area priorities are also expected to support some applications below the funding line submitted by "New Investigators" (see the RFA for definitions and eligibility requirements). 

After funding decisions have been finalized, applicants in most NIFA competitive programs receive copies of the written reviews of their application (with reviewer names removed to maintain reviewer confidentiality), the panel summary, and information on the relative ranking of their application. This information is commonly sent to the applicant only through email correspondence.