Guideline

Purpose

Scope

Background

• The Juvenile Justice and Delinquency Prevention (JJDP) Act of 1974, 42 U.S.C. 5601, et seq. (hereinafter referred to as the "Act"), as amended, requires that applications submitted to OJJDP for Part C discretionary funds be approved through a competitive process established by rule by the OJJDP Administrator. Programs carried out in declared disaster areas or programs that are uniquely qualified are exempt from this competitive application requirement.

• The Act further requires that programs be selected for OJJDP assistance through a formal peer review process using outside experts in fields related to the subject matter of the program, with the exception of assistance provided pursuant to Section 241(f) of the Act to an eligible organization comprised of member representatives of the State Advisory Groups.

• Accomplishing OJJDP’s mission to provide a comprehensive and coordinated approach to the problems of juvenile delinquency is dependent, to a large extent, on the success of the programs and projects OJJDP funds. To foster this success, OJJDP makes careful and informed selections of projects for funding. A very important element of the project selection process is peer review. Peer review is the technical and programmatic evaluation of projects and applications by experts from outside the Department of Justice who are qualified by training and/or experience to evaluate and make recommendations with regard to proposed programs.

Peer Review Policy

It is OJJDP’s policy to use peer review to assess all competitive assistance applications and, on an optional basis, applications for continued funding beyond a program’s original project period and noncompetitive awards to uniquely qualified applicants.

The following types of awards are specifically excluded from competition and peer review requirements under the terms of the OJJDP Competition and Peer Review Regulation:

• Funds transferred to OJJDP from another Federal agency to augment authorized juvenile justice programs, projects, or purposes.

• Funds transferred to other Federal agencies from OJJDP for program purposes as authorized by law.

• Procurement contract awards which are subject to applicable Federal laws and regulations governing the procurement of goods and services for the benefit and use of the Federal Government.

• Assistance awards from the 5 percent set aside of Special Emphasis funds under Section 261(e) of the Act.

• Assistance awards under Section 241(f) of the Act.

Peer review recommendations are advisory and do not bind the OJJDP Administrator to make the recommended decision. However, the Administrator will give full consideration to peer review recommendations in selecting projects for awards.

In special circumstances, a grant application may require a second review. When a second review is required, the cognizant Division Director will determine whether the second review panel will be composed of new reviewers, the original reviewers, or a combination of both. Circumstances that might necessitate a second review include:

• During the course of a review, prejudiced, misleading, or false information was presented to or used by the Peer Reviewers.

• A procedural error made the review process inconsistent with the program announcement, specific instructions to the applicants, or the OJJDP Competition and Peer Review Regulation.

Definitions

• A Peer Review Coordinator is an OJJDP employee designated to oversee all aspects of the peer review process.

• Competitive Awards are made under OJJDP program announcements (published in the Federal Register) informing the public of the availability of funds for specific purposes and inviting formal applications (or, in some instances, Concept Papers). The selection criteria to be applied by the Peer Reviewers to a specific application are listed in each Federal Register announcement.

• The Division Director is the director of any one of the following OJJDP components: Research and Program Development Division; Special Emphasis Division; State Relations and Assistance Division; Training and Technical Assistance Division; Information Dissemination Unit; Concentration of Federal Efforts Program; or Missing and Exploited Children’s Program.

• Financial Review refers to review by the Office of Justice Programs, Office of the Comptroller, to determine whether the budgeted costs presented in an application are reasonable, allowable, and cost effective for the proposed activities. All applicants must meet Office of Justice Programs (OJP) standards for fiscal integrity (as described in the current editions of the handbook on policies and procedures for OJP grants and the Financial Guide). A Financial Review is performed after the Administrator has decided to fund an applicant’s project. Financial Review does not obviate the need for the Peer Reviewers to rate the application’s response to the selection criteria for budget and cost effectiveness.

• An Internal Reviewer is an officer or employee of the Department of Justice qualified by experience and expertise to conduct appropriate application and program reviews.

• An Internal Review Group consists of Internal Reviewers selected to review Concept Papers or applications submitted to OJJDP in response to a competitive program announcement, review noncompetitive applications, or review and evaluate the recommendations of a Peer Review Panel as part of the internal review process.

• Noncompetitive Awards are made in the absence of program announcements inviting applications. These may include awards to continue a project’s funding beyond the original project period or awards for uniquely qualified projects not subject to peer review.

• A Peer Reviewer advises OJJDP on the merits of applications submitted for funding. A Peer Reviewer is an expert in a field related to the subject of a proposed program or in the implementation of that type of project and may not be an officer or employee of the Department of Justice.

• Peer Reviewer Recommendations consist of ratings or summary rankings of Concept Papers or applications for the purpose of making recommendations regarding the selection of applications for OJJDP funding.

• A Peer Review Panel consists of three or more experts selected to review, evaluate, and make recommendations on Concept Papers or applications submitted to OJJDP in response to a competitive program announcement.

• A Concept Paper is an abbreviated application. Concept Papers may be requested by OJJDP for competitive programs for which a large number of applications are expected. Concept Papers will be reviewed by OJJDP staff or others who have expertise in the program area in order to eliminate applications that fail to meet minimum program or eligibility requirements, as specified in a program announcement, or clearly lack sufficient merit to qualify as potential candidates for funding consideration. Concept Papers may be subject to peer review.

• A Program Announcement is a notice published in the Federal Register that invites applications for a specific program and set of requirements.

• The Program Manager is a member of the OJJDP staff who is directly responsible for the specific applications under peer review.

• A Ranking is an application’s relative position, based on summary ratings, to other applications submitted for a specific program announcement.

• Ratings are scores assigned by individual Peer Reviewers based on an application’s response to the selection criteria specified in the program announcement.

• Summary Ratings are the averages of the total scores assigned to each application by each Peer Reviewer.

Peer Review Procedures

Peer Reviewer Approval. The OJJDP Administrator approves qualified consultants to serve as Peer Reviewers for each application or group of applications based on recommendations provided by the Division Director.

Consultant Pool.

• An OJJDP support contractor maintains a pool of qualified consultants from which Peer Reviewers shall be selected. Any individual with requisite expertise may be selected from the pool with approval of the OJJDP Administrator or the Administrator’s designee. This pool is maintained for peer reviews and other technical assistance purposes, and includes a sufficient number of experts to meet OJJDP’s peer review needs.

• The Consultant Pool is managed by an OJJDP support contractor. Consultants are subcontractors employed by the OJJDP support contractor. Consultant experts are continually added to the pool to maintain a wide range of expertise, experience, background, ethnicity, gender, and geographic representation. Consultants performing peer review are reimbursed by the support contractor at a flat rate established by OJJDP.

• Individuals who wish to be considered for the Consultant Pool may submit their credentials to the Peer Review Coordinator or to the OJJDP support contractor, who will evaluate the consultants’ qualifications. If a consultant subsequently performs a peer review and fails to fulfill his/her obligation without substantial justification, the OJJDP Administrator may request that the support contractor remove the consultant from the Peer Review Pool. Reviewers who fail to satisfactorily complete their assignments will not be reimbursed for their work.

Selection of Peer Review Panels.

• The Program Manager may recommend qualified reviewers to the support contractor and will ask the support contractor to provide a listing of qualified reviewers in specific topical areas. A consultant expert must be enrolled in the Peer Review Pool to be eligible to serve as a reviewer.

• Based on the list received from the support contractor, the Program Manager and the Division Director will recommend potential reviewers from the Consultant Pool. The Administrator will approve reviewers from this list or ask for additional qualified consultant experts enrolled in the Consultant Pool.

• The Program Manager and the Division Director will submit their recommendations via a memorandum to the OJJDP Administrator. The proposed reviewers should be listed in order of preference with a brief biography attached to the recommendation memorandum. A copy of the memorandum shall be provided to the Peer Review Coordinator, who will notify the support contractor and the Division Director following approval of the Peer Reviewers.

• When considering candidates for a Peer Review Panel, the Program Manager and Division Director should recommend a highly qualified group that represents expertise related to the individual applications under review. Each panel should be structured to provide broad representation and many views on matters under the Peer Review Panel’s consideration. Some considerations that should help achieve reasonable balance on the Peer Review Panel are:

  • Each member of the panel should have expertise in or complementary to the subject area under review. This does not preclude using youth representatives.

  • When possible, the Peer Review Panel should be comprised of researchers, practitioners, and academics.

  • Panel members should be drawn from as wide a geographic area as is practical and should represent both urban and rural perspectives.

  • Special attention should be paid to recommending women and minorities who are qualified reviewers.

  • When appropriate, the Peer Review Panel should be comprised of a diverse group of experts from the public and private sectors, including community-based youth serving organizations.

Internal Review

• An internal review of applications or Concept Papers will be conducted by the Program Manager and/or by designated Department of Justice staff.

• The first stage of the internal review will determine if the application is in compliance with minimum program and statutory requirements. Applications that do not meet basic requirements will not be forwarded to a Peer Review Panel. Applicants whose proposals are rejected during the first internal review stage will be notified in writing of the reasons for the rejection. Examples of reasons for first stage rejection may include, but are not limited to, applications proposing activities other than those called for in the program announcement, applications proposing to serve a target population different than that specified in the program announcement, and applications from agencies or organizations that do not possess the qualifications specified in the program announcement.

• A second internal review will be conducted by the Program Manager after the completion of the external peer review. This may be supported by other Internal Reviewers and/or an Internal Review Group. Following the second internal review, the Program Manager will prepare a memorandum through the Division Director to the Administrator describing the review process, the conclusions and recommendations of the reviewers, the scores received by the application, any significant problems encountered during the review, suitability of the applicant, and significant recommendations for modifying or enhancing the application recommended for funding. The memorandum will provide a formal recommendation concerning applications recommended for grant awards.

Peer Review

• Peer reviews may be conducted by mail, in meetings, or through a combination of both; a peer review meeting is preferred when practical. These peer review meetings facilitate useful dialog among the experts, provide an opportunity for the reviewers to seek clarification from the Program Manager concerning program and technical requirements, and, through careful monitoring, assure that each application receives equal consideration.

• Infrequently, it may be necessary for Peer Reviewers and/or Program Managers to make site visits. In all instances OJJDP will determine the necessity of site visits. Should a Peer Review Panel believe that a recommendation cannot be finalized without a site visit, the Peer Review Panel should make a request to the Peer Review Coordinator who will present the request to the Division Director for approval.

• For peer reviews that involve meetings, Peer Review Panel members will be assembled for instruction, including a review of the program announcement, selection criteria, and peer review procedures. The Peer Review Coordinator will provide general oversight for the peer review meeting. The Program Manager will be available to provide interpretation of the program announcement, and will provide objective information concerning program requirements. The OJJDP support contractor will provide staff to facilitate and record the meeting and prepare a summary of the proceedings.

• Where time or other relevant factors, such as cost, preclude convening a Peer Review Panel meeting, reviewers will communicate with one another via mail, telephone, or electronic means.

Selection Criteria

All applications received by OJJDP are, at a minimum, rated on the extent to which they meet general selection criteria. The following selection criteria can also be enhanced to more clearly define the program requirements:

• The problem to be addressed by the project is clearly stated.

• The objectives of the proposed project are clearly defined and the outcomes are measurable.

• The project design is sound and contains program elements directly linked to the achievement of project objectives.

• The project management and overall organizational capability demonstrate the applicant’s capacity to successfully operate and support the project.

• Budgeted costs are reasonable, allowable, and cost effective for the proposed activities.

Each competitive program announcement will indicate any additional program-specific review criteria to be considered in the peer review for that program. The assigned points for each criterion will be specified in the program announcement.

Scoring Applications

The maximum score for each criterion shall be indicated in the program announcement, and the total possible score for all criteria shall equal 100 points. For example:

• Statement of the problem -- 20 points.

• Definition of objectives -- 10 points.

• Project design -- 30 points.

• Project management and organizational capability -- 25 points.

• Reasonableness of costs -- 15 points.

Competitive applications will be rated by each Peer Reviewer according to the selection criteria. Summary ratings will be calculated from the numerical scores assigned to each application by the individual reviewers. The ranking of each application will be based on its summary rating. The rating categories are as follows:

• 90-100 points -- Responsive with no revisions required.

• 80-89 points -- Responsive with minor revisions required.

• 70-79 points -- Responsive with significant revisions required.

• 60-69 points -- Minimally responsive with major deficiencies that would require extensive correction.

• 0-59 points -- Not responsive and not sufficient to receive funding.

Results of Peer Review

Peer Reviewers are encouraged to make suggestions for enhancing proposals.

Occasionally, supplementary reviews are necessary. Supplementary reviews are performed by a Peer Reviewer for particular programs or project applications for the following reasons:

• To address highly technical aspects of an application which initial Peer Review Panel members are not qualified to address.

• Conflicts of interest or other disqualifying circumstance within the Peer Review Panel resulted in an insufficient number of valid peer reviews.

Standards of Conduct

Conflicts of Interest

In addition to the general Department of Justice conflict of interest rules set forth in its Standards of Conduct, OJJDP Peer Reviewers are subject to the OJJDP Peer Review Policy with respect to conflicts of interest.

It is OJJDP’s policy to prohibit a Peer Review Panel member from participating in the review of any application when he or she has a real or potential conflict of interest, such as:

• The Peer Reviewer has been, or would be, directly involved in the project (e.g., as a current or past advisory board member, consultant, collaborator, or conference speaker whose expenses would be paid from the grant).

• The Peer Reviewer is employed by the same institution or organization as the applicant or was employed there within the past year.

• The Peer Reviewer and the applicant collaborated within the past year on work related to the proposal.

• The Peer Reviewer is or has been under consideration for a position at the applicant’s organization or institution.

• The Peer Reviewer served in an official capacity with the applicant’s organization within the past year.

• The Peer Reviewer’s organization has members or closely affiliated officials (e.g., board of trustees members) who serve in an official capacity with the applicant’s organization or institution.

• The Peer Reviewer and the applicant have a familial relationship.

• The Peer Reviewer had relations with the project director, or other key personnel identified in the application, as a student, thesis advisor, or post-doctoral advisor.

• The Peer Reviewer and applicant are known to be either close friends or open antagonists.

• The Peer Reviewer has a proposal planned for submission to OJJDP or currently under review by OJJDP within the same subject area as the proposed project.

• The Peer Reviewer was declined for an OJJDP project, had a substantial budget reduction in an OJJDP funded project, or incurred other unfavorable action from OJJDP.

• The Peer Reviewer is currently involved in a project closely associated with the proposed project.

• The aforementioned situations should be considered by the Program Manager before a Peer Reviewer is recommended for a peer review panel, and by the OJJDP support contractor and panelist before the proposed panelist accepts an invitation to serve on a specific review. Should a conflict of interest, or the appearance of a conflict of interest, develop after the individual has been selected, it should be brought to the attention of the Peer Review Coordinator by the Program Manager, Division Director, OJJDP support contractor, or Peer Reviewer.

• During the course of a review, should a Peer Reviewer question that he/she may have a conflict of interest or the appearance of a conflict, the reviewer should immediately notify the Peer Review Coordinator or the support contractor’s representative assigned to facilitate the review.

Confidentiality

Peer Reviewer Inquiries

Informing Applicants of Peer Reviewer Comments

Compensation

Managing the Peer Review Process

__________________________________________________
Shay Bilchik
Administrator

______________________________
Date