Incorporated into the Community Prevention Grants Program are mechanisms for evaluating whether desired program outcomes and systems changes have occurred and whether they are having their desired effects on risk factors and juvenile delinquency problems over time. Local grantees are required in their subgrant applications to specify "a plan for the measurement of performance and outcomes of project activities." This evaluation requirement is implemented at the individual program level by measuring desired program outcomes (e.g., enhanced parenting skills, improved test scores) and at the community level, by tracking changes in risk factors and juvenile problem behaviors. The prerequisite risk and resource assessment process creates a baseline from which changes can be measured during and after program implementation.

Local grantees recognize that there are many advantages to a good evaluation plan, especially one that includes tracking outcomes and risk factors. Tracking of outcomes assists stakeholders in their project cycle by feeding back information about program results relative to objectives, and enabling stakeholders to assess progress and refine their programs. Tracking risk factors helps communities further assess program effectiveness by monitoring the relationship between prevention strategies and long-term changes in community risk factors and youth problem behaviors. Nevertheless, for a variety of reasons including issues related to perspective and capacity, evaluation poses a particular challenge for many community members. In fact, as one community member described it, "Evaluation is the biggest barrier of the Community Prevention Grants Program at the local level."

5.1 Adopting a Long-term Perspective

As discussed in Section 2 above, conducting the initial risk and resource assessment can be challenging for many communities, particularly in terms of the time-consuming processes of finding and gaining access to community risk and protective factor data. Communities often confront these same challenges as they try to track risk factors over time. Additionally, many communities do not recognize that they should be tracking risk factors and juvenile delinquency problems over time as a key part of their evaluation process. The theoretical links between the community's programs, its short and intermediate outcomes, and the eventual changes in risk factors are not always clear.

A community's willingness to track outcomes and risk factors over time can be influenced by a perceived conflict between the theory and practice. That is, the theoretical model underlying Title V proposes a long-term perspective that fosters positive, sustained community change over time. Nevertheless, many communities perceive Title V as a short-term process. One factor underlying this conflict is that many communities are accustomed to traditional grants in which once grant money is expended, program operations end and monitoring of results cease. Recent trends toward comprehensive, community-based initiatives like Title V, however, encourage broader integrated efforts and long-term systems change intended to outlast the grant period.

The conflict between long-term theory and short-term practice is bolstered by the 3-year time limits on Title V funding, the uncertainty of funds from year to year, and the fact that risk factors take a long time to impact. The impact of prevention strategies -- such as home visitation for families with newborns -- on risk factors and adolescent problem behaviors may not be noticeable for as many as 10, 15, or 20 years (Developmental Research and Programs, 1999). Communities that are dealing every day with the consequences of serious juvenile crime and problem behavior often become frustrated at the prospect of waiting years for priority risk factors to change. Although short-term approaches produce short-term outcomes, communities feel validated when they can see the immediate effects of their efforts.

5.2 Building Local Capacity

Recognizing the challenges inherent in tracking risk factors and conducting evaluations, and the resource limitations of many communities -- specifically rural communities, low-income communities, and communities with few existing resources -- OJJDP and several States have developed solutions to help increase community evaluation capacity. OJJDP makes technical assistance and training available to States and communities on an as-needed basis.⁶ Assistance is available to help with all aspects of the Title V process, from strengthening the conceptual understanding of the risk-focused model of prevention to planning and evaluation. OJJDP has provided a valuable source of evaluation assistance with its Title V Community Self-Evaluation Workbook. The Workbook consists of easy-to-complete forms and step-by-step instructions that guide communities through key evaluation activities. In fact, the Workbook has been so well-received that 13 States including Arizona, Michigan, Florida, and Vermont, among others, now require communities to use the Workbook as their primary evaluation tool. Some States, such as West Virginia, also have promoted the use of the Workbook or related tools to grantees of other juvenile justice programs.

Several States are providing communities with local evaluation technical assistance support and training. For example, Colorado, Pennsylvania, and Michigan have TA and training contracts with local research and evaluation experts. Available to help communities meet their local evaluation needs, as well as Federal and State requirements, local TA and training providers have made significant progress in replacing evaluation fears with skills. To help communities develop measurable outcomes, Iowa is producing a set of evaluation protocols. Other States, including Washington and Nevada, have helped communities anticipate evaluation needs by requiring them to set aside a certain percentage of their grants for evaluation support.

Communities also are developing their own strategies to help overcome evaluation limitations. Using existing community-based resources, or creating their own, communities are securing the assistance they need to develop and implement evaluation plans. Some communities choose to use a portion of their grant money to hire a local evaluator. Others take advantage of local colleges or universities, seeking assistance from graduate students or faculty who are often willing to take on the evaluation as a collaborative research project. Still other communities recruit PPB members who possess evaluation expertise. Additionally, communities are taking advantage of available evaluation tools including surveys, data books, and supplemental materials designed to foster local evaluation capacity by groups such as DRP (Communities That Care) and the Search Institute (Youth Asset Model). A vast amount of information and other supplemental materials and resources are also available to communities through the Juvenile Justice Clearinghouse, OJJDP's information center for juvenile justice professionals, practitioners, policy makers, and the public.

5.3 Demonstrating the Power of Outcome Data

Despite the challenges of data collection and evaluation, communities have developed and implemented evaluation plans to help them monitor change in outcomes and risk factors. And, in evaluating their efforts, communities are demonstrating that through comprehensive prevention efforts, community change can, and does, occur. Local grant and evaluation reports indicate:

• The Gateway Initiative in Clinton, Iowa, implemented the Families and Schools Together (F.A.S.T.) program at nine local sites and experienced a 37 percent decrease in school behavior problems in the first program year, and a 31 percent decrease in the second program year.

• Of the 343 elementary school children served over the last three years by the Neighborhood Centers in Thurston County, Washington, pre-post test comparisons indicated that 66 percent of the students were "much improved" on academic achievement, and 25 percent were "much improved" in overall behavior.

• The School Reentry Program in Clark County, Washington witnessed a 39 percent decrease in gang involvement in participating students from 1995 to 1998.

• In Jackson, Tennessee, schools reported a 23 percent improvement in overall grade point averages for the 57 students consistently participating in the multi-agency delinquency prevention program and also a decrease in overall school suspensions. Additionally, vandalism cases in the targeted area dropped from 84 in the first year to 64 in the second.

• Following reassignment of police from patrol to a community resource center, residents in Ponca City, Oklahoma showed marked improvement in their fear of crime and perception of the police department. For example, after the initiative, less than one third (32%) of households surveyed indicated that they were "more afraid of crime than I have ever been" as compared to half (49%) before. More residents (83% versus 56%) agreed that the "police department does the best job it can against crime in this neighborhood."

• The Family Access Network -- school-based centers for child and family services -- in Bend, Oregon improved measured behaviors in 11 percent of the 2nd graders served by their mentor program. At one school, the peer mediation training program reduced discipline referrals by 75 percent.

• Students participating in the law-related education program in Layton City, Utah were less likely to be arrested for theft than were a control group. In addition, reported gang involvement was reduced by 37 students from year 1 to year 2.

1998 Report to Congress: Title V Incentive Grants for Local Delinquency Prevention Programs	OJJDP Report