Outcomes

Evaluation Overview and Highlights

Roca had an extremely exciting year in the evolution of its evaluation system uniquely designed to honor, respect, assess, and evaluate the process of change our young people go through over time toward meaningful and sustainable outcomes and to recognize the time and effort of staff it takes to stay in it with our young people. We have rigorously stayed on the course of holding ourselves accountable to ensure we are serving our young people in ways that have real value evidenced by positive results and long term impact.

As a result of our commitment to demonstrate that working with very high risk older teens and young adults is not hit or miss, but that when we engage them where they are, physically and developmentally, spend time building trust over years as necessary, and support them to move through stages of behavior change, they can, more often than not, succeed, we are beginning to see some remarkable results.

Roca actively served 664 youth and young adults through our High Risk Youth Intervention Model in FY09. While most programs struggle with serving and retaining this population, we don’t lose them:

• 91% of the target population participants who were initially engaged in FY08 were retained through Roca’s High Risk Youth (HRYI) Intervention Model through FY09 and are still engaged in relationships and programming to support their change processes

Of young people in Transformational Relationships (Phase 1 & Phase 2):

• 84% participated in stage based life skills, education, and/or employment programming.
• 72% of participants in a Phase 2 made positive progress through the stages of change related to specific behavior changes indicated on their service plans.
• 76% engaged in educational programming made academic gains.
• 88% engaged in pre‐vocational training achieved skill gains.
• 91% being worked with toward employment obtained employment.
• 81% who successfully completed transitional employment were placed in jobs. 74% of these retained their employment.

Of FY08 & FY09 graduates (Phase 3) of Roca’s HRYI Model assessed, sustained outcomes include:

• 96% sustain constructive adult relationships.
• 84% connected to education and/or are retaining employment.
• 98% sustained decreased criminal/delinquent behavior (no new arrests)

Roca’s progress in FY09 on our evaluation and performance based management systems for our High Risk Youth Intervention Model has dramatically improved our intentionality and capacity to drive young people toward positive outcomes. This year, Roca made improvements to our customized Efforts Towards Outcomes (ETO) software to track staff efforts, include essential performance indicators, and clarify all data points for participant outcomes directly linked to our theory of change which was developed to move disengaged young people toward long term outcomes including: sustained constructive adult relationships; sustained constructive peer relationships; educational advancement; connection to post secondary and/or vocational training; employment retention; decreased arrests; decreased pregnancy rates for those under the age of 24; increased social and emotional skills.

Roca tracks each participant's individual progress toward these outcomes over long periods of time (2‐3 years), with 2 years of follow up tracking to ensure the sustainability of the behavior changes and long term retention of the outcomes. Roca is working with the Crime and Justice Institute (CJI) on an implementation and impact evaluation to demonstrate our model as an evidence based intervention that consistently moves high risk young people toward these outcomes. The Schneider Institute at Brandeis University and Abt Associates are also partners in the evaluation.