Vera Inst of Justice Capital Change a Process Evaluation of Dc Juvenile Placement Reform 2011
Download original document:
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
CENTER ON YOUTH JUSTICE Capital Change: A Process Evaluation of Washington, DC’s Secure Juvenile Placement Reform Reagan Daly Tarika Kapur Margaret Elliott Vera Institute of Justice January 2011 © 2011 Vera Institute of Justice. All rights reserved. Additional copies can be obtained from the communications department of the Vera Institute of Justice, 233 Broadway, 12th floor, New York, New York, 10279, (212) 334-1300. An electronic version of this report is available for download on Vera’s web site, www.vera.org. Requests for additional information about the research described in this report should be directed to Reagan Daly at the above address or to email@example.com. Executive Summary A growing body of research conducted over several decades has persuaded most experts and many practitioners that punitive responses to juvenile offenders—particularly those placed in secure facilities—yield poor results for the youth involved and for public safety. Informed by this consensus, in 2005 officials in Washington, DC, began planning a comprehensive reform of the District’s responses to youth in secure placement (equivalent to prison in the adult context). By 2006, the District’s Department of Youth Rehabilitation Services (DYRS) began overhauling practices at its troubled Oak Hill Youth Center, a placement facility known for harsh and inhumane treatment of confined youth. By May 2009, DYRS closed Oak Hill and opened New Beginnings, a new secure facility designed to facilitate a Missouri-inspired treatment model that emphasizes group process, cooperative relationships between youth and staff, and positive youth development. With support from the Edna McConnell Clark Foundation, in April 2009 researchers from the Vera Institute of Justice’s Center on Youth Justice (CYJ) started a year-long process evaluation of the DYRS secure placement reforms. Drawing on documentation supplied by DYRS, as well as interviews and focus groups with DYRS leaders; personnel from New Beginnings; DC family court judges; trainers from the Missouri Youth Services Institute (MYSI), who served as consultants in the reform process; and staff from the Maya Angelou Academy (the school at New Beginnings), CYJ researchers sought to map DYRS’s reform strategy and identify factors that either negatively or positively affected implementation. The authors found that DYRS accomplished a tremendous amount during the period under study (2005 through mid-2010). In addition to closing Oak Hill, the agency established a new treatment model grounded in therapeutic principles and designed New Beginnings to embody those principles in both the programming and the physical environment. Despite the many accomplishments, however, a set of reforms as comprehensive in scope as that which DYRS began takes time to fully implement. Indeed, at the time of the study, implementation was an ongoing process and participants agreed that there was work still to be done to fully integrate the changes into standard practices at the facility and DYRS’s larger continuum of placement services. Below are some of the principal findings about the state of implementation, as of April 2010, and the factors advancing or hindering a cultural shift in the way the agency works with youth and families: The current state of implementation: The majority of programming elements included in the design of the reforms were being practiced regularly by staff at New Beginnings. The new education program received universal praise from facility staff and administrators, and the school played a large role in instituting the therapeutic treatment approach at the core of DYRS’s reform philosophy. Both behavioral (mental) health and school staff expressed a desire to be more integrated into daily programming. Vera Institute of Justice iii Both line staff and managers expressed concern that some elements of the reforms were being implemented inconsistently across the six model units (or groups) at New Beginnings. Juvenile justice system stakeholders expressed concerns that community-based aftercare services for youth released from New Beginnings were not well-implemented and that these implementation challenges negatively affected progress youth made while in secure placement. Factors affecting cultural change: External political forces—in particular support from elected officials and the creation of the Blue Ribbon Commission—were critical for initiating and implementing secure placement reforms. Strong, consistent leadership at both the senior- and middle-management levels helped ensure implementation of the reforms. Line staff varied in their acceptance of and investment in the reforms. While some had accepted the new therapeutic treatment approach and others were firmly against it, a large number of staff fell somewhere in the middle. Among the factors contributing to attitudinal differences among staff were their grasp of reform principles, variation in skill sets, and feelings of devaluation by senior management. Staff perceived variation in youth acceptance of the reforms as well. Staff said that while some youth were engaged in, and positively affected by, the group process, others were not. One of the most significant challenges to the implementation process was the temporary housing (in New Beginnings) of youth awaiting placement in an alternative setting. Staff felt that these youth—because they were not integrated into the new structure and approach of the facility and did not have to follow the same rules as other youth—were highly disruptive to the therapeutic group process. Given these findings, the authors recommend that DYRS enhance and build upon the reforms by developing alternative solutions for housing youth awaiting placement; providing more ways to connect staff to the agency philosophy and organizational culture; providing additional and more frequent training; better integrating school staff and behavioral health staff into New Beginnings programming; and formally documenting, in more detail, the New Beginnings program design. Finally, the authors recommend that, as a natural follow-up to this study, the agency conduct an outcome evaluation of the reforms (to assess public safety and positive youth development outcomes) as well as a process evaluation of the range of community-based alternatives to secure placement. Vera Institute of Justice iv From the Center Director Fortunately, we find ourselves in a time when increasing numbers of states and localities, heeding social scientific research on juvenile justice practices, are acknowledging that punitive, prison-like facilities for children are ineffective and often incredibly harmful. Juvenile justice officials from many of these same jurisdictions have visited Missouri, where a therapeutic, supportive approach to working with youth in custody has resulted in low recidivism rates and positive outcomes for youth, their families, and communities. But when officials return home, despite their praise for the Missouri model, few have taken on the daunting and politically charged task of transforming their own system. After a state or locality has decided that juvenile correctional reform is needed, what does the process of designing and implementing local change on the ground look like? This report explores that question by looking at one jurisdiction that went beyond asserting that reform was necessary and took the tough step of putting that assertion into practice: The District of Columbia. This process evaluation focuses on one critical aspect of DC’s reform effort: how the District’s Department of Youth Rehabilitation Services redesigned juvenile treatment, care, and rehabilitation facilities and practices for youth in secure custody following a court sentence, inspired and significantly informed by the Missouri model. Reform can be a long, arduous road, especially in a place such as DC where juvenile corrections has long been plagued with lawsuits and unacceptable conditions of confinement. While this report does not evaluate the outcome of DC’s reform effort, it does provide an objective analysis of the process, as well as an examination of the successes and challenges the District faced during that process. We hope the analysis that follows will help the District enhance its juvenile justice reform work and inform other jurisdictions as they embark on similar efforts. Annie Salsich Director, Center on Youth Justice Vera Institute of Justice v Acknowledgments This report would not have been possible without the support of many individuals. First, we would like to thank the Department of Youth Rehabilitation Services (DYRS) for its cooperation in and openness to this work. We would especially like to thank DYRS staff—and other DC stakeholders—who participated in interviews and focus groups for the study, and Barry Holman, director of the agency’s Office of Research and Quality Assurance, for his support and expertise along the way. Additionally, we are grateful for the contributions of our colleagues at Vera. Annie Salsich and Dan Wilhelm provided guidance and feedback throughout the drafting of the report, and our Communications Department—especially Alice Chasan, Robin Campbell, and Abbi Leman—were exceptional in helping us structure and edit the document. Finally, we would like to thank the Edna McConnell Clark Foundation for its generosity in funding this work. Vera Institute of Justice vi Table of Contents Introduction .................................................................................................................. 1 Background: Events Leading to Juvenile Justice Reform in DC ........................................... 3 Methodology ................................................................................................................. 5 Policy and procedure review ...................................................................................... 6 Interviews and focus groups ...................................................................................... 6 Findings ........................................................................................................................ 9 DYRS’s strategy for reform ........................................................................................ 9 Components of the reform design ............................................................................ 13 Assessment of implementation ................................................................................. 15 Implementation as an ongoing process ..................................................................... 17 Factors affecting cultural changes ............................................................................ 19 Recommendations ....................................................................................................... 26 Operations recommendations................................................................................... 26 Staff development recommendations ........................................................................ 27 Research and information-management recommendations ......................................... 30 Conclusion .................................................................................................................. 32 Appendix 1 .................................................................................................................. 33 Vera Institute of Justice vii Introduction Since the 1990s, when the notion of youth who got in trouble with the law as “super-predators” dominated the policy-making debate on juvenile corrections, practitioners and policy makers in the field have come to recognize that the prevailing punitive model fails to ensure either a safer society or a more positive future for troubled youth.1 The evidence of poor outcomes for young people who have been confined in prison-like environments is stark: an average of 55 percent of youth released from state custody in the United States are rearrested within a year, and an average of 25 percent are re-incarcerated in adult or juvenile custody within the same period.2 Although these alarming rates mask variation among jurisdictions, research shows that institutional confinement grounded in punitive principles has a negligible—and, in some cases, negative—impact on recidivism.3 A number of jurisdictions across the United States have been responding to these findings by shifting their focus from punishment toward new ways to help youth in the juvenile justice system change their behavior. A principal part of the shift has been an increasing reliance on community-based, therapeutically oriented alternatives to institutional placement, including programming that focuses on rehabilitation, skill development, and family engagement. A second critical component has been rethinking what experience youth in institutional facilities should have to produce the desired outcomes. One of the most prominent alternative juvenile justice confinement models to emerge in this context was established by the Missouri Division of Youth Services (DYS) in the early 1980s. Known nationwide as the Missouri Model, its hallmarks are individualized treatment and group processes, small home-like facilities, cooperative relationships between staff and youth, and an emphasis on positive youth development (PYD)—a philosophy that the route to healthy adolescent development is through activities and relationships that build on adolescents’ strengths. Jurisdictions across the country have expressed interest in Missouri’s approach not only for actively engaging youth and their families in the treatment process, but also for the relatively low recidivism rates associated with it.4 A few jurisdictions have begun to adopt their own versions of the Missouri Model in recent years. Among these, the efforts of the District of Columbia Department of Youth Rehabilitation Services (DYRS), which oversees juvenile secure placement in Washington, DC, may be best 1 Richard A. Mendel, The Missouri Model: Reinventing the Practice of Rehabilitating Youthful Offenders (Baltimore, MD: The Annie E. Casey Foundation, 2010). 2 H. N. Snyder and M. Sickmund, Juvenile Offenders and Victims: 2006 National Report (Washington, DC: U.S Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention, 2006). 3 Mark Lipsey, “The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A MetaAnalytic Review,” Victims and Offenders 4, no. 2 (2009): 124-147; Mark Lipsey and Frances T. Cullen. “The Effectiveness of Correctional Rehabilitation: A Review of Systematic Reviews,” Annual Review of Law and Social Science, 3 (2007), 297–320. 4 Only 22.5 percent of youth released from DYS in 2005 were re-incarcerated for a new offense or a violation within three years (Mendel 2010), and an average of 7 to 8 percent of youth are re-committed to DYS annually (Snyder and Sickmund 2006). Note that DYS measures recidivism as re-incarceration rather than rearrest. Vera Institute of Justice 1 known.5 DYRS undertook a series of significant institutional changes as part of a comprehensive set of reforms intended to transform itself into an agency that “improve[s] public safety and give[s] court-involved youths the opportunity to become more productive citizens by building on the strengths of youths and their families in the least restrictive, most homelike environment consistent with public safety.”6 Beginning in 2006, DYRS extensively reorganized Oak Hill Youth Center (Oak Hill), DC’s 220-bed facility for youth committed to secure placement. In addition to overhauling the physical building, agency administrators brought in consultants from Missouri to train staff in therapeutic and PYD approaches. They also contracted with the See Forever Foundation, which runs charter schools in DC, to revamp and administer the educational curriculum at Oak Hill. And they gave youth and their parents an active role in the case-planning process by engaging them in the development of treatment plans and seeking their input as youth progress through the treatment stages. At the same time, DYRS developed a community-based continuum of alternatives to secure confinement, intended to serve both medium- and low-risk youth and those who return to the community from institutional treatment. In May 2009, DYRS closed Oak Hill and opened the 60-bed New Beginnings Youth Development Center (New Beginnings) to house all youth placed in secure confinement. In April 2009, with funding from the Edna McConnell Clark Foundation, the Vera Institute of Justice began a year-long process evaluation of DYRS’s five-and-a-half-year planning and implementation period of reforms to institutional placement (2005 through April 2010). The goals of this process evaluation are to document DYRS’s strategy for reform, as well as to assess the implementation of the changes, which draw heavily from the Missouri Model. In particular, Vera’s researchers wanted to understand the factors that influenced the implementation process. While this evaluation does not explore how the facility transformations have affected recidivism and other outcomes among youth in New Beginnings, it provides an objective assessment of the state of implementation and the events and circumstances that affected the process. The evaluation also provides information to DYRS that agency leaders can use to continue to develop and enhance their reforms. Moreover, on a national level, the findings from this evaluation can help inform the work of other jurisdictions looking to make similar changes—in particular by highlighting the approach DYRS employed in introducing the reforms, the components the agency successfully launched, the obstacles it faced in long-term implementation, and strategies it used to address these challenges. Finally, this process evaluation is an essential precursor to an outcome evaluation, which will be necessary to help decision makers establish whether such reforms should be regarded as evidence-based practice. With these goals in mind, Vera’s researchers compared the principles behind the secure confinement reforms and their design in DC with actual practices at New Beginnings two and a 5 “Placement” in the juvenile justice context refers to post-adjudication (or conviction) disposition (or sentencing) to an institution. DYRS also oversees DC’s juvenile detention (jail) system. 6 DYRS’s mission can be found on the agency’s website: http://dyrs.dc.gov/DC/DYRS/About+DYRS/Who+We+Are. Vera Institute of Justice 2 half years after their implementation to qualitatively assess the extent to which the reforms were implemented as designed. In areas where the practice was consistent with the design, researchers probed for elements that contributed to successful implementation; in areas where practices diverged from the plan, they explored factors that interfered with the process. This report begins with a background section that describes the historical context and impetus for the DC changes. This is followed by an explanation of the methodology, a summary of findings that includes an overview of DYRS’s reform strategy and the process the agency undertook leading to initial implementation, a detailed description of the primary components of the reforms as designed, and the main findings on how they were implemented. The findings are divided into two areas: a discussion of implementation as an ongoing process and a summary of factors affecting cultural change. The report ends with recommendations for enhancing implementation and concluding thoughts. Background: Events Leading to Juvenile Justice Reform in DC Before 2005, the Youth Services Administration (YSA)—a division of the District of Columbia’s Department of Human Services—oversaw juvenile placement. Under YSA, conditions at Oak Hill were notoriously poor. Assessments of the facility described it as dangerous, overcrowded, and dirty, with no separation of pre-trial and committed youth.7 Furthermore, according to these sources, punishment and neglect characterized the culture at Oak Hill: staff frequently responded to bad behavior by confining all residents to their quarters for long periods and often locked down entire units.8 Limited program options coupled with minimal staff training translated into a chronic failure to address young people’s developmental needs.9 During this troubled period, YSA lacked consistent leadership. Despite Oak Hill’s deplorable conditions, it took years to build momentum for change. The impetus arose in 1985, when a class-action lawsuit, Jerry M. v. the District of Columbia (Jerry 7 Jerry M. v. District of Columbia (DC Superior Ct. Civil Div. March 1985), http://www.clearinghouse.net/detail.php?id=51&search=source|general;caseName|jerry%20m;orderby|caseState,%2 0caseName;; and Oak Hill Archive Project, Georgetown University, https://digitalcommons.georgetown.edu/blogs/oakhill/; Casey Strategic Consulting Group, Analysis of the Committed Youth Population at Oak Hill Youth Center Between July 1, 2004, and June 30, 2005 (Baltimore, MD: Annie E. Casey Foundation, 2005); Grace Lopes, The Special Arbiter’s Report to the Court on the Defendant’s Progress Toward Meeting the Requirements of the Third Short-Term Deliverables Plan and the Short-Term Safety Plan (December 2006); Testimony of Senator Mike DeWine to the U.S. Senate, July 29, 2005; Testimony of Senator Paul Sarbanes to the U.S. Senate, July 29, 2005; Senate Committee on Appropriations, District of Columbia Subcommittee, District of Columbia Subcommittee Hearing on Review of Deficiencies at the District of Columbia’s Youth Services Administration, March 30, 2004, testimony of Eugene Hamilton, senior judge of the Superior Court of the District of Columbia and chair of the Mayor’s Blue Ribbon Commission on Youth Safety and Juvenile Justice Reform of the District of Columbia. 8 Jerry M. v. District of Columbia (DC Superior Ct. Civil Div. March 1985), http://www.clearinghouse.net; Grace Lopes, The Special Arbiter’s Report to the Court on the Defendant’s Progress Toward Meeting the Requirements of the Third Short-Term Deliverables Plan and the Short-Term Safety Plan (December 2006). 9 Ibid. Vera Institute of Justice 3 M.), alleged that conditions in the facility violated youths’ Fifth and Eighth Amendment rights.10 A year later YSA, along with other government agencies, entered into a consent decree that placed the agencies under the supervision of a panel of juvenile justice experts and mandated swift compliance with an extensive list of changes that included steeply reducing the number of youth in Oak Hill and developing a community-based continuum of services for those who did not require secure placement.11 While the consent decree created strong external pressure for change, however, YSA did little to improve conditions at Oak Hill over the next several years. The parties disagreed about the terms of the consent decree, and juvenile justice reform took a back seat to other local priorities.12 At the same time that efforts to bring YSA into compliance with the Jerry M. consent decree stalled, a second wave of momentum for change began to build in the District under the leadership of Mayor Anthony Williams, who took office in 1998. Juvenile justice was a priority for Mayor Williams; in August 2000 he launched the Blue Ribbon Commission on Youth Safety and Juvenile Justice Reform to take a critical look at the District’s system and identify promising and model programs that could help transform the way that youth were served.13 At the conclusion of its investigation, the commission made sweeping recommendations: reduce overreliance on detention and placement; develop a continuum-of-care system of youth services including community-based alternatives to placement; and adopt better system oversight and accountability.14 In the years following the commission’s report, Mayor Williams and the City Council continued to push for reform. When the plaintiff’s counsel in Jerry M. moved to place YSA into court receivership in December 2003, the City Council responded by passing the Blue Ribbon Juvenile Justice and Youth Rehabilitation Act of 2004.15 Among other things, the act called for 10 Jerry M. v. District of Columbia (DC Superior Ct. Civil Div. March 1985), http://www.clearinghouse.net. Ibid.; Consent Decree, Jerry M. v. District of Columbia (DC Superior Ct. Civil Div. July 1986), http://www.clearinghouse.net. 12 Nancy Lewis, “Judge, Tired of Waiting, May Sock DC with Fine,” Washington Post, July 28, 1993, p.B6; Peter Slevin, “DC Juvenile Facility Begins Needed Overhaul,” Washington Post, March 16, 1998, p.B1. 13 Senate Committee on Appropriations, District of Columbia Subcommittee, District of Columbia Subcommittee Hearing on Review of Deficiencies at the District of Columbia’s Youth Services Administration, March 30, 2004, testimony of Eugene Hamilton, senior judge of the Superior Court of the District of Columbia and chair of the Mayor’s Blue Ribbon Commission on Youth Safety and Juvenile Justice Reform of the District of Columbia. 14 Casey Strategic Consulting Group, Analysis of the Committed Youth Population at Oak Hill Youth Center Between July 1, 2004, and June 30, 2005 (Baltimore, MD: Annie E. Casey Foundation, 2005). 15 DC City Council, “Blue Ribbon Juvenile Justice and Youth Rehabilitation Act of 2004,” (Bill 15-673). Receivership refers to a process in which a judge appoints a third party to assume legal and financial power over an agency or organization. Multiple agencies, including those responsible for the public housing system, the DC jail, the child welfare system, Lorton Prison Medical Facility, and the DC mental health agency, were already in receivership by the mid-1990s. Peter Slevin, “Receivership Sought for DC Youth Prison,” Washington Post, September 11, 1998, p.B5; Senate Committee on Appropriations, District of Columbia Subcommittee, District of Columbia Subcommittee Hearing on Review of Deficiencies at the District of Columbia’s Youth Services Administration, March 30, 2004, testimony of Eugene Hamilton, senior judge of the Superior Court of the District of Columbia and chair of the Mayor’s Blue Ribbon Commission on Youth Safety and Juvenile Justice Reform of the District of Columbia. YSA did not end up being placed into receivership; the plaintiffs withdrew their filing in 2007 based on DYRS’s significant progress in improving the conditions of the secure placement system. As of December 2010, however, the agency remains under the Jerry M. consent decree. 11 Vera Institute of Justice 4 the closure of Oak Hill by 2009 and clarified the role of DC’s juvenile justice system as one of preserving and strengthening families, rehabilitating youth, and serving them in their communities in minimally restrictive settings whenever possible. That same year, the council also passed legislation—the Omnibus Juvenile Justice Act of 2004—establishing DYRS as a cabinet-level agency responsible for the oversight of juvenile detention and placement and establishing goals for the agency that coincided with those outlined in the Blue Ribbon Act.16 In January 2005, Mayor Williams appointed Vincent Schiraldi as the agency’s first director.17 Schiraldi declared his intention to deliver on the mandates of the commission and DYRS, establishing legislation by reforming the physical facilities and establishing a new system of community-based services for committed youth; that same year, he began the process of transforming Oak Hill.18 Methodology Vera conducted this study from mid-2009 to mid-2010, exploring DYRS’s implementation of its secure placement reforms from the 2005 planning stages up to April 2010—a period of approximately five and a half years. In the interest of uncovering information about implementation that would be both useful to DYRS for enhancing its program of care and informative to the planning efforts of other jurisdictions, Vera’s researchers focused this process evaluation on three main questions: 1. What was DYRS’s strategy for overhauling its secure placement system? 2. To what extent were practices among staff at New Beginnings two and a half years after their implementation consistent with the principles and procedures of the reforms as they were designed? 3. What factors influenced the quality of implementation? Vera’s researchers explored these questions as they related to each of the components of the secure placement reforms—including the transformation of the physical building; the shift in institutional culture; and implementation of specific programming elements. Researchers probed for perspectives on specific elements of the implementation process that were either productive or counterproductive, such as training, staffing structure, and communication within the agency. They also looked for general perspectives on the content of the changes, to further assess the extent to which the agency had achieved a cultural shift among staff. 16 Lori Montgomery, “Juvenile Crime Bill Signed by Williams; Parents Facing New Accountability,” Washington Post, November 30, 2004. 17 Department of Youth Rehabilitation Services (DYRS), “Councilmember Fenty Announces Confirmation of Vincent Schiraldi as the Director of DYRS,” Press release (Washington, DC: DYRS, April 5, 2005). 18 Ibid. Vera Institute of Justice 5 Given that this evaluation was intended to explore the implementation of the secure placement reforms—as opposed to evaluating their impact on recidivism and other outcomes— Vera researchers employed qualitative techniques. In particular, the methodology for this study included a review of the newly developed policies and procedures for working with youth in secure facilities, as well as an analysis of qualitative data collected in interviews and focus groups. This approach was used to ensure that the elements of the reforms outlined in DYRS’s policies and procedures informed the specific areas that were explored. Policy and procedure review As the first step in assessing implementation of the new therapeutic approach for working with youth in DC’s secure placement facilities, Vera staff conducted a systematic review of policies, procedures, instruments, and other documentation related to the reforms. DYRS provided information on its guiding principles, youth rights and responsibilities, staff trainings, and procedures, in addition to copies of administrative forms and memos circulated to staff. The agency also provided documents related to the Jerry M. lawsuit—including the consent decree and work plan—and other reports, including a 2008 recidivism study that provided background information on the organization as a whole. Vera researchers also reviewed planning documents provided by the Missouri Youth Services Institute (MYSI)—a consulting group established by Missouri Model architect Mark Steward that helped DYRS develop and implement its reforms—and City Council testimonies from the years covered by the study. Researchers used all of this information to develop a description of both the reforms’ design—including underlying presuppositions, the reconceived physical environment, and new programming elements and procedures—that would serve as a reference point for assessing how the new approach had been put into effect in the facility and the reform process that DYRS undertook to implement the design. Interviews and focus groups Vera’s researchers relied most heavily on qualitative data that were collected in interviews and focus groups conducted between August 2009 and January 2010 with people who played a significant role in the reforms’ implementation. Researchers asked about how aspects of the reforms—in particular the programming elements and the therapeutic philosophy—were being practiced at New Beginnings, the nature of the collaboration between involved parties, the successes and challenges encountered during ongoing implementation, and general reactions to the reforms. Interviews. Vera staff conducted one-on-one interviews with senior-level DYRS administrators and unit managers who supervise New Beginnings’ model units. Because administrators and superintendents were heavily involved in the decision to effect the reforms, researchers interviewed them primarily to gain an understanding of the historical context, the Vera Institute of Justice 6 implementation strategy, and how the process had proceeded to that date.19 Researchers also conducted interviews with five of the six unit managers and one union official, all of whom provided information about implementation of the reforms on the ground and how unit staff received them. Focus groups. Vera’s researchers convened in-person focus groups with New Beginnings line staff, known as youth development representatives (YDRs), school staff, and family court judges; they also conducted telephone focus groups with MYSI trainers and behavioral health staff who they were unable to convene in person.20 They conducted these groups for a number of reasons. First, they wanted to gain insight from YDRs, school staff, and behavioral health staff into how the reforms were being executed in the facility, how their day-to-day jobs were affected, and how the system changed. Second, MYSI trainers were able to provide additional perspective on the strategy, planning, and training components of implementation; and third, judges provided an external perspective on the reforms. A summary of participants by group appears in Figure 1. Figure 1: Interview and focus group participants Group Type of interview Judges YDRs 1 focus group 2 focus groups; 5 one-on-one interviews One-on-one interviews One-on-one interviews 2 focus groups 1 telephone focus group Unit managers Administrators21 School staff Behavioral health staff MYSI trainers Total 1 telephone focus group Number of participants 3 15 5 9 6 3 2 43 Participant selection. Vera’s researchers selected participants using several sampling methods. For administrators, unit managers, family court judges, MYSI trainers, and behavioral health staff, they attempted to include as many people as possible in the study. Each of these roles is significant in the development, implementation, and/or delivery of the reforms, and none of the groups contained enough people to select a random sample. 19 This was supplemented with relevant information from the document review. The groups were organized by job description so that judges were interviewed with other judges, YDRs with other YDRs, and so on. Although the intent was to conduct focus groups with YDRs, some focus groups only had one staff member, thus creating more of a one-on-one interview setting. 21 The school principal and the behavioral health staff supervisor are included in the “Administrators” category, although they heavily informed analyses of the roles of school and behavioral health staff. 20 Vera Institute of Justice 7 Because there are large numbers of line staff and school staff at New Beginnings, Vera researchers interviewed samples of people from these two groups. They used a number of methods to recruit line staff for the focus groups, including sending e-mails, calling units, visiting units to ask for participants, and requesting announcements at team meetings. To recruit school staff, they asked the principal to send an e-mail explaining the study and requesting that interested parties e-mail or call Vera’s researchers to schedule a time to participate in a focus group. Researchers offered $25 gift cards as incentives for both the line staff and the school staff groups. Although they were eventually able to conduct interviews and focus groups with significant numbers of staff members, researchers had a number of communication challenges in their attempts to recruit staff and schedule meeting times. For example, although line staff had DYRS e-mail addresses, they were not required to activate the accounts or check them regularly; as a result, it was difficult to solicit participants using that approach. The schedule at New Beginnings also made it difficult to arrange interviews and focus groups. Line staff had very little down time, and there were several miscommunications about when staff would be available to speak with researchers. Because of these challenges, Vera’s researchers were unable to sample random participants and, instead, had to rely on interviews with those who were responsive. While this is a common way of recruiting participants for qualitative work (and happens to varying degrees even when a sample is selected randomly), it is important to keep in mind that the findings presented in this report reflect only the perspectives of those interviewed and may not be reflective of YDRs and school staff more generally. Researchers were careful to note this in the discussion of interviews and focus groups, stressing the prevalence of different perspectives and opinions among those who were interviewed. It is also of particular importance when considering findings from the interviews with YDRs, of whom only 15 of approximately 85 were interviewed for the study. Legal strictures prevented researchers from interviewing youth and families for the evaluation, so there is not firsthand information about how any of them viewed the changes to institutional practices or their impact on behaviors and dynamics inside or outside the facility. This was the most significant limitation of the study—especially given that youth and family engagement in treatment was a fundamental component of the reforms. While researchers did ask YDRs and unit managers for their perspectives on youth responses in interviews and focus groups, their perceptions may not reflect those of youth who spent time in secure placement. During focus groups and interviews, Vera’s interviewers tried to capture as much of the participants’ perspectives as possible through detailed note-taking and observation (interviews and focus groups were not recorded). Vera staff then analyzed qualitative data for content and identified overarching themes by synthesizing findings across interviews and focus groups. They paid particular attention to analyzing commonalities and differences in the successes and challenges of implementation, as well as respondents’ general opinions about the utility of the reforms. Vera Institute of Justice 8 Findings This section includes the findings from Vera’s research, beginning with an overview of DYRS’s strategy to reform its secure placement system. This is followed by an outline of the reforms’ program design, which was gleaned primarily from a review of DYRS documents as well as from information gathered in interviews and focus groups. Next are findings on the reforms’ implementation, drawn exclusively from perspectives gathered in interviews and focus groups with DYRS staff and other DC stakeholders. In particular, an assessment of the status of implementation at the time the study was conducted (April 2009 to March 2010) is offered, followed by a discussion of the factors that affected the process—either positively or negatively. The report focuses on factors that influenced cultural change within and outside of the agency, for two reasons. First, cultural change emerged in interviews and focus groups as essential to the successful implementation of such a transformative set of reforms. Second, the degree to which the reform effort had changed the culture of Washington, DC’s juvenile justice system varied in different sectors of the system and produced different results in the implementation of specific program practices. DYRS’s strategy for reform It took DYRS a number of years to plan and implement the reforms that are the focus of this evaluation (a detailed timeline of the process appears in Appendix 1). To help the agency identify its target population for a reconceived Oak Hill, DYRS Director Schiraldi brought in consultants from the Annie E. Casey Foundation (Casey) to assess the characteristics of the population placed in the facility between July 1, 2004, and June 30, 2005. Their investigation, the results of which were reported in November 2005, concluded that a substantial number of Oak Hill residents did not pose a significant public-safety risk, suggesting that the District could reduce the number of youth in secure placement. Moreover, it found that those with less serious presenting offenses and/or offense histories stayed longer than those who had committed more serious offenses.22 The consultants recommended that the agency overhaul its secure placement system, upgrade the facility, and limit the use of secure placement to those youth who pose a significant risk to public safety. Informed by Casey’s data-driven recommendations, Schiraldi began the process of reforming Oak Hill. Because the intended changes were comprehensive and required a cultural transformation among staff, Schiraldi began from the top of the agency down. As a starting point, he brought in a new team of high-level administrators—including a chief of staff, deputy director, chief of committed services, chief of detained services, director of research and quality assurance, and superintendent for Oak Hill. All of the new administrators had extensive knowledge of the Missouri Model; the new superintendent came directly from Missouri’s 22 Casey Strategic Consulting Group, Analysis of the Committed Youth Population at Oak Hill Youth Center Between July 1, 2004, and June 30, 2005 (Baltimore, MD: Annie E. Casey Foundation, 2005). Vera Institute of Justice 9 system. Schiraldi also turned to MYSI in April 2005 to help his team translate the philosophical shift at the top into changes on the ground. The agency signed a five-year contract with MYSI to provide intensive assistance on a number of fronts. On a broad level, the organization helped DYRS leaders through the process of developing their mission, vision, and principles.23 The majority of MYSI’s assistance consisted of advising DYRS on the specifics of the reform design. As with the Missouri Model, the DC juvenile justice reform model was built on a foundation of therapeutic values—including personal accountability, self-exploration, human dignity, and family involvement—that are manifested in two specific therapeutic techniques: positive youth development and group process. According to DYRS officials, the reform design was based on the fundamental belief that for youth to truly change and not recommit crimes, “they must go through a process of self-exploration that addresse[s] their history, family issues and challenges, and how this has influenced their present situation.”24 The design is rooted in the youth-services approach known as positive youth development (PYD), which focuses on building strengths and competencies while addressing any deficits—such as substance-use and mental-health problems.25 PYD places great emphasis on promoting positive interactions and relationships among peers as well as between youth and caring adults. According to the PYD framework, building these supports helps young people move further away from the delinquent behaviors that led them into the juvenile justice system. Agency leaders also placed the group process technique at the forefront of their reform plan. In group process, youth are treated as units of a whole rather than a collection of unrelated individuals. They are expected to listen respectfully to each other and help maintain stability within the group. Youth are also expected to support their peers as they work through issues that underlie delinquency. Similar expectations apply to relationships between staff and youth, and DYRS renamed its line staff accordingly, replacing “correctional officer” with “youth development representative” (YDR). Consistent with these new priorities, staff were expected to use cognitive-behavioral approaches aimed at changing youth’s negative thought patterns, and— in a marked change from Oak Hill’s longstanding punitive philosophy—they received training designed to minimize the use of force in volatile situations. Finally, the new treatment approach strongly encouraged family engagement. Families were expected to be involved in their children’s treatment plan from the outset. 23 Mission statement: The mission of the DC Department of Youth and Rehabilitation Services (DYRS) is to improve public safety and give court-involved youth the opportunity to become more productive citizens by building on the strengths of youths and their families in the least restrictive, most homelike environment consistent with public safety. Vision statement: DYRS will provide the nation’s best continuum of care for court-involved youth and their families through a wide range of programs that emphasize individual strengths, personal accountability, public safety, skill development, family involvement, and community support. 24 Department of Youth Rehabilitation Services, Unit Staff Orientation and Treatment Guide (October 2006), pg. 7. 25 Missouri does not explicitly refer to PYD as part of its approach the way that DYRS does, but the Missouri Model does emphasize the importance of building on youth strengths within the group process framework. Vera Institute of Justice 10 Therapeutic values also informed DYRS’s decisions about how to overhaul the physical building and revamp in-facility programming. MYSI played a pivotal role in guiding the Oak Hill redesign. After hearing the consultants’ recommendations on the types of changes that they concluded would be most useful, DYRS began working to transform the facility in 2006. Drawing on findings from the Casey assessment on the size of the target population group, agency leaders decided to create six model units for high-risk youth and redirect lower-risk youth into community-based programming.26 As each unit was refurbished, the unit staff learned the new approach for working with youth. Each unit took about six weeks to complete and was reopened with only two or three residents so that staff had an opportunity to practice the new techniques before the unit reached capacity. The first model unit was completed in June 2006, and by September 2007 all six units were fully refurbished. In June 2007, Maya Angelou Academy, a private school funded by the See Forever Foundation, replaced the DC public schools as the provider of academic services at Oak Hill. Another crucial step toward reforming secure placement, related to the physical building upgrades, was a reconsideration of Oak Hill’s organizational structure. Under the Youth Services Administration (YSA), line staff reported to shift superintendents, whose role consisted solely of security and administrative duties. There was no consistency to shift assignments; staff members often changed their posts rather than continue working with the same group of youth and coworkers. There was also a dearth of behavioral health staff. Under MYSI’s guidance, DYRS leaders reorganized line staff into six small teams of approximately 12 people corresponding to the six newly created units at the facility.27 They established the middle-management position of unit manager to oversee staff in each unit. A behavioral health clinician added to each team would provide individual therapy and support YDRs and unit managers in the group process.28 Each team was assigned a case manager to ensure that treatment and release planning were highly integrated with a youth’s progress at New Beginnings. Finally, MYSI trainers worked with DYRS leadership to develop and implement the new treatment program. Much of MYSI’s work in this arena focused on helping DYRS reorient Oak Hill’s institutional culture toward an emphasis on a therapeutic and strengths-based approach. A large part of this assistance involved training staff in the new treatment model. DYRS required training for all staff at Oak Hill except those who worked in the school (See Forever wanted to use its own training protocol for those staff). At the outset of implementation, MYSI advisers provided intensive six-week trainings on the Missouri Model and its application to the DC system to three unit teams, as well as an overview of the new approach to all high-level DYRS 26 The target population was 60 committed youth, down from 220 in Oak Hill before the reforms. Pili Robinson, “System Transformation: The Critical Components of a Juvenile System Reform” (PhD diss., Capella University, 2010). 28 DYRS officially took over the delivery of behavioral health services from the Department of Mental Health in October 2006. City Council of the District of Columbia, March 2, 2007, testimony of Vincent Schiraldi. 27 Vera Institute of Justice 11 administrators.29 They also simultaneously taught their techniques to all DYRS training staff (three in total at the time) so that they could train the three remaining units and other staff on campus, in addition to carrying out all subsequent trainings for the agency.30 After training the first three units, MYSI suggested that DYRS trainers shorten the initial course to two weeks and present the rest of the material to staff in a series of shorter sessions over a few months so they would not feel overwhelmed. A total of 191 staff members attended the initial round of training sessions (roughly 25 participated in each session) over a total of 108 days (13 days per session). The training materials were drawn directly from those used to train Missouri’s Division of Youth Services’ staff in the original Missouri Model, with content divided into several modules: systems theory, group dynamics, assessing and moving groups, correctional vs. treatment approach, group facilitation, human dignity, professional boundaries, reshaping behavior, crisis management, and experiential learning.31 MYSI consultants also provided hands-on assistance and coaching to unit staff every day for the first three years of implementation to help them apply the principles they learned in the initial training; and they trained unit managers to run their own refresher sessions for their units.32 The MYSI consultants worked with DYRS administrators to update policies and procedures, establish programming elements, and develop orientation manuals for staff and youth. DYRS leadership supplemented the assistance provided by MYSI with a number of independent steps to engage staff in the therapeutic model. When the agency reorganized Oak Hill staff into unit teams, Schiraldi replaced 11 mid-level managers because their backgrounds were not in line with the agency’s new approach, and, consistent with the DYRS Establishment Act, he later required that all newly hired staff have at least 30 college credits (as opposed to a high school education, the agency’s previous standard).33 He also began holding regular allmanagers meetings with DYRS leadership and middle management in 2006 to encourage collaboration and communication between the two groups. There would be new mechanisms as well for eliciting feedback from line staff and outside parties. Among them were weekly all-staff meetings, run by staff themselves but attended by administrators to communicate with them and hear their comments; monthly town hall meetings, which were led by Schiraldi or another administrator and gave individuals the opportunity to voice concerns and ask questions in a broader forum; community meetings, hosted by agencies such as the Consortium for Youth 29 All staff from the first three units, including YDRs and unit managers, as well as shift supervisors and case managers, attended the initial training sessions. Director Schiraldi’s executive team attended only the planning sessions and the overview of the reforms conducted by MYSI. 30 During the initial training, DYRS training staff consisted of three trainers. DYRS now has four trainers, and MYSI trained all of them. 31 Oak Hill Strategic Plan, April 2007. 32 MYSI consultants gradually decreased their coaching assistance over time, though they remained a strong presence at New Beginnings throughout their five years of work with DYRS. In 2010—the last year of their contract—they were still there 12 days out of every month. 33 Pili Robinson, “System Transformation: The Critical Components of a Juvenile System Reform” (PhD diss., Capella University, 2010). Vera Institute of Justice 12 Services, which represented contracted providers; and weekly visits by Schiraldi to Oak Hill to check in with staff there.34 DYRS completed its rollout of the reforms with the opening of Oak Hill’s sixth model unit in September 2007. The agency continued to refine and expand on its new system. Most notably, in 2008, the agency began construction on New Beginnings—a state-of-the-art facility designed to ensure security and foster a sense of well-being and nurturing for youth. New Beginnings’ physical environment represented a significant departure from the cell-like living quarters and restrictive conditions at Oak Hill. New Beginnings is organized into small units designed to hold no more than 10 residents, with a separate bedroom for each child (there are six units in total at the facility). Each unit includes a common area with a television and other furnishings, as well as an outdoor patio where young people can spend supervised free time. Additionally, the facility would have a gym, a school, workshop areas, and an auditorium. All these elements of the physical design were intended to promote relationship-building between youth and staff and among the young people. The $46 million facility, which opened in May 2009, helped solidify DYRS’s new approach to serving youth. MYSI trainers continued to provide on-the-ground assistance to unit managers and YDRs at Oak Hill and New Beginnings (their contract ended in October 2010), and agency leadership continued to refine procedures and policies as they received feedback from meetings and focus groups with staff and among themselves. Components of the reform design To assess how well DYRS’s reforms were implemented during the time under examination, Vera’s researchers first needed to understand what the reforms were intended to look like. More specifically, they needed to develop an understanding of the program as it was designed to promote the core principles and techniques of the model (such as the promotion of human dignity and a focus on group process) so that it could be compared to the actual practices described in staff interviews and focus groups. To examine the design, researchers relied primarily on documentation provided by DYRS, including staff orientation guides, agency procedures, and memos. Because the written materials on the agency’s new therapeutic approach were limited, however, researchers supplemented its documents with descriptions from interviews with agency staff and administrators wherever possible. The examination revealed four main programming components: group meetings, the level system, education, and recreational activities. Each of these design components is described in more detail below. Group meetings. The first element of the reform design conceived that residents would participate in daily group meetings that focused on developing positive peer relationships, healthy interactions between youth and adults, skill building, and self-awareness. Each meeting would tackle a specific topic relevant to the group. Either the staff or the youth could choose the topic, 34 City Council of the District of Columbia, March 2, 2007, testimony of Vincent Schiraldi. Vera Institute of Justice 13 including such subjects as resisting peer pressure or anger management. One meeting a week would be devoted to substance-abuse treatment, using a cognitive-behavioral approach known as the “Seven Challenges” to help youth address any issues they experienced in this area.35 In addition to scheduled meetings, youth and staff would be able to call impromptu meetings known as circle-ups whenever they wished. Developed as part of the Missouri Model, circle-ups were designed as a response to a group member’s need to discuss a concern or to work though a group crisis. Staff and youth in the unit would form a circle and talk openly until they feel it had been adequately addressed. The technique was structured to confront issues early, before they lead to larger problems in the unit, and to foster group support. Youth or staff could also call a circle-up to acknowledge a resident’s success or progress, such as getting a good grade on a test or having a positive interaction with a parent during visiting hours.36 The level system. New Beginnings introduced the level system to help youth better understand and address issues that underlie their delinquency and prepare them to return to the community. Like circle-ups, the level system was adapted from the Missouri Model and consists of six stages of self-exploration that residents must complete before they can leave the placement facility.37 The level system was designed so that all staff who have contact with a youth—collectively known as the Youth Development Team (YDT)—would make decisions about whether he has fulfilled level requirements. The team would include YDRs, unit managers, behavioral health specialists, case managers, and teachers. Family involvement would also be an important aspect of the level system, particularly in the final transition phase. As laid out in protocol, youth could petition to move to the next level after a month at a given level by presenting his case at team meetings, with preparatory help from his YDR advocate.38 The YDT would make a decision about the youth’s level during that meeting, and staff inform the youth of their decision the next day. If a youth was denied the opportunity to advance to the next level, he could request 35 The Seven Challenges is a developmentally appropriate, culturally sensitive evidence-based program designed for adolescents that focuses on critically looking at how substance use affects young people’s lives and developing the decision-making skills to make positive changes; www.sevenchallenges.com. 36 For both types of circle-ups, the group is supposed to come to a resolution or conclusion through the following six steps: (1) A young person or staff member calls a circle-up. The group stops whatever they are doing and comes together in a safe place. (2) Staff ensure that the dynamics in the circle are safe. (3) Staff reaffirm the group’s expectations (e.g., use “I” messages, speak one at a time). (4) The person initiating the circle-up provides the reason for doing so. (5) Staff allow the process to find a resolution without explicitly directing the process. (6) The group talks about next steps; after the circle-up, staff follow up on the next steps for resolution of the issue. 37 The six stages of the level system are the Learning Phase, when youth begin to learn about their surroundings and get oriented to the unit; the Self-Awareness/Discovery Phase, when youth work on learning about themselves and communicating clearly with others; the Self-Exploration Phase, when youth continue working on learning about themselves, including destructive patterns of behavior; the Growing Phase, when youth work on making constructive changes, such as clearly communicating their feelings and learning more productive ways of coping; the Collective Work and Responsibility Phase, when youth accept more responsibility in the unit, identify successes and challenges in changing their destructive behaviors, and begin to plan for reentry; and the Transition Phase, youth work with their staff, their social worker, and family members to finalize the details of their reentry plans, including education or employment arrangements. 38 Each resident is assigned a YDR to serve as his advocate, who is expected to develop a supportive relationship with the youth and help guide him through the program successfully. Vera Institute of Justice 14 to petition again at any subsequent team meeting and could work with staff to better prepare for that meeting. Education programming. In June 2007, the See Forever Foundation, which runs the charter schools in the District of Columbia, established the Maya Angelou Academy to provide education services at Oak Hill and subsequently at New Beginnings. As conceived, the school would have a staff that includes a principal, teachers, teaching assistants, and school advocates who would work closely with specific youth and represent the school on the Youth Development team.39 New Beginnings youth—called scholars in the education plan—would attend school with their units to reinforce group cohesion regardless of the range of ages and grade levels within the unit. Teachers’ assistants and YDRs would provide individual help for young people who were behind grade level or needed specialized attention. The curriculum would be structured along traditional subject lines, but eight month-long cross-curricular segments would stress specific themes (such as relationships, change, choice, power, justice, freedom, dreams, ethics, and systems). The plan also included a GED program as an alternative route to earning a high school diploma. School staff would use skill-building exercises, such as mock interviews, to develop and strengthen scholars’ core competencies. The design established a reward system to give students stars for each display of positive behavior, such as respect toward another student. Recreational programming. Productive recreational activity was a main program component for New Beginnings, both during the school day and after school in the units. School staff would organize activities such as metal shop, poetry sessions, and theater. Youth would have time every afternoon to play sports, such as basketball or football, and had constructive down time after dinner, during which they could play games and engage in group activities. Residents also frequently would have the opportunity to participate in outdoor recreational activities, such as rafting, fishing, caving, climbing, and biking. Assessment of implementation Overall, this study found that DYRS has accomplished a tremendous amount in the five and a half years since its reform planning process began. Not only did the agency succeed in closing Oak Hill, notorious for its terrible conditions and lack of programming; it established a new treatment model grounded in therapeutic principles and opened New Beginnings Youth Development Center, a facility where these principles are embodied in both the programming and the physical environment. Nevertheless, a set of reforms as comprehensive in scope as that which DYRS began takes time to fully implement. Indeed, in interviews and focus groups DYRS leaders said that despite the accomplishments to date, implementation was an ongoing process and there was room for improvement in the integration of the changes into both standard 39 Each unit has one designated school advocate, who works with the youth on developing educational goals and transitioning back into community schooling upon release. They are distinct from the YDR advocates, who act more as mentors. Vera Institute of Justice 15 practices at the facility and DYRS’s larger continuum of placement services. They also acknowledged that the reforms were evolving as they gained more insight into policies and practices that were working and those that needed rethinking. Broadly, the interviews and focus groups conducted by Vera researchers led to findings in two main areas: the current state of implementation (an ongoing process) and factors affecting the current state of implementation through cultural change within and outside of the agency. Below are some of the key findings. The remainder of this section describes these findings in more detail. The current state of implementation: The majority of programming elements included in the design of the reforms were being practiced regularly by staff at New Beginnings. The new education program received universal praise from facility staff and administrators, and the school played a large role in instituting the therapeutic treatment approach at the core of DYRS’s reform philosophy. Both behavioral health and school staff expressed a desire to be more integrated into daily programming. Both line staff and managers expressed concern that some elements of the reforms were being implemented inconsistently across the six model units at New Beginnings. Juvenile justice system stakeholders expressed concerns that community-based aftercare services for youth released from New Beginnings were not well-implemented and that these implementation challenges negatively affected progress youth made while in secure placement. Factors affecting cultural change: External political forces—in particular support from elected officials and the creation of the Blue Ribbon Commission—were critical for initiating and implementing secure placement reforms. Strong, consistent leadership at both the senior- and middle-management levels helped ensure implementation of the reforms. Line staff varied in their acceptance of and investment in the reforms. While some had accepted the new therapeutic treatment approach and others were firmly against it, a large number of staff fell somewhere in the middle. Among the factors contributing to attitudinal differences among staff were their grasp of reform principles, variation in skill sets, and feelings of de-valuation by senior management. Staff perceived variation in youth acceptance of the reforms as well. Staff said that while some youth were engaged in, and positively affected by, the group process, others were not. Vera Institute of Justice 16 One of the most significant challenges to the implementation process was the temporary housing of youth awaiting placement in New Beginnings. Staff felt that these youth— because they were not integrated into the new structure and approach of the facility and did not have to follow the same rules as other youth—were highly disruptive to the therapeutic group process. Implementation as an ongoing process Our interviews and focus groups with New Beginnings staff revealed that the model units were using the majority of programming elements that DYRS introduced during the initial years of the implementation. For example, most YDRs mentioned that they conducted circle-ups at least three times a day—once in the morning before youth left for school, once during the school day, and once in the evening. Staff also felt that there was an understanding among their team members and youth that circle-ups could be called at any time of the day and said that they were being used with increasing frequency across units. Generally, YDRs expressed that the emphasis on group work had helped bring about change in youth demeanor and attitudes, and the group process helped youth recognize and begin to address their personal issues. Staff also stated that the group process encouraged positive youth interaction and that residents were beginning to trust one another more. Additionally, despite the varying opinions among staff about the value of the level system, all units were using it. There was an hour of structured recreational time built into the daily schedule, and DYRS had established specific activities, such as the Shakespeare Project—a series of drama workshops—and the Guns to Roses art program, to expose youth to healthy forms of self-expression. The Shakespeare Project was established to prepare residents to perform Shakespeare productions in the community. In the Guns to Roses program, youth weld sculptures using metal from seized firearms. At the time the study was conducted, there were already several sculptures on display from the project. Finally, facility staff and DYRS leaders said that there had been a number of field trips for youth at the highest levels of the level system (levels five and six). These trips included white-water rafting, cave-climbing, and camping. The establishment of an expansive education program was perhaps DYRS’s greatest implementation success during the period studied. School staff regarded the See Forever School’s philosophy as very much in line with the reform design, and many of the school staff who were interviewed felt that See Forever played a significant role in instituting the therapeutic treatment approach that was at the core of the reform philosophy. School staff, along with unit managers, also felt that they might be in the best position to continue spreading the reform culture at New Beginnings. They gave a number of reasons for this optimism, the first of which was that See Forever was independent of DYRS and thus had considerable control over school staffing and programming. School staff also reported that they benefitted from having a strong principal who was fully supportive of the therapeutic and PYD ideals of the reforms and promoted these ideals among staff. They offered the example of the use of “scholars” rather than Vera Institute of Justice 17 “students” to refer to youth at the school, which they believed contributed to a more positive self-image. School staff also saw themselves in a position to play a pivotal role in facilitating cultural change at New Beginnings because they worked with youth intensively. They said that because the young people spent more of their day at school than in the units, teachers and other school staff could compensate for some of the YDRs’ apprehension about implementing the new model (discussed at length below). Finally, and perhaps most important, school staff said they had the opportunity to affect cultural change through their interactions with YDRs. According to them, when See Forever first came into Oak Hill in June 2007, most YDRs were very resistant to the therapeutic approach. Many of the school staff interviewed for this study felt that cultural change had been happening—albeit slowly—among New Beginnings unit staff over the past few years. Most of the school staff thought that strengthening inter-staff relationships through such activities as joint training and meetings would be mutually beneficial to both YDRs and themselves. Despite the integration of most core program design elements into practice at New Beginnings, administrators and staff who participated in the interviews and focus groups identified a number of design aspects that had not been implemented entirely according to the plan. For example, both DYRS administrators and staff said that behavioral health staff were not nearly as integrated into the daily routines of youth at the facility as the program design prescribed. Administrators intended for licensed clinicians specializing in behavioral health to play a critical role in the implementation of the correctional reforms by coaching YDRs and unit managers in therapeutic techniques and working directly with youth. In practice, however, it was difficult for behavioral health staff to establish a significant presence on the facility campus. The behavioral health staff said that they could play a larger role in training unit staff—especially those who were resistant to the approach—in therapeutic techniques. Additionally, a number of school staff mentioned that they would like to work more closely with behavioral health staff, particularly through integrating psychosocial education classes into the school system. However, they expressed concern that behavioral health staff were already spread too thin and thus could not have the level of involvement with youth and other staff described in the program design. Another area where program practice fell short of design, according to some of the people interviewed and focus group participants, was consistency in the implementation of reform practices such as crisis management and level assessments. Both YDRs and unit managers expressed concern that some elements of the reforms were being implemented inconsistently across the six model units at New Beginnings. While there was recognition that units would not operate identically, staff said that differences extended beyond nuances to the expectations that staff placed on youth. For example, a number of staff mentioned that units varied in their standards for passing youth between levels; some required them to demonstrate true personal growth while others passed them as long as they met the checkbox criteria for completing a level. Other staff had similar observations about variations in the circle-up process: some units Vera Institute of Justice 18 challenged youth to open up during meetings, while others engaged in more superficial dialogue.40 Finally, a number of study participants expressed concerns about the implementation of the continuum of community-based placement options that were introduced at the same time as the reforms at Oak Hill. In particular, family court judges, school staff, and YDRs all said they worried that the continuum of options was not fully implemented as designed and that, as a result, youth in the community were not getting the needed range of services. Furthermore, they raised concerns that the lack of adequate oversight in the community led to consequences for youth in New Beginnings. First, they feared that youth who returned to their communities after spending time in New Beginnings did not get necessary aftercare support, leaving them vulnerable to falling into old patterns. School staff mentioned that they saw youth coming back to New Beginnings for relatively minor reasons such as not attending school or failing to appear in court. YDRs said that youth who returned to New Beginnings were less trusting because they had not received the services they were promised in the community. Judges felt that many youth were recycling through the system because wrap-around services (services youth receive once they are released into the community) were inadequate. Most of the people interviewed said that better case management was critical for providing the support youth need upon returning to the community. Factors affecting cultural change As outlined earlier, the purpose of this process evaluation is to identify the factors related to the implementation of reforms that sought deep cultural change in the juvenile justice system, particularly those that improved or impeded the process. This section presents the findings from this inquiry, drawn from the perspectives and experiences of a diverse group of juvenile justice stakeholders in the District who were involved in or observed the reform process in some capacity. The findings are organized into four broad categories: external commitment to juvenile justice reform; senior and middle management’s investment in the reforms; staff responses to the reforms; and staff views on youth responses toward the reforms. Within each of these categories, both the factors that contributed to and presented obstacles to the implementation process are discussed. External commitment to juvenile justice reform. One of the primary themes that emerged in interviews and focus groups with DYRS administrators and MYSI trainers was the importance of political will—both outside DYRS and within the organization—for initiating and implementing the juvenile justice reforms. Their comments focused on the critical role that agency staff and external supporters played at different stages in the reform process. While external support was critical at every point, it emerged as a particularly important factor in the early stages, when the District government was trying to build momentum for the juvenile justice reforms. 40 Because the quality of circle-ups is a function of both staff and youth engagement, the perceived inconsistencies may not be entirely attributable to the way that staff ran the sessions. Vera Institute of Justice 19 According to DYRS leaders, two types of external political factors were critical for launching correctional reforms: elected officials’ support for juvenile justice reform in the years immediately preceding Schiraldi’s appointment and the creation of the Blue Ribbon Commission in 2000. Support from DC government officials, including Mayor Anthony Williams and the City Council, helped push juvenile justice to the top of the reform agenda and led to the creation of a departmental structure in DYRS that would effect change. Senior management who were interviewed also said that Schiraldi and his DYRS staff were able to leverage the political pressure that resulted from the Jerry M. lawsuit, allowing for bold change rather than just small scale improvements. While the Jerry M. consent decree had been in place for years before DYRS was created, they said that it became a driving force for change under Mayor Williams’ leadership. Administrators and MYSI trainers also cited ongoing support for the reforms from Mayor Adrian Fenty, who was a member of the City Council and the chair of the DYRS oversight committee when the reforms were first introduced.41 At the same time, according to DYRS administrators who spoke to Vera’s researchers, external political forces also created challenges for the agency in implementing its reform model. The conditions of the Jerry M. consent decree limited flexibility in the reform process, and at times DYRS was more focused on complying with the decree than on independently assessing what changes would best meet the needs of the youth. DYRS administrators were hopeful that this dynamic would change when the agency was no longer under the consent decree. In addition, while juvenile justice experts and children’s advocates around the nation praised DYRS’s reform efforts, some politicians, DYRS workers, and members of the general public voiced skepticism about the reforms’ effectiveness and ability to ensure public safety. The intense debate in the press has provided DYRS with both supporters and critics. DYRS also faced challenges collaborating with some DC agencies—in particular the Superior Court—during the time of this study. Both DYRS administrators and judges interviewed for this evaluation acknowledged that they often had different views about how to best serve youth in the District’s juvenile justice system, the types of youth that are most in need of secure placement, and the adequacy of the placement options. Among the concerns voiced by judges who participated in the study was that there were not enough residential placement options for DYRS youth—particularly for females, who must be sent out of Washington, DC, if 41 As with any significant reform effort, implementation of the juvenile justice reforms has been challenging. Throughout the period under study, DYRS leadership remained essentially stable. Since January 2010, however, there have been a number of changes to the agency’s senior management team—specifically to the director position. In January 2010, after five years with the agency, Vincent Schiraldi left the director post to become the commissioner of New York City’s Department of Probation. Marc Schindler—Schiraldi’s chief of staff—succeeded him as interim director, but in July 2010 Mayor Fenty replaced Schindler with former Deputy Attorney General Robert Hildum. In September 2010 Mayor Fenty lost his reelection bid to DC City Council chair Vincent Gray, who went on to win the mayoral election in November of the same year. Hildum resigned from the director position in December 2010, and Neil Stanley—DYRS’s general counsel since 2008—was appointed interim director. At the time of his inauguration, Mayor Gray had not announced his choice for DYRS director. These leadership changes have taken place against a backdrop of intense debate that surrounded the reforms from the time that DYRS first introduced them in 2005. Vera Institute of Justice 20 out-of-home placement is deemed necessary. Both judges and DYRS leadership also acknowledged that their differences were complicated by other changes that had occurred in the juvenile justice system over the past several years. In particular, around the same time DYRS was created and Director Schiraldi was appointed, the Council of the District of Columbia passed Bill 16-161, which removed the court’s discretion to assign youth to a particular type of placement (e.g., Oak Hill, residential treatment, or a non-residential community program) and placed it with DYRS. The judges who were interviewed felt strongly that this decision-making power belongs with the court, and while DYRS administrators were not responsible for initiating the change, it nevertheless created a difficult situation for both groups. Although this tension did not impede the initial implementation of new procedures, it created difficulties in ongoing implementation. Senior staff’s and middle management’s investment in the reforms. The need for strong, consistent support at all levels of DYRS to successfully implement the reforms was a dominant theme that emerged in the interviews and focus groups. Interviews with DYRS administrators and trainers suggested that internal commitment to reform—both at the senior and middlemanagement levels—had a tremendous influence on the implementation process. According to MYSI trainers, the stability and commitment of Schiraldi’s senior administration was a critical factor in the success of the process during the period of this study. Prior to Schiraldi’s appointment, YSA directors frequently tried to introduce changes at Oak Hill, but because top leadership changed frequently, facility staff viewed the reforms as nothing more than short-term initiatives that they could wait out. In contrast, MYSI trainers believed that having Schiraldi and his senior management team in place for five years communicated to Oak Hill staff that the changes represented a new way of serving youth in the system. MYSI trainers also stressed the importance of having a senior DYRS management team that both shared the director’s vision and was responsive throughout the reform process. Schiraldi bolstered his implementation efforts by selecting a deputy director, a chief of staff, a chief of committed services, and a superintendent for Oak Hill who fully supported the reforms and were committed to their implementation. The team’s flexibility under politically charged circumstances was evident in the way the agency worked with MYSI, according to MYSI trainers, allowing the consultants to take the lead in hands-on implementation of the reforms while DYRS senior managers handled the higher-level policy questions. Both DYRS leaders and MYSI trainers emphasized the role played by middle management, in particular unit managers, in supporting the implementation. They pointed out that by working directly with line staff, unit managers were in the unique position of being able to influence people who were responsible for, as one MYSI trainer put it, “practicing reforms” on the ground. At DYRS, most unit managers were hired after the initial implementation of the reforms, which benefited the agency in two ways, according to MYSI trainers. First, because the managers were not exposed to Oak Hill’s previous practices, they were generally very receptive to the new rehabilitative approach—in fact, they were recruited for this reason. Second, unit managers Vera Institute of Justice 21 worked hard to engage and invest in their new staff, which helped foster trust and collaboration. According to MYSI trainers, this combination of characteristics—openness to the correctional reforms and engagement with staff—helped the unit managers to enlist support from many of the YDRs. MYSI trainers cautioned, however, that progress could erode if there was middlemanagement turnover, which they said underscored the importance of consistency and stability across the board. Staff responses to the reforms. While the quality of New Beginnings’ middle-management leadership emerged from interviews with DYRS leaders and MYSI trainers as one of the most important forces behind the successful implementation of the reforms, the leaders and trainers said one of the most daunting aspects of implementation was garnering support from the line staff under their supervision. Both agency leaders and MYSI trainers said that staff members were in different places regarding their acceptance of and investment in the reforms. While a number of YDRs had accepted the agency’s therapeutic approach, others were firmly against the new model, and the majority of staff fell somewhere in the middle. This was consistent with what facility staff said. YDRs who said they saw the reforms’ value seemed to have a clear understanding of the principles at their core. For example, one YDR said that the new therapeutic approach gave staff a greater personal investment in the young people in their charge. Another said that group therapy helped youth build trust and recognize issues that may affect their delinquency; yet another mentioned the value of building positive relationships with peers in group activities. Even staff who admitted they were not entirely enthusiastic about the changes recognized the reforms’ benefits. For example, a few staff members said that while using a therapeutic approach required more work on their part, they could see that it was positively changing youth behavior and attitudes and helping the young people deal with their underlying problems. Despite the openness of some staff to the changes in treatment programming, others remained firmly resistant to the reforms. According to MYSI trainers and agency leaders, the majority of these staff were unable or unwilling to adjust to the changes in their roles wrought by the new institutional culture. MYSI trainers observed that many YDRs saw themselves as correctional officers and did not understand that part of their job was to engage—not just monitor—youth under their supervision. Interviews with staff suggested that much of this resistance to the new culture stemmed from a lack of understanding or belief in the underlying principles. For example, one person who was interviewed said that “before you can do therapy, you have to do corrections.” Others voiced concerns that the new system of discipline within the therapeutic model did not hold youth accountable and encouraged them to modify their behavior only for the duration of their stay at New Beginnings. (For example, they learned not to curse in the facility, but not to see cursing as a symptom of a larger problem.) While the interviews and focus groups with agency leaders, MYSI trainers, and facility staff revealed that many staff were supportive of the new therapeutic model, and many were wholly opposed to it, they also revealed a large majority who fell somewhere in between the two Vera Institute of Justice 22 extremes. Furthermore, the study revealed different reasons for this hesitation. According to DYRS leaders, unit managers, and MYSI trainers, some line staff were hesitant because they did not have the necessary therapeutic skills. One unit manager said that YDRs did not know how to discipline residents without using physical restraints—or, put another way, they had not been able to follow the reform model’s principle of “think[ing] creatively about discipline.” While all YDRs were trained in the principles and general strategies of positive youth development and group process, those who did not have a therapeutic background often struggled to apply the lessons in real conflicts, such as mediating fights between residents in the unit or working with a youth who was acting out. Both unit managers and MYSI trainers pointed out that MYSI’s continuing on-site support had helped to address these struggles; according to unit managers, MYSI trainers’ assistance in coaching and brainstorming with staff helped minimize backsliding to punitive responses. While most unit managers and YDRs who were interviewed appreciated MYSI’s ongoing assistance, a number also said they would have liked to learn more about the use of specific therapeutic approaches during the initial trainings. Among the suggestions made by YDRs for improving these trainings were more guidance in specific techniques for intervening when two residents are fighting and more role-playing exercises. This was echoed in some of the interviews with unit managers, who mentioned that staff needed more thorough training in counseling techniques. Both MYSI trainers and New Beginnings staff also mentioned that having longer and more frequent training would be helpful. During the initial implementation phase, trainings for staff lasted six weeks, but once DYRS trainers took over training responsibilities from MYSI they were scaled back to two weeks. While MYSI trainers endorsed this decision, they stressed the importance of following the initial two-week training with a series of additional trainings to cover the remaining modules, but DYRS was not able to carry over this component of the plan because of resource limitations. As a result, staff hired after the initial training only received two weeks of training on the agency’s therapeutic approach and a brief refresher training one year later, and they were not trained in all of the modules. A number of the YDRs interviewed for the study felt that this was not enough time to learn everything they needed to do the work effectively. Additionally, unit managers, YDRs, and trainers all mentioned that refresher courses would be helpful—especially considering that many YDRs at New Beginnings were trained in 2007 during the original session. Although some unit managers conducted condensed training sessions at unit meetings—as recommended by MYSI—and on an individualized basis, the agency has struggled to train unit managers across the board to perform this role because of turnover in the position. Unit managers, YDRs, and trainers agreed, though, that comprehensive staff refresher courses would be even more effective for reinforcing the principles and practices of the reforms. That said, a number of those interviewed said that they did not want to participate in additional training to hone their skills. Another factor that appeared to inhibit some staff from being open to the reforms was their sense of being undervalued by DYRS. While most of the YDRs who were interviewed said that they could trust and talk to at least one of the senior-level administrators at New Beginnings, Vera Institute of Justice 23 they also felt that they did not have support from DYRS leadership more generally. In particular, some of the YDRs felt that their input was discounted in decision making—a view that was echoed by some of the unit managers. Others felt that even when they were consulted on an issue, their input was not taken seriously, despite the fact that they were implementing the changes on the ground. Still others expressed concerns that New Beginnings YDRs were stigmatized as a result of the problems that had existed in Oak Hill, even though many of them were not involved in the questionable practices prior to the reforms. While this was not consistent with what was said in interviews with DYRS leadership, the perception nonetheless created tension between newer YDRs and “old guards” who, according to staff interviewed for the study, felt that people wrongfully assumed they had been abusive and neglectful during the pre-reform years. Interviews with unit managers and YDRs suggested that these feelings of being undervalued in some cases stemmed from a sense that there was poor communication between facility staff and agency leadership. When the reforms were first introduced, DYRS leaders invested in intensive training for Oak Hill and New Beginnings staff to introduce them to the principles and methods; they also developed an orientation manual to guide staff on day-to-day procedures and solicited input from the group in weekly all-staff meetings and monthly town hall sessions. Additionally, Director Schiraldi visited Oak Hill on a weekly basis to check in randomly with staff, and in 2008 he established “skip-level” meetings with line officers to give them an opportunity to communicate directly with him rather than through the unit managers. Despite these initial efforts, however, many of the line staff who were interviewed felt that agency leaders subsequently did not communicate with them frequently enough or in the most effective ways. Some said that they heard about things through the grapevine rather than through official channels, and others said that they would like to have more meetings with agency leadership where they would have an opportunity to give input and feedback on facility practices. In general, YDRs felt that they had important insights into which practices and procedures worked well and which were the least effective. Some YDRs and unit managers said some staff believed that agency leadership did not invest in staff development to the extent they invested in youth development. A number of the line staff who participated in the study—including those who agreed that it is good to invest in youth— voiced this concern. For example, one of the unit managers said the administration needed to take more time to understand the staff’s abilities and limitations rather than discounting them. Several YDRs also felt that the administration’s lack of investment in them signaled to the young people in the facility that they did not deserve respect. In particular, they perceived that the administration’s attitude toward staff contributed to youths acting out because they knew there were no negative consequences for their behavior.42 Not all facility staff shared this perspective, however. Some YDRs who were interviewed said that they had established their authority with 42 In 2009, the administration started responding to some of these concerns by establishing more positive reinforcements for staff. For example, they initiated an employee of the month program, and they also gave gift cards to staff members who performed well. Vera Institute of Justice 24 youth by making their behavioral expectations clear at the outset, using the level system to hold them accountable, and consistently reinforcing the idea that they are there to help youth succeed. Staff views on youth responses toward the reforms. A primary goal of DYRS’s reforms was that youth in the secure facility would become active participants in their own treatment and rehabilitation. Legal constraints on interviewing young people at New Beginnings made it impossible for Vera’s researchers to include this as a component of the study, so they were only able to learn about youth responses through the perceptions of facility staff who were interviewed. The findings outlined in this section should therefore be accepted cautiously. Despite the view among some staff that the reforms lowered the bar on youth accountability, others who were interviewed said that young people developed a greater sense of accountability and engagement through the group process. Many cited the level system for fostering this growth. One said that the prospect of being demoted a level seemed to deter youth from acting out; another said that the group process had given youth a tool for communicating more effectively with staff. In particular, staff who were interviewed credited group work with changing young people’s behavior and attitudes toward each other. According to one unit manager, youth were learning to open up to each other during circle-ups, and as a result “street conflicts” within the units had become less frequent. (Others noted, however, that the decrease in conflict resulted from more deliberate efforts to place youth in appropriate units and improvements in the physical environment.) Nevertheless, the staff who were interviewed felt that the implementation of the new model had not resulted in active engagement by all young people in the facility. While a number opened up to each other in group activities, there were others who had not yet fully developed trust in staff or the group treatment process, and some did not want to participate at all. Staff who were interviewed offered several explanations for some youth’s resistance to the reforms. Some said that it reflected youths’ poor understanding of the goals of the treatment program. Others blamed an absence of consequences for misbehavior. Still others thought that the resistant youth were mostly those who had spent time in Oak Hill before the reforms were implemented. Perhaps the most significant barrier to residents’ acceptance of the change in the juvenile justice culture, according to several staff who were interviewed, was the presence of youth awaiting placement who were temporarily housed at New Beginnings until DYRS could find them a spot in an alternative placement option. While New Beginnings had a unit dedicated to youth awaiting placement, staff said that they were often placed in the model units because the temporary unit was full. The staff felt that this deviation from the original model, coupled with the fact that the temporary residents did not have to follow the same rules or curriculum as the long-term residents, disrupted the group treatment process and negatively affected the experience of long-term resident youth. For example, YDRs could not require youth awaiting placement to do homework or to participate in recreational activities such as theater or sports. Many facility staff said that long-term resident youth picked up on the inconsistencies and became cynical about the importance of programmatic activities. Vera Institute of Justice 25 The New Beginnings staff and others who were interviewed had varying opinions about who was best positioned to address this problem. Judges and some school staff, for example, cited a need for more beds at New Beginnings. One unit manager said that DYRS had little control over the matter, noting that the court made placement decisions. Recommendations As discussed in the previous sections, DYRS took significant steps during the period under study toward its goal of establishing a secure placement model that treats youth with dignity and respect, builds on their strengths, engages them, and keeps them safe. It is no small accomplishment to identify a new model, secure funding and commitment, and work with existing agencies and staff to implement the changes. Yet any effort as broad in scope and culturally transformative as this one will require ongoing implementation; there will always be ways to improve, enhance, and build on the policies and practices. This section presents Vera’s recommendations to DYRS for moving forward with the implementation of existing facility reforms, based on the review of DYRS policies and procedures, findings from interviews and focus groups, and current best practices in implementing criminal and juvenile justice reforms. While these recommendations are based on feedback from all of the above-mentioned resources, some are easier to implement in the short term while others are more feasible as part of a longer-term planning process. It is up to DYRS to consider the practicability of each recommendation. The recommendations fall into three broad categories: operations; staff development; and research and information management. Operations recommendation Develop alternative solutions for housing youth awaiting placement. At the core of DYRS’s new treatment approach is the group process, which is grounded in the idea that peers should play a role in helping one another explore and address needs that underlie delinquency. At New Beginnings during the period under examination, however, the temporary housing within the model units of youth awaiting placement was a significant deviation from the reform design, and it has continued to be an issue since that time. (Between April and October 2010, an average of 32 percent of youth at New Beginnings were awaiting placement, according to DYRS’s statistical reports.) Because they were exempt from the rules and program requirements of youths in placement, the presence of youth awaiting placement in the model units disrupted trustbuilding and group cohesion. The following recommendations stem from the disruptions caused by this arrangement. DYRS should take steps to separate youth awaiting placement from youth placed in model units. Besides expanding the unit reserved for the population awaiting placement at New Beginnings, the agency could accomplish this by expanding its alternative housing options, Vera Institute of Justice 26 which currently include the Youth Services Center (YSC)—the agency’s detention facility for youth awaiting disposition—and other residential options.43 Given that most of the population awaiting placement is headed to non-secure placement,44 agency leaders should consider more community-based residential options before expanding the unit at New Beginnings. Expanding temporary housing options does not address the underlying problem of too many youth awaiting placement, however. Therefore, DYRS should also examine why these youth are being placed so slowly and, based on its findings, develop mechanisms for reducing the population. To ensure that the review is thorough, DYRS should—at a minimum—examine policies and procedures for managing youth awaiting placement, conduct a case-file review for a sample of youth, and explore how cases flow from the initial court appearance to final placement to pinpoint the causes of the delays. Staff development recommendations Invest more time and resources in building relationships with facility staff. The literature on best practices in implementing juvenile and criminal justice programs shows that gaining acceptance from staff is extremely important, particularly for reforms that involve a transformation of agency philosophy and institutional culture.45 Showing staff that the administration values their opinions and invests in their development can help. Therefore, DYRS should reinvigorate its efforts to engage staff to help spur the cultural transformation that agency leaders know is necessary for successful implementation of these reforms. A follow-up interview with selected agency leaders in spring 2010 revealed that DYRS had begun once again to increase its focus on staff engagement. For example, under interim director Marc Schindler the agency created Save DYRS, a contest that encouraged staff to suggest how the agency could work more efficiently. The agency also initiated a yearly staffwide summit meeting, began conducting focus groups with YDRs, and continued the skip-level sessions between the director and line staff.46 DYRS should continue these efforts and expand them to the level of intensity that existed when the reforms were first implemented. In particular, DYRS should increase the presence of executive staff at New Beginnings and should make a point of having at least one member of the executive team present at weekly unit team meetings to build more productive relationships with staff. DYRS should also consider investing additional resources in staff as a way to further engage them. For example, the agency could provide tuition assistance for YDRs who want to pursue further education to move to the next union grade. 43 As of December 2010, these other residential options include the Pines Residential Treatment Center and Jefferson Trail Treatment Center for Children. 44 According to awaiting placement reports provided by DYRS, only 13 percent of youth awaiting placement between April 1, 2010 and October 31, 2010 (in either New Beginnings or other housing options) went to a secure facility afterward. An additional 19 percent went to an RTC. 45 A.B. Cissner and D.J. Farole, Jr., Avoiding Failures of Implementation (New York: Center for Court Innovation, 2009; prepared under grant number 2007-DD-BX-K050). 46 It is unclear whether or not skip-level sessions continued under Interim Director Hildum. Vera Institute of Justice 27 Train staff more regularly, especially in specific techniques for working with youth. When DYRS initiated reforms at Oak Hill, agency leaders recognized that they were asking staff to make an enormous shift in their approach to working with youth, one that would require a great deal of intensive education, coaching, and on-the-ground support. DYRS leaders worked closely with MYSI trainers to develop and provide extensive six-week trainings at the outset of implementation and continued to provide on-site assistance to staff in model units during the period under examination. Interviews and focus groups revealed that this training—especially the continuing support—was critical for some staff to adjust to the cultural transformation. A number of staff, however, said that they did not feel they had the tools to hold youth accountable and that while trainings provided a lot of information on the reform principles, they did not provide enough information about how to apply these principles in their interactions with youth. This, in turn, appeared to be hindering acceptance of the reforms, at least among some of the staff. Based on these findings, DYRS should expand its training protocol for New Beginnings staff, first by identifying the greatest needs among staff and then by developing targeted trainings to address these needs. A survey of staff could identify areas with which they are struggling and those in which they would like additional training support. DYRS should also work with unit managers to elicit feedback on these questions in weekly team meetings. Once the agency has a clearer sense of the areas that need to be addressed, it should offer on-site training and troubleshooting within those areas—particularly on techniques for working with youth. (A sample topic is effective techniques for ending a circle-up when the group is not making progress.) In particular, DYRS should provide supplementary training in cognitive behavioral techniques because staff repeatedly requested this support in interviews and focus groups. Other areas will undoubtedly emerge from surveys and unit meetings. In addition to expanding the content of training, DYRS should institutionalize and document the original training process MYSI recommended: an initial two-week session on the fundamentals of the therapeutic approach followed by a series of shorter trainings over a few months, with refresher courses held annually for all staff. DYRS should continue to provide initial trainings; unit managers should conduct the refresher courses, after receiving instruction in how to do so according to the original reform design. Having a longer training period would allow staff more time to process and absorb information that is completely new for many of them. Refresher sessions, in turn, would reinforce the new treatment approach and be especially helpful for YDRs who were trained in the original 2007 session. DYRS would benefit from having MYSI consultants observe these trainings so that they could make further recommendations on how to enhance the training protocol. Finally, and closely related to the second recommendation, DYRS should give unit supervisors a greater training role, not only in refresher sessions, but also in new staff training. Not only do unit managers understand and support the therapeutic philosophy, but YDRs seem to Vera Institute of Justice 28 trust them and thus may be more open to the reform philosophy when hearing about it from the unit supervisor’s perspective. Enhance the integration of school staff into New Beginnings programming. The establishment of the See Forever School at New Beginnings was one of the greatest successes of DYRS’s reform efforts during the period under examination. In fact, the findings suggest that it was the area of programming that best exemplified the therapeutic and PYD principles underlying the DYRS reforms, even though it was not drawn directly from the Missouri Model. A great deal of See Forever’s success can be attributed to the reduction in class sizes that resulted from DYRS’s reorganization of youth into units under the reforms. It can also be attributed to the strong leadership at the school—in particular, a principal who fully supported these principles and actively promoted them among staff. The findings also suggest that the school’s success is partially attributable to See Forever’s status as an independent contractor and the control over programming that goes along with that independence. Although the decision to make See Forever independent was deliberate and produced great benefits in terms of the quality of programming and the institutionalization of a therapeutic culture among staff, it also led school staff to feel disconnected from other New Beginnings programming. School staff said that because they were not trained in DYRS’s new treatment approach they did not fully understand it, despite the fact that the principles overlapped a great deal with those of See Forever. They also mentioned that there were not enough opportunities for communication and exchange with YDRs. For this reason, DYRS should more fully integrate See Forever staff into New Beginnings programming and activities, first by mandating cross-training of DYRS and school staff. DYRS staff trained in See Forever’s classroom management techniques will be better able to support school staff in educational programming. Training school staff in DYRS’s therapeutic approach, in turn, will increase the likelihood that youth at New Beginnings receive consistent treatment and help integrate the school with the rest of the programming at the facility. DYRS should also create more opportunities for school staff to strengthen their relationships with other facility staff. For example, DYRS may want to consider holding monthly meetings or brown bag sessions for YDRs and school staff that would provide an opportunity for them to discuss their work experiences and draw on each other as resources. Finally, DYRS should include school staff in communications about the reforms. This would help school staff become more familiar with the larger framework of DYRS reforms. Enhance the integration of behavioral health specialists in the model units. From the outset of the reform process, DYRS sought to give behavioral health staff a significant role in treatment. Although agency leaders strongly supported group process, they felt that DC youth needed individualized attention, and they designed the treatment program so that each unit would have a behavioral health staff person on hand to provide this one-on-one guidance. Because of a shortage of staff, however, behavioral health specialists were spread across units—with some Vera Institute of Justice 29 staffing two at a time—and they tended to work with youth who had more serious mental health needs. The behavioral health specialists with whom Vera researchers spoke also felt that their work was disconnected from other aspects of the New Beginnings program—both because their role was not clearly defined and because they did not receive the same training as DYRS staff. To more fully integrate behavioral health staff into the model units, DYRS should assign one specialist to each unit. Also, DYRS should clearly define—and expand—behavioral health staff’s role in New Beginnings programming and train them in the overarching treatment approach. These changes will not only allow behavioral health specialists to provide the individualized treatment the reform program originally intended, but it will also create opportunities for them to build relationships with staff and youth in their respective units, making them a more integral part of the group process. Research and information-management recommendations Establish formal documentation of the New Beginnings program design. As discussed in the background section, one of the challenges Vera researchers faced in conducting this process evaluation was the limited amount of documentation on both the program model at New Beginnings and DYRS’s strategy for implementing the reforms. The agency did develop orientation guides for staff and youth, but these focused on guiding principles for behaviors, such as group process and positive youth development, rather than specific procedures for carrying out elements of New Beginnings programming (what to do if a unit group cannot resolve an issue raised in a circle-up, for example). Furthermore, there was very little documentation of the strategic planning process that led to the reforms’ implementation. The lack of documentation in both of these areas meant that researchers had to rely heavily on anecdotal information and self reports from individuals within DYRS whose perspectives and/or recollections often differed. The lack of an objective resource on the program design made it difficult to systematically assess the implementation’s fidelity to the plan. More distressing, however, is the negative impact of spotty documentation on DYRS operations. YDRs and unit managers frequently reported that staff were confused about their job expectations and described inconsistency in programming operations across units (different criteria used for advancing youth through the level system, for example). They also reported a lack of clarity on recommended responses to issues that arose at New Beginnings. All of these complaints could have been eliminated—or certainly minimized—if the original program design and operation design had been documented and disseminated, consistent with best practice in implementing criminal and juvenile justice reforms.47 Going forward, DYRS ought to begin formally documenting policy and operational decisions. In particular, the agency should develop a program manual for New Beginnings that, 47 A.B. Cissner and D.J. Farole, Jr., Avoiding Failures of Implementation (New York: Center for Court Innovation, 2009; prepared under grant number 2007-DD-BX-K050). Cissner and Farole also found that formal documentation of program models in criminal and juvenile justice contributes to their faithful replication and facilitates outcome studies. Vera Institute of Justice 30 beyond describing core principles and guiding steps, outlines specific procedures in each of the main program areas, from the point at which a youth enters through the point of exit. These programming components should include but not be limited to circle-ups, group meetings, the level system, education and the role of YDRs in the school, eligibility criteria, intake and classification, and training requirements. The agency should also document future planning efforts more formally. Conduct an outcome evaluation of secure placement reforms. While the findings from this evaluation provide insight into how DYRS effected a transformation of DC’s secure placement system over the past several years, they do not reveal anything about the impact of these changes on youth outcomes. It is important that DYRS answer this question for a number of reasons—not the least of which is that knowing the effectiveness of the reforms has significant implications for how the agency should work with youth in the future. The question of whether or not these reforms work has also been the subject of much intense public debate, both in DC and nationally, yet there is currently no empirical evidence on the effectiveness of the Missouri Model or any of its variations. For all of these reasons, DYRS should conduct an outcome evaluation. Of particular interest to DYRS and other juvenile justice stakeholders in the District is examining the changes’ impact on recidivism outcomes, but given their grounding in principles of positive youth development, it would be beneficial to examine outcomes in areas such as educational achievement and employment as well. Conduct a process evaluation of the agency’s continuum of community-based alternatives to secure confinement. This evaluation focused on the implementation of DYRS’s reforms to secure confinement. In the process, however, a number of people expressed concerns about the implementation of community-based alternatives—a crucial subject, but beyond the mandate of this analysis. DYRS should conduct a similar study on the community-based component of the reforms. Such a study will allow the agency to delve more deeply into the issues that surfaced during this research and develop an understanding of how youth are being served outside New Beginnings. It will also provide a more complete picture of the status of the reform effort, including how the community and secure placement reform implementation efforts have affected each other. The additional study will also better prepare DYRS to conduct an outcome evaluation, when the time comes to do so. With these considerations in mind, the scope of the evaluation should be as comprehensive as possible, examining the implementation of the following nonsecure placement options: the Lead Entity/Service Coalition Initiative (LESCs); other community programs that fall outside the purview of the LESCs, such as multisystemic therapy; and nonsecure residential programs, including residential treatment, group homes, and therapeutic foster homes. Vera Institute of Justice 31 Conclusion The goals of this evaluation were to document DYRS’s strategy for overhauling its secure placement system and to assess the implementation of the changes—in particular the factors affecting implementation. Vera researched the early phase of the reforms’ implementation and gathered staff perspectives about conditions on the ground to provide DYRS with information it could use to enhance its new system and inform the broader national dialogue on therapeutic models of juvenile justice reform. Based on the review of DYRS background documents and findings from interviews and focus groups, it is clear that the agency has made tremendous progress in the five and a half years since beginning the reform planning process but that there remains a great deal of work ahead. DYRS’s major accomplishments during this time included closing Oak Hill, a facility known for its poor conditions and lack of programming; establishing a new therapeutic approach for working with youth; opening New Beginnings, a new state-of-the-art facility that embodies the principles of this new approach; and instituting an educational program that shares the same values. Nevertheless, it faces impediments to the full realization of its reform design, such as resistance to the new model among both staff who work with youth at the facility and youth themselves; a growing number of youth awaiting placement who are temporarily housed in the model units; and inconsistent implementation of specific programming techniques across units. To both build on these successes and address these challenges, DYRS should implement further changes in operations, staff development, and research and information management as outlined in this report. Vera Institute of Justice 32 Appendix 1: Timeline of Reform Benchmarks December 2003 Jerry M. plaintiffs file motion to place YSA into receivership. August 2000 Mayor Anthony Williams forms District of Columbia Blue Ribbon Commission on Youth Safety and Juvenile Justice Reform. March 1985 Jerry M. v. District of Columbia lawsuit begins. July 1986 November 2001 Blue Ribbon Commission issues final report. Jerry M. parties enter into consent decree. November 2004 Blue Ribbon Juvenile Justice and Youth Rehabilitation Act of 2004 is passed, mandating closure of Oak Hill Youth Center. Early 2005 Youth Services Center is opened to house youth awaiting placement who were previously held in Oak Hill. January 2005 Vincent Schiraldi is hired to lead new cabinetlevel Department of Youth Rehabilitation Services. March 2005-June 2006 Senior-level DYRS administrators are hired. March 2005 Staff from DYRS and DMH travel to Missouri to tour the system. DYRS begins contract negotiations with Missouri Youth Services Institute. Vera Institute of Justice 33 November 2005 Annie E. Casey Foundation releases Analysis of the Mid-2005 DYRS hires Missouri Youth Services Institute to consult on reforms. April 2005 DYRS Establishment Act becomes law, establishing DYRS as a cabinet-level agency. April 2005 DYRS begins reducing population at Oak Hill by moving lower-risk youth to communitybased options. Early 2006 Director Schiraldi reorganizes Oak Hill staff into permanent teams of 12, corresponding with the six proposed model units. Committed Youth Population at Oak Hill Youth Center Between July 1, 2004, and June 30, 2005. Mid-2005 DYRS develops its mission, vision, and guiding principles in consultation with MYSI. July-September 2006 Thirty-one participants attend six-week MYSI training class, Group 2. MYSI trainers teach sessions. June 2006 First model unit opens at Oak Hill. March-May 2006 Twenty-five participants attend six-week MYSI training class, Group 1. MYSI trainers teach sessions. Summer 2006 DYRS holds first annual DYRS summit to share reform efforts with staff. Vera Institute of Justice 34 October 2006 DYRS develops Unit Staff Orientation and Treatment Guide. January 2007 DYRS issues an RFP to provide educational services at Oak Hill. October 2006 DYRS creates its own Department of Behavioral Health Services and hires a director. May 2007 Twenty-five participants attend two-week MYSI training class, Group 5. DYRS trainers teach sessions. February-March 2007 Seventeen participants attend four-week MYSI training class, Group 4. DYRS trainers teach sessions. January-February 2007 Twenty-five participants attend four-week MYSI training class, Group 3. MYSI trainers teach sessions. June 2007 See Forever Foundation's Maya Angelou Academy replaces DC public schools as provider of academic services at Oak Hill. June 2007 Twenty-five participants attend two-week MYSI training class, Group 6. DYRS trainers teach sessions. August 2007 Sixteen participants attend twoweek MYSI training class, Group 7. DYRS trainers teach sessions. Vera Institute of Justice 35 December 2007 Jerry M. plaintiffs withdraw September 2007 Final model unit opens at Oak Hill. motion to place agency into receivership. October 2007 Twenty-seven participants attend two-week MYSI training class, Group 8. DYRS trainers teach sessions. April 2008 DYRS is named one of Harvard Kennedy School's Top 50 Innovations in American Government and makes it to the semifinals of the Annie E. Casey Foundation Innovations Award in Children and Family System Reform. July 2010 Mayor Fenty replaces Interim Director Schindler with former Deputy Attorney General Robert Hildum. May 2009 New Beginnings Youth Center opens. January 2010 Vincent Schiraldi resigns from DYRS to become director of New York City Probation; chief of staff, Marc Schindler, is named interim director. December 2010 Robert Hildum resigns from his position of Interim Director of DYRS. The agency’s general counsel, Neil Stanley, is appointed new interim director. Vera Institute of Justice 36