Track: Thriving and Vibrant Nonprofit Sector
Commitment: Partnering with Purpose; Measuring that Matters
North Star: Commitment to Outcomes; Strategic Partnerships

Level of Learning: Learner

This roundtable will highlight the efforts of the San Francisco Wraparound Collaborative (SFWC) to answer a deceptively complex evaluation question: Are our Wraparound services (a high-intensity behavioral health intervention provided by multiple providers within and across multiple child-serving systems) effective? This seemingly straightforward question is actually very complicated in practice, requiring the linkage of mental health, child welfare, and juvenile justice data across multiple years and service providers. In order to tackle this evaluation challenge, we leveraged a multi-system private/public partnership between: 

  • Seneca Family of Agencies, a multi-service nonprofit providing a continuum of direct services to high need children, adolescents, and families
  • San Francisco’s Department of Public Health (which holds the contract)
  • San Francisco’s Juvenile Probation Department
  • San Francisco’s Human Services Agency

Despite long-running shared interest in assessing the effectiveness of these services, only recently has this unique data-sharing partnership enabled the service provider (Seneca) to access county-held datasets and analyze its own service outcomes. The panel will present the framework for this collaboration, share the technical approach leveraged to successfully link various datasets from disparate systems, and address some of the ongoing challenges these types of collaborative efforts face. We will share results from our analyses and discuss complications in understanding and applying findings from this evaluation to improve services at our organization. After a brief presentation from both Seneca staff and San Francisco county partners on the collaborative evaluation experience, we will invite audience members to share related experiences, successes, and challenges in getting at two big questions confronting a thriving nonprofit: 

  • How do we know that our services are effective?
  • How do we best use collaborative evaluation practices to improve services?

Learning Objectives

  • How one group of public and private child-serving organizations are collaboratively sharing data to meaningfully evaluate and explore service outcomes
  • How this framework for collaboration could be applied to other jurisdictions
  • Share and learn from others’ successful approaches to collaborative program evaluation and ways to overcome challenges

Presenter(s):

  • Sarah Accomazzo, assistant director of research and evaluation, Seneca Family of Agencies
  • Leticia Galyean, executive director of data, evaluation, and strategic initiatives, Seneca Family of Agencies
  • Jesh Harbaugh, assistant director of business intelligence, Seneca Family of Agencies

Sarah Accomazzo
Assistant Director of Research and Evaluation
Seneca Family of Agencies

Sarah Accomazzo is assistant director of research and evaluation at Seneca Family of Agencies, based in Oakland, California. Her research interests include mental health assessment and service delivery in public systems and agencies serving youth and families, strengths-based practice, data-driven decision making, and implementation and evaluation of evidence-informed interventions. Her practice experience includes evaluation, administration, and consulting roles in mental health and prevention organizations and public systems. She received her master's in social work and her Ph.D. from the School of Social Welfare at University of California, Berkeley.

Leticia Galyean
Executive Director of Data, Evaluation, and Strategic Initiatives
Seneca Family of Agencies

Leticia Galyean acts as Seneca Family of Agencies’ Director of Strategic Development and Evaluation. Galyean holds her master of social work and bachelor of social work degrees from the University of California, Berkeley. In Galyean’s 14-year tenure at Seneca, she has directed and provided services in over 30 distinct programs across ten California counties. She is an agency leader in wraparound, therapeutic behavioral services, intensive treatment foster care, adoptions, kinship support services, clinic-based therapy, and various evidence-based practices.

Jesh Harbaugh
Assistant Director of Business Intelligence
Seneca Family of Agencies

Jesh Harbaugh is the assistant director of business intelligence at Seneca Family of Agencies, a large social services provider serving youth and families in California and Washington State. He collaborates closely with executive and program leadership, IT, and the research and evaluation team to analyze and present meaningful and actionable outcome data to agency staff and stakeholders. His efforts center on the development of automated systems that improve the efficiency, accuracy, and accessibility of agency data analyses.