The federal Family First Prevention Services Act of 2018 (Family First Act) has changed the landscape of federal child welfare funding in significant ways to support children to remain safely in their homes and encourage family-based placement when foster care placement is necessary. With specific requirements around evidence-based programs and tools, the legislation expands opportunities to use research and data to drive decision making and direct funding to ensure that children and families receive the most effective services. This brief outlines these research and evaluation requirements and highlights next steps for state agencies, legislators, and researchers to achieve the Family First Act’s goals.
Passed in 2018, this landmark federal legislation advances key priorities, including:
For more detail on these and other provisions of the Family First Act, see Children’s Defense Fund (2018).
Title IV-E of the Social Security Act—the largest source of federal child welfare funding—previously reimbursed states for a portion of their eligible expenditures on child welfare services and activities. With the passage of the Family First Act, states have the option to use Title IV-E funds for services to prevent children from entering foster care. In the past, Title IV-E funding could only be used for prevention services through Child Welfare Waiver Demonstration projects approved by the U.S. Department of Health and Human Services. Now, under the Family First Act, children at imminent risk of entering foster care (“candidates for foster care,” as defined by each state), their parents or kin caregivers, and youth in foster care who are pregnant or parenting are eligible to receive time-limited, evidence-based prevention services to prevent foster care entry. By keeping children safely at home and preventing subsequent maltreatment, these services have the potential to greatly benefit children, families, and society.
When children must be removed from their homes, the Family First Act prioritizes the use of family-based settings, particularly with relatives. Evidence-based kinship navigator services, designed to connect relatives caring for children to information and supportive services, are funded through Title IV-E under the Family First Act. Further, the legislation establishes funding limitations on non-family placements. Reimbursement for the out-of-home placement needs of children will be limited to family-based settings, qualified residential treatment programs when indicated by a child assessment, and special treatment settings (e.g., for pregnant and parenting youth).
This brief represents our understanding of the research and evaluation provisions of the Family First Act, as of October 2019. We will continue to monitor the federal landscape for administrative guidance and/or legislative changes.
See the glossary for definitions of key terms.
The research and evaluation provisions of the Family First Act require that (1) prevention and kinship programs be evidence-based, (2) states rigorously evaluate and support implementation of prevention programs through data-driven continuous quality improvement efforts, and (3) assessment tools used to determine the appropriateness of residential treatment programs be evidence-based. This focus on evidence ensures that children and families receive services with demonstrated effectiveness, and is part of the larger movement toward evidence-based social policy (Supplee & Metz, 2015). The research and evaluation requirements of the Family First Act are summarized below for prevention services, kinship navigator services, and qualified residential treatment programs.
Keeping families together with services to support their safety, permanency, and well-being is a primary goal of the optional prevention services program of the Family First Act. Children who have been maltreated have a higher risk of experiencing a range of negative well-being outcomes, with significant direct and indirect societal costs,[i] and children who are removed from their home may experience additional trauma that can compound the effects of maltreatment (Kolko et al., 2010). Trauma occurs when children experience or perceive one or more negative events or circumstances that cause them emotional pain and impair their ability to cope (Bartlett & Steber, 2019). Expanding federal Title IV-E reimbursement to include prevention services is therefore a critical step toward supporting child and family well-being.[1]
States choosing to implement the prevention program of the Family First Act may be reimbursed for these services:
Prevention services must be documented in a written prevention plan for each child. The Family First Act requires that organizations delivering prevention services use a trauma-informed approach that understands, recognizes, and responds to trauma within a treatment framework that includes intervention(s) designed to address trauma (U.S. Department of Health and Human Services [U.S. DHHS], 2018a).
Under the Family First Act, states can receive federal reimbursement for specific prevention programs that meet the criteria for promising, supported, or well-supported practice. At least 50 percent of state expenditures must be on well-supported programs, and the remainder can be spent on promising and/or supported programs.
The federally funded Title IV-E Prevention Services Clearinghouse (Prevention Services Clearinghouse) continuously and systematically reviews the available literature to assign evidence ratings to programs (see Figure 1 below) or to determine that programs do not currently meet the criteria. Under the Family First Act, programs that do not meet the criteria are not eligible for federal reimbursement. The Prevention Services Clearinghouse released its first set of program ratings in June 2019.
The Family First Act requires that studies be rated through an independent systematic review and determined to be well-designed and well-executed to contribute to the evidence base. When determining ratings, the Prevention Services Clearinghouse considers evidence from moderate- and high-quality comparison studies (U.S. DHHS, 2018b).[2] These are limited to randomized controlled trials (RCTs) and quasi-experimental designs (QEDs) that meet standards outlined in the Title IV-E Prevention Services Clearinghouse Handbook of Standards and Procedures (Wilson et al., 2019).[3] In an RCT design, each unit has the same chance of being assigned to the intervention or control group, thereby creating equivalent groups that allow evaluators to determine cause and effect. Units of randomization may be children or families, or more expansive groupings such as offices or counties.[ii] QED studies rely on non-random assignment strategies to compare outcomes. For example, if rollout of a program’s implementation is staggered across multiple counties, the evaluator may compare outcomes for families in an early implementing county with outcomes for families in a similar county that has not yet implemented the program. Researchers may also use statistical methods to match children who participated in a program with similar children who did not participate in the program to compare outcomes.[iii]
The Prevention Services Clearinghouse casts a wide net in both the study population and outcomes considered in its review. Prevention programs are not required to be evaluated with a child welfare population; this means that the Clearinghouse’s review includes studies evaluating programs delivered through other child-, family-, and adult-serving systems (e.g., behavioral health or juvenile justice), regardless of whether the participants had current or past child welfare involvement (U.S. DHHS, 2018b).
Studies must examine one or more relevant target outcomes related to safety, permanency, and/or well-being. Safety and permanency encompass child welfare-specific outcomes such as substantiated maltreatment, placement stability, and reunification. Well-being outcomes may be for children (e.g., measures of behavioral, social, cognitive, and/or physical functioning) and adults (e.g., parenting, mental health, substance use, physical health, and/or economic/housing stability; Wilson et al., 2019).
Source: Wilson et al. (2019)
In recent decades, several clearinghouses have been established as repositories of evidence-based programs, including clearinghouses that focus primarily on interventions of interest to child welfare systems.[5] Although the Prevention Services Clearinghouse’s method of rating programs is informed by the methods that these other clearinghouses use, criteria and evidence ratings may differ across clearinghouses. For this reason, programs that are typically referred to as “evidence-based” may or may not satisfy all requirements of the Prevention Services Clearinghouse.
Programs with multiple variations or formal adaptations will be reviewed as separate programs by the Prevention Services Clearinghouse. For example, Multisystemic Therapy (MST) and Multisystemic Therapy – Child Abuse and Neglect (MST-CAN) were assigned separate evidence ratings as their manuals and model components differ. Program adaptations such as adding, subtracting, or significantly changing content; changing the format of service delivery; significantly changing content for cultural relevancy; and implementing the program with substantially different types of staff would all be considered new programs that require separate review by the Prevention Services Clearinghouse.
As the Prevention Services Clearinghouse continues to review interventions, the federal Children’s Bureau has released guidance that allows states to submit documentation of an independent systematic review concluding that a program not yet rated by the Prevention Services Clearinghouse qualifies as promising, supported, or well-supported. With approved documentation of an independent systematic review and all other requirements met, states can receive reimbursement (called “transitional payments”) for these services until the Prevention Services Clearinghouse completes its review of that program. However, the final evidence rating of these services will ultimately be determined by the Prevention Services Clearinghouse (U.S. DHHS, 2019a).
Tribal Title IV-E agencies are not required to adhere to the criteria for promising, supported, and well-supported prevention programs and must instead determine their own practice criteria for identifying prevention services that are relevant to their culture and context. These may include criteria such as longevity of the practice and community leader/elder approval (U.S. DHHS, 2018c).
For each promising, supported, or well-supported prevention program for which states are seeking federal reimbursement, the Family First Act requires states to outline (1) how they will implement the programs with fidelity using a continuous quality improvement (CQI) framework, and (2) how they will evaluate each program using a well-designed and rigorous process.
By implementing programs with fidelity, states will ensure that all key program elements associated with positive outcomes for children and families are maintained. Many program developers have created tools such as fidelity checklists to support supervisors and program administrators to regularly monitor these core elements. Used in combination with other program data, these measures feed into a cycle of continuous quality improvement through which states can strengthen prevention service delivery.
Beyond the fidelity requirements, states must develop and submit an evaluation plan for each prevention service. This requirement may be waived for well-supported interventions with documentation that the program has compelling evidence and the state will meet the CQI requirements. While the law does not define requirements for well-designed and rigorous evaluation strategies, states should consider the study quality standards established by the Prevention Services Clearinghouse to ensure that their evaluation activities will contribute to the evidence base. The Children’s Bureau has released a “tip sheet” for preparing evaluation plans. Meant to serve as a resource for states and their partners, the tip sheet outlines key components of evaluation plans and related reference materials (U.S. DHHS, 2019b). States can evaluate their prevention services or participate in a cross-site evaluation, if approved by the Administration for Children and Families (ACF).
Tribal Title IV-E agencies are afforded flexibility in their evaluation designs, with the recognition that some rigorous evaluation designs may not be feasible in the tribal context. Alternative evaluation strategies may include, for example, exploratory studies, qualitative studies, and community-based participatory research (U.S. DHHS, 2018c).
States must collect and report child-level data to the federal government. States are required to report the services that were provided to each child/family and the cost and duration of services, the placement status of each child 12 months after the prevention plan start date, whether the child entered foster care within two years of the prevention plan start date, whether the child is pregnant or parenting, and certain demographic characteristics. Some data elements are not required for pregnant or parenting youth (U.S. DHHS, 2019c).
The U.S. Department of Health and Human Services (HHS) may conduct evaluations and gather data to assess whether programs reduce foster care placements, increase kinship care, and/or improve child well-being. HHS may directly carry out this provision or enter into an agreement with external partner(s).
When children cannot remain safely in their homes, placement with kin provides an opportunity to keep children connected to their families and culture. Kinship navigator programs support kin caregivers to learn about, find, and use programs and services for the children in their care and themselves. Kinship navigator programs use a variety of approaches to connect kin caregivers to public benefits and community resources (e.g., designated staff, peer-to-peer support, help lines). Kinship navigators also serve an important role in educating child welfare agency and community partners about the needs of kin caregivers (Casey Family Programs, 2018).
Under the Family First Act, states can receive federal reimbursement for up to 50 percent of their expenditures on kinship navigator programs that meet the criteria for promising, supported, or well-supported practice. The evidence requirements for promising, supported, and well-supported are the same as those for prevention services, with the exception of the target outcomes. Kinship navigator programs are reviewed for their effects on child safety, child permanency, child well-being, adult well-being, access to services, referral to services, and/or satisfaction with services. Beyond the evidence requirements, several other practice requirements must be met for reimbursement. For example, kinship navigator programs must coordinate with other state/local agencies; consult with kin, youth, and community stakeholders in designing and implementing programs; and establish information and referral systems, among other requirements (U.S. DHHS, 2018d). The Prevention Services Clearinghouse reviewed two kinship navigator programs in its first round of review, and neither program met the criteria.
Complementary to its focus on supporting families through prevention and kinship navigator services, the Family First Act directs federal funding to encourage agencies to place children who must be removed from their homes in families, not group settings, when appropriate. Exceptions are made for children assessed to have needs that necessitate treatment in a qualified residential treatment program, or children and youth who qualify for specified settings. This policy shift is consistent with the research evidence that children in family settings fare better across a variety of outcomes, compared to children in group settings (Annie E. Casey Foundation, 2015; Casey Family Programs, 2017). The Family First Act defines qualified residential treatment programs (QRTPs) as programs that are designed to address the needs of children with serious emotional or behavioral disorders or disturbances, using a trauma-informed treatment model. Specific requirements related to assessments, program staff, family engagement, discharge planning, aftercare, and accreditation are outlined in the law.
The Family First Act specifies several requirements for assessing the appropriateness of every placement of a child in a QRTP; these include using a validated, evidence-based assessment tool to determine the appropriate placement setting. The assessment tool must be administered by a trained professional or licensed clinician who is not employed by the state agency or affiliated with a placement setting. While this provision does not require further research on the assessment tools being used, it is an important element of the Family First Act and reinforces the focus on effectiveness.
The Family First Act’s research and evaluation requirements ensure that children and families receive the most effective programs and services to meet their individual needs. The emphasis on evaluation, CQI, and evidence-based tools spurs child welfare stakeholders to work together to continue building the evidence base and monitoring program implementation and outcomes to ensure effectiveness.
As with any significant change in policy and practice, states are likely to face challenges in implementing provisions of the Family First Act. Challenges for research and evaluation may include the following:
For state agencies and legislators implementing the prevention, kinship navigator, and QRTP provisions of the Family First Act, key next steps related to research and evaluation include:
For researchers partnering with states to build the evidence base, key next steps include:
The Family First Act presents a significant opportunity to embrace the power of research to better serve children and families. It directs resources to what we know works best for children and families by supporting parents and kin caregivers to safely care for children in their homes and prioritizing family-based placements when children must enter the foster care system. The new requirements around evidence-based programs and tools hold great potential for improving the lives of children and families; however, significant work remains to ensure that states have the support they need to identify and implement evidence-based programs and evaluate their efforts in ways that continue to expand the evidence base. By deepening our collective understanding of what works and why, and channeling resources to proven interventions, the Family First Act contributes to ensuring that children and families receive high-quality, effective supports.
Comparison study: A study that compares a group receiving an intervention with a group that receives no intervention, minimal intervention, or services as usual. Wilson et al. (2019) define two types of comparison studies as follows:
Formal adaptations: “Alternative manualized versions of the original program designed to address particular issues or populations” (Wilson et al., 2019, p. 4). Changes to a program’s content or approach (e.g., changing from group to individual treatment) are examples of formal adaptations.
Target outcomes:
Usual care setting: A non-research setting where an existing provider delivers prevention or kinship navigator services. (Wilson et al., 2019)
Fidelity: Adherence to the specified components of a program.
Continuous quality improvement (CQI): An ongoing process of using data to identify and understand strengths and challenges and to test, implement, analyze, and strengthen solutions (National Child Welfare Resource Center for Organizational Improvement & Casey Family Programs, 2005).
Cross-site evaluation: An evaluation of the same program across multiple locations.
Annie E. Casey Foundation (2015). Every kid needs a family: Giving children in the child welfare system the best chance for success. Retrieved from https://www.aecf.org/m/resourcedoc/aecf-EveryKidNeedsAFamily-2015.pdf.
Casey Family Programs (2017). What are the outcomes for youth placed in congregate care settings? Retrieved from: https://caseyfamilypro-wpengine.netdna-ssl.com/media/SF_CC-Outcomes-Resource.pdf.
Children’s Defense Fund (2018). The Family First Prevention Services Act: Historic reforms to the child welfare system will improve outcomes for vulnerable children. Retrieved from https://www.childrensdefense.org/wp-content/uploads/2018/08/family-first-detailed-summary.pdf.
Child Welfare Information Gateway (2017). Child maltreatment prevention: Past, present, and future. Washington, DC: U.S. Department of Health and Human Services, Children’s Bureau. Retrieved from https://www.childwelfare.gov/pubPDFs/cm_prevention.pdf.
Child Welfare Information Gateway (2019). Long-term consequences of child abuse and neglect. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau. Retrieved from https://www.childwelfare.gov/pubpdfs/long_term_consequences.pdf.
Kolko, D.J., Hurlburt, M.S., Zhang, J., Barth, R.P., Leslie, L.K., & Burns, B.J. (2010). Posttraumatic stress symptoms in children and adolescents referred for child welfare investigation. Child Maltreatment, 15(1), 48-63.
Moore, K.A. & Metz, A. (2008). Random assignment evaluation studies: A guide for out-of-school time program practitioners. Retrieved from: https://cms.childtrends.org/wp-content/uploads/2008/01/Random-Assigment-Evaluations.pdf.
National Child Welfare Resource Center for Organizational Improvement & Casey Family Programs (2005). Using Continuous Quality Improvement to Improve Child Welfare Practice. Retrieved from: http://muskie.usm.maine.edu/helpkids/rcpdfs/CQIFramework.pdf.
Supplee, L. H. & Metz, A. (2015). Opportunities and challenges in evidence-based social policy. Social Policy Report, 28(4), 1-16.
U.S. Department of Health and Human Services. (2018a). ACYF-CB-PI-18-09: State requirements for electing Title IV-E prevention and family services and programs. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/pi1809.pdf
U.S. Department of Health and Human Services. (2018b). Attachment C to ACYF-CB-PI-18-09: HHS initial practice criteria and first list of services and programs selected for review as part of the Title IV-E Prevention Services Clearinghouse. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/pi1809.pdf
U.S. Department of Health and Human Services. (2018c). ACYF-CB-PI-18-10: Tribal Title-IV E agency requirements for electing Title IV-E prevention and family services and programs. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/pi1810.pdf.
U.S. Department of Health and Human Services. (2018d). ACYF-CB-PI-18-05: Fiscal Year 2018 funding available for developing, enhancing or evaluating kinship navigator programs. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/pi1805.pdf.
U.S. Department of Health and Human Services. (2019a). ACYF-CB-PI-16-06: Transitional payments for the Title IV-E prevention and family services programs. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/pi1906.pdf.
U.S. Department of Health and Human Services. (2019b). ACYF-CB-IM-19-04: Evaluation plan development tip sheet. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/im1904.pdf.
U.S. Department of Health and Human Services (2019c). Technical bulletin #1: Title IV-E prevention program data elements. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/tb_1_iv_e_data_elements.pdf.
Wilson, S. J., Price, C. S., Kerns, S. E. U., Dastrup, S. D., & Brown, S. R. (2019). Title IV-E Prevention Services Clearinghouse Handbook of Standards and Procedures, Version 1.0, OPRE Report # 2019-56, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved from https://www.acf.hhs.gov/sites/default/files/opre/psc_handbook_
v1_final_508_compliant.pdf.
[1] The services provided under the Family First Act can be categorized as tertiary prevention services, designed to support families already affected by child maltreatment. These interventions support child and family well-being by strengthening protective factors and reducing risk factors for maltreatment recurrence (Child Welfare Information Gateway, 2017). While states should consider a range of primary prevention efforts (i.e., services aimed at the entire population to prevent maltreatment before it happens) and secondary prevention efforts (i.e., services that are targeted at populations at higher risk for maltreatment), only tertiary prevention is covered under the Family First Act.
[2] The Prevention Services Clearinghouse considers a range of factors when determining study design quality, further outlined throughout the Title IV-E Prevention Services Clearinghouse Handbook of Standards and Procedures (Wilson et al., 2019). Examples of these factors include, but are not limited to, the similarity between the treatment and control groups at the start of treatment, and whether participants dropping out of the study could bias results.
[3] For a program to qualify as promising, supported, or well-supported, the Family First Act requires evidence of significant positive effects in well-designed and well-executed stud(ies). The Family First Act explicitly requires a QED or RCT study design for programs to be rated as supported or well-supported. According to the Act, promising programs may be evaluated using some form of control, with examples including an untreated group, placebo group, or waitlist study (section 471(e)(4)(C) of the Act). The Prevention Services Clearinghouse has interpreted the language of the Act to require an RCT or QED for all relevant research studies (Wilson et al., 2019).
[4] The Family First Act specifies that well-supported programs be evaluated in at least two studies (section 471(e)(4)(C)(v)). Because researchers often publish more than one analysis of the same sample, the Prevention Services Clearinghouse specifies that findings must come from two or more non-overlapping samples.
[5] Examples of related clearinghouses include the Department of Education’s What Works Clearinghouse, US DHHS’s Home Visiting Evidence of Effectiveness (HomVEE), Blueprints for Healthy Youth Development, and the California Evidence-Based Clearinghouse for Child Welfare.
[i] For a summary, see: Child Welfare Information Gateway (2019).
[ii] For more information on the design and implementation of randomized controlled trials, see Moore and Katz (2008).
[iii] For more information on the design and implementation of quasi-experimental evaluations, see Moore (2008).
