A Framework for Effective Child Welfare Practice Series Part 3 of 5


[MUSIC PLAYING] – WELCOME BACK TO OUR DISCUSSION
ABOUT BUILDING EVIDENCE AND SPREADING EFFECTIVE PRACTICE
IN CHILD WELFARE. IN THE FIRST VIDEO
IN THIS SERIES, MY PARTNER AND I TALKED
ABOUT THE NEED TO BUILD EVIDENCE IN CHILD WELFARE AND WHY
THIS CAN BE SO CHALLENGING. THEN WE INTRODUCED THE FRAMEWORK
TO DESIGN, TEST, SPREAD, AND SUSTAIN EFFECTIVE
CHILD WELFARE PRACTICE, A PRACTICAL GUIDE
DESIGNED TO HELP CHILD WELFARE DECISION MAKERS,
EVALUATORS, AND FUNDERS SUPPORT EVIDENCE-BASED PRACTICE. IN A NUTSHELL, THE FIVE PHASES
OF THE FRAMEWORK FORM A COMPLETE PROCESS
THAT INCLUDES IDENTIFYING AND EXPLORING
A PROBLEM, DEVELOPING AND TESTING
A SOLUTION, RIGOROUSLY EVALUATING
THE INTERVENTION, ADAPTING AND SPREADING
INTERVENTIONS THAT WORK, AND APPLYING, MONITORING, AND CONTINUOUSLY
IMPROVING THEM OVER TIME. – IN THE SECOND VIDEO,
WE PRESENTED THE FIRST OF THE FRAMEWORK’S FIVE PHASES–
IDENTIFY AND EXPLORE. IF YOU HAVEN’T SEEN
THOSE VIDEOS, IT MAY BE HELPFUL TO WATCH THEM
BEFORE VIEWING THIS ONE, THE THIRD IN A FIVE-PART SERIES. – TO BEGIN,
WE’RE GOING TO IMAGINE THAT WE’VE ALREADY
CAREFULLY EXAMINED A PROBLEM AND THOUGHTFULLY RESEARCHED
POSSIBLE SOLUTIONS BASED ON OUR THEORY OF CHANGE DURING THE IDENTIFY
AND EXPLORE PHASE. – WHICH PHASE OF THE FRAMEWORK
WE ENTER NEXT DEPENDS ON THE INTERVENTION
WE’VE SELECTED, ITS EVIDENCE BASE, AND WHETHER OR NOT IT’S ALREADY
BEEN IMPLEMENTED. – IN THIS CASE, WE’VE DECIDED THAT EXISTING EVIDENCE-SUPPORTED
INTERVENTIONS ARE NOT AVAILABLE TO ADDRESS OUR PROBLEM AND THE
NEEDS OF OUR TARGET POPULATION. SO WE’RE GOING TO DESIGN
A NEW INTERVENTION. – THAT MEANS OUR NEXT PHASE
WILL BE DEVELOP AND TEST. – WHETHER THAT’S A NARROWLY
FOCUSED, DISCRETE PRACTICE, LIKE USING A PARTICULAR
FAMILY CONFERENCING APPROACH; A LARGER SYSTEMIC CHANGE, LIKE ADOPTING A CASEWORK
PRACTICE MODEL; OR A SWEEPING POLICY REFORM,
LIKE EXTENDING SERVICES TO YOUTH IN FOSTER CARE
BEYOND THE AGE 18; OUR INTERVENTION MUST BE
CLEARLY DEFINED SO THAT EVERYONE UNDERSTANDS WHAT IT IS
AND HOW IT’S DIFFERENT FROM WHAT WE’RE ALREADY DOING. – THIS MEANS SPECIFYING THE
INTERVENTION’S CORE COMPONENTS, OR THE ELEMENTS
THAT ARE MOST IMPORTANT FOR THE INTERVENTION TO WORK. – LET ME OFFER AN EXAMPLE. OUR COUNTY HOPES TO REDUCE
THE NUMBER OF CHILDREN EXPERIENCING BRIEF AND REPEATED
STAYS IN FOSTER CARE. FOR THE PAST SIX MONTHS, WE’VE
WORKED WITH PROGRAM EXPERTS, PRIVATE PROVIDERS,
AND COMMUNITY PARTNERS TO DESIGN A NEW COMMUNITY-BASED,
FAMILY PRESERVATION MODEL. WE WANT TO PREVENT
UNNECESSARY REMOVALS AND KEEP CHILDREN SAFE
IN THEIR HOMES. WE’VE BEEN REVIEWING
ARTICLES AND REPORTS ABOUT INTENSIVE FAMILY
PRESERVATION APPROACHES AND OTHER HOME-BASED SERVICES,
LIKE HOME VISITING, BUT THERE ISN’T AN EXISTING
EVIDENCE-SUPPORTED PROGRAM THAT REALLY MEETS OUR NEEDS. INSTEAD, WE’RE BUILDING
ON WHAT’S BEEN LEARNED IN THESE PROGRAMS, IDENTIFYING
THE CORE COMPONENTS, AND SETTING OUR OWN STANDARDS
FOR THINGS LIKE PROGRAM ELIGIBILITY,
STRUCTURE, AND CONTENT, AS WELL AS WORKER QUALIFICATIONS
AND TRAINING. OUR AGENCY’S CURRENT
FAMILY PRESERVATION SERVICES ARE UNSTRUCTURED AND
INCONSISTENT BY COMPARISON. – CAN YOU SAY MORE ABOUT HOW YOU’RE DEVELOPING
THE INTERVENTION? – SURE. EACH CORE COMPONENT IS TIED
TO OUR THEORY OF CHANGE ABOUT WHY KIDS ARE REPEATEDLY
BEING PLACED IN FOSTER CARE FOR SHORT PERIODS OF TIME. WE’VE LEARNED THAT CHILDREN
WHO BOUNCE BACK AND FORTH
BETWEEN THEIR HOMES AND SHORT-TERM PLACEMENTS
IN OUR COUNTY OFTEN HAVE CHALLENGING BEHAVIORS
AND SPECIAL NEEDS. THEIR CAREGIVERS FEEL
OVERWHELMED AND INEFFECTIVE WHEN PARENTING, PARTICULARLY
WHEN FAMILY CONFLICT ESCALATES INTO CRISIS. THESE FAMILIES
HAVEN’T BEEN SUCCESSFUL, DESPITE HAVING ACCESS
TO OUR CURRENT SERVICES. BY PROVIDING HIGH-INTENSITY
IN-HOME SERVICES WITH CORE COMPONENTS THAT
INCLUDE WEEKLY CASEWORKER VISITS,
24-HOUR ON-CALL SUPPORT, HOME-BASED PARENT EDUCATION
AND COACHING, AND ACCESS
TO ADDITIONAL RESOURCES, LIKE YOUTH MENTORING
AND TRAUMA-FOCUSED THERAPY, WE EXPECT TO MEET THESE
FAMILIES’ NEEDS WHILE KEEPING CHILDREN SAFE
IN THEIR HOMES AND PREVENTING FUTURE PLACEMENT
IN FOSTER CARE. AFTER WE DEFINE CLEAR
AND MEASURABLE STANDARDS FOR EACH CORE COMPONENT,
THEY WILL BECOME THE BASIS FOR A PROGRAM MANUAL AND AN
IN-HOME CURRICULUM FOR FAMILIES. WE’LL ALSO HAVE TO DESIGN
EVALUATION INSTRUMENTS WITH YOUR HELP TO TRACK WHETHER
WE’RE ACTUALLY DELIVERING ALL OF THE CORE COMPONENTS
AS INTENDED. – SO THAT’S
THE DEVELOPMENT PART. – THAT’S RIGHT. AFTER THAT, WE WILL
PILOT THE PROGRAM WITH ONE OR MORE SMALL SUBSETS
OF WORKERS AND FAMILIES. – THE “TEST”
OF “DEVELOPMENT AND TEST.” – YES. STARTING WITH A SMALL SUBSET, WE’LL TEST PROCESSES
LIKE PROGRAM REFERRALS AND IDENTIFY BARRIERS
TO IMPLEMENTATION. THIS ALSO SERVES AS A KIND
OF TEST RUN TO MAKE SURE THAT ALL OF THE CORE COMPONENTS
ARE IN PLACE AND THE PROGRAM IS WORKING THE WAY WE’D PLANNED,
AND IF NOT, IT’S A GREAT TIME
TO MAKE ADJUSTMENTS. – IT CERTAINLY IS. THIS IS ALSO AN OPPORTUNITY TO
ENSURE THAT SHORT-TERM OUTCOMES ARE BEING ACHIEVED AND TO REFINE
THE CORE COMPONENTS IF NEEDED. – THE DEVELOP AND TEST PHASE
ENDS WHEN THE INTERVENTION HAS BEEN DEFINED WELL ENOUGH
THAT OTHERS CAN REPLICATE IT, THE CORE COMPONENTS ARE CLEAR
AND NO LONGER NEED ADJUSTMENT, AND EVALUATION FINDINGS SUGGEST
THAT THE INTERVENTION IS ASSOCIATED
WITH IMPROVED OUTCOMES. – THIS LEADS US INTO THE NEXT
PHASE OF THE FRAMEWORK– COMPARE AND LEARN. THE PURPOSE OF COMPARE AND LEARN
IS TO DETERMINE WHETHER, ON AVERAGE,
THE INTERVENTION RESULTS IN BETTER OUTCOMES
FOR CHILDREN AND FAMILIES THAN AN ALTERNATIVE. – BASED ON
THE EVALUATION FINDINGS, DECISION MAKERS, FUNDERS,
EVALUATORS, AND OTHER STAKEHOLDERS
DECIDE WHETHER THERE IS CONVINCING EVIDENCE
THAT THE INTERVENTION WORKS BETTER THAN
SERVICES AS USUAL. STEPS IN THIS PHASE INCLUDE
DESIGNING A RIGOROUS EVALUATION INVOLVING AN INTERVENTION
AND A COMPARISON GROUP, COLLECTING DATA; PERFORMING ANALYSES
TO DRAW CONCLUSIONS; AND USING DATA TO DIG DEEPER,
WHEN POSSIBLE, TO DETERMINE WHETHER
THE INTERVENTION WAS MORE OR LESS EFFECTIVE
FOR CERTAIN GROUPS OF PEOPLE OR UNDER CERTAIN CONDITIONS. – THE RESEARCHER IN ME
LIKES THIS PHASE. – I THOUGHT YOU MIGHT. DO YOU HAVE ANY EXAMPLES
THAT WOULD FIT? – YES. I THINK I MIGHT
HAVE THE PERFECT ONE. I ONCE WORKED WITH AN AGENCY
THAT WANTED TO TEST THE EFFECTIVENESS OF A PARENTING
PROGRAM FOR FIRST-TIME PARENTS. A ONE-HOUR-A-WEEK
CLASSROOM-BASED PARENTING PROGRAM
WAS CONSIDERED STANDARD FOR FIRST-TIME MOTHERS
AND FATHERS INVOLVED WITH THIS AGENCY, AND A WEEKLY HOME-VISITING
PARENTING PROGRAM WAS THE NEW INTERVENTION
IT WANTED TO TEST. – THIS SOUNDS LIKE
A GREAT EXAMPLE. WHAT WERE THE RESULTS? – HOLD ON. I THINK
YOU’RE SKIPPING A FEW STEPS. IN ORDER TO DO A GOOD JOB OF
COMPARING THE STANDARD PROGRAM WITH THE NEW INTERVENTION,
WE USED A LOTTERY PROCESS TO ASSIGN ALL PARENTS
REFERRED TO THE AGENCY TO EITHER
THE CLASSROOM-BASED PROGRAM OR THE NEW
HOME-VISITING PROGRAM. THIS RANDOM ASSIGNMENT
HELPED TO ENSURE THAT THE TWO GROUPS OF PARENTS
WERE AS SIMILAR AS POSSIBLE. FOR EXAMPLE,
THE LOTTERY SYSTEM PROTECTED AGAINST THE POSSIBILITY
THAT PARENTS IN THE NEW HOME-VISITING PROGRAM
WERE MORE MOTIVATED AT THE START THAN PARENTS
IN THE REGULAR PROGRAM. – SO WHAT SORT OF DIFFERENCES IN
OUTCOMES ARE WE TALKING ABOUT? – WELL, THE CLASSROOM-BASED
TEACHERS AND HOME-BASED TEACHERS KEPT TRACK OF ATTENDANCE
AND PARTICIPATION, AND I MEASURED
PARENTS’ CONFIDENCE IN THEIR ABILITIES
TO CARE FOR THEIR CHILDREN. WE ALSO TRACKED
LONGER-TERM OUTCOMES, SUCH AS CHILDREN’S ENTRY
INTO FOSTER CARE DURING AND AFTER
THE PROGRAM ENDED. THE AGENCY ALSO GATHERED
PERSONAL STORIES AND FEEDBACK THROUGH INTERVIEWS
AND FOCUS GROUPS WITH PARENTS. PARENTS IN THE HOME-VISITING
PROGRAM SAID THAT BEING ABLE TO INTERACT WITH THEIR CHILDREN
WHILE LEARNING THE PARENTING TECHNIQUES
MADE A BIG DIFFERENCE. ON THE OTHER HAND,
PARENTS WHO ATTENDED THE REGULAR PARENTING CLASSES
REPORTED THAT THEY HAD DIFFICULTY REMEMBERING
THE TECHNIQUES WELL ENOUGH TO BE ABLE TO USE THEM
WHEN THEY RETURNED HOME. ULTIMATELY, THE OUTCOME DATA
SEEMED TO SUPPORT THESE STORIES. COMPARISON TESTS
REVEALED IMPORTANT, STATISTICALLY SIGNIFICANT
DIFFERENCES IN OUTCOMES BETWEEN THE TWO PROGRAMS. PARENTS WHO PARTICIPATED IN THE
HOME-VISITING PARENTING PROGRAM REPORTED GREATER CONFIDENCE
IN THEIR PARENTING SKILLS, AND THEIR CHILDREN EXPERIENCED
FEWER ENTRIES INTO FOSTER CARE. – THAT IS SOME AMAZING WORK, AND AS A RESULT, IT SOUNDS LIKE
THE AGENCY GAINED CONVINCING EVIDENCE
THAT CHILDREN OF PARENTS WHO RECEIVED HOME-BASED SERVICES
WERE SAFER, IN COMPARISON TO THOSE WHO RECEIVED
CLASSROOM-BASED SERVICES. THIS KIND OF STRONG EVIDENCE
IS REALLY HELPFUL TO HEAR. – WE ALSO COLLECTED
IMPORTANT INFORMATION ABOUT HOW CLOSELY THE AGENCY
FOLLOWED THE CORE COMPONENTS OF THE HOME-VISITING PROGRAM,
ITS DELIVERY, AND PARENTS’ PARTICIPATION. THIS LOYALTY
TO INTERVENTION AS PLANNED IS WHAT WE CALL
INTERVENTION FIDELITY. TRACKING FIDELITY MAY ALLOW US
TO LOOK MORE CAREFULLY AT WHY SOME PARENTS
FARED BETTER THAN OTHERS WITHIN THE HOME-VISITING GROUP. BECAUSE THE AGENCY
MADE THE COMMITMENT TO COMPLETE COMPARE AND LEARN
AND PARTNERED TO MAKE THE EVALUATION
A SUCCESS, IT WAS ABLE TO JUSTIFY
INFORMED DECISIONS ABOUT PARENTING PROGRAMS
IN THE FUTURE. THAT SAID,
IT’S IMPORTANT TO REMEMBER THAT BECAUSE THIS EVALUATION
WAS ONLY CONDUCTED WITH PARENTS SERVED
BY ONE AGENCY, MORE STUDY WILL BE NEEDED
TO DETERMINE IF SIMILAR RESULTS CAN BE EXPECTED FOR OTHER
POPULATIONS, LIKE MOMS AND DADS WITH DIFFERENT
DEMOGRAPHIC CHARACTERISTICS AND LIVING IN OTHER
GEOGRAPHIC REGIONS. – THANKS. YOU SEAMLESSLY TRANSITIONED
INTO OUR NEXT PHASE, WHICH IS REPLICATE AND ADAPT. UNFORTUNATELY, WE HAVE RUN OUT
OF TIME FOR THIS SEGMENT, BUT BE SURE TO WATCH
OUR NEXT VIDEO, WHERE WE DISCUSS THE NEXT TWO
PHASES IN THE FRAMEWORK TO DESIGN, TEST, SPREAD,
AND SUSTAIN EFFECTIVE CHILD WELFARE
PRACTICE.

Daniel Yohans

Leave a Reply

Your email address will not be published. Required fields are marked *