Research Summary: Life Skills Training Program
Background
Botvin Life Skills Training (LST) is a school-based prevention program that targets the early drug and alcohol use of adolescents. The main goals of the LST program are to prevent substance use amongst adolescents and to promote healthy alternatives to risky behaviour through activities designed to teach youth the necessary skills to resist peer pressures to smoke, drink alcohol, and use drugs.
The middle school program is delivered over 15 sessions in the first year, 10 sessions in the second year, and 5 sessions in the third. Each session lasts approximately 45 minutes. The youth modules have three major component areas, which address domains associated with decision-making skills to prevent the use of alcohol, tobacco and drugs. The first component is designed to teach students a set of general self-management skills, the second focuses on teaching general social skills, and the third includes information and skills that are specifically related to the problem of substance abuse.
Booster sessions in subsequent years enable youth to practice the social and personal skills components of the program that were delivered in the primary year.
Ben Calf Robe is a community organization in Edmonton, Alberta, that was chosen to implement the LST program using an after-school format, and tailored to an Aboriginal population. Ben Calf Robe targeted youth who displayed behavioural problems at an early age, displayed hostility, aggression and a deficit in social skills, used drugs/alcohol, and/or have been involved with the law or Children’s Services at a young age. The screening form was used by the program as a general guide did not measure risk factors.
The program at Ben Calf Robe was adapted and delivered to students in grades 4 through 9, beginning in grade 4 or 7. Four cycles of participants were tracked, with evaluation outcomes focusing on the latter two cycles.
Public Safety contracted an independent firm, R.A. Malatest & Associates, Ltd., to conduct an impact evaluation of LST. The impact evaluation study took place from September 2010 until March 2014.
Method
The evaluators employed a mixed-method approach using quantitative and qualitative methods. A matched comparison group with a pre-post-post design was used to measure change over time on key outcomes.
Because the impact evaluation team was not in place when the LST program was initiated (in the 2009/10 school year), the comparison group for Ben Calf Robe was only identified about 20 months later. Prior to the start of each cycle of the program, LST youth and comparison youth were generally comparable in terms of demographics and pre-test scores.
Several measurement tools were used such as the: LST Questionnaire, Rosenberg Self-Esteem Scale, Decision Making Scale, and the Casey Life Skills American Indian Assessment Supplement. Evaluators also conducted key informant interviews, focus groups, case studies and photo-voice activities.
Findings
Findings were reported for process and impact-related outcomes. The actual number of participants was substantially less than anticipated (116 of 160 youth participated, with only 13 receiving the full LST program with booster sessions), which affected the evaluation scope.
Results indicate that program participants had moderate levels of self-esteem that improved over time. Median scores on the Rosenberg Self-Esteem scale increased over time (27 to 30.5 from cycle 3 pre-test to cycle 4 post-test) as did those of the comparison group. Qualitative findings show that program staff reported noticeable improvements in most youth’s self-image, and this was echoed by others in focus groups and case-study interviews.
In terms of substance use, program participation had observable effects on elementary youth’s anti-smoking knowledge, with scores increasing from cycle 3 pre to post test. Further, the program group had higher overall, anti-smoking and life-skills knowledge scores compared to the comparison group at the cycle 4 pre-test. The findings suggest that knowledge gained during cycle 3 was retained, and that the increases were due to more than maturation.
For Jr. High program participants, there were higher overall knowledge scores over the course of cycles 3 and 4. The program participants had higher overall and anti-drug knowledge scores in relation to the comparison group. Program staff identified improved understanding of the effects of substance use amongst almost all participating youth.
Survey findings did not demonstrate any increases in cultural knowledge or affiliation. Data from the Casey American Indian Youth Assessment scale suggests that youth demonstrated moderate knowledge of and affiliation with their culture, and that these levels were similar between program and comparison youth.
There is evidence to suggest that the program helped youth maintain, rather than learn, resistance skills in relation to substance use. At cycle 3 and 4 pre and post tests, program and comparison youth scored highly on drug refusal skills. During cycle 3, both program and comparison youth achieved higher drug refusal scores at the post test than the pre test. High scores were maintained in the program cycle 4 pre-test. None of the between-group and within-group comparisons were statistically significant.
Qualitative findings suggest that the program had a positive effect in helping youth to resist pressure towards substance use. There was a lack of statistical evidence to suggest the development of anti-smoking, anti-drinking and anti-drug attitudes among program youth. Many participants in the program group scored near the maximum on these indicators and therefore the ceiling effect comes into play. The lack of statistical significance on most observations suggests that any observed changes in scores could be a result of chance variation.
Focus groups and parent interviews found that parent communication with their children generally improved over the course of the program.
The findings from the evaluation suggest that the LST program has demonstrated modest success in meeting its short and medium-term outcomes. Qualitative findings were overwhelmingly positive for the outcomes of interest, but the quantitative analyses were limited by low sample sizes limiting the ability to make conclusive statements about program efficacy.
Implications
The evaluation of the LST program at Ben Calf Robe was limited by low sample size numbers due to reduced target enrolment, combined with challenges of participant retention and drop-out. This was an issue for both the program and comparison sites. It is the consistency and calibre of the qualitative data that is promising. It shows that there is potential for LST to produce positive changes in risk and protective factors amongst participants, including Aboriginal youth.
Source
R. A. Malatest & Associates (March 2014). Final Evaluation Report of the Life Skills Training Program (LST). Submitted to Public Safety Canada.
For more information on research at the Community Safety and Countering Crime Branch, Public Safety Canada, to get a copy of the full research report, or to be placed on our distribution list, please contact:
Research Division, Public Safety Canada
340 Laurier Avenue West
Ottawa, Ontario
K1A 0P8
PS.CPBResearch-RechercheSPC.SP@ps-sp.gc.ca
Research Summaries are produced for the Community Safety and Countering Crime Branch, Public Safety Canada. The summary herein reflects interpretations of the report authors’ findings and do not necessarily reflect those of the Department of Public Safety Canada.
- Date modified: