Evaluation Summary Community Cadets Corps (CCC) Program

Evaluation Summary Community Cadets Corps (CCC) Program PDF Version (471 KB)

© Her Majesty the Queen in Right of Canada, 2015

Cat. No.: PS113-1/2015-7E-PDF

ISBN Number: 978-0-660-02202-4

Introduction

The Community Cadets Corps (CCC) program evaluation was undertaken to examine the effectiveness of Cadets as a crime prevention measure for Aboriginal youth. The CCC program was designed to increase the protective factors of Aboriginal youth ages 10 to 18 years old that are at risk of police contact, and/or engaging in criminal offences, anti-social activities, having conduct problems or truancy at school. Through the National Crime Prevention Strategy (NCPS), Public Safety Canada contributed $2.8 million in funding to implement 4 CCC crime prevention pilot projects in Aboriginal communities: Cross Lake, Manitoba; Kahkewistahaw First Nation, Saskatchewan; Touchwood Agency Tribal Council, Saskatchewan; and Hobbema, Alberta. Funding from the NCPS began in 2010 and ended in 2013.

Program Description

The CCC program aims to enhance youths’ ability to make positive decisions, and lead to a productive and crime-free lifestyle. The program’s structure was influenced by the core principles underlying the theory of the Circle of Courage^{Footnote 1}. This model provides a framework for healthy, holistic culturally authentic alternatives that will build a sense of identity and community (belonging) as well as provide opportunities to enhance skills (mastery), develop a sense of independence, and support youth to practice generosity. These are the foundations for positive youth development ‒ empowering Aboriginal youth to develop pro-social competencies, attitudes and behaviours, thereby reducing the risk of delinquent activity and criminal involvement.

The CCC program is composed of the following five main program components:

• Drill and Deportment: This component provides youth with a structured activity that takes place for approximately 30 minutes each week for 40 weeks. It comprises a series of techniques that must be executed in a specific way, requiring practice and focused attention from participants. Drills/marching are modeled after RCMP or military activities.
• Group and Individual Mentoring: This component takes place through drill and deportment, life skills training, community leadership, and cultural and recreational activities. Community Elders, community workers and others provide youth with positive role models, inspire youth toward future career directions, and provide support in the areas of personal, academic and career development, social and athletic growth. Youth should meet with mentors at least 1 to 4 hours or more each week.
• Life Skills Training: In this component, youth take part in various workshops/activities focused on improving their social and self-management skills. It is recommended that youth meet on a weekly basis, 9 months per year (36 weeks in total) in school settings.
• Community Leadership/Involvement: Activities under this component aim to engage youth in the community, to enhance a sense of purpose and belonging as well as to develop
• leadership, planning, and implementation skills. During 1 week night per month and daily sessions during the summer months, Cadets will work towards benefitting the community through various activities.
• Cultural and Recreational Activities: This component incorporates cultural adaptations and activities associated with the tradition, values, practices, beliefs, attitudes, and behaviours of Aboriginal youth involved in the program.

Evaluation Objectives

The original intent of this evaluation was to determine whether the intervention had an impact on key outcomes, including increased attachment to and attendance/performance in school; enhanced life, leadership, communication, decision-making and problem-solving skills; increased pro-social behaviours and reduced anti-social and delinquent behaviours; increased physical fitness and employability; and increased attachment to and engagement in Aboriginal culture. Due to data collection challenges in most of the sites, the data that was most reliable included implementation related information^{Footnote 2}.

Three independent third party consultants were contracted by the project recipient to conduct monitoring and process evaluations of the four CCC pilot projects. The objectives of these evaluations were as follows:

• Assess the feasibility of reaching the target population.
• Assess the extent to which the project was implemented as intended (e.g., intake process, reach, dosage, program fidelity, resource utilization, partnerships).
• Determine how satisfied program participants, staff, volunteers, and partners were with the program and its different elements.
• Identify challenges and recommendations for strengthening the evidence base of the intervention.

Evaluation Research Methods

For the Cross Lake CCC program, data were collected through the administration of three questionnaires. Information regarding Cadets’ participation was based on attendance records. For the Kahkewistahaw First Nation Community CCC program and the Touchwood Agency Tribal Council CCC program, data were collected through the implementation of a basic attendance record-keeping system, in-person interviews and focus groups. For the Hobbema CCC program, data were collected from program observation, documents, reports and other program collected data. Interviews were conducted with key stakeholders.

Findings

Intake Process^{Footnote 3}

For Kahkewistahaw First Nation Community and Touchwood Agency Tribal Council, the CCC program was open to all youth despite their perceived level of risk for criminal involvement. Youth were admitted based on interest in the CCC program. In Hobbema, interested children and youth and their parents completed an application/consent form. If they met the eligibility criteria (aged 6-17, have a Hobbema connection, and are attending school) and parents signed the consent form, the children and youth were accepted. For Hobbema, the program was open to new participants throughout the year.

Target Group

Table 1 (below) provides a summary of the target population reached for each project site. The table indicates that a total of approximately 565‒580 youth were enrolled and participated in the CCC program across all project sites. All participants were Aboriginal and living on reserve, and the vast majority fell within the target age range of 10‒18 years old. No risk level assessment data were recorded at the time of program registration.

Dosage^{Footnote 5}

Program dosage, in a process evaluation, is collected to gain an understanding of how much of the intervention is necessary to potentially effect change in crime prevention outcomes. There is no consistent expected standard of dosage across program sites, and it is important to note that some activities overlap and the dosage received by participants varied by site.

The largest percentage of Cross Lake CCC program Cadets (42.3%) participated in 6 to 10 Cadet activities. They were followed by those who participated in 11 to 15 activities (26.9%), 1 to 5 activities (17.3%) and 16 or more activities (13.5%). On average, each Cadet participated in 9.7 different activities during their time in the program. In addition to the activities carried out weekly during the Drill Nights, many Cadets also participated in both summer and winter training opportunities.

On average, Kahkewistahaw First Nation Community CCC program Cadets received a total of 108.6 hours of programming over 22 months. The average number of hours spent at Drill Nights was 20.2 and the average number of hours spent in Life Skills Training was 41.6. The average length of stay in the program was 97.7 weeks – just over two years of program exposure.

The Touchwood Agency Tribal Council CCC program had challenges tracking attendance records. Based on the data available, the frequency of participation was slightly less than once per month – the average number of Drill Nights attended was 12.8 (out of approximately 40). Program dosage and intensity fell below what was expected. Cadets on average attended 32% of the Drill Nights that were offered to them. The average length of stay was 61.7 weeks which equals an average of 15 months of program exposure.  

Program Fidelity^{Footnote 6}

The Cross Lake CCC program implemented all core components of the CCC program as planned.

The Kahkewistahaw First Nation Community CCC program implemented all core components of the CCC program. In addition, the project director and staff consulted with Cadets to determine what worked and then made changes to the program based on this feedback. For example, in one instance, Cadets organized, researched, and directed their own skits on the use of drugs. In another instance, the project incorporated the Duke of Edinburgh's Award program into their programming as an incentive to encourage Cadets' engagement and recognize achievement.

The Touchwood Agency Tribal Council CCC program did not implement all core components of the CCC program. Only three program components were delivered including Drill Nights, three summer camp trips and some weekend learning events. While informal life skills events were offered and trips were provided, the key learning events to be included in the program, such as formal life skills training and community service learning projects, were not implemented.

The Hobbema CCC program, since funding was re-instated in November 2010, has emphasized the drill component and large games activities, has partially delivered the cultural component and has not formally developed or delivered any aspect of the life skills component of the CCC program.

Resource Utilization ^{Footnote 7}

The Cross Lake CCC program activities were facilitated by four staff and numerous military and civilian volunteers. Volunteers assisted with training and chaperoning trips out of town. Many of these volunteers self-selected for their roles, as opposed to being actively recruited. It was concluded that they possessed the training and skills required to effectively deliver the programs and services based on their broad range of fields, former training and credentials, and life experiences. Military experience was one area of deficiency noted.

The human resources available for the Kahkewistahaw First Nation Community were adequate in terms of management committee involvement and all staff positions with the exception of the Coordinator position. It was difficult to attract and retain a fully qualified candidate for this role partly because many skills are required for this position. With respect to the level of volunteerism, it was felt that more volunteers were needed and that volunteers should be more present at all Cadet events. Finding ways to get more parents to volunteer was identified as an area for improvement. Finally, at times, delivering Cadets was problematic and the project did not have a dedicated space.

The Touchwood Agency Tribal Council did not have the full range of resources to effectively implement the program. Key inputs that were missing included human resources, namely people to take on the roles of Program Manager and Evaluation Data Entry Clerk. As a result, key activities such as data collection, entry and reporting only occurred if and when there was time left over after meeting basic needs such as planning drills, transporting Cadets, and dealing with issues when they arose. A lack of leadership in terms of managing the Cadets during programming was also identified – there were only a few volunteers present on a regular basis (an average of three volunteers per year), and there was little parental involvement. Further, the RCMP officers expressed concern that when they showed up for the 'drill instruction' sessions the other facilitators/coordinators would 'take a back seat' and let the RCMP take over.

Partnerships

The Cross Lake CCC program was a community partnership from the start, when the Cross Lake Town Council agreed to be its sponsor. Every aspect of its development thereafter was undertaken in collaboration with multiple sectors of the community (local, band council, provincial and federal political levels). They included Cadets' parents, community Elders, professionals within the education, police, health care and the emergency responder sectors and representatives from the corporate sector. For the Kahkewistahaw First Nation Community CCC program and the Touchwood Agency Tribal Council CCC program, the main partnership was with the RCMP officers who helped with the 'drill instruction' component, and in certain instances, they assisted in chaperoning some of the field trips. For Hobbema, some formal partnerships were established through the development of more formal MOUs. The strongest working relationship was with the RCMP. It appears that limited effort was expended on developing linkages with the local schools or creating community service opportunities for the participating young people.

Project Satisfaction

Based on the qualitative feedback obtained through Cadet questionnaire responses, interviews, and focus groups, and key informant interviews with staff, volunteers and other stakeholders, it may be concluded that the CCC program was able to meet the needs of participants. There were positive perceptions of the program by youth and parents, and it appeared to address a need in the community – the apparent lack of positive alternative social and recreational activities. The CCC program has made a positive impact on many of the youth who attended it. Participants indicated that the program (and its environment) was welcoming, friendly, safe and secure. It provided them with opportunities to grow and develop their skills, competencies and interests (e.g., conflict resolution and anger management). They felt that they were respected, appreciated, supported and rewarded while participating in the program. Further, Cadets indicated that the program allowed them to make new friends – increasing their association with peers who possessed significantly more pro-social attitudes and behaviours than did their peers prior to Cadets. Finally, participants believed that the program engages youth in proactive activities, keeping them busy and 'out of trouble'. Cadet staff members, volunteers and other stakeholders also noticed positive attitude and behaviour changes in the Cadets – they are more outgoing and have more confidence, are better behaved at school, are more disciplined at home, and are more likely to interact with the RCMP in the community. Cadets were positive, overall, when it came to assessing their global satisfaction with the program – virtually all Cadets were satisfied with their experiences through the program (67.3% of Cross Lake participants and 77% of Hobbema participants rated their level of satisfaction as “very satisfied”). Almost all Cadets indicated that they were likely to recommend the program to a friend or relative and to encourage others to join it.

Conclusions

Challenges

Fidelity in Program Implementation – An overall challenge in the process evaluation was the lack of development of a tool to measure implementation fidelity. Another concern was program adherence as two of the project sites did not implement all of the key components of the CCC program. Finally, determining program exposure was an issue as the number, length and frequency of each of the key component sessions/activities was difficult to determine in some sites, and a clear understanding of overall program dosage was lacking. This may be due in part to the limited previous literature on the theory of change undertaken by this program.

Data Collection and Data Entry – The tools and plans for collecting and entering detailed monitoring and process evaluation data were not utilized and implemented as planned (e.g., detailed attendance records). As a result, the detailed information typically reported in this type of evaluation was not available. Overall, it is believed that a lack of human resources available to help manage, oversee and implement the CCC program resulted in this deficiency.

Program Staffing and Volunteers – Staff turnover, position vacancy and ongoing changes was a dominant reality throughout program implementation. For example, turnover and then vacancy in the Data Entry Clerk position in one site meant that no evaluation related data entry/cleaning work was completed by program staff, which was a fundamental requirement of the evaluation. Volunteer and parent involvement was viewed as an important element but one that was lacking in the operation of the program. There was very little involvement from anyone other than staff, RCMP, a few volunteers and the Elders. More leadership may have resulted in greater success.

Transportation and Program Space – Transportation was a significant issue. Based on the remoteness and distance between the participating communities, a significant amount of travel was necessary for implementation of the Cadet Program. Due to the limited number of drivers/vehicles and the inclement weather, several Cadet events were poorly attended, cancelled, or never planned. Having a permanent (and appropriate) space for 'Cadet Nights' was also viewed as a major challenge. In one site, concern was expressed for the current space, suggesting a need for a higher quality and a larger space.

Recommendations

Recommendation 1: Prior to program implementation, a community readiness assessment should be conducted.
An effective implementation strategy requires communities to assess their needs, resources in place, and the level of commitment. Before implementing a CCC program, the community's level of cohesion and stability needs to be assessed as these are necessary elements for successful program implementation. For the Kahkewistahaw First Nation Community CCC program, one of the greatest assets was the Kahkewistahaw community itself.

Recommendation 2: The program design should account for the social context of the targeted First Nation community.
The design of a First Nation CCC program must take into consideration the values, dynamics and realities of the community in which it is being implemented. For example, while the CCC program is geared towards helping high-risk youth develop pro-social values, in small Aboriginal communities the 'fishbowl effect' is evident where individuals are exposed to a higher level of scrutiny and attention than in larger centers where a new program is just one of many and operates in a highly populated setting where anonymity is more easily achieved. The Kahkewistahaw First Nation Community CCC program and the Touchwood Agency Tribal Council CCC program both had an 'open door' policy where youth of all risk levels were accepted. If this had not been the case, it would have had the effect of publicly naming these youth as 'problematic', potentially upsetting and offending the youth and their families.

Recommendation 3: During the planning of the intervention, MOUs with various partners should be implemented to enhance community engagement.
Partnerships are important to the successful implementation of CCC programs. First Nation Chief and Councils need to be engaged from the planning and proposal-writing stage through to implementation of the program. The Hobbema CCC program noted that regardless of the funding source, a partnership with the schools is a critical element in any effort to ensure the CCC program values are generalized to more than one environment.

Recommendation 4: Prior to the implementation phase, a thorough and realistic assessment of workforce stability and expected employee turnover should be conducted.
An organization's past turnover rate should be a key consideration in determining a community's capacity to take on a long-term multi-component project. Further, funding for CCC projects should be conditional on meeting and maintaining a minimal staffing requirement. For example, in the Touchwood Agency Tribal Council CCC program, one staff member acted as the Coordinator/Manager/Drill Instructor and was not able to keep up with all the demands these three roles entailed, and ended up neglecting overall program management and adherence to the requirement for collection and reporting of information. In the future, funding for CCC programs should be conditional on having one person assigned to each of these three key positions.

Recommendation 5: Parent/guardian engagement and volunteerism should be a formal component of CCC programs.
The support and involvement of parents/guardians is important for successful program implementation and also for reinforcement of the pro-social values being taught in the program. Because Cadet events involve a variety of activities and attendance of many youth, having volunteers is extremely beneficial to the program. For the Hobbema CCC program, this was an area to be improved, not only in terms of numbers of volunteers but also in terms of their willingness to participate.

Recommendation 6: Future funding agreements for CCC programs should include formal training for staff and volunteers.
Receiving adequate training is central, and everyone involved in implementation should participate. Given the fact that the Cadet Coordinator position is central to program success and candidates may not have all of the many key competencies required for the position, it is important to offer training upon hiring. Further, as military experience was one area of deficiency noted by the Cross Lake CCC program, having staff participate in the 2-week Cadet Cadre Instructor Training would be beneficial. While the Kahkewistahaw First Nation Community CCC program offered one-on-one training to volunteers, it was felt that more formal training should be provided so that all volunteers receive the same information regarding how to interact with the Cadets and promote the values being taught to program participants.

Recommendation 7: CCC programs should find a balance between structured and unstructured programming.
It is possible to find fun and creative ways to learn and teach others about important life skills. The Hobbema CCC program suggested the use of guest speakers to address various health issues, addiction and mental health issues, and issues of bullying, gang involvement, etc. Further, the implementation of community service learning projects was an important component which can have a significant effect on both the Cadets and the communities they are helping. 

Recommendation 8: While an 'open door' admission policy may be appropriate for First Nation communities, the opportunity to remain in the program should be conditional on regular attendance.
For the Touchwood Agency Tribal Council CCC program, while well-intended, the decision to allow sporadic attendance ended up harming the program. Having youth come and go limits the progression of drills which affected the amount of drill-related learning that occurred, but left both the RCMP staff and youth feeling frustrated with the repetition and lack of progress. Further, when attendance is sporadic it creates uncertainty and chaos. For example, important implementation considerations such as transportation needs, size of space needed, snacks needed, number of staff or volunteers needed for supervision etc., are difficult to determine.

Recommendation 9: If a CCC program includes both children and youth participants, separate program sessions and activities should be available for these two groups.
There should be separate sessions for different age groups with activities requiring various levels of discipline (reflecting levels of maturity of participants). During focus group discussions, older Kahkewistahaw First Nation Community CCC program Cadets said they thought two Drill Nights should be offered so that Cadets ages 11 and younger attend one night and Cadets ages 12‒18 attend on another night.

Recommendation 10: Future funding agreements for First Nation CCC programs should include the provision of hot healthy meals at Drill Nights and the establishment of reliable transportation.
Since the Drill Nights last approximately 1.5 hours and involve over an hour of physical activity such as marching, pushups and physical fitness or sport activities, it is important that Cadets have the nutrition necessary for this level of physical activity. Reliable transportation must also be in place for the duration of program implementation. The availability of reliable transportation is a key predictor of success for many programs, and in the case of CCC programs this is particularly true.

Recommendation 11: A communication strategy should be developed and maintained (reporting on findings and successes, and highlighting community and stakeholder support/endorsement) in order to maintain buy-in and support, and to secure additional funding to promote program sustainability.
One of the most important elements for ensuring support relates to CCC program initiated communication. The Hobbema CCC program suggested producing a regular newsletter with current information regarding program events and activities along with testimonials from youth. The Cross Lake CCC program suggested the implementation of a communication strategy to share these process evaluation findings with prospective funders and government departments and organizations.

When these recommendations are addressed, there will be an increased likelihood that this intervention will be able to address the outcomes related to increased attachment to and attendance/performance in school; enhanced life, leadership, communication, decision-making, and problem-solving skills; and increased pro-social behaviours and reduced anti-social and delinquent behaviours.

Appendix A

Date modified:

2022-08-09