Evaluation of an Innovative Cyberbullying Intervention: STOPit: Summary Report

Evaluation of an Innovative Cyberbullying Intervention: STOPit: Summary Report - PDF Version (482 KB)

Table of contents

• Introduction
  • Background
  • Evaluation Questions
  • Methods
    • Methodological Limitations
• Results
  • Environmental Scan and Comparative Analysis
  • Potential Relevance
    • Simulated Scenarios
  • Potential Performance
    • STOPit App
    • DOCUMENTit System
• Discussion
  • Implications
• Next Steps
  • Opportunities for further research
• References

by Wendy Ryan and Donna Smith-Moncrieffe

Derived from Evaluation Report by John Gelder (Senior Evaluator) and Eva Maxwell (Evaluator), Gelder, Gingras and Associates

Abstract

Cyberbullying can have serious negative effects on the emotional, social, and physical health of children and youth. There is little research on cyberbullying prevention and intervention programs. This evaluation aimed to assess the potential relevance and performance of one innovative cyberbullying platform called STOPit developed to encourage the anonymous reporting of cyberbullying. Several key findings were identified: Youth found the STOPit platform to be highly functional, useful and innovative, particularly with regards to its anonymous feature and the possibility of sending fast text or video/image reports; youth also indicated that trust, reputation, and credibility were necessary precursors for their potential use and acceptance of the platform. Technical functionality and capabilities, reporting and follow-up features, ease-of-use, and privacy protection are also highly important to youth. An important next step would be to evaluate the impact of STOPit or a similar application in Canadian school settings. A randomized control trial or quasi-experimental design using matched comparison schools within a school board would provide concrete data regarding the impact of STOPit or a similar application on the prevention of cyberbullying.

Author’s Note

The views expressed are those of the authors and do not necessarily reflect those of Public Safety Canada. Correspondence concerning this report should be addressed to:

Research Division, Public Safety Canada
340 Laurier Avenue West
Ottawa, Ontario K1A 0P8
Email: PS.CPBResearch-RechercheSPC.SP@ps-sp.gc.ca

Acknowledgements

This research report is a summary of the following contract deliverable:

Gelder, Gingras & Associates (June 2017). Evaluation of a Cyberbullying Prevention/Intervention Initiative: Final Evaluation Report. Prepared for Public Safety Canada, Community Safety and Countering Crime Branch, Research Division.

We would like to acknowledge the invaluable contributions of Professors Andrea Slane, PhD and Jennifer Martin, PhD who served as advisors to this project.

We would also like to thank the STOPit organization for its cooperation on the project, including providing access to the platform for evaluation purposes. This evaluation would not have been possible without their support.

Introduction

Background

Cyberbullying has been defined as intentional, repeated harm inflicted through the use of computers, cell phones, and other electronic devices (Hinduja & Patchin, 2015). The intent of cyberbullying incidents is to threaten, harass, embarrass, or socially exclude individuals using online technology (Williams & Guerra, 2007).  Cyberbullying can have serious negative effects on the emotional, social, and physical health of children and youth. Those who are cyberbullied may feel intense feelings of isolation, fear, loneliness, and despair, which may lead to acts of peer or self-harm (Prevnet, 2015).

Although there are similarities between online and offline bullying, certain contextual factors characterize cyberbullying and may lead to even greater negative impacts for the victim. These include anonymity, increased dissemination, lack of supervision present on electronic media, and greater accessibility to the target (Hinduja & Patchin, 2008).  These contextual factors have implications for the development of appropriate cyberbullying interventions.

In their research, Hinduja and Patchin (2015) found that students are often reluctant to discuss their cyberbullying experiences with adults and many fear potential negative consequences if their peers find out that they “snitched”. Therefore, these researchers suggest that an effective response to cyberbullying may be to create an anonymous reporting system through which students can quickly alert school staff to harmful or inappropriate behaviours. One such system, called STOPit, has recently been developed.

The Stopit application (app) allows victims and witnesses to submit text messages or screenshots of bullying/cyberbullying incidents using the device. These are received by a designated school staff member who has access to the DOCUMENTit app and can then respond to the student by text message and follow up on the incident. The STOPit app allows students to remain anonymous and seek help or guidance from an adult experienced with using the application.  The app can be accessed via smartphone, tablet or computer.

Evaluation Questions

The purpose of this evaluation was to shed light on the potential relevance and performance of an electronic platform to prevent or intervene in cases of cyberbullying online among youth. The questions guiding this evaluation are listed below:

• How does the STOPit application compare with other cyberbullying initiatives in terms of cost?
• Potential Relevance: How relevant is the STOPit platform as a cyberbullying prevention/intervention platform in the context of Canadian schools?
• Functionality: How do potential users perceive the functionality of the app?
• Potential Performance: How effective could STOPit be in preventing or reducing the incidence of cyberbullying among school-aged children and youth?

Methods

In this evaluation, the STOPit platform’s potential relevance and performance were assessed using a series of focus group sessions among school-aged youth (12-17) and adult professionals. To provide added context to the evaluation, an environmental scan was conducted including a review of literature pertaining to cyberbullying prevention technologies and a comparative analysis of similar technologies and apps. Each of the initiatives found were assessed for country of origin, type of initiative, technological compatibility, cost, intended audience, type of intervention, anonymous features, and point of intervention (preventative or reactive^Footnote1 ).

Four (4) focus group sessions were conducted, involving a total of 29 participants (7-8 youth per session).  The participating youth were divided into groups by age and gender (males 12-14; males 15-17; females 12-14; females 15-17). Participants were all Anglophone and resided in the National Capital Region of Canada (i.e., Ottawa or Gatineau).

To stimulate thinking and provide the participants with an opportunity to test the STOPit app, the evaluators designed and pre-tested a set of three scenarios, including scripts and videos.

• Scenario 1: WhatsApp group conversation between a group of friends. The group is speaking negatively about another student at school and planning to physically harm him.
• Scenario 2: iMessage conversation between two friends. One friend is exhibiting signs of depression and thoughts of self-harm.
• Scenario 3: Instagram feed in which multiple students are shaming and bullying another student by posting explicit and hurtful comments on her pictures, uploading altered and embarrassing images, and threatening to share sensitive images.

All scenarios were fictional (created by the evaluation team), and the content was validated for realism by two school-aged youths. Each simulation (video) lasted 1:50 to 3:00 minutes. During and following each simulation scenario, participants were given an opportunity to use the STOPit report function to send in an incident report to the DOCUMENTit users. Participants were instructed to consider how they might use the app in an actual social situation. The scenarios stimulated participant engagement with the app and elicited test reports. In total, 29 “test” reports and 69 reports responding to simulated scenarios were submitted over the four sessions for a total of 98 reports. Reports were accompanied by reciprocal instant messaging between the DOCUMENTit users and focus group participants. 

Four or five adult professionals (including a mix of educators, counselors, and social researchers) were recruited for each of the four focus group sessions. They were invited to test the DOCUMENTit software and interact with the students through the software, observe the youth focus groups, and provide feedback during their own focus group sessions that took place after the youths’. In total, there were 10 observers, some of which were present for more than one session. These professionals observed the youth focus group sessions and tested out the software in a separate viewing room with one-way mirrors.

Methodological Limitations

While the simulation exercises used during the focus group sessions allowed the evaluation team to arrive at findings related to the potential utility and relevance of the platform, the approach does not allow for direct attribution of results or assessment of impact. For example, this evaluation did not determine if the app could prevent harm or stop cyberbullying more than another app.

Another methodological limitation is the non-representative sample recruited for the focus group sessions. For instance, the sample of participants was drawn from one linguistic group (Anglophones) in one geographical area (the National Capital Region). It is possible that differences may exist depending on geography and linguistic group. As well, the number of youth participants (N= 29 participants or 7-8 per session) may not be reflective of the larger Canadian youth population and may not represent the cultural or socio-economic diversity of Canadian youth.

Finally, readers are reminded that this evaluation included the examination of one tool/initiative (the STOPit platform). STOPit was found to be reasonably representative of similar tools and technologies used for similar age groups (e.g., in terms of design, target population, and intended purpose and use); however, the environmental scan showed that there are many different tools/initiatives intended for related purposes which have different features and benefits. Thus, findings for the specific tool cannot necessarily be generalized to similar or related tools and technologies that may exist for this purpose.

Results

Environmental Scan and Comparative Analysis

Thirty-four (34) cyberbullying initiatives were found in the literature. These were analyzed and categorized into seven (7) program types based on the primary method used to intervene in cases of cyberbullying. The most common types of initiative are monitoring/tracking tools (35%) which inform parents/guardians of activity on a youth’s cell phone, tablet or computer. Also popular are educational initiatives (20%) which seek to inform technology users of strategies and techniques to prevent or intervene in cases of cyberbullying. Fifteen percent (15%) of the initiatives reviewed in this study were identified as school-wide programs, including STOPit. These programs are intended to be purchased and activated by the school to inform and equip all students and staff with the information and tools necessary to prevent or intervene in cases of cyberbullying. Purchasing the STOPit program, for example, provides schools with the software to be used by all students and staff, access to 24/7 customer service, and a suite of informational and educational tools, materials, and guides. Also reported in the literature are blocking tools (9%), reporting tools (9%), games (6%), and text altering programs (6%).

The most expensive initiatives were the school-wide programs; each provides licensing for the initiative based on a fee-per-student model (e.g., $0.50 to $1 per student), usually with a minimum cost for smaller schools. Minimum annual costs for these programs fall between $499 to $795; for example, Bullybox ($499), STOPit ($500), Tootoot ($681), and CyberBullyHotline ($795).

Monitoring tools were the second most expensive type of initiative, with costs varying from $19.99 (Safe Eyes) to $179.88 (My Mobile Watchdog) per year per family, with two being offered for free (GoGoStat and Web Safety, Inc.).

Educational apps, strategies, and tools often charge a small fee for access to content, ranging from $2.59 (Bully Proof Assistant Prof) to $19.00 (Professor Garfield Cyberbullying), but some are free. Many blocking tools, games, online reporting tools, and text altering tools are available for free download through the Apple App Store or Google Play Store.

In this review of cyberbullying initiatives, the overwhelming majority of programs, including STOPit, originated from the United States (70%). Using US software in Canada may have some implications for privacy that need to be further explored.

Although most of the initiatives reviewed originated in the United States, the majority (67%) can be accessed and downloaded worldwide.

The STOPit platform is a comprehensive, accessible application that is similarly priced or less costly than other school-wide cyberbullying programs that were reviewed.

Potential Relevance

Simulated Scenarios

When asked how they felt about the simulated situations, most youth stated that the situations were serious and that they were similar to situations they encounter in their day-to-day lives. When asked to reflect upon which incident, if any, they would report if the situations had been real, participants confirmed that the likelihood of reporting depends on the context and severity of the situation. For instance, the participants were likeliest to report the scenario in which a student was exhibiting serious signs of depression and self-harm. The results are presented in Table 1 below:

N=29

With respect to relevance, youth focus group participants indicated that current approaches to preventing or intervening in cases of cyberbullying, and indeed bullying in general, are minimal in their effectiveness or lacking altogether. Existing approaches tend to provide information but do not provide adequate solutions or tools. Youth noted several types of presentations and activities which take place within their schools, including assembly-type presentations; school-wide awareness activities (e.g., Pink Shirt Day); classroom discussions with school counsellors or teachers (e.g., in leadership or health classes); and plays, videos, or presentations by external organizations, including police officers. During these discussions and presentations, youth have been advised that the best solution to cyberbullying is to tell someone (e.g., an authority figure).

Only two of the 29 participants mentioned having access to a formal reporting tool for cyberbullying (e.g., an anonymous school-initiated website form or the application PS Tolerance Zero) in their schools.

Participants agreed unanimously on the usefulness of an accessible (i.e., on their smartphones or online) and anonymous platform. With regard to features, anonymous reporting was the most important point for them. As one participant stated, “the screen protects you each way. If it is easier for someone to bully in front of a screen, then it is easier to report in front of a screen.”

Youth mentioned two different ways in which they would potentially use the app. The first would be to seek advice or to reach out to someone with whom to talk. Advice could take the form of methods of resolving conflicts or intervention. The second way would be to report incidents of bullying or cyberbullying. In this regard, the STOPit platform has been designed to be used for both purposes (providing advice/guidance and responding to reports).

Although almost all participants thought that STOPit was useful (28/29), a smaller number of participants thought that the app was needed in their schools (21/29). Significantly more female group participants (93%) than male participants (53%) felt that there was a need for the platform.

Among those who did not think there was a need for the platform, reasons included small school size (e.g., only 300 students) and supportive school environments (e.g., highly involved and proactive teachers). Thus, the perceived need for STOPit or for similar platforms appears to depend, at least to some degree, on specific attributes and contextual factors specific to a particular school.

Some male participants added that although the platform might be useful, they would hesitate to depend on it as a stand-alone solution, preferring instead a hybrid approach (e.g., reporting through the platform as well as in person) or having a call-in option. This is mainly because they were not completely sure that they could trust the platform or because they feared that others might abuse or “troll” the app. Female participants emphasized that the platform could potentially be useful for those who do not have anyone else to speak with and that it could be a useful safety precaution. Despite these somewhat different viewpoints, almost all participants agreed that every school has serious problems offline and online that need to be addressed and that many incidents currently go unreported.

Potential Performance

The STOPit platform includes both the STOPit app, which allows student reporting, and the DOCUMENTit app, which is used by the designated school official who can then respond to the student and follow up on the incident.

STOPit App

The majority of youth focus group participants found the STOPit platform to be highly functional, useful and innovative, particularly with regards to its anonymous feature and the possibility of sending fast text or video/image reports (e.g., for capturing proof via screenshots).

Most participants saw potential in the STOPit app to prevent harm in schools. A phenomenon of note is the “upstander” effect wherein students take a stand, often publicly, for each other. Some of the participants gave examples of students known in their schools or on social media for defending victims of bullying or taking a strong stance. They noted that STOPit would be another way to empower students and give a voice to upstanders while fostering a culture of proactive reporting.

Youth indicated that trust, reputation, and credibility are necessary precursors for their potential use and acceptance of the platform. Technical functionality and capabilities, reporting and follow-up features, ease-of-use, and privacy protection (anonymity) are also highly important to youth.

Each youth focus group discussed the issue of trust in both the technology itself and in those who would be receiving their reports (responders). For instance, various participants indicated that acceptability among their peer groups, knowledge that the app was used in other schools or school boards, advertising, promotion, and reputation would increase their likelihood of using the app. Of great concern to all participants across all four groups was the level of trust in the person/people receiving the reports. In this respect, assurance about who would be receiving the reports and how qualified and competent they were to address incidents were important considerations. For instance, participants felt it would be important to know that there was a “real” person at the receiving end, especially one that had a connection to them/the school. Another source of reassurance would be to know that the responder had experience, could act, had the knowledge and education required to address reports, and had the correct authority/position to be receiving reports.

Youth noted some of the platform’s limitations. First, it was noted that STOPit only offers the possibility of reporting the actions of other students in the same school, although cyberbullying may frequently involve others outside of the school system or students operating under aliases or anonymous profiles. Second, while STOPit is seen to have the potential to empower youth, there may be a risk of also abusing the system (e.g., false reporting). In this respect, participants stressed the importance of monitoring and managing the system. Finally, there was some concern that although the platform offers anonymous reporting, bullies could find a way to discover who reported (“snitched”) and that this could have negative implications.

With respect to possible changes, additions, or areas of improvement, some youth mentioned that users could experience some confusion between the report and the messaging features, as well as some ambiguity regarding the correct use for the messaging feature. It was suggested that a possible remedy to this issue is the addition of an in-app tutorial, FAQ, or information page to explain the functions and their purpose. Additionally, it was suggested that it may be useful to have a separate message history for each report. One participant stated that the disclaimer at the beginning of the app was too long, too formal, and cumbersome.

Concerns about privacy including keeping messages/reports hidden from others was common across all groups. In this regard, suggested additions or changes to the platform included adding a password protect option, masking/hiding the app from prying eyes, and offering the possibility of deleting old conversations from smartphones (while retaining message history in the school’s database for proof and tracking).

Since knowing and trusting those on the receiving end of a report was seen to be critical for youth, some suggested adding an “About Us” section which would include profiles of the respondents as well as the capability of choosing who receives the report.

DOCUMENTit System

DOCUMENTit users found the case management system useful for capturing and tracking incident records as well as for enabling statistical analysis, although some participants experienced difficulty navigating the online system.

DOCUMENTit users in this study outlined several issues and concerns which may be generalized to any cyberbullying prevention/intervention platform. Most significant among these are questions about capacity (i.e., school time and resources for managing the platform) and accountability (i.e., setting appropriate parameters and defining the scope of the school’s mandate to deal with issues both online and offline).  

Discussion

Implications

Findings from the focus group sessions indicate that cyberbullying is a serious problem among school-aged youth and that current approaches to prevention and intervention are having little effect. Findings also indicate that platforms such as STOPit may be useful, under the right conditions, to facilitate reporting and management of incidents. Future research would be beneficial to further validate how these platforms contribute to mitigating the broad and complex risks associated with cyberbullying.

Findings from the environmental scan indicate that the most optimal approach to addressing cyberbullying is one that is holistic in nature. This means that a tailored mix of prevention (e.g., awareness building, education, blocking), intervention (e.g., reporting, addressing incidents), and post-incident resolution methods (e.g., counselling, mediation) is likely to be most effective compared to a one-size fits all approach.

Findings from the environmental scan and from the focus group sessions indicate that the multi-faceted issue of cyberbullying is one that requires a coordinated approach from a community of professionals and practitioners, including school administration, educators, health professionals, families, law enforcement, technology providers, researchers, and the private sector.  

Next Steps

Opportunities for further research

Given the breadth and complexity of challenges associated with cyberbullying (e.g., with respect to mental health, adverse effects, anonymity, privacy concerns, accountability), this evaluation was undertaken to shed light on cyberbullying prevention/intervention initiatives. The scope of this evaluation was limited to one initiative (STOPit). Although results indicate the potential relevance and usefulness of this initiative in the school context, findings point to the need for ongoing research, including the following:

Impact Evaluation: A randomized control trial or quasi-experimental design using matched comparison schools within a schoolboard would provide concrete data regarding the impact of STOPit or a similar application on cyberbullying. Such a research project would require buy-in at the schoolboard level and school level as well as support by students and their parents.  

Although the focus group approach identified the acceptability, potential relevance, and usefulness of a cyberbullying prevention/intervention application among youth and among adults and practitioners, focus group sessions alone cannot provide direct or conclusive information about the impact of this technology. An important next step would be to evaluate the impact of STOPit or a similar application in Canadian school settings using a rigorous research design.

Research on privacy laws: Privacy laws in different jurisdictions (e.g., US and Canada) have potential implications for the use of cyberbullying prevention/intervention initiatives. Further research could shed light on the implications of using non-Canadian technology within a Canadian context. From a broader perspective, comparative research on privacy legislation across jurisdictions could help to inform future decision-making regarding policy and legislation.

References

Hinduja, S., & Patchin, J. W. (2008). Cyberbullying: An exploratory analysis of factors related to offending and victimization. Deviant Behavior, 29, 129-156.

Hinduja, S. and Patchin, J.W. (2015). Bullying Beyond the Schoolyard: Preventing and Responding to Cyberbullying, 2nd Ed. Thousand Oaks, California: Corwin, Sage Publications.

Prevnet (2015). Cyberbullying. Retrieved from: http://www.prevnet.ca/bullying/cyber-bullying.

Williams, K.R. and Guerra, N.G. (2007). Prevalence and Predictors of Internet Bullying, Journal of Adolescent Health, 41, S14-S21.

Date modified:

2022-07-21