Welcome to Condom Country Campaign: Interim Evaluation Report
AIDS Committee of Toronto
Toronto has witnessed a recent and dramatic rise in the proportion and absolute number of new HIV infections among men who have sex with men (MSM). In 1995, MSM accounted for 56% of all new HIV infections in Canada. By 1999, this rate had dropped to 49%. In the first six months of 2000, however, the proportion of positive tests attributed to MSM in Canada had increased to 60%.
Dr. Robert Remis of the University of Toronto has apparently modeled the annual incidence of HIV infections among Toronto MSM. His models are reported to have estimated that the number of new HIV infections among this group of men rose from 350 new HIV infections in 1996 to 700 new HIV infections in the year 2000.
Amidst reports of this rise in infection rates, the AIDS Committee of Toronto (ACT), in co-operation with partner agencies, prepared a proposal for the Ontario Ministry of Health and Long Term Care’s AIDS Bureau to launch a new awareness campaign. The Welcome to Condom Country campaign was launched in June 2001 to coincide with Toronto’s annual Pride Week activities. Funding for the campaign was received from Ontario’s Ministry of Health and Long Term Care and Health Canada. The campaign included media exposure through a variety of approaches and the production of related materials and events. The campaign concluded in December 2001.
This campaign was designed to be a new type of response to the community’s need for education. ACT, along with other AIDS service organizations (ASOs), have continued to be involved with AIDS education and outreach activities over the years but the rise in infection rates caused people to stop and think about new approaches that should be attempted. The following quote from the funding proposal summarizes the situation:
“Over the past five years, the visibility of HIV prevention materials has diminished considerably within gay community venues and within the media. While ASOs still provide direct outreach in bars, bathhouses and public parks, and maintain information boards in gay commercial venues, our HIV prevention materials cannot compete with ads from commercial advertisers who saturate the gay press and commercial venues.”
The challenge, therefore, was clear. How to get the message about a rise in HIV infections to men who have sex with men in Toronto in the midst of a market saturated with other ads and information? How to get people’s attention? How to make people notice a new message?
In addition, the challenge was made more complex by the fact that Toronto is an extremely diverse city. Is it possible to create one message and one campaign that would be successful at having an impact on men who have sex with men who live in a variety of social and cultural contexts?
The challenge of this campaign was significant. Toronto was experiencing a resurgence of new HIV infections while ASOs continued to engage in prevention education activities. It was impossible to say whether the prevention activities were becoming less effective or if there were other factors at work. It was incumbent on service providers to review their activities to ensure that new opportunities were explored for getting the message out. The Welcome to Condom Country campaign was a new approach to an old problem.
This report is the interim report of the evaluation of the Welcome to Condom Country campaign. Shea & Company was engaged by ACT to conduct an independent evaluation of the campaign to determine the level of success of the campaign.
Serendipitously for the campaign’s evaluation, a study was being planned by researchers at the HIV Social, Behavioural and Epidemiological Studies Unit of University of Toronto’s Faculty of Medicine. This study, Ontario Men’s Study (OMS), is the largest ever Canadian study of men who have sex with men. This venue-based survey is being administered in the first quarter of 2002 and is expecting to have a sample size of approximately 5,0000, with 50% coming from the Toronto area. The survey is being administered in eight languages. Investigators for the study agreed to include a series of questions directly related to the campaign. The OMS is currently underway and results are due in the Fall of 2002. Once these results are available, they will be analyzed and included in a Final Evaluation Report.
It is important to note and thank three other contributors to this report:
· James Murray, ACT – provided the information regarding community marketing activities
· Cohn & Wolfe (public relations firm) – provided the information regarding media exposure
· Robert Vitella (senior media planner) – provided the information regarding the advertising activities
2.0 Campaign Plan
ACT assembled a group of volunteers in an effort to ensure that the people who were involved would be people with experience from the private sector that could be helpful in crafting a new response to the need for heightened HIV awareness. The campaign was developed by an Emergency Task Force (ETF), in consultation with a Community Advisory Panel (CAP). The ETF consisted of a group of volunteers from the marketing, advertising and communications sectors. This group was charged with the responsibility of developing and implementing the campaign. The CAP consisted of representatives from partner agencies with a particular focus on ethno-specific communities. The CAP provided advice to the ETF and to ACT at various points in the process.
The campaign plan was established and a launch date was chosen to coincide with Pride Week at the end of June in Toronto. The campaign was to have several components:
· TV ads and PSAs
· Subway and Subway Station Posters
· Transit Shelter Ads
· Newspaper and Magazine Ads
· Posters and Handouts (e.g., introduction cards, condom inserts, drink coasters, etc.)
· Street and Building Banners
· ACT Website
· Presence in Pride Parade
The Welcome to Condom Country campaign was established with articulated objectives and a clear goal.
To reduce the rate of new HIV infections among Toronto MSM via a comprehensive social marketing campaign.
· Raise awareness of the increase in HIV infections among Toronto MSM
· Encourage the adoption and/or re-adoption of risk-reduction/safer sex practices
· Provide support for MSM that would like to practice safer sex
· Make safer sex practices a priority health consideration
· Reach beyond gay venues and media to MSM who may not identify as gay
3.0 Evaluation Framework
Before beginning the evaluation process, a framework was established to ensure there were agreed upon evaluation principles, and an appropriate data collection plan and logic model.
3.1 Evaluation Principles
The following principles have guided the development and implementation of the evaluation:
¨ Independent: to ensure independence of the evaluation, data collection, analysis and evaluation report writing will be conducted by external contributors and consultants;
¨ Practical and Feasible: It should be possible to conduct within the resources and time available. This includes setting reasonable objectives for the type and amount of data to be collected; and
¨ Focused: the evaluation will focus on clearly defined purposes, rather than attempt to gather a wide range of information that might be useful, but for which no identified purpose/evaluation question has been identified.
3.2 Data Collection Plan
Data Source & Method
|Has the campaign raised awareness of the increase in HIV infections among Toronto MSM?||Percentage of participants who indicated raised awareness as a result of the campaign||OMS &|
|Has the campaign had an impact on the adoption and/or re-adoption of risk reduction/ safer sex practices among Toronto MSM?||Self -reported impact of campaign on adoption and/or re-adoption of rr/ss practices||OMS & |
|Has the campaign provided support for Toronto MSM who would like to practice safer sex among those who don’t currently practice safer sex?||List of activities|
Extent to which participants perceived the campaign message as affirming and supportive of a positive self-image for MSM
|Focus Groups |
& Activity Report
|Has the campaign contributed to making safer sex practices a priority health consideration?||Extent to which participants believe the campaign has caused people to “stop and think”||OMS &|
|Has the campaign reached beyond the gay community venues and media to MSM who don’t identify as gay?||Proportion of the GTA pop reached (Men 18+) |
Amount of media coverage generated
Amount of media exposure outside the gay community
|Has the campaign had any unintended impacts/benefits?||Extent and type of unintended impacts/benefits as identified by participants and PR report||Cohn & Wolfe|
|Are there any ways that the process of developing and implementing the campaign could be improved? (A separate report of the Process Review was provided to ACT in January 2002.)||Extent and type of process concerns and suggestions for improvements||Process Review|
3.3 Logic Model
A logic model is often developed to assist in program evaluation. Logic models help to illustrate how the objectives and activities in a program, for example, are connected. In short, a logic model is a graphic representation of the logic behind a particular program or campaign. The following is the logic model prepared to illustrate the logic of the Welcome to Condom Country campaign:
This section of the report provides the results of the various data collection methods outlined in the data collection plan. Together, these results provide all of the evidence that contributes to the campaign’s evaluation.
4.1 Advertising Activities
Information Source: Robert Vitella, Senior Media Planner
Advertising exposure was a critical part of the campaign strategy. The planned elements of the advertising campaign were television ads, print ads, indoor posters, outdoor posters and Internet ads. All of the elements were executed except the Internet activity. It appears that the time restrictions and some internal organizational issues contributed to this part of the campaign not being followed through.
Television ads were targeted to specific television shows, including nationally broadcast shows of interest to men who have sex with men, local television news broadcasts and local ethno-cultural media. Posters were placed indoors in bars and outdoors on garbage/recycling boxes, transit shelters, subway platforms and inside subway, bus and streetcars. Many of the suppliers provided additional exposures at no-charge.
It should be noted that Naked Creative, the advertising agency that developed the Welcome to Condom Country campaign, donated their services and their work has been recognized for creative excellence within the industry.
The following are the results from the advertising activities:
· The total number of planned impressions (i.e., the number of times that the ad was seen by someone) over the period of June 2001 to December 2001 was 50,630,200. The total number of actual impressions (including all exposure delivered at no-charge) was 56,616,200. A total of 5,986,000 (i.e., approximately 12%) additional impressions were delivered.
· Transit shelter ads accounted for approximately 76% of all impressions.
· Media exposure spanned the targeted and general markets to ensure that the campaign would reach beyond the traditional gay market. For example, the interior subway, bus and streetcar ads, local television news broadcasts and ethno-cultural media exposures all provided access to MSM who are not identified with the gay community.
· The total cost of this media buy was $136,941, while the actual value was calculated at $204,608. This represents a 49% bonus over the paid advertising. This indicates just how much the campaign, as a community-based non-profit effort, was able to leverage donated advertising.
4.2 Media Exposure
Information Source: Cohn & Wolfe
The campaign was launched in the context of the release of a white paper outlining the current situation and setting the stage for a response. This was a critical element to the public relations strategy that was aimed at ensuring that the message that “HIV is on the rise” would be heard.
The strong creative that was used for the campaign created some controversy. The cowboy images were recognized as being related to a well-known image used by a cigarette company. For quite a period of time after the launch of the campaign, there was significant discussion in the media, locally, nationally and internationally, about the particular images chosen for this campaign. This controversy also created opportunities for additional media exposure that could include the key messages of the campaign.
All the major newspapers in Toronto and many newspapers across the province and across the country ran stories on the campaign. In addition, many actually printed one of the ads to run with the story. Of particular note is the fact that the Toronto Sun ran the ad on the front page of the paper on June 19, 2001. This provided significant additional exposure for the campaign. It has been noted that the fact that the ads were not sexually explicit enabled them to be accepted more easily by news organizations and advertising venues, such as the transit commission.
The following are the key results from the public relations activities:
· Over 43 million media impressions were generated. This is “free advertising” that is equivalent to what would have cost $114,000 to buy. The total funds expended for PR activities was $46,500.
· 82% of the coverage was in Ontario and the remainder was spread throughout the other provinces.
· 82% of the coverage was in print, followed by 10% on TV and 8% on radio.
· 51% of the coverage was positive, 34% was neutral and 15% was unfavourable.
· 61% of “letters to the editor” in various publications were positive, 31% were negative and 8% were neutral.
Cohn & Wolfe, in their Public Relations Results Report, made the following conclusion:
“This campaign was truly breakthrough – few social marketing campaigns garner the media’s attention for such a prolonged period. Although the campaign launched in June, coverage continued into September and even December, when it was highlighted in the Globe & Mail wrap up of the year’s most provocative campaigns.”
4.3 Community Marketing Activities
Information Source: James Murray, ACT
Community marketing activities enabled the campaign to have a very local and personal component in addition to the media and advertising exposure. These activities began with the launch of the campaign during Pride Week. The Pride Parade included a float, along with 200 volunteers marching in t-shirts and cowboy hats and distributing condom packs. A Condom Country theme party was also held during that week at a restaurant/bar on Church Street. There were several bar and bath outreach nights where Condom Country materials were distributed.
The following table indicates the distribution level for each of the community marketing materials:
How many distributed
|Large Banner Signs|
|Business card promos (greet cards)|
|Business card (SM promo)|
|Condom inserts placed in packs|
4.4 Focus Groups
Information Source: Focus Group Notes
Three focus groups were conducted by Shea & Company in February 2002. A total of 23 men participated in the focus groups. Participants were recruited through announcements that appeared in local gay press and information faxed to all local AIDS service organizations. Participants were required to register in advance by email or phone. It was a requirement that participants be gay and bisexual men who had seen some of the campaign materials and who were between the ages of 25 and 44. Although demographic information was not collected, it appeared that participants spanned the target age range. All participants were paid $20 for their time.
The following are the key results from the focus groups:
· All participants were asked to indicate which campaign materials they recalled seeing. 20% of the responses indicated exposure to ads in newspapers. This was followed by 16% for outdoor signs on Church Street, 15% for pamphlets and handouts and 14% for posters in bars. The detailed results follow in the table below:
# times mentioned
% of total mentions
|Church Street Outdoor Signs|
|Pamphlets & Handouts|
|Posters in Bars|
|Mural on Side of The Barn|
· It is important to recognize that the focus groups did not consist of a random or representative sample of the target audience and therefore their responses are not necessarily representative of the kind of exposure or recognition in the general target audience. However, there are some interesting qualitative observations that can be made nonetheless. For example, five participants noted that they had seen the Condom Country mural painted on the side of one of the bars on Church Street. As it turns out, the mural is actually an advertisement for a beer company but it uses similar creative with two cowboys as the central feature of the image. These five participants all thought that ad was part of the campaign. This may speak to the power of the Condom Country image.
· Also noteworthy from the above table is how it appears to have been a good idea to have multiple channels for the message as different people seem to have been exposed through different channels.
· The majority of participants indicated that they believed the main message of the campaign was about reminding people about the need to use condoms because HIV has not gone away. In addition, some people saw the use of cowboys to also have a message specifically about barebacking. Some found the men in the image to be so attractive that it either glamorized HIV or it made you think that it’s ok to use condoms because these attractive guys use condoms.
· Participants were asked if they thought the campaign had had the effect of raising awareness about the rise of HIV in the community. Approximately half of the participants agreed that the campaign had raised awareness. Some noted that their own behaviour had changed as a result of their awareness being raised. Others noted that it may have reinforced something that people already knew about. In addition, some people thought that the image created barriers to people receiving the message. For example, the use of cowboys for an urban setting was problematic to a few. For a few others, the image communicated to them that all gay men have HIV. In these cases, the message may have gotten lost.
· Only a few participants indicated that they thought the campaign provided support for people to practice safer sex. Although several people thought the campaign itself was supportive of the idea of safer sex, participants generally thought that it lacked a clear message about how to get more information to support actual behaviour change. Others noted that more practical information, for example about choosing the right condom, would have been helpful.
· Most participants indicated that the campaign had been successful at getting safer sex back into conversations. A few participants noted that they had had conversations with friends about condom use as a result of the campaign. Others noted that the controversy about the campaign and the image used in the campaign were more the subject of conversations that people were having.
· Participants sighted unintended impacts or benefits of the campaign. Some thought the campaign re-enforced negative stereotypes of gay men (i.e, that all gay men have HIV) or the idea that only gay men get HIV. Many others noted that the unintended benefit of the controversy related to the images used in the campaign is that there was much more coverage in the media as a result. A few participants were concerned that the image was so beautiful that the message may have gotten lost.
· There was little consensus among participants regarding the relevance of the campaign and its images to their communities. Some said they could relate to the images because they were obviously gay men. Others thought the cowboy image was too distant from the urban multicultural experience to be relevant. Still others found the multi-racial models were a good way of making the image relevant to more people.
5.0 Summary and Conclusions
The interim results noted above provide a good picture of the information available to date to be able to provide an interim judgment about the success of the campaign. It is important, however, to recall that results from the Ontario Men’s Survey will enable the evaluation to be more robust. Therefore, the summary and conclusions provided in this section should be interpreted as interim results only.
Evaluation Question #1 - Has the campaign raised awareness of the increase in HIV infections among Toronto MSM?
The primary source of information for this question will be the results provided by the Ontario Men’s Survey. Focus group information indicates, however, that it appears that the campaign has had an impact in this area. Approximately half of the participants indicated that they thought the campaign had raised awareness and the majority had understood the main message to be about raising awareness and reminding people to use condoms.
A number of focus group participants also noted that the use of the cowboy images and the resulting controversy may have had the effect of re-directing people’s attention such that they did not have their awareness raised regarding the rise in HIV infections.
Interim Conclusion: While there is only limited data available to assess this question at this time, it appears the campaign may have had at least a moderate impact on raising awareness, at least among gay men, about the rise in HIV infections.
Evaluation Question #2 - Has the campaign had an impact on the adoption and/or re-adoption of risk reduction/ safer sex practices among Toronto MSM?
Once again, the Ontario Men’s Survey will provide the results required to be able to answer this question. Focus group participants were less enthusiastic about the campaign’s effects in this area. Many noted that they have not noticed a change in behaviour but a few noted that there has been an impact on their own behaviour and those of their friends.
A number of focus group participants also noted that the cowboy images that were used actually presented a barrier for some people. This barrier, in their opinion, prevented the risk reduction message from getting through to some people. The barriers noted related to the use of images that were either not relevant to an urban multicultural environment or too glamorous to have any impact.
Interim Conclusion: While there is only limited data available to assess this question at this time, it appears the campaign may have had at least a minor impact on the adoption and/or re-adoption of risk reduction/safer sex practices.
Evaluation Question #3 - Has the campaign provided support for Toronto MSM who would like to practice safer sex among those who don’t currently practice safer sex?
This evaluation question, and the related objective, is trying to get at whether the campaign provided people with information that would enable them to get the support they need to make desired behaviour changes. Focus group results indicate that people generally felt that this objective was not achieved. Many noted that there was no obvious link to practical information. However, it is important to note that focus group discussions tended to centre on the advertising materials more than the community marketing materials.
Some of the focus group participants also noted that they thought the campaign images presented a very positive image of gay men, one that would help to make them feel good about themselves. This could be considered as a “support” to the practice of safer sex in that people who feel good about themselves are more likely to take necessary precautions to protect their health and safety.
Others noted that they thought the campaign images actually presented a negative image of gay men. The fact that HIV has been associated with gay men in Canada and that only men were used in the images, caused some participants to think that the ads were reinforcing the idea that only gay men are at risk of HIV. Some thought this might actually re-enforce a stereotype image that keeps non-gay identified men from thinking they are at risk.
Interim Conclusion: It is difficult to make a conclusion on this evaluation question. Did people feel supported as gay men? Some did and some didn’t. What about non-gay identified men? It is possible that the use of what people have referred to as “gay images” may have presented a barrier for non-gay identified MSM. Generally, the information to date indicates that this objective has not been well achieved.
Evaluation Question #4 - Has the campaign contributed to making safer sex practices a priority health consideration?
This question is concerned with the idea that the campaign was meant to get the topic of safer sex back into popular conversation. There is no doubt that this campaign made people stop and look. It grabbed people’s attention, as is evidenced by the amount of press coverage and the focus group discussions.
Focus group participants noted overwhelmingly that they were aware of or had participated in conversations with friends about the Condom Country campaign. Again, it is important to note that some of this conversation may have been more related to the controversy or the images themselves, rather than about safer sex. However, it can be said that even this type of conversation occurs within the context of a general understanding of the message that is being conveyed. In that sense, the controversial elements of the campaign could actually fuel greater conversation that includes a raised awareness of the key message.
Interim Conclusion: The campaign appears to have had a significant impact on conversations, at least among gay-identified men. Evidence suggests that the campaign sparked many conversations that brought safer sex back into discussions among friends.
Evaluation Question #5 - Has the campaign reached beyond the gay community venues and media to MSM who don’t identify as gay?
This campaign was designed to get access to MSM who are not gay-identified as this group of people has traditionally been difficult to reach with HIV awareness activities. Choices were made to place ads on such television shows as local news broadcasts in order to reach MSM who are not gay-identified and may not be exposed to ads and posters on Church Street or in the gay press.
While it is extremely difficult to survey non-gay-identified MSM, the Ontario Men’s Survey is attempting to capture some information from this group. It is not difficult, however, to ensure that there is some level of exposure to the general population. The assumption is that this would include exposure to non-gay-identified MSM.
Media buy reports indicate that there was significant exposure of the television ads on a variety of both national and local programming that is not gay-specific. In addition, ads were placed in NOW and EYE magazines, local news and entertainment weeklies, and ads appeared in public transit vehicles that traversed the city. Of particular note is the fact that ads appeared in some of the local ethno-cultural newspapers. In at least one case, it was the first time that an ad that involved AIDS or gay men had appeared in such a publication.
Interim Conclusion: The campaign had significant success in reaching beyond the gay community venues and media to reach MSM who do not identify as gay.
Evaluation Question #6 - Has the campaign had any unintended impacts/benefits?
The simple answer to this question is yes. It would be difficult to execute this type of campaign without having unintended impacts or benefits. The challenge is to have them be positive benefits. The Condom Country campaign created some controversy as a result of the decision to use an image that was identified as having similarities to a popular cigarette advertising image. This controversy certainly had the benefit of creating much media coverage and causing many conversations to ensue.
For some, it seems the images used may have created a barrier to fully appreciating the message. It is important to note that many understand that creating an image or campaign that will reach everyone in this type of diverse community is virtually impossible. However, there is general support from the evidence to date to indicate that the resulting coverage was mostly positive and supportive of the key message.
Interim Conclusion: The campaign had some unintended impacts but the evidence suggests that they did not significantly detract from the key message.
In conclusion, from an interim evaluation perspective, it appears the Welcome to Condom Country campaign had the following results:
· More evidence is required, however, it appears the campaign may have had at least a moderate impact on raising awareness, at least among gay men, about the rise in HIV infections.
· More evidence is required, however, it appears the campaign may have had at least a minor impact on the adoption and/or re-adoption of risk reduction/safer sex practices.
· The campaign appears to have not had a significant impact on the provision of support to practice safer sex.
· The campaign appears to have had a significant impact on conversations, at least among gay-identified men. Evidence suggests that the campaign sparked many conversations that brought safer sex back into discussions among friends.
· The campaign had significant success in reaching beyond the gay community venues and media to reach MSM who do not identify as gay.
· The campaign had some unintended impacts but the evidence suggests that they did not significantly detract from the key message
There are only two objectives that appear to have not been achieved or where there appears to have been only a minor impact. Both objectives relate to the intention that the campaign actually have an impact on changing behaviour. Overall, the campaign’s success should not be tarnished by this particular failure. It is possible that these particular objectives were beyond the capacity of this type of campaign. Awareness raising is the first in a series of steps that need to be taken in social marketing. The next campaign or related efforts may need to help people take the next steps towards behaviour change.
While it is reasonable to say, at this time, that the Welcome to Condom Country campaign has at least been moderately successful at achieving its stated objectives, this interim assessment will need to be updated once the data from the Ontario Men’s Survey is available.