In this episode, Allie and David walk us through how Political Traction determines the top issues for the show every week.

Allie also interviews Navigator’s research lead, Chris Kelly, to discuss  the methodology and how it compares to traditional research methods.

To make sure our methodology is crystal-clear, we’ve provided a detailed breakdown of our process below.

Measuring the top issues in Ottawa:

To determine the top political issues that political leaders, pundits, and media are discussing in Ottawa every week, we track Question Period, political broadcasting, and political columnists.

Question Period:

To calculate the top issues from Question Period, we assign points based on who answers the question on each issue. We assign more points to the issue if the Prime Minister answers a question than a Minister, and if a Minister answers a question on an issue, we assign more points than other members of parliamentary. We weight the answers this way, because, for the most part, it’s soundbites of the  Prime Minister and his Cabinet from Question Period that get airtime on political broadcasts and nightly newscasts.

 

We break down the points like this:

Position Points per answer
Prime Minister 5
Minister 3
MP 1

In practice it works like this: let’s say that the Energy East Pipeline was discussed during Question Period. If the Prime Minister answered one question on the issue and a Minister answered one question on the same issue, we would allocate 8 points to the issue.

Issue Answer Points
Energy East pipeline Prime Minister 5
Energy East pipeline Minister 3
Total 8

 

Political Broadcasting:

Measuring the top issues from political broadcasting is very straightforward. For every minute an issue is discussed, we assign it one point.

If we use our example of the Energy East Pipeline again, it looks like this:

Issue Broadcasting minutes Points
Energy East pipeline 24 24

 

Political Columns:

To measure the top issues from political columns, we track a number of political columnists and assign one point for every article that is written on a given issue.

Issue Number of columns Points
Energy East pipeline 3 3

If we apply this system to multiple issues, it looks like this:

 

Question Period:

Issue Answer Points
Energy East pipeline Prime Minister 5
Energy East pipeline Minister 3
Canadian action against ISIS Prime Minister 5
Government deficit Minister 3
Trans Pacific Partnership Minister 3
Electoral reform MP 1

Broadcast:

Issue Broadcast Minutes Points
Energy East pipeline 24 24
Government deficit 22 22
Canadian action against ISIS 20 20
Electoral reform 15 15
Trans Pacific Partnership 10 10

Political columns:

Issue Number of Articles Points
Canadian action against ISIS 3 3
Government deficit 2 2
Energy East Pipeline 2 2
Trans Pacific Partnership 1 1
Electoral reform 0 0

Then, looking at all three mediums, our top issues for each look like this:

Question Period Broadcast Political columns
Energy East pipeline Energy East pipeline Canadian action against ISIS
Canadian action against ISIS Government deficit Government deficit
Government deficit Canadian action against ISIS Energy East Pipeline
Trans Pacific Partnership Electoral reform Trans Pacific Partnership
Electoral reform Trans Pacific Partnership Electoral reform

To determine the top issues for the overall conversation, we combine our three lists. We weight each medium equally, to ensure that if, for example, one item received a lot of attention in Question period but not very much broadcast time, it would still be considered as part of the conversation. To weight each medium, we look at our three lists and assign points based on their rank on each list. Therefore, the top issue for Question Period receives 10 points. As does the top issue for broadcast, and for political columns. The second issue receives 9 points – and so on and so forth, all the way down the list. It looks like this:

Question Period Points Broadcast Points Political columns Points
Energy East pipeline 10 Energy East pipeline 10 Canadian action against ISIS 10
Canadian action against ISIS 9 Government deficit 9 Government deficit 9
Government deficit 8 Canadian action against ISIS 8 Energy East Pipeline 8
Trans Pacific Partnership 7 Electoral reform 7 Trans Pacific Partnership 7
Electoral reform 6 Trans Pacific Partnership 6 Electoral reform 6

Then, we add all of the points together for each issue. Using our example, it looks like this:

Question Period Points Broadcast Points Political columns Points
Energy East pipeline 10 Energy East pipeline 10 Canadian action against ISIS 10
Canadian action against ISIS 9 Government deficit 9 Government deficit 9
Government deficit 8 Canadian action against ISIS 8 Energy East Pipeline 8
Trans Pacific Partnership 7 Electoral reform 7 Trans Pacific Partnership 7
Electoral reform 6 Trans Pacific Partnership 6 Electoral reform 6

Our master list of the top issues from Ottawa looks like this:

Issue Points
Energy East pipeline 28
Canadian action against ISIS 27
Government deficit 26
Trans Pacific Partnership 20
Electoral reform 19

 

Measuring the Canadian conversation:

To measure how much traction issues are getting with Canadians across the country, we created a sample of the Canadian conversation on politics and government with a sample of 1000 Twitter users who are interested in these issues. This sample is distributed to reflect the distribution of the Canadian population by province and gender.

Using our social media software that collects data from Twitter, we searched people who have been active on Twitter for the past year with interests in Canadian politics and government – both federal and provincial. We generated a random list of users who fit these interests.

From there, we sorted through the list for people who used Twitter to actually engage with issues – so people had to be creating their own content, they couldn’t just be retweeting others.

Also, on Political Traction we look at these issues week over week, so they had to post with a certain level of frequency so that there wouldn’t be weeks where our Canadian sample had no activity.

And finally, we filtered out people who work in politics, in some fashion  – so people who work for mainstream news outlets, people who work for the government or hold public office, or people who work for lobby groups. Also, we sifted out any spam accounts.

We take what we have measured as the top issues in Ottawa and we go to our Twitter panel. Using our social media software, we look for keywords related to the issues within our panel’s online conversation. That way, we can see how much traction these issues got across Canada.

We also look at what issues our panel is talking about outside of the top issues from Ottawa, in case there is something that Canadians are focusing on, completely outside of the Ottawa conversation.
In practice, it looks like this:

Music in this episode:

Political Traction Theme Song, by Andrew Polychronopoulos   

Anameli High Quality, Pixelland, and Quirky Dog, by Kevin McLeod from royalty-free music site incompetech.com

Viper, by Ray Rude