UW CIP x AARP
Background
Team:
Maya Nair: Undergraduate Research Intern
Connor Klentchy: Undergraduate Research Intern
Jevin West: Lead PI
Connor and I were chosen for this project as we both had prior experience as Teaching Assistants for the informatics class, INFO 270, Data Reasoning in the Digital World.
Context:
In the summer of 2020 I joined the UW Center for an Informed Public research lab as an undergraduate researcher. This multidisciplinary research center is aimed at “resisting strategic misinformation, promoting an informed society and strengthening democratic discourse.” One of the projects that was most important to me personally while at the CIP was the CIP x AARP Washington Factcheck Ambassador Training Program.
My team was responsible for 3 main tasks. First, we curated course materials which were used to teach a group of ~500 AARP members new techniques to spot misinformation in the media.
Next, we were responsible for designing an intuitive google form & supplemental materials which trained AARP ambassadors could use to upload potential misinformation they came across pertaining to the 2020 election, which would then be analyzed by a team at the UW CIP for further investigation by another team within the CIP.
Lastly, our team was asked to create first iteration wireframes for a potential social media style site, where users could upload potential pieces of misinformation for professional review. In theory, this site would then be maintained by Fact Checkers at the CIP to help verify the various posts.
Research Goals
Overall, a large goal of this project was to reduce the spread of strategic misinformation by empowering AARP members with tools to help discern information they are consuming throughout the day.
After teaching them core tools, we also wanted to provide AARP members with a form to connect them with a team of experts who they could send potential misinformation to for further investigation, more specifically pertaining to the 2020 election.
Methods
Inital Literature Review
First, we put gathered a list of fact checking sites to explore, specifically looking pertaining to open source fact-checking. This helped my research partner and I gain an understanding of some of the core websites and resources in the space. We began with this task as we were still investigating whether or not an open source fact-checking tool was already available, and if so, what it looked like. However, after our search, we found there were not any obvious matches to this idea with the closest sites being things such as Yahoo answers, Facebook fact-checking, and Quora.
Next, my partner and I reviewed ~15 articles pertaining to digital media literacy and older persons. From this, we found that existing research had shown that needs-based approaches to improving digital literacy were best for creating self-efficacy (Vroman et al., 2015; Lam & Lee, 2007; Livingstone et al., 2005). This meant it would be important to provide ample resources for the members, to help them navigate more of the UI rather than process the actual content being taught to them.
We decided that in conjunction with creating the google form where they could upload tips, we needed to create clear documentation that would help walk them through filling out the form. This further proved that holding (live) virtual information sessions as well as ‘office hours’ for our ambassadors would be best in helping them feel most supported through the learning process.
Community & Demographic Survey
After completing our first round of initial background research, we began creating a survey for our AARP members. This survey was meant to us better understand the members needs and habi regarding digital media literacy. Being that our team had experience teaching about misinformation, we planned on touching on all of the content covered in Data Reasoning in the Digital World and more. However, this survey helped us best decide what concepts we should place emphasis on, and served as a springboard for understanding a bit more about our target demographic.
Survey Questions
-
How old are you?
50-59
60-69
70-79
80-89
90+
Goal:
Understand the age group we are receiving information/results from.
Blindspot:
Only collecting results from 1 age group (having skewed results).
-
1) On what social media sites have you seen posts flagged for “false” or “potentially misleading” information?
Facebook
Twitter
Instagram
Other
I have not seen a flagged post on any of these sites
I do not use social media
Goal :
Find out what platforms users most frequently use or view.
Blindspot:
Assumes that our audience uses social media platforms enough to have seen posts that were flagged.
2) Do you trust social media as a reputable source of factual information?
Yes
No
Sometimes
Goal :
Gauge whether or not people perceive social media as being a reliable source or are already in the habit of questioning what they see.
Blindspot:
Assumes our audience uses social media enough to even use it for factual information.
3) What are your main reasons for using social media?
Work related
News
Keep in touch with family and friends
Meeting new people
Other (fill in the blank)
I don’t use social media
Goal :
Understand the intent/purpose social media serves in our user groups life.
Blindspot:
Assumes our the use of social media.
4) How frequently are you to using social media for your primary source of information?
Daily
2-3 days a week
Weekly
Monthly
Seldom to never
I do not use social media
Goal:
Understand the frequency of use for our user group so we can better decide how much content about discerning information on social media we should add into our modules.
Blindspot:
We don’t ask if they use social media as a supplement to other forms of news such as watching TV or reading the paper.
5) How long do you spend on social media daily?
0-30 minutes
30 minutes-1hour
2-3 hours
4+ hours
Goal:
Understand the daily social media usage of our users, which may be quite different than that of other age groups
Blindspot:
Assumes semi consistent use of social media.
6) What social media sites do you use the most?
Facebook
Twitter
Instagram
Snapchat
Other
I do not use social media
Goal:
Help us understand what social media sites our user group are most involved in/what content they might be seeing based on the social media they use.
Blindspot:
We don’t have any way of knowing the relative amount of time spent on each site.
-
1) In what forms do you most frequently get your news?
TV/Broadcasted television
Desktop websites
Mobile Apps (apple news)
Magazines/newspapers
Radio
Goal:
Better understand what forms of media people most frequently consume as it may play a role in what information they are coming into contact with.
Blindspot:
A fair amount of people don’t watch the news/have a regular routine for surrounding this, so they may use all sources equally.
2) What news sources and outlet(s) do you most frequently obtain your news from?
(check all that apply)
MSNBC
The New Yorker
Huffington Post
Vox
CNN
Washington Post
Politico Time
New York Times
NPR
Associated Press
Bloomberg
USA Today
Wall Street Journal
Reason
Fox News
Examiner
Daily Wire
New York Post
Daily Mail
NewsMax
Goal:
Understand what news sources our users look at and whether or not they may be more slanted. This will help us create content which encourages looking at news sources that are more “centered”.
Blindspot:
People may just pick familiar names but actually the ones they read/view.3) Are you more likely to watch local news stations or major news stations (MSNBC, FOX NEWS, etc.)
Local News Stations
Major News Stations
I do not watch news stations
I watch equal amounts of local and major news stations
Goal :
Helps us best understand whether or not our users view national/global news or primarily local news stations. This will help us decipher what information we are more likely to add to the modules.
Blindspot:
This assumes that users watch the news.
4) How frequently do you use fact checking sites such as Snopes/Politifact/etc
Daily
2-3 days a week
weekly
Monthly
Seldom to never
I do not use fact checking sites
Goal:
This question helps us understand whether or not our population of users is familiar with fact-checking sites and frequently use them.
Blindspot:
Assumes that there is trust in fact-checking sites, and that our users know they exist as a resource.
-
1) Would you be interested in fact-checking?
Yes
No
Not sure/need more information
Goal:
Understand whether or not our users want to be more versed in fact checking.
Blindspot:
Being that our users are apart of the ambassador program, we can assume a certain level of interest in the topic.
2) How frequently do you read research articles/journals?
Daily
2-3 days a week
Weekly
Monthly
Seldom to never
I do not read research articles/journals
Goal:
Help us understand how many of our users consume academic articles, which in the case that they do, we will add more information about predatory journals, peer reviewed journals, etc.
Blindspot:
Journals/research articles are not normally read by within the industry or topic that the article surrounds. Ex: If you work on a farm for a living, how likely are you to read medical journals?
3) What fact checking sites do you use?
Snopes
Politifact
Factcheck.org
Allsides
Other
I don’t use fact checking sites
Goal:
Find out what fact checking sites our users are aware of and use so we can better understand whether or not we add more information about fact-checking sites.
Blindspot:
Assumes use and knowledge of existing fact checking sites.
Outcomes
Based on our literature review and demographic research, we concluded that we would need to focus our efforts on making sure our curriculum and form were concise and intuitive, with many supplemental documents.
Based on the literature we reviewed, we also determined teaching the material in a real-time virtual setting would be best with making sure the members felt adequately supported as they navigated the course materials. We also determined that supplemental documents would be helpful to refer to after each seminar but not as primary learning materials.
From our survey, we found that many of our ambassadors reported that they had held professional careers, with some stating that they still consumed information from academic journals in their field when possible. Many were already online media consumers as well, which did not come as a large surprise due to their willingness to be apart of this initiative.
Because of these factors, we were less concerned about our group's ability to learn the content, but more so how they would be able to navigate the online materials as this was being done completely virtually.
Form & Guide
To combat the potential UI barriers for our user group (especially when using Misinformation reporting form), we decided to create a step-by-step document which members could refer to.
This form was iterated on as we worked with and interviewed AARP members on our initial google form which they used for uploading tips. We curated this form based on the more frequent questions we were seeing in our office hours and seminars with members.
This step-by-step form ended up being extremely helpful. Once the Google form was rolled out, we were able to collect the responses and pass the tips along to another team in the CIP working on the election integrity project.
Mock-Ups
Lastly, Connor and I constructed a list of must-have and must-not-haves, which we used when creating our first iteration of wireframes for the theoretical fact-checking social media site.
Though this aspect of the project had less practical application, constructing the wireframes and ideating on this was a highlight of the project. Due to time constraints, this site was not developed but served as a great exercise in user research.
