This summer and fall we’re interviewing members to understand why they support media organizations and what other sites might do to entice them. You can use and remix the materials we bring to this work, including an interview discussion guide and exercises to learn what your prospective members value.
After talking to staff at publications and starting an open database of news organizations with membership programs, we’ve begun to transition our research to talk to members of media organizations, starting with De Correspondent supporters. For each hour we spend interviewing members, we put many more into the activities before and after each conversation, including user recruitment, question writing, synthesis, and team presentation. (Interested in how we’re using human-centered design in our research more widely? Look no further.)
A few other publications’ staffers have asked to compare how we each conduct research into members’ news needs. Below you can find information used to guide our first set of planned member interviews. These materials will likely change as we interview more members around the world. We ask that you’ll let us know your thoughts if you try out any of these tools—let’s add to and improve them.
If you’re eager to learn more about research approaches and tools, see this set of resources for human-centered design methods compiled by Sonja Marziano of Smart Chicago and Jordan Wirfs-Brock of Inside Energy. You may also be interested in their user feedback roadmap exercise for news product development. Also, Erika Hall’s research methods “cookbook” Just Enough Research is a great primer if you want more.
We start WITH A GUIDING BRIEF
To get our team aligned across two continents, we create a short, sub-three page research brief document. It includes research goals, relevant background information for the project, details on the methods we want to pursue, recruitment and venue plans, a conservative timeline and budget, names of team members, and potential next steps. We create this as a Google Doc so collaborators can add ideas and questions about proposed approaches.
How detailed should you be when writing your brief? I like to keep this document high level to inspire teammates’ interest, as you can see in the introductory elements of the member research brief:
Our Research Goals
In looking to design an effective contract between organizations and members, we’re conducting research to understand members of two groups: people who might contribute and organization staff. In this process we want to learn about potential members’ motivations, behaviors, attitudes, and unmet needs. These learnings will inform plans for The Correspondent launch as well as principles we share with other news organizations for their own membership planning.
Background
We’re currently developing hypotheses about what matters most to members based on a widespread literature review and Jay’s initial conversations with members. We’ll be looking to validate (or disprove!) these hypotheses throughout the summer in conversations with current, potential, and former members of De Correspondent; other news, media, and entertainment organizations; and other non-media organizations with membership programs.
You can see the complete brief here with additional detail about our methodology and the discussion guide we created. Once our team is ready to move forward, I create another more detailed document that includes language for our participant invitation, screener, and communication with potential interviewees.
We Clarify Who WE Want to Talk To
In a recent conversation with Umbreen Bhatti, manager of KQED’s innovation lab, we both noted that recruiting current, past, and potential users can be the trickiest part of research to get right. The quality of all of your learnings hinges on finding and scheduling relevant people, and seeking “extreme users” can be highly beneficial, as detailed here by the Stanford design school. When you only have a limited number of spots, it’s imperative to recruit selectively and to ensure that an appropriate balance of individuals is represented. It’s worth reading Local News Lab’s note about the importance of avoiding recruitment bias.
In our latest round of research, we had time for 13 60-minute sessions with individuals and three 120-minute group sessions across three cities. It’s good practice to talk to no fewer than seven relevant users to be able to identify trends and outliers. We wanted to ensure a blend of occupations, ages, educational backgrounds, membership tenure, and behaviors on and off platform. In our participant invitation (which we posted in Dutch on De Correspondent), we directed interested individuals to submit responses to screening questions through Google Forms. We were then able to review confidential answers to these inquiries to ensure we didn’t skew too heavily in any direction, such as seeking only super fans or people working in the social sciences.
- What is your name?
- What is your preferred email address?
- What city do you live in?
- What is your occupation?
- What is your age?
- What news do you read, watch, or listen to weekly or more often?
- Are you a paying member of De Correspondent?
- Yes
- No [Route out to thank you note]
- How are you involved with De Correspondent as a member? Please select all that apply.
- I read
- I comment
- I share stories
- I’ve been a source
- I’ve contributed sources
- I’ve contributed proofreading
- I’ve contributed other research help (please describe)
- Other (please describe)
- In 200 words or less, why are you a member of De Correspondent?
- What would you change about being a member of De Correspondent, if anything?
- What other types of organizations are you a member of?
- In 200 words or less, how does the experience of being a member of De Correspondent compare with these other memberships?
- During which of these time blocks are you able to meet in person for an interview? (with time options as radio buttons)
We Gather Participants & TRY TO Make Them Comfortable
Once you’ve prioritized potential participants, reach out with any follow-up questions and logistical information about sessions. It’s important to also send a kind note to the people you aren’t planning to schedule with details about how they might stay involved in your project.
In a perfect world you could meet participants in their homes to observe how they act and see them on the devices they use regularly. This can involve a decent amount of travel, and booking a research facility with observation rooms for teammates is a good alternative option. If you’re on a budget, you can reserve a neutral space where you can record the session with participants’ consent. The New York Times’ research recruiting lead Sarah Soffer introduced me to Breather spaces for American and Canadian cities, and many countries have similar services. (No coffee shops! Having other people around can inhibit what participants are willing to share with you, and your audio or video recordings will likely be unintelligible later.)
Encourage your participants to arrive early so they can get settled. Have a teammate capture a shoulders-up photo of each participant that you’ll attach to session notes so everyone on the team can distinguish them: this is especially important for de-anonymizing participants with teammates who aren’t able to observe sessions in real time. If possible, assign a note taker who will sit in the room or in a separate observation space with quality AV to capture takeaways from the conversation. Try to solicit participant consent to record either audio or video for review and fact-checking purposes later.
Introduce yourself to your participant(s) and your project and ask if they have any questions before getting underway. You can see the questions we asked over 60+ minutes in the interview discussion guide. You’ll see questions for interviewees categorized by introduction (including sufficient get-to-know-you time), background information, media habits, membership interactions and ideas, and parting thoughts.
Don’t worry if you don’t get to all of the questions you intend to ask. Coming away with a rich set of learnings–including confirmations of what you expected to hear, rejections of your hypotheses, and surprise takeaways (which we call “a ha!” moments)–is far more important than being completist.
We Use Exercises to Learn What They Value
In planning your sessions, there is no shortage of design exercises you might use to draw information of interest. You can show rough sketches to garner user reactions or use provocation statements to generate discussion. Less is more: each exercise can take 10 minutes or more, and sometimes your conversation is rich enough to warrant forgoing your plans entirely.
For this research round, we planned two exercises. You’re welcome to repurpose these materials:
A card sort exercise to learn what members value in organizations:
We created individual cards listing various values based on our team’s literature review and conversations with media experts. You can download the cards here, and you’ll see that they include options like “Advocacy on my behalf” and “Organizational authenticity.”
We wanted to understand how important members consider each of these items in the organizations they consider joining in media and beyond. You could ask your participants to rank them from least to most important. Note that the conversation itself can be as valuable if not more than the “data” gathered in your interviewees’ organization of the cards.
For the sake of speed, we had De Correspondent members group cards into three categories for discussion purposes:
- Things I get value from
- Things I might get value from in the future
- Things I don’t value
The first and third groups can be highly illustrative, especially to identify trends across participants, and the second group can point out participants’ aspirations.
We saw near unanimous responses from the Dutch and Belgian members we interviewed. Almost all who took part in this exercise shared that they value an appropriate financial ask, brand design/visual appeal, a sense of uniqueness, and a chance to offer the world something they think should exist. Opportunities for physical branded goods and a sense of exclusivity (like members-only content and access to staff) were the two options that these members said they find least important.
A letter writing exercise to understand what’s essential:
To know what members value—as well as what they’d change—about the journalism and the platform, we asked them to pen a short letter to De Correspondent. We found that many members were initially uncomfortable with this exercise, so we mentioned it briefly at the beginning of sessions as a “goofy American custom” that we asked them to partake in at the end of our time together. You can see one member’s letter in this post. We also changed the format after the first day of research from a blank sheet of paper (which could seem somewhat overwhelming) to a postcard with purposefully limited space. After a few minutes of solo writing, we asked participants to read the postcards aloud. One asked for time to think about his response and promised to send it by post to De Correspondent’s offices, an offer we were happy to extend.
This exercise serves two purposes: it provides a (sometimes emotional) summary of their thoughts about the publication, and it offers an artifact that can be shared across the team. The importance of tangible evidence of affinity is something that the Guardian US has also recognized with the certificates it mails to members featuring art by data editor Mona Chalabi. It’s a compelling example of reciprocity:
We Recognize Participants’ Contributions
We explained to our interviewees that the information they shared would be used in the following ways: We’ll be publishing our findings for the benefit of the news industry, particularly to help independent media that are launching or revamping their own membership programs. The insights you share will help us create promoted practices for news organizations, including but not limited to DC.
In sharing insights from the research we quote participants by name after confirming that they’re comfortable having what they said published. We guard what they share with us and with participating De Correspondent staff carefully, including not disclosing their contact details. For more information on research ethics considerations, look into the large body of literature around user privacy, including Dr. David Travis’ article on informed consent.
Ideally you will give your recipients cash or a gift card to thank them for their time. In our research we compensated travel (up to €20 with proof of payment); provided a De Correspondent tote bag, postcards, and a copy of one their staff writers’ books; and offered dinner for participants we met in the evening.
We weren’t able to interview everyone who we might have learned from. We’re soliciting members’ opinions in responses on this post and will be surveying members of De Correspondent and other publications in the coming months.
Research synthesis—making sense of what you hear, see, and infer—is its own field of study and warrants another post. We’ll be sharing additional research resources this fall, including an open database of media organizations with robust membership programs and a library of relevant literature. In the meantime, if you have ideas for us and for others interested in learning about members’ motivations and ideas, we welcome them!
Editor’s note: We have added additional research resources that you are welcome to put to use, including these survey questions and this values exercise. You can see research insights that came from using these approaches here.
Special thanks to Maaike Goslinga and René Clerc for help facilitating this member research.
