-In Loving Memory –
Forgive the tangent, but I am taking a much-needed moment here. An underlying theme of this endeavor has been that life happens no matter what goals you're pursuing. This project is no different and comes after LIFE hit my small family in a tremendous way. We lost our sweet little girl, Sophie, after only seven short years with her.
This post and the book that follows will be dedicated to her. She taught us so much and brought so much love into our lives. She kept us present, taught us to enjoy the little moments, reinforced the need to stretch, and appreciated the best of snuggle sessions.
We are all the better because of her. Rest in peace, sweet girl. We will keep you in our hearts, always.
Thank you for your understanding; let's get back into it. We've now entered the leg of the dissertation process, where we're gathering data to either test our hypotheses or build a framework in response to our research questions. Depending on your methodology, this process can go by quickly or be an extensive process. Accordingly, this section will discuss different data-gathering processes and a few logistic-based realities that can occur while gathering data. The first bit will cover qualitative research, and the next post will cover quantitative studies.
Data Gathering Methods
Let's begin this journey by reviewing the different methods you can employ in qualitative research. Qualitative research is designed to reveal any underlying meanings or rationales behind people's behaviors, experiences, or perceptions. More simply, the beauty of qualitative research is it reveals insight into the WHY behind phenomena, trends, or experiences. What it does not do is prove causality, but it can lead to the formulation of theories that can be tested for causality later on.
Methods to gather qualitative data include interviews, focus groups, observational studies, document analysis, and meta-analyses of other qualitative studies. We'll review each of these:
Interviews
Interviews are one of the most common ways to gather qualitative data. This task involves sitting down with a participant and asking them open-ended questions to gain insight into their thoughts, feelings, and experiences. Interviews can be structured or unstructured, depending on the research question and the information sought.
It's important to recognize four levels of interview format formality (i.e., structure): unstructured, semi-structured, structured, and standardized (Creswell & Creswell, 2018). We'll review these from the casual interview to the completely formal format:
Unstructured interviews involve open-ended questions, allowing the interviewer to explore topics in-depth without a predetermined format. Subjects may ask interviewers questions, which feels more like a conversation than a fully formalized interview.
Semi-structured interviews have a more defined structure, with questions or topics guiding the discussion while allowing for flexibility and follow-up questions.
Structured interviews have a highly defined format, with a fixed set of questions and a specific order of presentation. Interviewers will go through questions here with a few deviations and redirect subjects to answer the questions.
Standardized interviews involve a set of highly structured questions that are asked in the same order and manner for each participant. No deviations occur in this format, and the interview is treated as running through a script.
Focus Groups
Focus groups are another way to gather qualitative data. This involves bringing together people with a shared experience or characteristic, such as a specific age group or profession. The moderator (you) would guide the discussion and encourage participants to share their thoughts (perspectives, opinions, and interpretations) and experiences with the group.
There are two factors concerning focus groups that researchers need to be aware of:
The first of which involves the confidentiality of the participants in the study. While you can take every step possible to safeguard the personal identifying information in the study, this in no way guarantees that subjects will follow suit. Because of this, you need to follow the guidelines when communicating to participants that confidentiality cannot be guaranteed 100%. It's best to use the methods suggested by your training here and, of course, your chair's guidance.
The other factor to consider concerns the phenomenon of groupthink. Groupthink occurs when a group becomes too focused on consensus and harmony to the point where they begin to suppress or ostracize dissenting opinions. Worse still, groups could ignore evidence that does not fit their espoused and newly formed perspective. Subsequently, groupthink can skew your data. To mitigate the impact of groupthink in focus groups, you can use techniques such as devil's advocacy or the Delphi method to encourage dissenting opinions and generate a more robust discussion (Stewart & Shamdasani, 2014).
Observational Studies, Document Analysis, and Meta Analysis
Observational studies are a method of gathering qualitative data wherein you watch people in their natural environment. This research can be done through direct observation or video or audio recordings. Observation can provide valuable insights into human behavior and can help researchers understand how people interact with their environment. Full disclaimer: I've not seen any examples of this outside of marketing research, but it IS a method, which is why we're talking about it.
Document analysis is another method of gathering qualitative data. This method involves analyzing written or visual materials, such as diaries, letters, or photographs, to gain insight into the experiences and perspectives of the people who created them. Document analysis can be instrumental when studying historical events or cultural artifacts. One study I came across had subjects write a letter to their future selves. The researchers then used the letters as a source of qualitative data, so there is precedent for this approach.
Qualitative meta-analysis is a research method that combines the results of multiple qualitative studies to synthesize the findings and gain a deeper understanding of a particular phenomenon. This approach involves systematically reviewing and analyzing the data from each study to identify common themes, patterns, and insights. The resulting meta-synthesis can provide a more comprehensive and nuanced understanding of the research question than any individual study alone (Sandelowski & Barroso, 2007).
The use of qualitative meta-analysis has become increasingly popular in recent years, and there are many examples of successful applications across various disciplines, such as healthcare (Thomas et al., 2008), education (Mitteness et al., 2019), and social sciences (Miles et al., 2020). Setting clear guidelines for study inclusion or exclusion becomes critical when using this method. Additionally, as a cautionary note, if you are interested in pursuing a meta-analysis for your dissertation, clear this with your chair to ensure it is sufficient and appropriate.
Finding and Recruiting Study Participants
This part is, by far, the most nail-biting, stressful part of running a qualitative study. If you're here, hang on and have faith. You're likely worrying about getting a response or scheduling your confirmed participants. All the issues in other spheres of life apply here: people will drop off, you will be ghosted, your responses may not be as fruitful as you'd like, and you will need to redirect folks back to the questions you asked initially.
Your goal is to hit theoretical saturation (the point where no new insights are presented from additional interviews) while ensuring your group is representative of the population you're targeting. To add more fun-ness to this task, theoretical saturation is not some concrete, clearly defined number; it differs from study to study. Give yourself a little credit here; this is a tall order.
This section will discuss each step and hopefully give a few helpful tips.
Interview Design: To design an effective interview, you need to consider several factors, including the research question, the study population, and the level of structure. The research question should guide the selection of the interview questions and the overall interview design. For example, if the research question is exploratory, an unstructured or semi-structured interview may be appropriate, while a structured or standardized interview may be more suitable for confirmatory research.
The study population also plays a role in interview design, as you need to consider factors such as the participant's age, gender, culture, and language proficiency. Finally, the level of structure should be determined based on the research question(s), the desired level of detail, and the required consistency in the data.
To ensure the quality of the data collected through interviews, you must carefully consider the interview process, which includes selecting a suitable location for the interview, establishing rapport with the participant, and ensuring that the interview questions are clear and relevant. Virtual interviewing through a platform like Zoom or Microsoft Teams is great for this, but make sure that the platform you use is HIPPA-compliant to ensure participant confidentiality.
Platforms and Postings:
When planning a research study, using social media platforms to find participants can effectively reach a broad audience and diversify the sample. Social media platforms like LinkedIn, Facebook, or Instagram can be handy for studies targeting specific age groups or communities, as they allow for targeted advertising and outreach. This trick is especially useful when targeting professional groups on these platforms.
Remember, like the interviewing platform, it's essential to consider the accessibility and security of each social media platform and the nature of the study. You should choose a platform that's easy to use for the target population and has robust security measures to protect participants' privacy.
Network Effect:
One benefit to growing your professional social media reach is in instances like these where you must cast your message as far and wide as possible to recruit the most representative circle you can. The network effect refers to the phenomenon of a product or service becoming more valuable as more people use it (Katz & Shapiro, 1985). In the context of recruiting research subjects, as more people become aware of the study and share it with their networks, the more likely it is that the study will reach its target audience.
This is the point where you ask your connection to help spread the word regarding your study or use the professional groups available on the platforms to tap into prequalifying membership pools. Start by creating a post about your study on social media and encourage people to share it with friends and followers. This post can generate buzz and increase the study's reach beyond your immediate or first connection network. Additionally, using targeted hashtags and groups can help reach people interested in the study but not yet connected to your network. By utilizing the network effect, you can tap into social media's (insert animated voice here) phenomenal cosmic power to recruit a diverse and representative sample of research subjects (Kaplan & Haenlein, 2010).
Scheduling:
Scheduling is more an art than a science, but there are tips that can help this process. Naturally, when trying to schedule times with other professionals, you are beholden to their schedules and your own. Plan on 2-3 weeks to conduct the interviews needed to hit theoretical saturation and be ready to roll with the punches (metaphorically, of course). Here are a few rules (ahem, as Capt. Barbossa says, more like guidelines) to help make this process manageable or at least not overbearing:
Rule #1: Do not schedule more than two interviews each day.
Why, you may ask? Each interview will take thirty minutes to an hour, so three hours may not seem that bad for three interviews. You make an excellent point here, but here's what you need to remember: that's three hours of focused attention; in addition, you must transcribe three hours' worth of audio afterward. Best to pace yourself here. For every hour, plan on at least 2-4 hours' worth of work behind it.
Rule #2: Be ready with probative follow-up questions.
Given the nature of the interview environment, you need to get people comfortable and ease them into talking through your questions. At first, the responses will likely be brief and could be dichotomous (i.e., yes or no), even to open-ended questions. This means you'll need to be ready and able to come at these questions more effectively by breaking them down and asking for more information.
There's a great book that helped me with this by Rubin and Rubin called Qualitative Interviewing.
Rule #3: Plan for overtime.
If one challenge is getting people to talk, another end of the spectrum might be sticking to your time limitations if people talk a lot.
There's a healthy balance concerning these conflicting demands. On the one hand, a person's subject is giving you a lot of great information; on the other, you are running over time and will need to sort out the usable information from the off-topic notes after transcribing it. If you plan on having a one-hour interview, reserve at least 90 minutes on your schedule to have some flex. You'll need it.
Rule #4: Be flexible but firm in the cutoff point when rescheduling.
Scheduling issues will come up. While going through this stage, I put out requests and posts for three weeks before hearing back from anyone. It takes persistence and patience, but you need to be ready once folks come rolling in. In my case, I had a few interested folks reach out after I hit saturation and needed to thank them for their interest but not take on any more interviews. It's all a balance between your data needs and your deadlines. Remember, the seven-year countdown is still in effect.
Rule #5: If needed, expand the scope.
Backup plans are great, ami-rite? You should have a way to open the qualified subject pool slightly if reaching your initially intended subject pool has proven problematic. This response was something I was getting ready to do myself.
Initially, I targeted creative post-secondary faculty, defined as faculty working for institutions that were a part of the Association of Independent Colleges of Art and Design (AICAD). After waiting three weeks without a response, I decided that deferring to a larger US Higher Education faculty pool was necessary.
I switched the subject pool, and low and behold, those in my original targeted sample came forward, so I was lucky enough to stick to my OG plan. Had I not had a backup plan, this could have delayed my study or forced me back to the drawing board.
Steps to Protecting Research Subject Anonymity and Confidentiality
Protecting the confidentiality and anonymity of research subjects is required to ensure ethical and responsible research practices. As we've discussed previously, it's one of the crucial concerns of the IRB, so we need to do everything we can as researchers to protect our participants. Here are a few steps that you can take to protect the confidentiality and anonymity of your subjects:
1. Obtain informed or implied consent:
Before beginning the study, researchers should obtain informed or implied consent from participants. This process involves explaining the purpose of the study, the procedures involved, and the potential risks and benefits. This step helps to establish a clear understanding between you and the participant and sets the foundation for maintaining confidentiality and anonymity throughout the study (American Psychological Association, 2017).
2. Use anonymous identifiers:
You should separate PII from the responses by using anonymous identifiers, such as numeric codes or pseudonyms, to replace participants' real names in any data collection or analysis. This helps ensure that the participant's identity is not revealed in published reports or research findings (National Institutes of Health, 2018). Another layer of obfuscation I used here was to use they/them/theirs pronouns when referring to any participant to reduce identifiability by participants' gender identities.
3. Protect data storage and transmission:
This is a BIG one. You should take measures to protect the storage and transmission of participant data. This includes using secure servers or encrypted cloud storage to store data and password-protecting any electronic devices used to collect or analyze data (American Psychological Association, 2017). Due to their data usage policies, you should also not use publicly 'free' platform services like Google.
4. Limit access to data:
You should limit access to participant data only to those directly involved in the research project. In other words, you should be the only one who knows about your participants. Bonus points if you obscure this data from yourself. This limitation includes using password-protected files or folders to store data and limiting access to electronic devices that contain participant data (National Institutes of Health, 2018).
5. Obtain a Certificate of Confidentiality:
This was new for me, but hooray research. Did you know you can obtain a Certificate of Confidentiality from the National Institutes of Health to provide additional legal protection for participant data? This certificate helps to prevent the disclosure of identifiable participant data in response to legal demands (National Institutes of Health, 2018).
Data Gathering Faux Pas
With all the things you should be doing listed in this post, there are equally things you shouldn't do. When gathering qualitative data, you need to be aware of potential pitfalls or faux pas that can compromise the quality and validity of their findings:
Faux pas #1: Coming into the research with an outcome or hunch in mind.
One common mistake is to impose preconceived notions or biases on the data collection process, leading to a confirmation bias that may skew the results. To avoid this, approach data collection with an open mind and be willing to explore unexpected findings and perspectives. This issue also requires declaring these biases ahead of time and working consciously to seek the truth and not confirm your theory. One professor hailed this as passionate dispassion concerning the outcomes. Any answer you come to, even one that counters your theories, is still considered an answer.
Faux pas #2: Myopic research is not cogent research.
As a researcher, you must look far and wide when building your literature review, combining many perspectives and approaches to your research topic. One trap you should avoid falling into is relying too heavily on a single data source or method. This faux pas leads to a limited and incomplete understanding of the research question. To avoid this, mix it up and use multiple methods and sources to triangulate the data and ensure the findings are robust and comprehensive.
Faux pas #3: Research ethics.
You should be mindful of potential ethical considerations when conducting qualitative research. This awareness includes obtaining informed consent from participants, ensuring confidentiality and privacy, and minimizing potential harm or discomfort to participants. You also should consider any power dynamics and reflexivity issues, particularly in studies involving vulnerable or marginalized populations. IRBs are VERY STRINGENT concerning vulnerable groups, but you should also be diligent about the ethical implications of your study as it impacts your participants. In many cases, subjects put themselves out in some way, shape, or form to help you. Please respect this and honor the risk they took by looking out for their best interests.
Bringing your research practices all together
Gathering qualitative data requires you to carefully plan your approach, maintain your attention to detail, and commit to completing your study ethically and transparently. The learning curve is tremendous, and you'll grow so much from this process. In the next post, we'll cover similar areas for quantitative studies. Until next time!
References:
American Psychological Association. (2017). Ethical principles of psychologists and code of conduct. Retrieved from https://www.apa.org/ethics/code/
American Psychological Association. (2020). Ethical principles of psychologists and code of conduct. https://www.apa.org/ethics/code/
Baxter, L. A., & Babbie, E. (2016). Basics of communication research. Cengage Learning.
Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589-597.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.
Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field methods, 18(1), 59-82.
Kaplan, A. M., & Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of social media. Business Horizons, 53(1), 59-68. Katz, M. L., & Shapiro, C. (1985). Network externalities, competition, and compatibility. The American Economic Review, 75(3), 424-440.
Maxwell, J. A., & van der Vorst, R. (2018). Designing qualitative research. SAGE Publications.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2020). Qualitative data analysis: A methods sourcebook. Sage publications.
Mitteness, C., Kilgore, A., Kecskes, K., & Poling, T. (2019). A meta-synthesis of qualitative research on teachers of gifted students. Gifted Child Quarterly, 63(3), 199-216.
National Institutes of Health. (2018). Protecting research participants: What everyone should know. Retrieved from https://www.nih.gov/health-information/nih-clinical-research-trials-you/protecting-research-participants-what-everyone-should-know
Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice. SAGE Publications.
Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing qualitative research. Springer Publishing Company.
Stewart, D. W., & Shamdasani, P. N. (2014). Focus groups: Theory and practice. Sage publications.
Thomas, J., Harden, A., & Oakley, A. (2008). Integrating qualitative research with trials in systematic reviews. BMJ, 337, a2290.