Chapter 12: Primary Data: Surveys, Interviews and Observation

A.    Checklist for Survey

   1.    Planning: thinking through ‘who’, ‘where’, ‘when’, ‘how’ and ‘what’. Have you considered:

  • Who you plan to speak about (population) – and gather data from (sample)
  • How you will reach your sample
  • How you plan to control your biases
  • How you might develop the skills/resources needed to carry out your survey, i.e. statistics proficiency
  • Ethics/ethics approval
  • The aspects of your research question that can be answered through a questionnaire.
  • Details – distribution, reminders, response rates, and data management
  • Contingencies – i.e.) having a back-up plan ready to go if response rates are low

   2.    Developing your questionnaire. Have you:

  • Operationalized your concepts
  • Explored existing survey instruments
  • Drafted questions
  • Decide on response categories
  • Considered whether your questions are ambiguous, leading, confronting, offensive, based on unwarranted assumptions, double-barreled, or pretentious
  • Rewritten your questions
  • Appropriately ordered your questions
  • Written instructions
  • Worked on layout
  • Written a cover letter/ introductory statement

   3.    Piloting. Have you:

  • Done a run-through
  • Reflected on the piloting process
  • Sought feedback
  • Trialled your stats package

   4.    Modifying. Have you:

  • Made appropriate modifications
  • Considered revisiting your planning, development and piloting process (necessary if modifications are substantial).

   5.    Implementing. Have you:

  • Distributed your questionnaires. Be sure to include instructions for return (address and return date) and possibly a self-addressed stamped envelope.
  • Sent reminder letters – send these out if response rates are low.
  • Initiated a low response rate plan – put this into action if not enough data has been gathered by your deadline.

   6.    Managing and analysing. Have you:

  • Organized/collated your data
  • Analysed your data. Most survey data will be analysed statistically, but you will need to engage in thematic analysis for any open-ended questions.

B.    Checklist for Interviewing

   1.    Planning: Thinking through ‘who’, ‘where’, ‘when’, ‘how’ and ‘what’. Have you considered:

  • Who you plan to speak about (population) – and gather data from (sample)
  • Access - how you will reach your sample
  • How you will present yourself
  • How you plan to control your biases
  • How you might develop the skills/resources needed to carry out your interview
  • Ethics/ethics approval
  • Data – exactly what it is you want to elicit from your respondents
  • Details – appointments, timing (travel time, interview time, wait around time), location, recording methods, etc
  • Potential cultural/ language barriers
  • Contingencies – i.e. having a back-up plan ready to go if interviewees cannot be accessed

   2.    Developing an interview schedule/recording system. Have you:

  • Drafted questions and/or themes
  • Considered whether your questions are ambiguous, leading, confronting, offensive, based on unwarranted assumptions, double-barrelled, or pretentious
  • Rewritten your questions
  • Appropriately ordered your questions
  • Prepared additional information – i.e. instructions, prompts, or probes
  • Decided on recording methods
  • Trained any note takers/translators

   3.    Piloting. Have you:

  • Done a run-through
  • Reflected on the piloting process
  • Sought feedback
  • Reviewed notes/transcribed

   4.    Modifying. Have you:

  • Made appropriate modifications
  • Considered revisiting your planning, development and piloting process (necessary if modifications are substantial).

   5.    Implementing. Have you:

  • Taken care of preliminaries – make appointments early, arrive early for your interview so that you have time to set up, and check any equipment.
  • Made your interviewee as comfortable as possible
  • Eased into main questions/themes
  • Kept a balance between keeping on track and exploring interesting tangents
  • Wound down and closed the interview

   6.    Managing and analysing. Have you:

  • Organized/collated your data
  • Analysed your data. Most interview data will be analysed thematically, but if you have 30 or more interviews you may want to engage is some level of statistical analysis.

C.    Checklist for Facilitating a Good Interview

   7.    Do your homework

  • Be prepared to talk about your research – be ready to clearly articulate the rationale, aims, objectives, and methods
  • Prepare a brief outline of your project – individuals or organizations may want to have a document they can consider and/or present to ‘gatekeepers’.
  • Have a letter of introduction – a letter of introduction from your supervisor can professionally answer questions like ‘So who are you and where are you from?’
  • Find out about appropriate protocols – sometimes the contacts that are most willing to help do not have the authority to authorize access. Finding out about appropriate protocols can help avoid awkward situations.

   8.    Be professional

  • Be respectful – choose the right time for your approach, be prompt, dress appropriately, and be modest in your initial requests.
  • Plan for the unexpected – very rarely does the research process run smoothly, especially when you are dealing with individuals; be prepared for glitches.
  • Leave doors open – many researchers swear they’ve collected all the data they are going to need, but later wish they could go back and ask just a few more questions.

   9.    Offer something back

  • Don’t disappear – let your contacts know how things are progressing and/or send a note of thanks.
  • Make results available – it is quite natural to have a sense of curiosity about studies of which you are a part; the results of your study can be quite valued by those who have facilitated your research.
  • Facilitating honest and open responses even though your interviewees may want to impress, be liked, or maintain privacy.
  • Suspending all judgement – if respondents feel judged, ashamed, or offended, it is difficult to gather credible data.
  • Figuring out how attributes such as race, gender, ethnicity, class, and age of interviewer and interviewee alike might affect the interview process.
  • Communication miscues – moving from questions to answers is anything but a straightforward process. Misunderstandings and misinterpretation are all too common.

D.    Checklist for Observation

   1.    Planning: Thinking through ‘who’, ‘where’, ‘when’, ‘how’ and ‘what’. Have you considered:

  • The type of observation study you will do – do your goals and context lend themselves to an observation study that is candid or covert; participant/nonparticipant; structured/unstructured – and of what duration?
  • Population and sample/ respondent/ participants – who you plan to speak about (population) – and gather data from (sample)
  • Access to the group/ situation/ activities you wish to observe
  • How you will present yourself
  • How you plan to control your biases
  • How you might develop the skills/resources needed to carry out your observation
  • Strategies for ensuring credibility
  • The tools you will use, i.e.) an observation schedule or if unstructured, any relevant themes to explore
  • Details – what timeframe will you be working towards? If you will observe on one occasion, multiple occasions, or will your study involve prolonged engagement?
  • How you will record your data
  • Ethics/ethics approval
  • Contingencies – i.e. having a back-up plan ready to go your original plan does not pan out

   2.    Implementing. Have you:

  • Eased into the observation situation
  • Prepared yourself to accept a range of sensory input- use all your senses, and possibly your intuition, to gather data
  • Invested significant time in your observations
  • Looked for saturation – try to ensure your observations no longer yield new knowledge before ending the process

   3.    Recording. Have you:

  • Recorded your observation as soon as possible. If using schedules, they should be filled in while observations occur. If you are more immersed in your research context, you may want to record your observations when removed from the situation either on data sheets or in a journal.

   4.    Reviewing. Have you:

  • Reviewed the process and noted any difficulties encountered
  • Reviewed your observation records
  • Confirmed - checked with an insider, ask another observer to compare notes, or triangulate your observational data with other data types

   5.    Refining. Have you:

  • Made modifications - based on your own review of the process; any confirmation strategies you have attempted; and the quality of the data generated
  • Kept reviewing and refining - observation takes practice; keep refining until you are comfortable with the process and the data collected
  • Considered major issues - if there are major issues, you will need to openly discuss with your supervisor and consider modifications

   6.    Managing and analysing. Have you:

  • Organized/collated your data
  • Analysed your data. Most data collected in observation can be quantitative (using checklists) or can be much more qualitative (using journaling)