This project was a sub-project of a larger cluster randomized trial (Primary and Secondary Prevention of Cardiovascular Disease and Stroke-Translating Research into Practice [PPRNet-TRIP-II], funded by the Agency for Healthcare Research and Quality. The project was underway within 20 primary care practices (Ornstein, 2001) that were part of a practice based research network comprised of users of a common electronic medical record (EMR). The practices sent quarterly data extracts to the network for benchmarking and quality improvement research. The PPRNet-TRIP-II study design included site visits, network meetings and practice performance reports as the intervention being tested. Practice performance on 21 quality indicators (related to cardiovascular and stroke prevention) would be measured to evaluate the impact of the intervention. A process evaluation was needed to learn what practices did as a result of their involvement with this project. This quality improvement model had not previously been tested, thus there was a significant opportunity for qualitative research to learn how and what practices did.

I was a student in a nursing PhD program at the time, and within this project posed the question: How do practices make changes in practice to adopt clinical guidelines for cardiovascular disease and stroke prevention? The quantitative data that demonstrated changes in practice from the baseline of the project did not explain what processes the practices used to implement quality improvements for these conditions. I worked with the investigators to propose this subproject as part of the evaluation of the research, and was granted entrée into this research team.  My previous experience was within hospitals, not primary care, but largely focused on quality improvement and outcomes management.  I had comfort in the world of hospitals but had biased opinions of how things worked in these settings due to my previous experience. This was not the same in primary care, where I did not previously practice. I was open to seeing things with fresh eyes, and was not tainted in knowing how things should be.  I was also not an everyday user of the electronic medical record that was being used in these settings, nor had I participated in the work of a research network before. 

I worked closely with the project evaluator for the PPRNet-TRIP-II research study to delineate the scope of this specific project and conducted a literature review on the process of change and quality improvement in practice settings. Within this review, the Institute of Medicine’s study on Microsystems provided a useful framework to begin an exploration of the implementation of change in health care practices. This study provided a great starting point to evaluating selected aspects related to the process of change, and an interview tool was adapted to frame my interview. (Donaldson & Mohr, 2000) There was a body of literature on barriers to using clinical guidelines (Cabana et al., 2000; Cabana et al., 1999) that helped me frame my semi-structured interview with a diverse group of practice staff (physicians, nurse, medical assistants, office staff and practice managers).  Additionally, research from a business perspective regarding leading change (Kotter, 1996) provided a useful set of concepts to consider as I reviewed the data within my interviews.

I was a novice qualitative researcher at this point, so I consulted with Ben Crabtree, a leading qualitative researcher in the field of primary care medicine (Crabtree & Miller, 1999) who reviewed my initial proposal and agreed to participate with me in the dissertation process.  I was surprised at the collaboration and cooperation I was receiving in my scholarly endeavor, and I set off armed with an Olympus DS-330 digital voice recorder, a brand new software package  NVivo 2.0 and tremendous enthusiasm for the project I was beginning.

Adapting an interview from the Institute of Medicine’s study of Microsystems, I specifically chose to hone in on three areas of focus. These were level of performance, investment in improvement and leadership. I developed a specific interview guide to start the initial process of data collection (see Table 2).

The Olympus DSS-330 digital voice recorder was essential to provide a record of each interview. Initially I thought I would use a voice transcription system to convert the interview recordings into a transcript to be analyzed. This proved much more difficult and too time-consuming to be effective, despite having purchased the software and headset for this process. To do voice transcription, one needed to train the software for many hours using your own spoken voice. This involved listening to the interview, stopping to speak the words I heard and recite it into the computer. The output was not close to what I had spoken and needed many edits to be able to create a good verbatim transcription of the interview. A transcriptionist would be needed to help make this data ready for coding with the NVivo 2.0 program that I had purchased. This was very frustrating and a difficult deviation from my initial plan to be efficient in transcribing my own interview files.

I also created a set of demographic questions to be able to describe my sample, their age, gender, roles in practice, etc. These were important to characterize my sample, and I created pseudonyms for my participants so that I could keep their identities anonymous. Table 1 in the manuscript http://www.implementationscience.com/content/3/1/3/table/T1 provides this detail. Only I had access to the names of the individuals, but this was never needed in the future.

A grounded theory paradigm was chosen because I was so new to this practice setting, and there were not any published reports of how practices made change in practice. There were previous reports related to quality improvement interventions and the compelling need for quality improvement in health care that led to recent publication of a suggested a conceptual framework (Microsystems). The Microsystems approach was developed in large health care systems, not the primary care community small practices that were involved in the PPRNet-TRIP-II intervention. Using a purposive sample of participants from within each practice, including representatives of nursing staff, office staff and managers, physicians and nurse practitioners I sought a broad set of perspectives to help me to learn how different groups perceived the experience of implementing guidelines for cardiovascular disease prevention and how they used their electronic medical record system to help them. Throughout the interview process a theoretical sampling method was used to add diversity to the data being obtained. In theoretical sampling, one looks for instances of what has developed as a hunch in a previous data collection. The interviews evolved to provide data for emerging concepts, and new questions. My use of grounded theory included using a systematic set of procedures: coding and categorizing data, re-reading the interviews, recoding, and collapsing categories, writing memos and asking more questions of the data to construct meaning from what the participants had to say.

Having coded, and reduced data in an iterative process, examining memos, field notes and interview transcripts, a model developed that explained the phenomena. Using this method to analyze the qualitative data, key concepts emerged and enabled the relationships to be modeled. The usefulness of this modeling is the resulting framework, which allows future developers of evidence-based improvement projects to test an implementation strategy that engages team members to be actively involved.

I had not previously used any qualitative software, but was determined to learn how to use NVivo to enable a thorough analysis of the data I had developed. I had to search for a trainer to help me to learn how to make sense of this new and complicated program I had purchased. There were several resources on the software’s website that helped to give me a basic understanding of how the software worked, yet I kept feeling like I would need to have someone who used the software show me how to set up a coding and node structure. I located a nursing faculty member listed as a consultant/trainer for the NVivo program within one of the cities I was visiting for the interviews. She provided a private session for me and the project evaluator of PPRNet-TRIP-II to learn the key features of NVivo. During this very helpful session I learned the basics of how to begin with some confidence to use this software to start coding and locating codes and themes in my files.

The initial codes were developed using the body of literature on barriers to implementing clinical guidelines, leading change and Microsystems as previously referred to (in Setting up the Data). These initial codes were set up as a ‘tree’ structure )category, sub-category etc) to start off with each document. These codes had a numeric identifier in the code list. As I began to code the documents, I found that new ideas continually emerged to be attributed to specific comments within the interviews. When the words occurring in the text were used to label the codes, this process was referred to as coding “in-vivo” within the software. I labeled these as free codes (in NVivo’s terminology, nodes) that would be reviewed at a later time. Eventually a large number of codes were present, and many of these seemed redundant. This required condensing the codes that were similar to be able to get a manageable set that would lead to the development of new concepts within this data. So, the lesson learned was start specifically, then go broadly, then refine the data so the overlapping themes would be reduced.

Additionally, it was important to create attributes for the participants in each interview. Using the demographic characteristics that I collected I was able to look at the data in different ways when thinking about ways in which I might make interpretations of the data. For example, one idea was to consider if a certain code or concept was found in the interviews of only physicians or only young female assistants, I could search based on some of those characteristics to check out the emerging ideas I was starting to have in relation to the data. While playing with the features of the software proved interesting and fun, I had to scale back on some of these ideas so I could focus on the work of coming to some conclusions with the data.

These are the coding structure and quantity of codes originally created. 

To develop a pilot process for the evaluation of my data, I analyzed nine of the interviews first (as these were the first interviews transcribed). This was very helpful as I had to present a poster for Student Research Day. I wanted to get a sense of how I was setting up the analyses, to make sure I was on track. I developed the goal of creating a conceptual framework as the product of my research; and this needed to be evaluated to see if indeed I was coming up with concepts that could be related to a process.  

Using the existing codes, I reviewed the categories with the highest numbers of citations and reviewed this data to create some new concepts out of the codes. This provided the opportunity to work with preliminary ideas and then start transforming the raw data. One of my nursing faculty members had some time to spend with me, and reviewed the coding with me to ensure that the work was credible and that the codes I developed made sense. Together we refined some key concepts and themes, and I worked to map the concepts out so a flow diagram or process of change could be visualized. 

Shared Leadership for Practice Change 

Upon viewing this flow diagram, I felt ready to talk about this work as the pilot study of the analytic process that I would use for the dissertation.  Visual representation of the concepts looked credible and I had direct quotes to confirm that the concepts were grounded in the data. I presented this early framework at Student Research Day very successfully, and I was awarded the first place prize for PhD posters in my category (plus a cash prize!) This gave me confidence with the direction I was proceeding in, and I felt ready to continue to code the remaining interviews. My poster prize afforded me with the funds to pay a transcriptionist for the work I needed completed. Next, I presented the poster (as a work in progress) at a primary care research meeting (North American Primary Care Research Group) and at a nursing research meeting (Southern Nursing Research Society) in the student section, to get some ideas from conference participants. I found the input extremely useful to the remaining process that needed to be undertaken.

I continued with the coding process and completed the rest of the interviews, then added other materials which were gathered in the process of the research evaluation of the parent study. This included site visit notes and impressions related to the practices that were visited. There was also quantitative data regarding the changes in practice that were occurring in relation to the use of the clinical guidelines that were being implemented related to primary and secondary prevention of cardiovascular disease and stroke. All of the data was considered within the multi-method analysis of the PPRNet-TRIP II project which was simultaneously underway and was informing my impressions about how these practices were achieving changes in practice.

To develop the final conceptual framework, I developed a plan to have a weekend immersion in the data with two of my dissertation committee members. This took some time to arrange, but as I was being guided in an important new process of analysis, I wanted to make sure it was the best and most credible qualitative analysis that could be done. One of my mentors came in from California and the other from New Jersey. We convened the analysis at my home, sitting on rocking chairs on the front porch to read some of the transcripts aloud to fully immerse and crystallize the meaning of these interviews. The process of immersion and crystallization was used to refine the theoretical concepts that were established in the early conceptual model. Immersion and crystallization is a process whereby the individual can open up new insights through induction of a self-reflective, interpretive cycle (Borkan, 1999). Through iterative processes of immersion and crystallization with the data, creative and intuitive ideas flowed enabling the construction of new meanings that extended earlier explanations. As these new concepts were considered, critical insights emerged. Immersion in the data provided a means to re-live the experience that generated the data, with a fresh perspective.

This analytic process provided the opportunity to reflect, revise and re-consider the original framework and help differentiate the concepts more fully. I redrafted the conceptual framework and the three of us were in agreement with the revised conceptual frameworkCrystallization of the concepts in a group process provided a sense of intellectual honesty that helps to validate the findings. For each of the concepts, a series of quotes from the participants elucidated the meanings and perspectives related to each concept which are provided in the final manuscript.

This is final conceptual framework as shown in the final manuscript. 

After completion of writing and defending a dissertation, I went into a hibernation mode for a period to recuperate from the dual obligations of a full time position within hospital/nursing administration and a full time doctoral student. I submitted abstracts for presentation within nursing research and primary care research meetings and reported my final results in poster and podium presentations. I wanted a break from the writing, and it took me almost a year from my graduation to the completion of a manuscript to briefly describe the results of my research. A new journal was launched during 2005, (the year I completed my dissertation) and it appeared to be a perfect fit for my work. I set my goal for publishing within th open access journal, Implementation Science. This journal required suggesting peer reviewers for your topic, which was a new concept for me. Open access was exciting, as the author retains copyright on the material. With complete electronic access, the knowledge developed within one’s work has an opportunity to be broadly shared.

I had submitted my manuscript in early June, 2006 and in late July I received the first reviews of the paper. A major revision would be needed, but there were areas where the two reviewers substantially differed. This left me baffled and a bit paralyzed about where to start. It took me nine months before I had another draft. After the second version was completed, both reviewers still raised concerns about changes that were needed. Another six months passed before I overcame the inertia to complete the revision. Eighteen months after I began the process to publish my research within this journal, it was finally accepted.

Implementation Science provided a good home for this work. I was encouraged to pursue this venue for publication when the first calls for papers came out, and the the journal’s purpose seemed to be so closely aligned to the field of health services research that I was engaged in. Another collaborator that I worked with also pursued publishing a paper in this journal, and his paper was accepted and published quickly. This may have helped me in having co-authored a paper in the journal before my paper was accepted. His paper and my paper were both designated a “highly accessed paper” receiving a large number of downloads within the first month of publication.  My paper remains on the list of the highly accessed articles for the past year. In the end, I was significantly humbled by the experience of publishing in this venue as I highly regarded the work being generated in this very high quality journal (the unofficial impact factor for 2008 was 4.43).

The manuscript was late to the field, partly because of my inertia between reviews, and also because I did the initial research early within my doctoral trajectory. As a result, I had to hold off publishing results until my dissertation was approved. The body of knowledge expanded during this time, so that the question about whether or not my work was “new” was being raised within the publication process. There was not another project such as mine reported in the literature on implementing change in primary care practices, and there were not visual models of the process of leading change using electronic medical records.  Additionally, I chose to report the research in a format that allowed the voices of the participants to be heard, to provide strength to the argument that a framework for change in primary care practice was needed. By elaborating the meanings of the key themes through the words of the participants, I intended to create a credible and practical framework that could be used by others within the field of primary care improvement.

The conceptual model  was adopted as one of the three components of the PPRNet-TRIP QI model which includes an intervention model, an improvement model and a practice development model. PPRNet research has evolved since this time, and we now have a significant portfolio of funded research which is underway within primary care practices throughout the United States in 140 practices within 38 states. 

Borkan, J. (1999). Immersion/Crystallization. In B. F. Crabtree & W. L. Miller (Eds.), Doing Qualitative Research (pp. 177-194). Thousand Oaks: Sage.

Cabana, M. D., Ebel, B. E., Cooper-Patrick, L., Powe, N. R., Rubin, H. R., & Rand, C. S. (2000). Barriers pediatricians face when using asthma practice guidelines. Archive of Pediatrics and Adolescent Medicine, 154(7), 685-693.

Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H., Abboud, P.-A. C., et al. (1999). Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA, 282(15), 1458-1465.

Crabtree, B. F., & Miller, W. L. (1999). Doing Qualitative Research (2nd ed.). Thousand Oaks: Sage.

Donaldson, M. S., & Mohr, J. J. (2000, November 3, 2000). Exploring innovation and quality improvement in health care micro-systems: A cross-case analysis.   Retrieved 3/23/2002, from http://www.nap.edu/openbook/NI000346/html/65.html

Glaser, B. G., & Strauss, A. (1967). The Discovery of Grounded Theory. Strategies for Qualitative Research Hawthorne, NY: Aldine De Gruyter.

Kotter, J. P. (1996). Leading change. Boston: Harvard Business School Press.

Ornstein, S. M. (2001). Translating research into practice using electronic medical records the PPRNet-TRIP project: primary and secondary prevention of coronary heart disease and stroke. Topics in Health Information Management, 22(2), 52-58.

1. Three years on: research developments

Relevant subsequent research by you or others – where was it done and how different was it to your earlier project?

The initial study was done 2003–2005 and published in 2008. Since then, a body of research has been conducted within PPRNet (the practice based research network that the original research was done within). Ten additional studies have used the conceptual framework to investigate how practices have improved on selected quality measures relevant to primary care (colorectal cancer screening, medication safety, alcohol screening, brief intervention and treatment, reducing inappropriate antibiotic prescribing, increasing immunizations, preventive screening and diabetes care measures). I have recently completed a qualitative synthesis of seven of these studies to refine the PPRNet-TRIP QI model as it has evolved over the past decade, as advances in health information technology and team development have occurred. 

What does it add to or alter in the understanding of your project’s topic?  What is new and why? (Different method? Different theoretical approach? ….)

A new framework was developed that specifies four main concepts that are needed to improve primary care using health information technology (IPC-HIT). The model concepts include:

• Develop a team care practice.
• Adapt and use HIT tools.
• Transform practice culture and quality.
• Activate patients.

If there isn’t anything new in the area since your work, what does that indicate about the field, or maybe the decisive brilliance of your work?

This new model is particularly useful now that the US healthcare system has developed financial rewards and incentives to practices that ‘“meaningfully use’ HIT in their practice. A movement is underway within the primary care environment for practices to become recognized as Patient Centered Medical Homes,  which develop and utilize their teams to set goals for improvement, in a transformed culture of quality using performance data for learning and further improvement.  Practices are now being rewarded through health care payers (commercial, as well as Medicare and Medicaid) for achieving high performance on clinical quality measures of outcome and process for meaningful use of their health information technology.  Patient engagement is key to the process of achieving success with these transformed primary care teams.

1. In hindsight

If you were to design and conduct that project now, what would you do differently? Why? (Different method or location? Different theoretical approach? ….)

The initial project was designed well, and its’ relevance a decade later indicates that the results were important.

What did the project need to be more satisfying to you, more adequately addressing questions that matter? 

The project has been important to me, and I think important in the field of implementation science, yet not recognized entirely for this.  As I started this work as a doctoral student, my own level of sophistication as a researcher was being developed, and I could have better stated the rigor of the methods I used, and added additional levels of review and consultation.

1. Software tools

Looking back, could you have used the qualitative software available to you more effectively – and how? (Please do honestly contribute any reflections on your own or team members’ software use and advice to others.)

The level to which I used NVivo 2.0 software in my initial project was elementary. I did not fully understand all of the features and still continue learning to be a more effective user.

How would current versions of software tools have helped?  (The answer may be simple – e.g., not at all! – or quite complex, if new tools have appeared to do what you were unable to do 3 years ago.)

Additional features in NVivo 10 have increased the experience of qualitative analysis for researcher.  I struggled for a number of years that I was only using Macbook Pro and partitioning a Windows side of my computer to do NVivo. This was difficult and slowed my learning of how to most efficiently use it. 

Another chance to express annoyance or despair if all these years on, the available tools still would not do what you needed?

NVivo tools are excellent, but you need to spend consistent time working in the program to be able to make the most of it.  If you are away from your project for a couple of months, it seems you must relearn it to be facile in organizing your materials and remembering the nuances and power of the program. I still after ten years of using NVivo feel as if I am not up to full use, and would love to spend a week with the power users to become the expert I should be at this point.

Lynne S. Nemeth, PhD, RN has been involved in healthcare from many different perspectives throughout her career. I broadened my view of the healthcare system by working within clinical practice, management and administration of healthcare delivery and research. As a nurse in these various settings, I focused on quality improvement within the healthcare system, and had many questions and observations regarding why some groups are more successful than others in accomplishing improvement. I have been involved in qualitative research for the past seven years, during which time I completed my doctoral studies. Immersed within the study of the process of change, my dissertation findings explained how primary care practices evolved to improve quality using their electronic medical record systems. I continue to be engaged in primary care research within a practice-based research network of users of a common electronic medical record. Within this network, I am the qualitative evaluator for several completed and ongoing research studies relevant within this setting. I am currently an associate professor at the Medical University of South Carolina, in the College of Nursing. I travel extensively with my husband and family, and have a passion for art and fitness, and being near the ocean.