Nursing care documentation in electronic health records (EHRs)

Evidence of Progress in Making Nursing Practice Visible Using Standardized Nursing Data: a Systematic Review

Tamara G. R. Macieira, BSN, PhD student1, Madison B. Smith, BSN, RN, PhD student1 Nicolle Davis, BSN, RN, SCRN, PhD student1, Yingwei Yao, PhD1, Diana J. Wilkie, PhD,

RN, FAAN1, Karen Dunn Lopez, PhD, MPH2, Gail Keenan, PhD, RN, FAAN1 1University of Florida, Florida; 2University of Illinois at Chicago, Illinois

Abstract

Nursing care documentation in electronic health records (EHRs) with standardized nursing terminologies (SNTs) can facilitate nursing’s participation in big data science that involves combining and analyzing multiple sources of data. Before merging SNTs data with other sources, it is important to understand how such data are being used and analyzed to support nursing practice. The main purpose of this systematic review was to identify studies using SNTs data, their aims and analytical methods. A two-phase systematic process resulted in inclusion and review of 35 publications. Aims of the studies ranged from describing most popular nursing diagnoses, outcomes, and interventions on a unit to predicting outcomes using multi-site data. Analytical techniques varied as well and included descriptive statistics, correlations, data mining, and predictive modeling. The review underscored the value of developing a deep understanding of the meaning and potential impact of nursing variables before merging with other sources of data.

Introduction

The main frontline providers of care are nurses who also represent the largest category of health workers in the hospital setting. Among the 2.8 million registered nurses currently working in the United States (U.S.), 61% work in hospitals1 whereas 19% of 297,1002 pharmacists and 41.9% of 854,698 physicians in practice work in hospitals.3,4

Nurses are responsible 24 hours each day for continuously identifying care issues, implementing and adjusting care prescribed by themselves and other providers to achieve desired patient outcomes. To date, however, it has been difficult to effectively evaluate the impact of nursing on patient outcomes. The growing use of electronic health records (EHRs) to document care now offers the opportunity to use the data captured in practice for discovering knowledge to transform health care. Thus, the documentation entered by nurses into EHRs, for the first time ever, is a potential source for discovering the impact of nursing care on patient outcomes and using the knowledge to improve care. In this article, we report our systematic review of studies that utilized nursing EHRs data to answer a variety of research questions from describing nursing care for a specific population to predicting patient outcomes. The publications reviewed provide a foundation for identifying future paths of inquiry involving nursing and other data retrievable from EHRs.

The use of standardized nursing terminologies (SNTs) to document nursing care enables the easy retrieval and analysis of nursing data while also representing the nurse’s clinical reasoning.5 The integration of nursing data into large datasets requires the frequent and rapid input of new valid information from EHRs.6 These can be achieved through the use of controlled vocabularies in EHRs, which helps overcome the major challenges of aggregation, processing and analysis associated with unstructured text data.7-8 In nursing, SNTs are controlled vocabularies that represent nursing care as nursing diagnoses, interventions and outcomes.8 The SNT coded data retrieved from EHRs can be analyzed alone or merged with other EHRs data. The use of SNTs to document nursing practice is a big step toward supporting the aggregation of nursing data to large datasets and big data science.

Different sets of SNTs are used to document nursing care. The American Nurses Association (ANA) recognizes and supports the use of certain nursing terminologies to guide and document care if those have clear and unambiguous concepts, are coded with a unique identifier per concept, and if those terminologies were tested for reliability, clinical usefulness and validity.9 The following nursing terminologies are recognized by ANA: NANDA-International (NANDA-I)10; Nursing Interventions Classification (NIC)11; Nursing Outcomes Classification (NOC)12; International Classification for Nursing Practice (ICNP)13; Omaha System14; Clinical Care Classification (CCC)15; and the Perioperative Nursing Data Set (PNDS).16 While ICNP, Omaha System, CCC and PNDS sets contain diagnoses, interventions and outcomes terms; NANDA-I (diagnoses), NIC (interventions) and NOC (outcomes) are three separate terminologies. Since NANDA-I, NIC and NOC are very often used together, we will refer to them as a terminology set (NNN).

Systematically reporting and analyzing studies that used SNTs nursing data retrieved from EHRs is important to understand the analytic issues related to the complexity and richness of data generated from the use of these

1205

 

 

!

terminologies. Given the growing emphasis on using existing health care datasets, there is a need for new statistical, computational, and visualization methods to analyze EHRs data, given their complexity and volume.6 For nursing to join this effort, an important first step is to identify and examine the studies that analyze EHR data coded with SNTs.17

To date, there are few reviews that report secondary analysis of SNTs nursing data. One recently published systematic review18 described study focus, sample characteristics and frequency of publications that studied SNTs. The authors also identified a limited set of studies in which SNTs nursing data were being analyzed.18 However, substantial evaluation and discussion on the analysis of SNTs nursing data were not performed. Another review19 described characteristics of nursing research data (ranging from patient demographics, social history, medical history, medications, among others) and evaluated if a specific index metadata system represents sufficiently nursing data. Our systematic review differentiates from the previous investigations as we focused only on studies that analyzed nursing data coded with SNTs retrieved from EHRs. The earlier literature review19 included all types of unstructured nursing data, and different types of studies, such as controlled clinical trials. The authors also did not restrict their selection criteria for nursing data retrieved from EHRs.19

The systematic review presented below was conducted to describe and critically analyze the body of studies in which secondary analyses of data coded with the ANA recognized SNTs was performed. These nursing data were documented during the delivery of nursing care and retrieved from EHRs. We believe that the findings from the present systematic review will emphasize the importance of coded nursing data and will encourage a wider use of SNTs that will allow greater participation of nursing in big data science initiatives.

Objective

The objective of this systematic review was to uncover the state of the science related to the use of standardized nursing data (coded with SNTs) retrieved from EHRs to answer research questions, describe the analytical techniques employed, and outline the lessons learned applicable big data science and nursing.

Methods

Search strategy

A comprehensive literature search was conducted to identify publications in which secondary analysis was performed on data extracted from EHRs and documented using terms from the ANA recognized nursing terminologies (or sets). There were two phases used in this process: 1) study selection and 2) data collection process.

Building from the work of Tastan et al.18 that reported secondary uses of SNTs documentation in studies up to 2011, the databases PubMed and CINAHL were searched using keywords encompassing all ANA recognized SNTs, along with “nursing” and “electronic health records” from 2010 to 2017. Keywords were defined for each database with the help of a librarian, who was a specialist in Consumer Health, Nursing and Health Education and Behavior. The limiters since 2010, abstract available and published in English were used. Potential publications identified in both databases were downloaded into a reference management program (EndNote X7, Thompson Reuters ISI ResearchSoft), in which duplicates were deleted and abstracts were reviewed. Finally, grey literature search was conducted using Google Scholar to identify possible publications not captured by the traditional search methods, including relevant publications not in PubMed or CINAHL, but in computer science databases like IEEE. Name of authors of publications already reviewed and included in this study’s sample were individually searched in Google Scholar. Publications pertinent to the subject were reviewed and included.

Study selection

The studies identified using the specified search strategy were submitted to abstract review according to the following inclusion criteria: 1) publications that conducted secondary analysis of nursing data retrieved from EHRs, 2) nursing data were coded with one of the ANA recognized nursing terminologies and documented at the point-of-care, as part of the institution’s routine, and 3) publications were published in English. Following PRISMA guidelines20, Figure 1 shows the number of publications first identified, retained after selection from abstracts, and reviewed as full text.

The team utilized a two-stage selection process. In the first stage, two doctoral students in nursing informatics (TM, ND) independently conducted the literature search in the databases using the same keywords and found the same number of studies. A guide created by the authors was used to systematically review the abstracts. The reviewers first identified if the article contained at least one of the ANA recognized SNTs. If the abstract described the use of a SNT, the reviewer would continue to verify if the data analyzed were retrieved from EHRs. In the case of an affirmative answer, the reviewers would further analyze the abstract to assess if the SNT coded nursing data were documented as

1206

 

 

!

part of routine clinical practice and not solely for research purposes. Publications selected by one reviewer but not the other were discussed until agreement was reached. In the second stage, TM and MS (doctoral students) conducted a full-text review for further eligibility screening.

Data collection process

Data from the eligible publications were independently collected by TM and MS. Tables were created to summarize and analyze the content of the articles. The following information was extracted: study foci; sample characteristics (number of health records); variables (dependent, independent, descriptive); and statistical analysis. Data extracted from the selected articles were also reviewed by two faculty members, one specialist in nursing informatics and one statistician. Consensus agreement was reached. The present study aimed to report the diverse statistical analysis methods utilized and studies foci addressed through the use of SNTs coded nursing data. We did not intend to report the results of those publications.

Quality appraisal

Quality appraisal of publications is common in systematic reviews, however, it is challenging when the publications being reviewed are of the secondary data analysis nature. For the purpose of this review, quality appraisal of articles was evaluated regarding completeness of reporting of information. An adapted version of the STROBE Statement- checklist for observational studies21 was used, including the following items from the original checklist: study size, clear definition of variables, and description of all statistical methods used. Clear definition of variables was scored as 1 (not defined), 2 (partially defined), and 3 (well defined). These ratings considered if the variables in the publications were described with sufficient clarity to be replicated (collected) in further studies. As the previous stages of the review, authors independently (TM, MS) evaluated the completeness of the items mentioned in rounds until agreement was achieved.

Results

Abstracts of 1,809 publications were examined and 75 publications were retained according to the inclusion criteria. In the data collection process (second stage), of the 75 publications remaining, 40 were excluded due to different reasons as described in Figure 1. A total of 35 publications were included in our systematic review.

The studies included represent considerable diversity in terms of sample sizes, ranging from 29 to 379,601 health records. The largest sample sizes represent more than one unit in a single hospital or multiple hospitals, and more than one primary health care center. In these cases, the same EHR and SNTs were implemented across the institutions. Types of records portrayed in the studies encompass daily entries, or shift entries of nursing information into the EHRs at point-of-care. Episodes of care represent the entire history of a hospitalization, typically consisting of several entries for a unique patient hospitalization. For the purpose of this study, health records were defined as representing unique patients and health records entries were defined as daily/shift documentation of care.

In this review, we found studies for only three of the five terminology sets. There were no studies found for CCC and PNDS. There were 21 NNN articles, 10 Omaha System articles, three articles on ICNP, and one article that evaluated both NANDA-I and ICNP. Although all articles were published in English, we identified studies conducted in countries where English is not an official language. All articles on ICNP were conducted in South Korea, while all articles using the Omaha System were conducted in the US. The studies focusing on NNN were from Spain and U.S.

Descriptive, dependent and independent variables of each study were collected. In studies of descriptive nature, the variables analyzed were the nursing diagnoses, interventions and outcomes relevant to each set of SNTs. For studies in which statistical tests for associations were conducted, the majority of independent variables were the sets of SNTs, along with other factors related to delivery of care and the healthcare environment. Dependent variables in those studies varied significantly, including for example, occurrence of falls22, pain23-24, gender, age and medical diagnoses25, nursing cost26-27, among others.

 

 

1207

 

 

!

 

!

! ! ! ! ! ! ! ! ! ! ! ! ! !

 

 

 

 

 

Figure 1. PRISMA flow diagram illustrating the selection process of publications.

Research focus

Research foci encompassed different questions that could be answered by SNT coded nursing data. Twelve studies focused on characterization of units and patients regarding the most common nursing diagnoses, nursing interventions and nursing outcomes28-39 (Table 1). In two studies, authors described potential flaws in the terminologies regarding their content40-41 (Table 1).

Study of the nursing care through the use of SNTs included differences between standardized and non-standardized plans-of-care38; comparison of nursing care provided and standard nursing care from guidelines32; and testing different search strategies using nursing interventions to analyze the incidence of a medical complication.42 The benefits of using SNTs in EHRs to increase availability, validity, and reliability of data for statistical analyses and other research purposes were also described.34

SNTs NOC and Omaha System were used across different studies, measuring changes in outcomes from admission to discharge.43-45 Both terminologies have a grading scale to be used by nurses daily to rate the nursing outcomes for each patient. In the studies43-45, authors compared ratings given at admission and discharge to patients as a way of measuring effectiveness of care. Association of those ratings with other variables from EHR was also performed.43 One study focused on demonstrating differences between two statistical methods in describing changes in ratings for the Omaha System terminology.46

 

 

Publications after duplicates removed (n = 1,809)

PubMed + CINAHL (n = 2,364)

!

Additional publications Previous systematic review (n = 10)

Grey literature (n = 4)

Abstracts screened (n = 1,809)

Abstracts excluded based on defined exclusion criteria

(n = 1,734)

Full-text articles assessed for further eligibility (n = 75)

Full-text articles excluded: mapping free-text nursing documentation to a SNT; used solely

paper-based documentation; focused on something other than use of SNTs; full-text

not available (n = 40)

Publications included in qualitative synthesis

(n = 35)

1208

 

 

!

Table 1. Research foci. SNT Research focus

NNN

To describe availability of plans-of-care data34 To describe most prevalent NNN of an SNT-based electronic nursing documentation37 To describe the most frequent NNN documented for hospitalized older patients with a primary discharge diagnosis-related group of pneumonia30 To describe the most frequent NNN documented for hospitalized older patients with a discharge primary diagnosis of heart failure33 To report pain care from admission to discharge or death for end-of-life patients24 To determine statistically significant changes in mean scores of outcomes from admission to discharge for specific medical diagnoses45 To determine changes in outcomes scores from admission to discharge, and describe interventions for pediatric patients with dehydration44 To measure the cost of delivering high surveillance, among hospitalized elders at risk for falling27 To measure association among patient characteristics, unit characteristics, medical, pharmacy, and nursing interventions, and falls for older adults22 To measure the association between high surveillance and low surveillance and the occurrence of failure to rescue48 To establish association between degree of severity of problems and psychiatric diagnosis, number of nursing diagnoses, age, gender25 To determine associations between the variables nursing characteristics and nursing cost per acute care episode26 To identify patient and nurse-related factors associated with nursing outcomes for end-of-life patients with death anxiety51 To report of NOC scores linked with the most frequent nursing interventions, age and length of stay36 To describe the difference between plans-of-care with NNN and plans-of-care without NNN in relation to intermediate health outcomes38 To measure association between the variable nurse continuity and the occurrence of pressure ulcers56 Describe data mining techniques to predict whether a patient would meet the expected pain related outcomes23 To develop predictive models that show how patient and nursing variables impact comfortable death outcome in end-of-life patients52 To predict current NOC rating for each nurse shift based on previous shift ratings and factors related to nurse shift53 To create predictive models that determine whether a patient with pain problems will be re-admitted to a hospital55 To describe infomarkers that mark a shift in pain outcomes from standard care to palliative care among patients that died during hospitalization54

OS

To describe client problems, interventions, categories and targets, along with outcomes documented for patients in a low-birth weight program28 To describe all Omaha System data elements across multiple homecare settings using two different EHR vendors29 To describe client risk index, family home visiting interventions between groups of low- and high-risk31 To describe free text entry for the category Signs and Symptoms of the Omaha Classification40 To describe free text entries associated with Omaha System on a software41 To describe ratings of knowledge, behavior, and status on admission and discharge for Latina adolescent and adult mothers with mental problems43 To demonstrate differences between p-values and Cohen’s d in describing clinically meaningful changes in scores for Omaha System problems46 To determine which group of interventions predicts the outcome hospitalization for frail and non-frail elders47 To measure association between patient, support system factors and nursing interventions and improvement in urinary and bowel incontinence49 To determine the association between nurses, Omaha System interventions and client characteristics, and variability in health literacy50

ICNP To describe variability in nursing intervention to prevent and treat pressure ulcer patients32 To describe nursing preventive pressure-ulcer interventions and to compare with measures from two published guidelines35 To describe narrative nursing statements of cancer patients treated with cisplatin-based chemotherapy35 To test different computerized search strategies, to analyze the incidence and clinical characteristics of contrast media hypersensitivity42

N/ICNP To describe key nursing diagnoses from the two terminologies for patients with heart failure39 Key = NNN (NANDA-I, NOC, NIC); OS (Omaha System); N/ICNP (NANDA-I and ICNP).

1209

 

 

!

NOC and NIC were used to estimate nursing cost.26-27 Jenkins and Welton26 used the NOC rating scale as a measure of nursing intensity, and in turn, nursing cost. A different measure of nursing cost was used by Shever et al.27 In their study, NIC was used to measure the cost of delivering the nursing intervention surveillance. Surveillance was used as a measure of intensity of treatment, based on the average number of times a day that it was delivered (low or high). The number of times the intervention was performed was associated with the total hospital cost for the population under study.27

In two studies the researchers examined relationships between nursing interventions and outcomes.47-48 Other (n=4) studies in the sample focused on establishing associations among a broader number of variables, such as patient characteristics, support system factors; nursing characteristics, nurse continuity, degree of severity of problems; medical diagnoses, pharmacy and nursing interventions, and nursing outcomes.22,25,50-51 Lodhi et al.51, Lodhi et al.52, and Lodhi et al.53 identified patient and nurse-related factors associated with meeting expected nursing outcomes.

Almasalha et al.23 focused on the use of SNTs to predict whether a patient would meet the expected outcomes by the end of a hospitalization period. Yao et al.54 identified specific elements of SNT related to changes in outcomes that could be used as an indicator for the adjustment from standard nursing care to palliative nursing care. Lastly, Lodhi et al.55 explored the creation of different predictive models for re-admission of patients that were diagnosed with pain during their last hospitalization.

Table 2. Progression of nursing science and knowledge.

Key = * health records; ** health records entries

Statistical analysis

A total of 15 studies used only descriptive statistics to analyze their data. The following statistical analysis methods were present among the remaining studies: propensity scores26,48; logistic regression and other regression methods25- 27,47-48,52,55-56; generalized estimating equations22,27; data mining techniques22,47,51-53,55; chi-square23-24,51-53; t-test23,39,44-46;

Dataset Year Sample* Statistical analyses

1 university hospital, 2 large

community hospitals, and 1

small community hospital

201234 40,747 Descriptive statistics 201323 569 Descriptive statistics, Data mining, Chi-square test, t-test 201324 596 Chi-square, Wald tests 201451 432 Data mining, Chi-square 201552 438 Data mining, Logistic regression, Chi-square 201553 160 Data mining, Logistic regression, Pearson’s two proportion z-test 201556 840 Descriptive statistics, Logistic regression 201554 901 Known-group comparative analysis, ANOVA, Tukey’s post-hoc test 201755 2,300 Data mining

3 community hospitals

200744 29 Descriptive statistics, t-test 201130 451 Descriptive statistics 201133 302 Descriptive statistics

1 academic medical center

200827 7,851 Propensity scores, Regression with generalized estimating equations 201122 7,851 Generalized estimating equations

1 homecare and 1 maternal home

visiting program

201040 3,388 Descriptive statistics 201141 61,701** Descriptive statistics

1 public health agency

200728 75 Descriptive statistics 201131 486 Descriptive statistics 201346 1,016 Paired samples t-test, Effect sizes (Cohen’s d) 201550 141 Logistical mixed-effects model

15 home health agencies

201029 2,900 Descriptive statistics 201147 1,750 Data mining, Logistic regression 201149 2,072 Descriptive statistics, Logistic regression, Data mining, Chi-square

1 university hospital

 

201132 41,891 Descriptive statistics 201242 759 ** Descriptive statistics 201235

Study 1: 427 Study 2: 355

Study 1: Descriptive statistics, ANOVA Study 2: Descriptive statistics

1210

 

 

!

ANOVA35,54; Pearson correlation25,53; Tukey’s post-hoc test54; logistical mixed-effects model50; effect sizes (Cohen’s d)46; general linear mixed methods models43; and known-group comparative analysis.54

Data from same institutions were used across different publications to answer a diverse number of research questions. Table 2 shows the progression of science and knowledge in nursing across the years. Publications are grouped by dataset (each group of institutions used the same EHR). It can be seen (see Table 2) that earlier publications used mostly descriptive statistics and advanced to generalized estimating equations, logistic regressions, and data mining procedures. Sample sizes varied across publications and more specific and complex statistical analysis were related to smaller parts of the datasets.

Quality of publications included

Among the 35 publications included in this systematic review, all described their study size, which was represented by number of health records or health records entries, and listed statistical analyses used. Definitions of variables were scored for the 35 publications; 74% of the publications22-25,27-33,37,39-40,44-48,51-56 scored a 3 (variables well defined), 23% of the pubications26,34-35,38,41,42,49-50 scored a 2, and 3% of the publications43 scored a 1. Overall, the studies were rated as good for completeness of reporting of information.

Discussion

The present systematic review provides important information about the types of studies that have been conducted on SNTs coded nursing data retrieved from EHRs. The 35 studies were identified and reviewed through a comprehensive systematic process. The SNTs NANDA-I, NOC, NIC and Omaha System were those implemented most often across different EHRs. Specifically, NNN was represented more in the hospital setting, while Omaha System was present in the primary level of healthcare. Worth noting however is that both of these terminologies can be used interchangeably between hospitals and primary health care centers.

There was great diversity in the questions addressed in each study and the variables analyzed. Characterization of nursing care through the most common nursing diagnoses, interventions and outcomes was the aim of some studies. Others focused on determining significant changes in outcomes from admission to discharge, measure cost of nursing interventions, establishing associations using variables inherent to nursing care and other variables found in EHRs. The creation and use of predictive outcomes models were also studied. This range of study foci show the potential of obtaining and translating new knowledge from nursing data coded with SNTs and the holistic approach of nursing care. It also brings the attention of researchers to the possibility of asking new questions aimed at improving patient outcomes, nursing practice and the healthcare system.

Among the studies included in the sample, most investigators either utilized descriptive analysis (mean, standard deviation, frequency, percentage, etc.) to describe terminology usage in different healthcare settings or relied on classical tests (e.g., ANOVA, Tukey’s Post Hoc, Chi-square, t-test, Mann Whitney U test, correlation test) to determine bivariate associations. Although of limited nature, descriptive studies are a first step to understand a dataset and the target population under study. Descriptive statistics are also very efficient and helpful in the identification of predictors with low variability that could be excluded in further analysis.57 The descriptive studies included in this systematic review provide a foundation for the nursing scientific community to move forward to the use of more sophisticated statistical methods.

The association analyses were those between the adoption of SNTs and patients’ outcomes, between nursing diagnosis and episode length, and between nursing interventions and patient outcomes. A number of articles conducted multivariate analysis utilizing regression analysis (e.g., generalized linear mixed regression, generalized estimating equations, logistic regression, etc.) incorporating techniques such as propensity score and clustering to examine the effects of patient, nurse, and care characteristics on patient outcomes. Finally, there were a few studies51-53, 56 using data mining methods to identify hidden patterns among nursing care data or to construct predictive models for patient outcomes. Interesting results were reported in these studies. For example, the pain management, medication, management, and positioning interventions were collectively associated with pain relief among end-of-life patients.23 Continuity of nursing care was not associated with occurrence of pressure ulcers56 and predictive modeling on large EHR SNTs data can be used as foundation in nursing decision support.53