Advertisement

NOTICE: We are experiencing technical issues with Academy members trying to log into the JAND site using Academy member login credentials. We are working to resolve the issue as soon as possible. Alternatively, if you are an Academy member, you can access the JAND site by registering for an Elsevier account and claiming access using the links at the top of the JAND site. Email us at [email protected] for assistance. Thanks for your patience!

Reprint of: Development and Evaluation of a Global Malnutrition Composite Score

      Quality measures proliferated in the late 1990s and early 2000s and were first tied to financial performance incentives with the establishment of quality reporting programs for hospitals and then physicians.
      • Marjoua Y.
      • Bozic K.J.
      Brief history of quality movement in US healthcare.
      Quality measurement has since expanded to virtually all provider areas of health care in the United States. Despite this growth, one area where a major deficit persists has been nutrition care. This article outlines the process pursued by the Academy of Nutrition and Dietetics (Academy) and Avalere Health (Avalere) to develop the first of its kind electronically specified composite measure addressing malnutrition care for hospitalized adults.

      Quality Measurement in Malnutrition Care

      In the United States, national surveillance data from 2016 indicates that as many as 8% of hospitalized adults have a diagnosis of malnutrition.
      • Barrett M.L.
      • Bailey M.K.
      • Owens P.L.
      Non-maternal and non-neonatal inpatient stays in the United States involving malnutrition, 2016.
      However, previous studies suggest that malnutrition and malnutrition risk may actually be found in as many as 20% to 50% of hospitalized patients, indicating a significant gap in the identification of malnutrition.
      • Pereira G.F.
      • Bulik C.M.
      • Weaver M.A.
      • Holland W.C.
      • Platts-Mills T.F.
      Malnutrition among cognitively intact, noncritically ill older adults in the emergency department.
      Malnutrition is a critical predictor for inpatient adverse outcomes given its association with 30-day readmissions, length of stay (LOS), complications, and mortality.
      • Barrett M.L.
      • Bailey M.K.
      • Owens P.L.
      Non-maternal and non-neonatal inpatient stays in the United States involving malnutrition, 2016.
      ,
      • Barker L.A.
      • Gout B.S.
      • Crowe T.C.
      Hospital malnutrition: prevalence, identification and impact on patients and the healthcare system.
      Despite this major gap in identification, no public quality reporting programs include performance measures focused on nutrition care or malnutrition.
      Driven by the consistent and expanding evidence of the high prevalence of malnutrition in hospitalized patients across the United States, the Academy, along with Avalere and other stakeholders, developed and implemented the Malnutrition Quality Improvement Initiative (MQii). The MQii was established largely in response to the need for assessment quality of care provided to hospitalized patients who are malnourished or at risk of malnutrition.
      Academy of Nutrition and Dietetics Malnutrition Quality Improvement Initiative.
      Through a dual-pronged approach, the MQii supports quality improvement (QI) for malnutrition care based on a set of four malnutrition-focused electronic clinical quality measures and a complementary MQii Toolkit that includes resources guiding implementation of QI activities.
      • McCauley S.M.
      • Barrocas A.
      • Malone A.
      Hospital nutrition care betters patient clinical outcomes and reduces costs: the Malnutrition Quality Improvement Initiative story.
      ,
      • Silver H.J.
      • Pratt K.J.
      • Bruno M.
      • Lynch J.
      • Mitchell K.
      • McCauley S.M.
      Effectiveness of the Malnutrition Quality Improvement Initiative on practitioners malnutrition knowledge and screening, diagnosis, and timeliness of malnutrition-related care provided to older adults admitted to a tertiary care facility: a pilot study.
      A multistakeholder collaboration identified measure gaps in malnutrition care, which were translated into a set of individual electronic clinical quality measures (eCQMs). As part of the measure evaluation process, a technical expert panel had also been convened to weigh in on the initial measure concepts from both a clinical and technical perspective regarding data feasibility. These eCQMs were subsequently piloted at a large hospital in the Midwest, and the testing results demonstrated that the measures were usable for identifying key improvement areas in malnutrition care related to identifying risk, assessing for clinical malnutrition, developing the appropriate care plan, and ensuring the diagnosis of malnutrition is documented to support follow up care.
      • Nepple K.G.
      • Tobert C.M.
      • Valladares A.F.
      • Mitchell K.
      • Yadrick M.
      Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.
      The individual eCQMs that were tested are outlined in Figure 1.
      Figure 1Individual malnutrition-focused electronic clinical quality measures (eCQMs).
      eCQMMeasure nameMeasure description
      1Completion of a malnutrition screening within 24 h of admissionPatients aged ≥18 y who received a malnutrition screening and results are documented in their medical record within 24 h of their admission to the hospital
      2Completion of a nutrition assessment for patients identified as at risk for malnutrition within 24 h of a malnutrition screeningPatients aged ≥65 y who were identified to be at risk of malnutrition from a screening were provided a nutrition assessment within 24 h of the screening
      3Nutrition care plan for patients identified as malnourished after a completed nutrition assessmentPatients aged ≥65 y who were assessed and found to be malnourished should also have a documented nutrition care plan in their medical record
      4Appropriate documentation of a malnutrition diagnosisPatients aged ≥65 y who were assessed and found to be malnourished should have a physician-confirmed diagnosis of malnutrition documented in their medical record to ensure care plan implementation and transfer of necessary medical information upon discharge
      The initial pilot testing of these novel malnutrition-focused eCQMs demonstrated that it was feasible to collect the data from existing hospital electronic health record systems, and that the measures met minimum reliability and validity testing requirements as established by expert consensus.
      National Quality Forum
      Measure evaluation criteria and guidance for evaluating measures for endorsement.
      Subsequently, the tested measures were adopted by a national learning collaborative of hospitals all implementing the principles of the MQii. A group of 27 US hospitals reported use of the four eCQMs to guide various QI projects focused on improving care provided to hospitalized patients who are malnourished or at risk of malnutrition.
      • Valladares A.F.
      • Kilgore K.M.
      • Partridge J.
      • Sulo S.
      • Kerr K.W.
      • McCauley S.
      How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.
      The participating collaborative hospitals reported changes in measure performance based on implementation of cyclical quality improvement initiatives at their respective institutions. With this new aggregate data, multivariate analyses were conducted to identify the relationships between performance on these implemented eCQMs with patient outcomes of 30-day readmission and LOS. The study results concluded that the measures could be successfully implemented in a cohort of diverse hospitals in the United States. Furthermore, the study demonstrated that when supported by QI tools, the hospitals were able to see meaningful improvements in measure performance. In addition, the multivariate analysis demonstrated that all four measures were significantly associated with outcomes of 30-day readmissions and patient LOS.
      • Valladares A.F.
      • Kilgore K.M.
      • Partridge J.
      • Sulo S.
      • Kerr K.W.
      • McCauley S.
      How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.

      The Global Malnutrition Composite Score

      These initial studies were crucial in establishing the evidentiary basis for the four malnutrition-focused eCQMs, which were being adopted across dozens of hospital systems throughout the country. As work with the Centers for Medicare and Medicaid Services (CMS) continued, an external panel of experts from CMS provided feedback to develop a composite measure—The Global Malnutrition Composite Score (GMCS)—constructed from the four individual eCQMs. CMS defines a composite measure as a performance measure representing a “combination of two or more component measures, each of which individually reflects quality of care, into a single performance measure with a single score.”
      CMS Measures Management System Blueprint v14.1.
      Composite measures facilitate the grouping of multiple quality of care constructs into a single value that more comprehensively assesses quality. The intent is that the composite performance score can be influenced in some way by each component score and have a summary score that can reflect the totality of the components. The feedback produced by a composite measure condenses a broader range of metrics that would be more challenging to otherwise assess comprehensively.
      Best practices for composite measure development
      CMS Measures Management System Blueprint v14.1.
      include ensuring all included component measures have a common orientation or focus (eg, domains of a specific area of health care quality) that may be broad in nature (overall quality of care in a disease state) or narrow (adherence to a specific set of guidance), composite component measures are justified by clinical evidence, presence of a demonstrated a gap in care or outcomes, and component measures are empirically evaluated for reliability and validity.
      Learning from the experience of building and testing the individual malnutrition-focused eCQMs referenced above, the Academy and Avalere (the measure development team) launched a composite measure development process to implement the feedback received from the external review.

      Developing the GMCS

      The measure development team studied existing composite measures and identified an initial framework and objective for the eventual composite measure. Informed by the experience of the hospitals implementing the individual malnutrition-focused eCQMs, the development team determined the focus of the proposed composite measure would be on optimal malnutrition care for adults aged 65 years and older who are admitted to inpatient service and receive care appropriate to their level of malnutrition risk and/or malnutrition diagnosis if identified.
      The GMCS includes four component measures that are first scored separately as proportion measures. The four component measures (Figure 2) represent slight variations from the original individual eCQMs (Figure 1). The composite measure components were established using empirical testing by determining which individual components would most contribute to a sound overall composite score. The overall composite score is derived from averaging the individual performance scores for the following component measures:
      • 1.
        Screening for malnutrition risk at admission;
      • 2.
        Completing a nutrition assessment for patients who screened for risk of malnutrition;
      • 3.
        Appropriate documentation of malnutrition diagnosis in the patient’s medical record when this is indicated by the assessment findings; and
      • 4.
        Development of a nutrition care plan for malnourished patients, including the recommended treatment plan.
      Figure 2Global malnutrition composite score component measure details.
      Component measure nameDenominatorNumerator
      Screening for Malnutrition Risk at AdmissionAll patients in the measure population with a documented malnutrition screening no more than 48 h before admission to the hospitalAll patients in the measure population who are documented as at risk for malnutrition via the completed malnutrition screening
      Completion of a Nutrition Assessment for Patients who Screened for Risk of MalnutritionPatients from the measure population who are documented as at risk for malnutrition via the completed malnutrition screeningPatients at risk of malnutrition who have a completed nutrition assessment documented
      Appropriate Documentation of Malnutrition Diagnosis for Patients Identified with MalnutritionPatients from the measure population who have a completed nutrition assessment documented with findings of moderate or severe malnutritionPatients who have been identified as moderately or severely malnourished by the nutrition assessment who also have a documented medical diagnosis of malnutrition in their medical record
      Development of a Nutrition Care Plan for Malnourished PatientsPatients from the measure population who have a documented medical diagnosis of malnutrition in their medical recordPatients with a documented medical diagnosis of malnutrition in their medical record who have a documented nutrition care plan with treatment recommendations to address malnutrition
      The process for risk identification, diagnosis, and treatment of malnutrition necessitates a multidisciplinary care team that begins with identification of an initial risk population for more thorough assessment by a registered dietitian nutritionist (RDN). An RDN, in turn, provides the necessary treatment recommendations to address nutritional status and the clinical indicators that inform a medical diagnosis of malnutrition documented by a physician. As described above, the four component measures individually only provide a fraction of the necessary information on quality of care for patients at risk of malnutrition. For example, knowing which patients have been assessed out of those who were initially identified as at risk, but not knowing whether or not the appropriate proportion of patients were screened upon admission would be an insufficient assessment of quality of care. Therefore, the composite measure offers a more comprehensive assessment of follow-through on best practices for patients at risk of malnutrition. One analogous example can be found in a composite measure developed to better assess quality of care for patients with type 1 diabetes. Although glycated hemoglobin level is the traditional marker for glycemic control, guidelines promote several other steps to care for patients with type 1 diabetes and track their outcomes more comprehensively. Therefore, the reported composite measure includes three domains of care with corresponding component measures: management tools, diabetes care assessment, and complications risk.
      • Indyk J.A.
      • Buckingham D.
      • Obrynba K.S.
      • Servick C.
      • Gandhi K.K.
      • Kramer A.
      • et al.
      The Type 1 Diabetes Composite Score: an innovative metric for measuring patient care outcomes beyond hemoglobin A1c.
      Composite measures reflect the individual component measures of which they are comprised. Consequently, a composite measure’s validity and usefulness is dependent on the accuracy of the individual measures as well as the methodology for combining the individual measures into a combined measure. The National Quality Forum outlines four key steps to developing a composite measure:
      National Quality Forum
      Composite performance measure evaluation guidance.
      • 1.
        Defining the composite measure’s purpose and theoretical framework,
      • 2.
        Selecting the appropriate individual component measures for inclusion,
      • 3.
        Establishing the methodology for combining the selected component measures, and
      • 4.
        Empirically testing the validity and reliability of the overall composite
      Similar to individual performance measures, composite measures must also be tested to demonstrate reliability (that the measure is well defined and can be implemented consistently within and across measured entities), and validity (that the measure logic and scoring accurately captures the intent of the measure).
      CMS Measures Management System Blueprint v14.1.
      In the case that individual components have not been empirically tested, developers should first ensure that the individual components are empirically sound and robust before considering their inclusion in the composite. After being evaluated individually, the components can be incorporated in one of the many aggregation approaches outlined above and tested as a set to ensure reliability and validity in predetermined combination.

      GMCS Development Methodology

      The findings from the outcome analyses were used to inform the inclusion and exclusion of data elements and modifications to the existing individual malnutrition eCQMs in a newly proposed GMCS.
      • Valladares A.F.
      • Kilgore K.M.
      • Partridge J.
      • Sulo S.
      • Kerr K.W.
      • McCauley S.
      How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.
      As determined by the empirical validity testing outlined in the Composite Measure Evaluation section below, each of the main components of this measure is strongly correlated with outcomes that have been empirically associated with malnutrition, including 30-day readmissions and hospital LOS. The measure development team identified that each component was correlated in a significant way to both malnutrition as a clinical outcome as well as the sequelae of untreated malnutrition, including readmissions and longer LOS. Components that were excluded from the proposed GMCS included process-oriented timing intervals for admission-to-screening and screening-to-assessment to reduce methodological complexity of the GMCS.

      Rationale for Measure Scoring

      The approach to measure scoring is considerably important for composite measures because they are a mathematical computation of multiple individual metrics. In practice, there are several ways that composite measures may be scored via the included components. Common methods include all-or-none scoring where a binary outcome of performance met or not met only occurs when performance is met on all components, any-or-none scoring where the performance on the composite is met if at least one of the component measure’s performance criteria is met, opportunity scoring is a patient-based scoring method that is based on a particular number of care events being met for a patient in the measure, linear scoring where the composite score is based on a sum of component scores in which the performance is met, and weighted scoring where each component score is assigned a weight factor and the overall composite score is the sum of the weighted scores.
      The measure development team proposed constructing the composite measure as an arithmetic average of the four components weighed equally, given that all components were significantly correlated to the important outcomes of malnutrition, 30-day readmissions, and LOS. In clinical practice, all four steps are critical components of the nutrition care process. Patients who are diagnosed and treated for malnutrition by a care team are often first identified by a nutrition screening for malnutrition risk around the time of admission. Next, based on the screening results, the patient is referred to an RDN for assessment and recommendations for malnutrition diagnosis and nutrition intervention. The names, denominators, and numerators of the final composite measure components are outlined in Figure 2. Each measure component is a proportion with a possible performance score of 0 to 100%. After each component score is calculated, an unweighted average of all four scores is completed to determine the final composite score with a total score ranging from 0 to 100%.

      Composite Measure Evaluation Methodology

      As outlined by National Quality Forum’s Measure Evaluation Criteria and Guidance,
      National Quality Forum
      Composite performance measure evaluation guidance.
      all performance measures should be tested for reliability and validity to ensure they are precise or repeatable (indicating reliability), and they accurately reflect quality of care provided and can identify differences in quality (indicating validity). To that end, a large analytic patient dataset was assembled from data reported by 56 acute care hospitals across 10 states (N = 179,336). Data were collected at the encounter level and included information on each patient encounter: LOS, discharge status, and 30-day readmission flag, screening for malnutrition risk, nutrition assessment and subsequent nutrition care plan development by an RDN, and diagnosis of malnutrition by the attending physician. Data quality was a concern for the time-to-screening data point for patients above the 99th percentile and were therefore excluded from the analysis. The capture of screening data longer than 48 hours before admission was also not included in the dataset because they are considered not to be clinically reliable.

      Reliability Testing

      The goal of reliability testing of performance measures is to demonstrate that the calculated composite score can detect true differences between measured entities (eg, hospitals or clinicians) from random measurement error.
      National Quality Forum
      Composite performance measure evaluation guidance.
      Mathematically, a reliability coefficient is estimated as the ratio of true variance over the sum of the true variance and error variance. The value is constrained to fall between 0 and 1, with values closer to 1 indicating stronger reliability. In most instances, a reliability index of 0.70 or greater is regarded as acceptable for drawing conclusions about groups, although a value of 0.80 or higher is desirable. In the present study, GMCS reliability was assessed by fitting an intercept-only generalized linear mixed model (GLMM) to the composite score data. The GLMM variance components were then used to estimate the intraclass correlation coefficient (ICC), a variant of the Pearson correlation coefficient that is widely used to assess the reliability of group- and cluster-based measurements.
      • Adams J.L.
      The reliability of provider profiling: a tutorial.

      Validity Testing at Composite Score Level

      Composite measures validity testing seeks to confirm that the composite measure score is highly associated with outcomes that have been supported by the evidence (30-day readmissions and LOS). The overall composite measure was first tested for construct validity at the score level by constructing a hierarchical linear regression model. The hierarchical linear regression model was conducted to demonstrate that the predictability of the model significantly improved when the components in aggregate were included into the model over standard predictors of these outcomes such as patient characteristics and primary diagnoses. A stepwise approach was taken to measure the explanatory power of the malnutrition-associated measure components (ability of the components to explain the outcome). Initially, the hospital 30-day readmissions and LOS models were estimated using only the demographic and clinical variables. Later, the models were re-estimated, including the malnutrition variables. This approach allowed the measure development team to estimate the incremental improvement in goodness-of-fit from including the malnutrition variables. Model goodness-of-fit was reported as adjusted-R2 for the hospital LOS model and the concordance statistic (c-statistic) for the 30-day readmissions model.
      A secondary analysis was conducted to specifically assess the association between the main clinical end point of the composite measure (nutrition care plans for patients with a diagnosis of malnutrition) and the outcomes most associated with malnutrition (30-day readmissions and LOS). The analysis intended to understand the association of having a nutrition care plan with a malnutrition diagnosis vs not having a nutrition care plan.

      Validity Testing for Component Measures—Critical Data Elements

      Validity was also studied for the component measures by developing a generalized linear (logistic) regression model where medical diagnosis of malnutrition was the response variable and screening (completion and result) and assessment variables (completion, timing, and result) were the predictor variables. An additional test was conducted to ensure the overall linear model for predicting diagnosis was also predictive of the nutrition care plan. The hypothesis for this test is that all predictor variables would be correlated to the outcome of malnutrition diagnosis and together would be a strong predictor of the malnutrition outcome, supporting the validity of including these components in the malnutrition composite.
      In addition to testing the components of the measure for validity toward the outcome of the composite measure, testing was completed to assess the correlation between the components and outcome of the composite measure with the clinical outcomes of patient LOS and 30-day readmissions. This phase of testing assessed the predictive relationship between the set of measure components and LOS and readmissions, adjusting for differences in patient characteristics. A generalized linear mixed model approach was utilized to conduct the analyses.

      Testing Composite Measure Denominator Exclusions

      The two main exclusions for this measure are a LOS <24 hours because those patients are not in the hospital long enough to receive proper assessment, and intervention care plan for malnutrition. Patients who are transferred or discharged to hospice

      Schwartz DB, Posthauer ME, Maillet JO. Advancing nutrition and dietetics practice: dealing with ethical issues of nutrition and hydration. J Acad Nutr Diet. September 25, 2020 [Epub ahead of print]. https://doi.org/10.1016/j.jand.2020.07.028.

      ,
      • Dorner B.
      • Friedrich E.K.
      Position of the Academy of Nutrition and Dietetics: Individualized nutrition approaches for older adults: long-term care, post-acute care, and other settings.
      have significantly different requirements for nutrition support
      American Society for Parenteral and Enteral Nutrition (ASPEN) definition of terms, style, and conventions used in ASPEN board of directors–approved documents.
      ,
      Academy-Avalere Health-Defeat Malnutrition Today dialogue proceedings/advancing patient-centered malnutrition care transitions March 2018.
      and those treatment plans are highly dependent on patient preferences.
      The measure development team tested measure exclusion criteria for both influence on the measure performance score and validity statistics for the individual malnutrition eCQMs when they were first developed.
      • Nepple K.G.
      • Tobert C.M.
      • Valladares A.F.
      • Mitchell K.
      • Yadrick M.
      Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.
      The measure development team tested the measure specifications with a set of hypothetical measure exclusions that were determined by consensus agreement of the Technical Expert Panel (consisting of a group of clinical and technical experts whose guidance was sought by the measure development team) but were not explicitly identified in the evidence review. The measure performance score of each respective testing site was assessed with and without the exclusion criteria to determine the exclusion criteria’s influence on the facility’s score.

      Composite Measure Evaluation Results

      Reliability Testing Results

      Component measures were calculated using data provided by 10 health systems composed of 47 affiliated hospitals. To be included in the reliability testing, a reporting entity was required to have sufficient data collected on a minimum of 20 encounters for at least three of four GMCS component measures. Moreover, patients were required to be eligible for each of the four individual component measures, thereby ensuring the completeness of data. After removing low-frequency entities (N = 7), the GLMM was fitted to the data, and the ICC was then estimated, as described previously, using the model variance components. The resulting ICC value was 0.839, indicating that GMCS reliability falls within the acceptable range
      • Mehta S.
      • Bastero-Caballero R.F.
      • Sun Y.
      • et al.
      Performance of intraclass correlation coefficient (ICC) as a reliability index under various distributions in scale reliability studies.
      and thus, can be regarded as suitable for differentiating provider groups along the measurement continuum.

      Validity Testing Results

      The composite measure validity testing revealed that the malnutrition indicators are significantly related to LOS and readmissions after controlling for the other variables that were included in the model (ie, patient demographics, primary diagnosis, and comorbidities) known to be predictive of those outcomes. The R2 statistic for the LOS model was found to be 0.063 before the inclusion of the four aggregate measure components and 0.288 after (P < 0.001), and the c-statistic for the 30-day readmissions model was 0.614 before their inclusion and 0.625 after (P < 0.01).
      However, to better characterize the predictability of current malnutrition outcomes model for LOS and readmissions, the predictability was compared for the Hierarchical Condition Category risk-adjustment model implemented by CMS. The Hierarchical Condition Category model predicts total annual health care costs and demonstrates the predictive ability for individuals of prospective diagnosis-based models had R2 values ranging from 0.0186 to 0.1246.
      • Pope G.C.
      • Kautter J.
      • Newhart C.
      • et al.
      Evaluation of the CMS-HCC risk adjustment model.
      Given the statistics shared above, the strength of predictability of this model and overall measure is adequate and comparable to those already being implemented by CMS for similar purposes.
      The secondary analysis of the relationship between a documented nutrition care plan and risk of 30-day readmissions in patients with a malnutrition diagnosis showed a statistically significant relative risk reduction of 24% (21.4% vs 26.5%, respectively) in the likelihood of 30-day readmissions (odds ratio = 0.74, 99% CI 0.558 to 0.941). For LOS, hospitalized patients with a malnutrition diagnosis who had a nutrition care plan had, on average, a 3-day longer LOS than malnourished patients without a nutrition care plan (LOS of 9.46 vs 6.46 days, respectively; P = 0.0001). Consequently, it was identified that 30-day readmissions risk was inversely associated (care plans suggest a protective effect for malnourished patients), LOS was not. This is because, although patients who were diagnosed with malnutrition, a significant portion never saw received an RDN assessment and therefore did not have a nutrition care plan documented. These patients often were discharged earlier than those who did receive an RDN assessment and the proper nutrition treatment plan. An item-level analysis demonstrated, a patient with a nutrition care plan has a LOS than a patient who does not have a care plan regardless of malnutrition status. More studies need to be conducted that thoroughly examine this phenomenon surrounding malnourished patients, nutrition care plans, and LOS in the data that continue to be reported by hospitals around the country.
      As reported in the results of both analyses, the composite measure results are strongly correlated to important clinical outcomes associated with malnutrition in the literature, 30-day readmissions, and LOS. Furthermore, the secondary analysis demonstrated that nutrition care plans may be associated with a reduced risk of 30-day readmission for those with malnutrition vs those who are diagnosed with malnutrition but do not have a nutrition care plan.

      Validity Testing for Component Measures—Critical Data Elements

      As outlined in Table 1, all main effects and 2-way interactions were highly significant (all p-values <.0001), consistent with our hypotheses. The c-statistic of 0.828 indicates an excellent fit of the model to the malnutrition diagnosis and nutrition care plan. A c-statistic above 0.8 normally indicates a very strong predictive model. In addition to the results in Table 1 results of analyzing the composite measure components demonstrated that all components of the malnutrition composite measure, including the outcome of the malnutrition composite measure (malnutrition diagnosis and nutrition care plan) were significantly predictive of the outcome of LOS (P < 0.0001) and 30-day readmissions (P < 0.0001).
      • Valladares A.F.
      • Kilgore K.M.
      • Partridge J.
      • Sulo S.
      • Kerr K.W.
      • McCauley S.
      How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.
      Table 1Results of generalized linear regression model on composite outcome
      EffectdfWald χ2P value
      Screening result275.1< .0001
      Time to assessment21094.5< .0001
      Assessment result22006.8< .0001
      screening result × time to assessment4480.9< .0001
      screening result × assessment result4609.0< .0001
      EffectFisher exact test
      Malnutrition diagnosis × nutrition care plan17584.5< .0001
      c-statistic
      Fit of the overall score.
      0.828
      a Fit of the overall score.

      Composite Measure Denominator Exclusions Testing Results

      In the original measure testing of the individual components, it was identified that neither of the exclusion criteria had significant influence on the performance scores. Because measures were constructed with and without exclusions, no P values reached significance when a two-tailed t test was performed on the difference between the performance scores (Table 2).
      Table 2Composite measure denominator exclusions t test results
      Component measure namet test P value
      Malnutrition Screening> 0.3
      Nutrition Assessment> 0.4
      Malnutrition Diagnosis> 0.8
      Nutrition Care Plan> 0.3

      Recommendations for Composite Measure Use

      Composite Performance Score Interpretation

      The results of the validity testing at both the component and overall composite level support the inclusion of each of the component measures into the composite. Each component is independently associated with the overall quality construct and is predictive of outcomes of interest. In aggregate, the components together are better predictors of important patient outcomes of care than just patient characteristics alone. Therefore, the unweighted statistical average of the four components which results in the overall performance score for each reporting entity represents a quality construct that can be interpreted as representing a comprehensive assessment of quality of care for this population.
      To facilitate a simpler way to compare providers on an equal footing, a three-tier scoring scale was established. Participating hospitals were grouped into three tiers that reflect organizations with composite measure performance scores above, overlapping, or below the 95% estimate generated in a bootstrap analysis described below. If a hospital’s composite score was assigned a Tier 3 score, it was above the estimated confidence interval (CI) and implies that the specific hospital’s performance was above the average of the estimate developed from the aggregate of all reporting sites. A hospital receiving a Tier 2 score means their performance was not meaningfully different than the estimated mean. Finally, a hospital receiving a Tier 1 score implies that their composite performance score fell below the mean estimate interval reflective of lower-than-expected performance.
      To construct this tiering system, a bootstrap resampling methodology was employed to generate a 95% CI around the composite score mean. The 95% CI is then used to group providers into performance categories (ie, high, medium, or low). Among hospitals that meet the case minimum of 20 patients and at least three reportable measures, 44.7% of hospitals were in the highest performing Tier 3, 14.9% were in Tier 2, and 40.4% were in Tier 1. This tiering approach informed by the bootstrap sample derived from the observed performance measures was used to appropriately distinguish sites with varying degrees of performance among the component measures. These differences ultimately translated to variation in performance on the overall composite measure. The sample of sites used in the development of this composite measure is relatively homogeneous because the participating hospitals have been targeting improvement on these quality measure constructs for 1 to 3 years.

      Data Capture and Reporting

      As outlined in consensus guidance from the Academy and the American Society for Parenteral and Enteral Nutrition, it is recommended that hospital care teams properly map out their existing workflows to best identify the existence and structure of the necessary data elements for reporting this malnutrition composite measure.
      • Kight C.E.
      • Bouche J.M.
      • Curry A.
      • et al.
      Consensus recommendations for optimizing electronic health records for nutrition care.
      One hospital group reported their experience and lessons learned with initial implementation of the individual component measures that served as a helpful guide for measure implementers to reference.
      • Doley J.
      • Phillips W.
      • Talaber J.
      • Leger-LeBlanc G.
      Early implementation of malnutrition clinical quality metrics to identify institutional performance improvement needs.
      Furthermore, the pilot study site that originally tested the individual malnutrition eCQMs informed the development of GMCS reported that the improvements to the electronic health record workflows would best be pursued in a continuous quality improvement process.
      • Nepple K.G.
      • Tobert C.M.
      • Valladares A.F.
      • Mitchell K.
      • Yadrick M.
      Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.
      This would allow hospitals the time necessary to successfully modify existing workflows, pilot test the changes, and gain clinician acceptance and compliance with workflow adjustments.
      • Nepple K.G.
      • Tobert C.M.
      • Valladares A.F.
      • Mitchell K.
      • Yadrick M.
      Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.
      Furthermore, other expert consensus groups that have focused on composite measure development and use have recommended that any composite measure that is developed should be capable of deconstruction to facilitate performance improvement.
      • Peterson E.D.
      • DeLong E.R.
      • Masoudi F.A.
      • et al.
      ACCF/AHA 2010 position statement on composite measures for healthcare performance assessment: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to Develop a Position Statement on Composite Measures).
      More specifically, apart from reporting the overall composite score, it is important that providers can also see the performance on individual component measures to allow for targeted quality improvement efforts that reflect the areas of improvement unique to the reporting entity.

      Composite Measure Limitations

      Of note, a potential challenge or limitation of composite measures is that without a transparent and understandable methodology, they may be limited in providing actionable information to stakeholders on a clinician’s performance with respect to individual components. Moreover, the variance in performance within specific measures may make them highly vulnerable to a subset of components depending on the weighting methodology. Therefore, composite measures need a strong evidence base to inform their development, a clearly outlined methodology, and a robust testing for accuracy and usefulness for them to serve as an effective tool for measuring quality of care.

      Policy and Practice Recommendations

      Ultimately, the results of testing the GMCS indicate that it is a compelling and comprehensive tool for assessing the quality of care provided to hospitalized patients, aged 65 years and older, at risk of malnutrition. This GMCS summarizes the key steps in the malnutrition care workflow that are necessary to identify and mitigate malnutrition risk in a timely and effective manner. For hospital providers, this composite measure may very well serve as an effective tool to understand the state of malnutrition care in their institution and help identify critical areas for quality improvement. As shown in the empirical testing, the collection of these indicators of malnutrition care may also be used to further study the influences of clinical malnutrition on patient outcomes and the effect of risk mitigation on adverse outcomes. Given the established relationship between malnutrition and critical outcomes that have substantial implications for health economics, further assessment of the performance improvement on this measure should consider influence on health care costs.
      In the existing Inpatient Quality Reporting Program implemented by CMS, although there is considerable focus on some of the sequelae of malnutrition such as pressure ulcers, infections, 30-day readmissions, and mortality, no measure considering the role of nutrition has ever been included in this program. Given the proliferation of the GMCS component measures to dozens of hospitals around the country through the national hospital learning collaborative as part of the MQii and the rigorous empirical evaluation standards met by the measure development team, policymakers should consider the GMCS for provider pay-for-reporting and pay-for-performance programs like the Inpatient Quality Reporting Program. The inclusion of the GMCS could provide valuable information to providers, consumers, and federal stakeholders on nationwide performance on standards of nutrition care practice that have considerable implications for clinical and economic outcomes.

      Acknowledgements

      The authors thank Kathryn K. Hamilton, MA, RDN, CSO, CDN, FAND, Linda Kluge, RDN, CPHQ, Susan Roberts, DCN, MS, RDN, LD, CNSC, FAND, Dana Buelsing, MS, and Carol J. Gilmore, MS, RDN, LD, FADA, FAND, for their critical review of the manuscript. The Malnutrition Quality Improvement Initiative (MQii) is a project of the Academy of Nutrition and Dietetics, Avalere Health, and other stakeholders who participated in and provided guidance and expertise in this collaborative partnership. C. D’Andrea, M. Kahn, and S. M. McCauley are employees of the Academy of Nutrition and Dietetics. K. Kilgore, K. Mitchell, and A.F. Valladares are employees of Avalere Health. The Academy of Nutrition and Dietetics does not receive funding for the MQii. Avalere Health’s work to support the MQii and drafting of this manuscript was funded by Abbott.

      Author Contributions

      A. F. Valladares, K. Mitchell, and S. M. McCauley provided technical assistance on the Global Malnutrition Composite Score. S. M. McCauley and M. Khan provided clinical guidance on the development of the project. K. Kilgore and A. F. Valladares collected and analyzed the data. A. F. Valladares, S. M. McCauley, M. Khan, and C. D’Andrea developed the outline and the first draft for the manuscript. All authors reviewed and commented on subsequent drafts of the manuscript.

      References

        • Marjoua Y.
        • Bozic K.J.
        Brief history of quality movement in US healthcare.
        Curr Rev Musculoskelet Med. 2012; 5: 265-273
        • Barrett M.L.
        • Bailey M.K.
        • Owens P.L.
        Non-maternal and non-neonatal inpatient stays in the United States involving malnutrition, 2016.
        • Pereira G.F.
        • Bulik C.M.
        • Weaver M.A.
        • Holland W.C.
        • Platts-Mills T.F.
        Malnutrition among cognitively intact, noncritically ill older adults in the emergency department.
        Ann Emerg Med. 2015; 65: 85-91
        • Barker L.A.
        • Gout B.S.
        • Crowe T.C.
        Hospital malnutrition: prevalence, identification and impact on patients and the healthcare system.
        Int J Environ Res Public Health. 2011; 8: 514-527
      1. Academy of Nutrition and Dietetics Malnutrition Quality Improvement Initiative.
        • McCauley S.M.
        • Barrocas A.
        • Malone A.
        Hospital nutrition care betters patient clinical outcomes and reduces costs: the Malnutrition Quality Improvement Initiative story.
        J Acad Nutr Diet. 2019; 119: S11-S14
        • Silver H.J.
        • Pratt K.J.
        • Bruno M.
        • Lynch J.
        • Mitchell K.
        • McCauley S.M.
        Effectiveness of the Malnutrition Quality Improvement Initiative on practitioners malnutrition knowledge and screening, diagnosis, and timeliness of malnutrition-related care provided to older adults admitted to a tertiary care facility: a pilot study.
        J Acad Nutr Diet. 2018; 118: 101-109
        • Nepple K.G.
        • Tobert C.M.
        • Valladares A.F.
        • Mitchell K.
        • Yadrick M.
        Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.
        J Acad Nutr Diet. 2019; 119: S32-S39
        • National Quality Forum
        Measure evaluation criteria and guidance for evaluating measures for endorsement.
        • Valladares A.F.
        • Kilgore K.M.
        • Partridge J.
        • Sulo S.
        • Kerr K.W.
        • McCauley S.
        How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.
        JPEN J Parenter Enteral Nutr. 2021; 45: 366-371
      2. CMS Measures Management System Blueprint v14.1.
        • Academy of Nutrition and Dietetics
        Electronic clinical quality measures: Global Malnutrition Composite Score.
        • Indyk J.A.
        • Buckingham D.
        • Obrynba K.S.
        • Servick C.
        • Gandhi K.K.
        • Kramer A.
        • et al.
        The Type 1 Diabetes Composite Score: an innovative metric for measuring patient care outcomes beyond hemoglobin A1c.
        Pediatr Qual Saf. 2020; 5: e354https://doi.org/10.1097/pq9.0000000000000354
        • National Quality Forum
        Composite performance measure evaluation guidance.
        • Adams J.L.
        The reliability of provider profiling: a tutorial.
      3. Schwartz DB, Posthauer ME, Maillet JO. Advancing nutrition and dietetics practice: dealing with ethical issues of nutrition and hydration. J Acad Nutr Diet. September 25, 2020 [Epub ahead of print]. https://doi.org/10.1016/j.jand.2020.07.028.

        • Dorner B.
        • Friedrich E.K.
        Position of the Academy of Nutrition and Dietetics: Individualized nutrition approaches for older adults: long-term care, post-acute care, and other settings.
        J Acad Nutr Diet. 2018; 118: 724-735
      4. American Society for Parenteral and Enteral Nutrition (ASPEN) definition of terms, style, and conventions used in ASPEN board of directors–approved documents.
      5. Academy-Avalere Health-Defeat Malnutrition Today dialogue proceedings/advancing patient-centered malnutrition care transitions March 2018.
        • Mehta S.
        • Bastero-Caballero R.F.
        • Sun Y.
        • et al.
        Performance of intraclass correlation coefficient (ICC) as a reliability index under various distributions in scale reliability studies.
        Stat Med. 2018; 37: 2734-2752
        • Pope G.C.
        • Kautter J.
        • Newhart C.
        • et al.
        Evaluation of the CMS-HCC risk adjustment model.
        • Kight C.E.
        • Bouche J.M.
        • Curry A.
        • et al.
        Consensus recommendations for optimizing electronic health records for nutrition care.
        J Acad Nutr Diet. 2020; 120: 1227-1237
        • Doley J.
        • Phillips W.
        • Talaber J.
        • Leger-LeBlanc G.
        Early implementation of malnutrition clinical quality metrics to identify institutional performance improvement needs.
        J Acad Nutr Diet. 2019; 119: 547-552
        • Peterson E.D.
        • DeLong E.R.
        • Masoudi F.A.
        • et al.
        ACCF/AHA 2010 position statement on composite measures for healthcare performance assessment: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to Develop a Position Statement on Composite Measures).
        J Am Coll Cardiol. 2010; 55: 1755-1766

      Linked Article

      • Development and Evaluation of a Global Malnutrition Composite Score
        Journal of the Academy of Nutrition and DieteticsVol. 122Issue 2
        • Preview
          Quality measures proliferated in the late 1990s and early 2000s and were first tied to financial performance incentives with the establishment of quality reporting programs for hospitals and then physicians.1 Quality measurement has since expanded to virtually all provider areas of health care in the United States. Despite this growth, one area where a major deficit persists has been nutrition care. This article outlines the process pursued by the Academy of Nutrition and Dietetics (Academy) and Avalere Health (Avalere) to develop the first of its kind electronically specified composite measure addressing malnutrition care for hospitalized adults.
        • Full-Text
        • PDF
        Open Access