Advertisement

Validity and Educational Impact of Clinical Entrustable Professional Activities Linked to the Nutrition Care Process for Work-Based Assessment of Entry-Level Dietetics Students: Evaluation of a 3-Year Implementation Study in Australia

Published:September 08, 2022DOI:https://doi.org/10.1016/j.jand.2022.09.009

      Abstract

      Background

      Entrustable professional activities (EPAs) describe key workplace activities and are increasingly being used for student work-based assessment. An EPA-based work-based assessment tool offers potential to increase understanding of dietetics student skill development and opportunity for standardised work-based student assessment.

      Objective

      To determine construct validity and educational impact of an EPA work-based assessment tool for dietetics placement students in clinical settings.

      Design

      Using a time series design, supervisor and student self-assessment data collected from an EPA-based assessment tool from three cohorts of dietetics students and supervisors from 2017-2019 was analyzed.

      Participants and setting

      Dietetics students (n = 145) from an accredited dietetic training program in Australia and affiliated metropolitan and rural hospitals.

      Main outcome measures

      Construct validity was determined through analysis of supervisor evaluation of student performance against EPAs over time. Educational impact was determined through comparing student performance across EPAs to predicted milestones to identify areas of least entrustment.

      Statistical analyses performed

      Student performance over time and differences between student self-assessment and supervisor assessment for each EPA were investigated using Wilcoxon signed-rank tests and linear mixed-model analysis. Descriptive statistics summarised student performance against each EPA.

      Results

      Performance significantly increased over time in 35 out of 37 EPAs. Significant differences between supervisor and student self-assessment were evident in 9 out of 37 EPAs. Dietetics student performance varied across EPAs with 88.2% of students achieving entrustment for nutrition management EPAs compared with 100% for professionalism EPAs.

      Conclusions

      The tool’s construct validity was established. EPAs identified areas of Nutrition Care Process skills development requiring additional support. Data collected by the tool can inform teaching.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of the Academy of Nutrition and Dietetics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Ten Cate O.
        • Tobin S.
        • Stokes M.-L.
        Bringing competencies closer to day-to-day clinical work through entrustable professional activities.
        Med J. Aust. 2017; 206: 14-16
        • Meyer E.G.
        • Chen H.C.
        • Uijtdehaage S.
        • Durning S.J.
        • Maggio L.A.
        Scoping review of entrustable professional activities in undergraduate medical education.
        Acad Med. 2019; 94: 1040-1049
        • Lau S.T.
        • Ang E.
        • Samarasekera D.D.
        • Shorey S.
        Development of undergraduate nursing entrustable professional activities to enhance clinical care and practice.
        Nurse Educ Today. 2020; 87104347
        • Bramley A.L.
        • Thomas C.J.
        • Mc Kenna L.
        • Itsiopoulos C.
        E-portfolios and Entrustable Professional Activities to support competency-based education in dietetics.
        Nurs Health Sci. 2021; 23: 148-156
        • Haines S.T.
        • Pittenger A.L.
        • Stolte S.K.
        • et al.
        Core entrustable professional activities for new pharmacy graduates.
        Am J Pharm Educ. 2017; 81: S2
      1. Carraccio CL, Benson BJ, Nixon LJ, Derstine PL. From the educational bench to the clinical bedside: translating the Dreyfus Developmental Model to the learning of clinical skills. 2008;83(8):761-767.

        • Ten Cate O.
        Competency-based education, entrustable professional activities, and the power of language.
        J Grad Med Educ. 2013; 5: 6-7
        • Dhaliwal U.
        • Gupta P.
        • Singh T.
        Entrustable professional activities: teaching and assessing clinical competence.
        Indian Pediatr. 2015; 52: 591-597
        • Chen H.C.
        • van den Broek W.E.
        • Ten Cate O.
        The case for use of entrustable professional activities in undergraduate medical education.
        Acad Med. 2015; 90: 431-436
        • Bramley A.L.
        • McKenna L.
        Entrustable professional activities in entry-level health professional education: a scoping review.
        Med Educ. 2021; 55: 1011-1032https://doi.org/10.1111/medu.14539
        • Tekian A.
        • Hodges B.D.
        • Roberts T.E.
        • Schuwirth L.
        • Norcini J.
        Assessing competencies using milestones along the way.
        Med Teach. 2015; 37: 399-402
        • Pangaro L.
        • Ten Cate O.
        Frameworks for learner assessment in medicine: AMEE Guide No. 78.
        Med Teach. 2013; 35: e1197-e1210
        • Tekian A.
        • Ten Cate O.
        • Holmboe E.
        • Roberts T.
        • Norcini J.
        Entrustment decisions: implications for curriculum development and assessment.
        Med Teach. 2020; 42: 698-704
        • Bramley A.
        • McKenna L.
        Entrustable professional activities: focus on assessment methods.
        in: Nestel D. Reedy G. McKenna L. Gough S. Clinical Education for the Health Professions: Theory and Practice. Springer Singapore, 2020: 1-13
        • Dietitians Australia
        Accrediation standards for dieteitcs eduation programs.
        2017
        • British Dietetic Association
        Practice based learning.
        2021
        • Academy of Nutrition and Dietetics, Accrediation Council for Education in Nutrition and Dietetics
        ACEND accreditation standards for nutrition and dietetics co-ordinated programs.
        2016
        • Ash S.
        • Palermo C.
        • Gallegos D.
        The contested space: the impact of competency-based education and accreditation on dietetic practice in Australia.
        Nutr Diet. 2019; 76: 38-46
        • Palermo C.
        • Dart J.
        • Begley A.
        • et al.
        Dietetics students’ construction of competence through assessment and placement experiences.
        Nutr Diet. 2018; 75: 307-315
        • Jamieson J.
        • Palermo C.
        • Hay M.
        • Gibson S.
        Assessment practices for dietetics trainees: a systematic review.
        J Acad Nutr Diet. 2019; 119: 272-292.e223
        • Rhodes L.A.
        • Marciniak M.W.
        • McLaughlin J.
        • Melendez C.R.
        • Leadon K.I.
        • Pinelli N.R.
        Exploratory analysis of entrustable professional activities as a performance measure during early pharmacy practice experiences.
        Am J Pharm Educ. 2019; 83: 6517
        • Englander R.
        • Flynn T.
        • Call S.
        • et al.
        Toward defining the foundation of the MD degree: core entrustable professional activities for entering residency.
        Acad Med. 2016; 91: 1352-1358
        • Van Der Vleuten C.P.M.
        The assessment of professional competence: developments, research and practical implications.
        Acad Med. 1996; 1: 41-67
        • Van Der Vleuten C.P.M.
        • Schuwirth L.W.T.
        Assessing professional competence: from methods to programmes.
        Med Educ. 2005; 39: 309-317
        • Bramely A.
        • Forsyth A.
        • McKenna L.
        Design, implementation and evaluation of novel work-based clinical assessment tool: an e-portfolio with embedded Entrustable Professional Activities.
        Nurse Educ Today. 2021; 10715101https://doi.org/10.1016/j.nedt.2021.105101
        • Bramely A.
        • Forsyth A.
        • McKenna L.
        Development and evaluation of Entrustable Professional Activities embedded in an e-portfolio for work-based assessment in community and public health dietetics. Health & Social Care in the Community.
        (Published online August 8)2022https://doi.org/10.1111/hsc.13967
        • Whittaker R.
        • Merry S.
        • Dorey E.
        • Maddison R.
        A development and evaluation process for mHealth interventions: examples from New Zealand.
        J Health Commun. 2012; 17: 11-21
        • Mulder H.
        • Ten Cate O.
        • Daalder R.
        • Berkvens J.
        Building a competency-based workplace curriculum around entrustable professional activities: the case of physician assistant training.
        Med Teach. 2010; 32: e453-e459
        • Chang A.
        • Bowen J.L.
        • Buranosky R.A.
        • et al.
        Transforming primary care training—patient-centered medical home entrustable professional activities for internal medicine residents.
        J Gen Intern Med. 2013; 28: 801-809
        • Hauer K.E.
        • Soni K.
        • Cornett P.
        • et al.
        Developing entrustable professional activities as the basis for assessment of competence in an internal medicine residency: a feasibility study.
        J Gen Intern Med. 2013; 28: 1110-1114
        • Lacey K.
        • Pritchett E.
        Nutrition Care Process and Model: ADA adopts road map to quality care and outcomes management.
        J Acad Nutr Diet. 2003; 103: 1061-1072
        • Porter J.M.
        • Devine A.
        • O'Sullivan T.A.
        Evaluation of a Nutrition Care Process implementation package in hospital dietetic departments: NCP implementation evaluation.
        Nutr Diet. 2015; 72: 213-221
      2. PepplePad Software. Version 5. E-Innovation Center. University of Wolverhapmton, 2017
        • Ten Cate O.
        Nuts and bolts of entrustable professional activities.
        J Grad Med Educ. 2013; 5: 157-158
        • Valentine N.
        • Wignes J.
        • Benson J.
        • Clota S.
        • Schuwirth L.W.
        Entrustable professional activities for workplace assessment of general practice trainees.
        Med J Aust. 2019; 210: 354-359
      3. SPSS. Version 26. IBM-SPSS, Inc; 2019.

      4. Version 0.14.1. University of Amsterdam, 2021
        • Peters H.
        • Holzhausen Y.
        • Maaz A.
        • Driessen E.
        • Czeskleba A.
        Introducing an assessment tool based on a full set of end-of-training EPAs to capture the workplace performance of final-year medical students.
        BMC Medical Educ. 2019; 19: 207
        • von Elm E.
        • Altman D.G.
        • Egger M.
        • Pocock S.J.
        • Gøtzsche P.C.
        • Vandenbroucke J.P.
        The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.
        J Clin Epidemiol. 2008; 61: 344-349
        • Shorey S.
        • Lau T.C.
        • Lau S.T.
        • Ang E.
        Entrustable professional activities in health care education: a scoping review.
        Med Educ. 2019; 53: 766-777
        • Dart J.
        • McCall L.
        • Ash S.
        • Blair M.
        • Twohig C.
        • Palermo C.
        Toward a global definition of professionalism for nutrition and dietetics education: a systematic review of the literature.
        J Acad Nutr Diet. 2019; 119: 957-971
        • Karupaiah T.
        • Reinhard T.
        • Krishnasamy S.
        • Tan S.-P.
        • Se C.-H.
        Incorporating the Nutrition Care Process model into dietetics internship evaluation: a Malaysian university experience.
        Nutr Diet. 2016; 73: 283-295
        • Holzhausen Y.
        • Maaz A.
        • Cianciolo A.T.
        • Ten Cate O.
        • Peters H.
        Applying occupational and organizational psychology theory to entrustment decision-making about trainees in health care: aconceptual model.
        Perspect Med Educ. 2017; 6: 119-126
        • Ten Cate O.
        • Scheele F.
        Competency-based postgraduate training: can we bridge the gap between theory and clinical practice?.
        Acad Med. 2007; 82: 542-547
        • Davis D.A.
        • Mazmanian P.E.
        • Fordis M.
        • Van Harrison R.
        • Thorpe K.E.
        • Perrier L.
        Accuracy of physician self-assessment compared with observed measures of competence: a systematic review.
        JAMA. 2006; 296: 1094-1102
        • Violato C.
        • Lockyer J.
        Self and peer assessment of pediatricians, psychiatrists and medicine specialists: implications for self-directed learning.
        Adv Health Sci Educ Theory Pract. 2006; 11: 235-244
        • Eva K.W.
        • Regehr G.
        Self-assessment in the health professions: a reformulation and research agenda.
        Acad Med. 2005; 80: S46-S54
        • Hays R.B.
        • Jolly B.C.
        • Caldon L.J.M.
        • et al.
        Is insight important? Measuring capacity to change performance.
        Med Educ. 2002; 36: 965-971
        • Wijnen-Meijer M.
        • Ten Cate O.
        • van der Schaaf M.
        • Burgers C.
        • Borleffs J.
        • Harendza S.
        Vertically integrated medical education and the readiness for practice of graduates.
        BMC Med Educ. 2015; 15: 229
        • Rekman J.
        • Gofton W.
        • Dudek N.
        • Gofton T.
        • Hamstra S.J.
        Entrustability scales: outlining their usefulness for competency-based clinical assessment.
        Acad Med. 2016; 91: 186-190
        • Ten Cate O.
        • Taylor D.R.
        The recommended description of an entrustable professional activity: AMEE Guide No. 140.
        Med Teach. 2020; : 1-9

      Biography

      A. Bramley is Allied Health Education Lead, Monash Health, Clayton, Vic, Australia, and an adjunct senior lecturer, Dietetics and Human Nutrition, School of Allied Health, Human Services, and Sport, La Trobe University, Bundoora, Victoria, Australia.

      Biography

      A. Forsyth is an Associate Professor, Dietetics and Nutrition, Australian Catholic University, City, State, Australia, and an adjunct associate professor, Dietetics and Human Nutrition, School of Allied Health, Human Services, and Sport, La Trobe University, Bundoora, Victoria, Australia.

      Biography

      L. McKenna is Dean and Head, School of Nursing and Midwifery, College of Science, Health, and Engineering, La Trobe University, Bundoora, Victoria, Australia.