Advertisement

Insights from Washington State’s COVID-19 Response: A Mixed-Methods Evaluation of WIC Remote Services and Expanded Food Options Using the RE-AIM Framework

Published:March 23, 2022DOI:https://doi.org/10.1016/j.jand.2022.03.013

      Abstract

      Background

      In response to the COVID-19 pandemic, Washington State’s Supplemental Nutrition Program for Women, Infants, and Children (WA WIC) adopted federal waivers to transition to remote service delivery for certification and education appointments. WA WIC also expanded the approved food list without using federal waivers, adding more than 600 new items to offset challenges participants experienced accessing foods in stores.

      Objective

      This study aimed to assess the reach and effectiveness of the programmatic changes instituted by WA WIC during the COVID-19 pandemic; the processes, facilitators, and challenges involved in their implementation; and considerations for their continuation in the future.

      Design

      A mixed-methods design, guided by the RE-AIM framework, including virtual, semi-structured focus groups and interviews with WA WIC staff and participants, and quantitative programmatic data from WIC agencies across the state.

      Participants/setting

      This study included data from 52 state and local WIC staff and 40 WIC participants across the state of Washington and from various WA WIC programmatic records (2017-2021). The research team collected data and conducted analyses between January 2021 and August 2021.

      Analysis

      An inductive thematic analysis approach with Dedoose software was used to code qualitative data, generate themes, and interpret qualitative data. Descriptive statistics were calculated for quantitative programmatic data, including total participant count, percent increase and decrease in participation, percent of food benefits redeemed monthly, and appointment completion rates.

      Results

      All WA WIC participants (n = 125,279 in May 2020) experienced the programmatic changes. Participation increased by 2% from March to December 2020 after WA WIC adopted programmatic changes in response to the COVID-19 pandemic. Certification and nutrition education completion rates increased by 5% and 18% in a comparison of June 2019 with June 2020. Food benefit redemption also increased immediately after the food list was expanded in April 2020. Staff and participants were highly satisfied with remote service delivery, predominantly via the phone, and participants appreciated the expanded food options. Staff and participants want a remote service option to continue and suggested various changes to improve service quality.

      Conclusions

      Participation in WIC and appointment completion rates increased after WA WIC implemented service changes in response to the COVID-19 pandemic. Staff and participants were highly satisfied with remote services, and both desire a continued hybrid model of remote and in-person WIC appointments. Some of the suggested changes to WIC, especially the continuation of remote services, would require federal policy change, and others could be implemented under existing federal regulations.

      Keywords

      Research Questions: What did the transition to remote services and expansion of the food list during the COVID-19 pandemic look like for the Washington State Supplemental Nutrition Program for Women, Infants and Children (WA WIC), and how did the changes impact programmatic reach and effectiveness? In addition, do WA WIC staff and participants want these programmatic changes to continue in the future and, if so, how?
      Key Findings: Compared with December 2019 (prior to the COVID-19 Pandemic), overall WA WIC program participation was increased in December 2020. In addition, appointment completion rates were increased in June 2020 compared with June 2019. Both staff and participants appreciate the convenience of remote appointments and would like to see a hybrid model combining both in-person and remote appointments continue beyond the COVID-19 pandemic.
      The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) is a federal program that provides nutritious foods, health screening, nutrition education, breastfeeding support, and referrals to approximately 6.2 million low-income and nutritionally vulnerable women, infants, and children across the United States, including nearly one-half of all infants and 25% of young children.
      The State of WIC
      WIC Research, Policy and Practice Hub.
      Forty-two percent of WIC participants are Black, Indigenous, and People of Color
      WIC racial-ethnic group enrollment data 2016
      US Department of Agriculture, Federal Nutrition Service.
      and the percent of eligible families participating in WIC is higher in rural than urban communities.
      Rural hunger in America: Special Supplemental Nutrition Program for Women, Infants, and Children.
      WIC participation is associated with healthier diets and improved infant morbidity and mortality rates.
      • Carlson S.
      • Neuberger Z.
      WIC Works: Addressing the nutrition and health needs of low-income families for more than four decades. Center on Budget and Policy Priorities.
      ,
      • Soneji S.
      • Beltrán-Sánchez H.
      Association of Special Supplemental Nutrition Program for Women, Infants, and Children with preterm birth and infant mortality.
      Despite program benefits, only approximately one-half of eligible individuals enroll in WIC.

      Gray K, Trippe C, Tadler C, Perry C, Johnson P, Betson D. National-and State-Level Estimates of WIC Eligibility and WIC Program Reach in 2017, Final Report: Volume I.; 2019:86. Accessed August 6, 2021. https://fns-prod.azureedge.net/sites/default/files/resource-files/WICEligibles2017-Volume1.pdf

      The barriers to using WIC services are well-documented and include required physical presence at WIC offices for appointments, limited WIC-approved foods, and negative shopping experiences.
      • Woelfel M.L.
      • Abusabha R.
      • Pruzek R.
      • Stratton H.
      • Chen S.G.
      • Edmunds L.S.
      Barriers to the use of WIC services.
      • McKinney S.
      Improving WIC usage: Office procedures, food package, and nutrition education.
      The COVID-19 pandemic initially exacerbated these barriers, as people were encouraged or required to stay home to limit coronavirus transmission, and grocery stores experienced shortages of WIC-approved foods.
      The State of WIC
      WIC Research, Policy and Practice Hub.
      ,

      WIC Innovates to Support Maternal and Child Health During the Pandemic. Robert Wood Johnson Foundation. Published May 7, 2021. Accessed July 23, 2021. https://www.rwjf.org/en/blog/2021/05/wic-innovates-to-support-maternal-and-child-health-during-the-pandemic.html

      The outbreak also created economic conditions that increased rates and severity of food insecurity among vulnerable families.
      • Wolfson J.A.
      • Leung C.W.
      Food Insecurity and COVID-19: Disparities in early effects for US adults.
      In March 2020, Washington State WIC (WA WIC) began offering WIC certifications and other appointments remotely (together referred to as “remote services”), using waivers provided through the Families First Coronavirus Response Act.
      FNS responds to COVID-19. US Department of Agriculture, Food and Nutrition Service.
      WA WIC also expanded the list of allowable foods by more than 600 items in April 2020; WA WIC intentionally expanded the list without the use of federal waivers to ensure the expanded access to allowable foods would continue beyond the COVID-19 pandemic. These changes allowed eligible households to participate in WIC when clinics were closed to the public and improved access to WIC foods during retail shortages.
      COVID-19 pandemic–related changes to WIC services provide an opportunity to gain insight into improving WIC services over the long-term. This study aimed to assess the reach and effectiveness of the programmatic changes instituted by WA WIC during the COVID-19 pandemic; the processes, facilitators, and challenges involved in their implementation; and considerations for their continuation in the future.

      Methods

      This mixed-methods study used semi-structured interviews with WIC participants, focus groups and key informant interviews with WIC staff, and participant-related programmatic data to evaluate WA WIC’s transition to remote services and expanded food options during the COVID-19 pandemic. Study protocols were determined exempt from human subjectsreview by the University of Washington Human Subjects Division and Washington State Institutional Review Board.

      Framework

      The RE-AIM framework (see Figure 1) guided the research aims and design, providing a structure for evaluating the programmatic changes’ reach, effectiveness, adoption, implementation, and maintenance.
      • Glasgow R.E.
      • Harden S.M.
      • Gaglio B.
      • et al.
      RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review.
      Using both qualitative and quantitative data within the RE-AIM dimensions generated a thorough examination of programmatic changes
      • Rutherford Z.
      • Zwolinsky S.
      • Kime N.
      • Pringle A.
      A mixed-methods evaluation of CARE (cancer and rehabilitation exercise): A physical activity and health intervention, delivered in a community football trust.
      because qualitative data provided context for quantitative trends.
      • Rutherford Z.
      • Zwolinsky S.
      • Kime N.
      • Pringle A.
      A mixed-methods evaluation of CARE (cancer and rehabilitation exercise): A physical activity and health intervention, delivered in a community football trust.
      Figure 1Constructs of the RE-AIM
      • Glasgow R.E.
      • Harden S.M.
      • Gaglio B.
      • et al.
      RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review.
      framework, construct definitions, and data sources for the constructs used to analyze the Washington State Special Supplemental Nutrition Program for Women, Infants, and Children (WA WIC) pandemic response.
      ConstructDefinition applied in this studyData source
      ReachThe number and proportion of WA WIC participants who experienced the program adaptations
      • Programmatic participation data
      EffectivenessThe extent to which WA WIC adaptations were associated with maintained or improved:
      • WIC participation
      • Appointment completion
      • Food benefit redemption
      • Programmatic participation data
      • Appointment completion data
      • Food benefit redemption data
      • Staff focus groups
      Adoption
      • Programmatic changes adopted by WA WIC
      • Reasons for adopting these changes
      • Staff focus groups
      Implementation
      • How the adaptations were implemented
      • Supports needed to implement the adaptations
      • Facilitators and challenges to implementation
      • Satisfaction of WA WIC participants and staff including advantages and disadvantages
      • Programmatic data
      • Staff focus groups
      • Participant interviews
      Maintenance
      • Staff and participant interest in maintaining the adaptations
      • Supports needed to continue or improve the adaptations
      • Improvements to WA WIC remote services and food list
      • Staff focus groups
      • Participant interviews

      Sample and Recruitment

      WIC Staff

      Convenience sampling was used to recruit state and local WIC staff to participate in focus groups and key informant interviews. Existing contacts within WA WIC identified state-level staff involved with programmatic changes. To recruit local WIC staff, the research team selected 10 of the 57 local WIC agencies from which to recruit staff based on rurality, caseload, region, and service status (eg, if offering curbside services as of May 2020). The lead staff at the state WIC office e-mailed study information to coordinators at each local agency and asked them to share study information with other WIC staff at their agency. The final sample included 52 WIC staff, 10 who worked at the state level and 42 who worked at the local level. In addition to interest in sampling WIC staff from the state agency and a diverse array of local WIC agencies as detailed above, recruitment intentionally focused on staff roles in the program; specifically, the team sought perspectives of staff involved in program coordination, certification, nutrition education, or breastfeeding education at the local level and staff involved in developing the food list and supporting the transition to remote services at the state level. Recruitment continued until invitations of at least 1 staff member in each role were sought from each of the identified agencies. Race and ethnicity, sex or gender, and age information for WIC staff participants were not collected.

      WIC Participants

      WIC participants were recruited through a banner placed on the WICShopper app in English and Spanish for 1 day, as the WICShopper app was used by approximately 90% of Washington WIC households at the time of recruitment.

      USDA extends waivers: WIC continues to provide remote services. Washington State Department of Health. Published September 29, 2020. Accessed August 3, 2021. https://www.doh.wa.gov/Newsroom/Articles/ID/2397/USDA-extends-waivers-WIC-continues-to-provide-remote-services

      In that time, 147 WIC participants volunteered by clicking on the banner, which brought them to a webpage with the study procedures and a link to provide contact information and self-report demographic and other key characteristics, including race and ethnicity, length of time on WIC, breastfeeding status, how many (if any) infants and children enrolled in WIC, and the WIC agency they visit. No data on sex and gender or age of WIC participants were collected.
      Seventy-two survey respondents were contacted using a maximum variation approach to aim for diverse perspectives of the WA WIC participant population regarding race, ethnicity, rurality, and WIC participant enrollment category (whether they were enrolled, their infant was enrolled, or their child/children were enrolled). The study team made a particular effort to contact participant volunteers identifying as American Indian and Black because these racial groups experience the largest nutrition
      • Wiesman J.
      • Lofy K.
      Washington State Health Assessment.
      and infant and maternal health disparities in the state.

      Report to the Legislature: Washington State Maternal Mortality Review Panel: Maternal Deaths 2014-2016. Washington State Department of Health, Prevention and Community Health Division. Accessed April 1, 2022. https://doh.wa.gov/sites/default/files/legacy/Documents/Pubs//141-010-MMRPMaternalDeathReport2014-2016.pdf

      In addition, the aim was to recruit 10 WIC participants whose primary language was Spanish to ensure that the perspectives of participants who speak a language other than English were included. Spanish is the most commonly spoken language after English in Washington State, and an estimated 6.5% of households receiving cash, food, and medical benefits in the state consider Spanish their primary language.
      Participant recruitment continued until the goal of 40 total interviewees, 10 of whom spoke Spanish, was reached; the goal was based on prior similar work that indicated 40 interviews would be sufficient to reach data saturation.
      • Guest G.
      • Bunch A.
      • Johnson L.
      How many interviews are enough? An experiment with data saturation and variablity.
      Of the 72 respondents contacted, 40 completed interviews, 6 declined participation, 11 were scheduled but did not complete interviews, and 15 were unreachable after up to 4 contact attempts.

      Data Collection

      Programmatic Data

      Nonidentifiable programmatic data were provided by WA WIC to examine reach and effectiveness through temporal trends in participation and food benefit redemption. WA WIC also shared reports from local WIC agencies from May and August 2020 with data on WIC service status (eg, number of staff and number of sites offering curbside or face-to-face services), supports needed, and challenges experienced. These data were explored so emergent quantitative trends could inform questions during qualitative data collection and to help triangulate focus group and interview findings.

      Staff Focus Groups and Interviews

      Ten focus groups were organized to include staff members with similar roles within WIC as able. One focus group included state staff managing the transition to remote services (n = 5), 1 included state staff managing the food list expansion (n = 5), 2 included local program coordinators (n = 7, n = 6), 2 included breastfeeding support staff (n = 3, n = 2), 1 included nutrition educators (n = 5), 1 included certification staff (n = 4), and 2 included staff from a variety of positions (n = 7, n = 6). In addition, 2 local staff participants (program coordinator and nutrition educator) were not able to participate in focus groups due to scheduling constraints, so key informant interviews were conducted with these individuals. The study team developed the semi-structured focus group guide (Figure 2; available at www.jandonline.org) using the study aims and the RE-AIM framework; this guide was also used for the key informant interviews. The questions in the focus group guide asked staff about processes, facilitators, and challenges that accompanied the switch to remote services and the expanded food list. Staff were also asked about the feasibility and support needed to continue aspects of remote services. Two female researchers with masters- or doctoral-level training and experience conducting research in public health and nutrition conducted and recorded approximately 70-minute focus groups via the Zoom platform from January 2021 through February 2021.

      Participant Interviews

      The semi-structured interview guide for WIC participants was informed by the study aims, RE-AIM framework, and emerging themes from the staff focus groups. Questions were developed by the research team and reviewed by WA WIC. Participants were asked about their history with WIC, their most recent remote WIC appointments, their experiences with the expanded food list, their interest in continuing remote appointments, and how WIC could be improved in the future (Figure 3; available at www.jandonline.org). Interviews were conducted by the same researchers from March through April 2020 via Zoom and were recorded with interviewees’ permission. Participants were aware that researchers were affiliated with the local university and were working in partnership with the state WIC program. Interviews were conducted in English or Spanish (with a professional interpreter). Interviews with English-speaking WIC participants averaged approximately 20 minutes and interviews with Spanish-speaking WIC participants and interpreters averaged approximately 40 minutes. There were no repeat interviews with any participants. Interviewees were compensated differentially for this additional interview duration (English-speaking participants received $20 and Spanish-speaking participants received $40), but participants were not aware of the compensation differentials.

      Data Analysis

      Programmatic Data

      Programmatic data were analyzed in Microsoft Excel
      using descriptive statistics and visualized in Tableau
      to assess trends over time for total participation by women, infants, or children; percent increase and decrease in participation; percent of food benefits redeemed monthly; and appointment completion rates.

      Focus Groups and Interviews

      Focus groups and interviews were transcribed using the Zoom automated transcript function. Study team members reviewed and cleaned transcripts for accuracy, referring back to the audio recording as necessary. Transcripts were not reviewed by participants. The team used Dedoose
      software to apply deductive codes generated from the RE-AIM constructs, adapting the codebook to include emergent parent and child codes.
      Four team members engaged in an iterative process of independently coding a small subset of the transcripts and then comparing code applications to refine code definitions. Coding inter-rater reliability between 2 pairs of coders was assessed in Dedoose; on reaching κ-statistics signifying “good” inter-rater reliability (ie, .79 or greater), team members coded independently. Data saturation, meaning no new themes emerged from the data,
      • Fusch P.I.
      • Ness L.R.
      Are we there yet? Data saturation in qualitative research.
      occurred before all transcripts were coded; however, the remaining transcripts were coded because the staff and participant samples intentionally captured a diverse set of experiences with WIC. After coding, team members summarized coded text.
      Reflexivity, an evaluation of researcher’s experiences and identities that could influence the research is important in qualitative research.
      • Berger R.
      Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative research.
      There were ways in which the study team’s experiences and identities differed from the study participants, which had the ability to influence study design, data collection, and interpretation. To account for these differences, the team debriefed regularly throughout the research process; wrote reflections after focus groups and interviews; consulted with co-researchers to ensure precise analysis; and discussed findings with WA WIC staff to cross-check interpretation, a process known as “member-checking.”
      • Nowell L.S.
      • Norris J.M.
      • White D.E.
      • Moules N.J.
      Thematic analysis: Striving to meet the trustworthiness criteria.

      Results

      Sample Characteristics

      Characteristics collected for WIC staff and WIC participants are detailed in the Table. The majority of WIC participants interviewed for this study were enrolled with 1 or more children and had experience with WIC before the beginning of the COVID-19 pandemic. The majority of WIC staff who participated in interviews or focus groups were employed at the local level.
      TableCharacteristics of participants and staff from the Washington WIC
      WIC = Special Supplemental Nutrition Program for Women, Infants and Children.
      program who participated in interviews and focus groups on programmatic adaptations made in response to the COVID-19 pandemic
      Categoryn (%)
      WIC participant characteristics (n = 40)
      Race/ethnicity
      Categories are not mutually exclusive.
      2020 Washington state race and ethnicity statistics: 78% were White, 13% were Hispanic, 10% were Asian, 5% were 2 or more races, 4% were Black, 2% were American Indian and Alaskan Native, and 1% were Native Hawaiian or other Pacific Islander.27,28
       American Indian or Alaska Native3 (8)
       Asian3 (8)
       Black/African American3 (8)
       Hispanic14 (35)
       Multiracial3 (8)
       White21 (53)
       Other4 (10)
       Missing3 (8)
      WIC enrollment
      Categories are not mutually exclusive.
       Self17 (43)
       1 or more infants14 (35)
       1 or more children30 (75)
      Urbanicity
      Based on WIC agency from which they received services.
      Using the US Census Bureau and Office of Management and Budget definitions, SHARE-NW classified the Washington State counties into rural and nonrural and reported each county’s population; in 2013, 85% of Washington population lived in nonrural counties and 15% of Washington population lived in rural counties.29
       Urban33 (83)
       Rural7 (17)
      Geography
      Based on WIC agency from which they received services.
       Eastern WA13 (32)
       Western WA27 (68)
      Experience with WIC
       Received WIC services at some point before the COVID-19 pandemic (before March 2020)35 (88)
       Received WIC services for the first time during the COVID-19 pandemic5 (12)
      WIC staff characteristics (n = 52)
      Agency jurisdiction/service area
       State10 (19)
       Local42 (81)
      Urban22 (52)
      Urban and rural13 (31)
      Rural5 (12)
      Tribal2 (4)
      WIC staff role based on assigned focus group
      Many staff had multiple roles in WIC at their agency. Only 1 role is reflected here; categories are mutually exclusive.
       State agency
      Remote services5 (10)
      Expanded food list5 (10)
       Local agency
      Program coordinators14 (27)
      Certifiers11 (21)
      Breastfeeding educators5 (10)
      Nutrition educators6 (12)
      Mixed staff roles6 (12)
      a WIC = Special Supplemental Nutrition Program for Women, Infants and Children.
      b Categories are not mutually exclusive.
      c 2020 Washington state race and ethnicity statistics: 78% were White, 13% were Hispanic, 10% were Asian, 5% were 2 or more races, 4% were Black, 2% were American Indian and Alaskan Native, and 1% were Native Hawaiian or other Pacific Islander.

      Population by race. Office of Financial Management. Published 2020. Accessed February 28, 2022. https://ofm.wa.gov/washington-data-research/statewide-data/washington-trends/population-changes/population-race

      ,
      Population of Hispanic/Latino origin
      Office of Financial Management.
      d Based on WIC agency from which they received services.
      e Using the US Census Bureau and Office of Management and Budget definitions, SHARE-NW classified the Washington State counties into rural and nonrural and reported each county’s population; in 2013, 85% of Washington population lived in nonrural counties and 15% of Washington population lived in rural counties.
      Data dashboard: Demographics. Share-NW.
      f Many staff had multiple roles in WIC at their agency. Only 1 role is reflected here; categories are mutually exclusive.

      Reach

      Based on data reported by agencies and focus group respondents, all WIC participants who had an appointment or were certified during the COVID-19 pandemic experienced some aspect of remote services. Because the food list expansion was implemented statewide, all WIC participants also experienced the additions to the allowable food list. By May 2020, the WIC participant population reached with these changes included 26,891 women, 26,378 infants, and 72,010 children in Washington State; the population was predominantly White (70%), followed by multi-race (10%), Black or African American (8%), American Indian or Alaskan Native (5%), Asian (5%), and Native Hawaiian or Pacific Islander (2%); 42% of WIC participants identified as Hispanic.

      Effectiveness

      WIC Participation

      As seen in Figure 4, participation had been declining steadily since 2017. Participation began increasing just before the start of the COVID-19 pandemic and continued to increase after the programmatic changes were made to WIC in March 2020. There was a 5.9% increase in participation from December 2019 to December 2020, and participation increased by 2% from March 2020 (when the COVID-19 pandemic began) to December 2020.
      Figure thumbnail gr1
      Figure 4Overall participation in Washington State Special Supplemental Program for Women, Infants, and Children (WA WIC) broken down by participant type (2017-2020). Participation data provided by WA WIC.
      Between December 2019 and December 2020, participation increased for children by 11%, but decreased slightly for women (0.1% decrease) and infants (0.1% decrease). Participation growth also differed by race; from December 2019 to December 2020, participation increased among children identifying as Native Hawaiian or Pacific Islander (18% increase), Black or African American (17 % increase), multi-race (14 % increase), Hispanic (12% increase), White (11% increase), and Asian (9% increase), but participation decreased for children identifying as American Indian or Alaskan Native (6% decrease).

      Appointment Completions

      Local and state staff described increased appointment completion rates after implementing the programmatic changes, hypothesizing that the increase related to the convenience of remote services. As described by 1 WIC staff, “Our show rates are much higher because there isn’t the barrier of gathering all of your kids and your things and either driving or taking public transportation to get to WIC” (staff 02). Programmatic data from sample months before and during the implementation period corroborated this statement; nutrition education completion rates increased from 78% in June 2019 to 96% in June 2020, and the certification completion rate increased from 72% in June 2019 to 77% in June 2020.

      Redemption of Food Benefits

      Staff reported that the expansion of the allowable foods helped increase access to approved foods during COVID-19 pandemic–related food shortages. Participant interviewees echoed this statement. One participant shared that “at the beginning, it was hard to get milk, cheese, and cereal, but then WIC updated the list of available foods and now I am able to get them with no problem” (participant 30S).
      Food benefit redemption data throughout 2020 (Figure 5) demonstrated that the average percent of all food benefits redeemed by WIC participants declined at the start of the COVID-19 pandemic from March to April, then increased almost back to pre-COVID-19 pandemic levels right after the food list was expanded. Redemption rates then dipped once more before returning to rates near those experienced in early 2020.
      Figure thumbnail gr2
      Figure 5Average food package redemption rates (%) of Washington State Special Supplemental Nutrition Program for Women, Infants, and Children (WA WIC) participants by month in 2020. Food package redemption data provided by WA WIC

      Adoption

      State staff described 3 federal waivers that made remote services possible during the COVID-19 pandemic, listed in Figure 6.
      US Department of Agriculture
      Request for WIC flexibility in response to COVID-19.
      US Department of Agriculture
      Request for WIC flexibility in response to COVID-19.
      US Department of Agriculture
      Request for WIC flexibility in response to COVID-19.
      These waivers allowed WIC services to continue while offices were closed to the public and reduced the administrative burden on clinics that had staff pulled away to COVID-19 pandemic response. In addition, WA WIC had transitioned to an electronic benefit transfer (EBT)–based system for benefit issuance throughout 2019 and had just started loading benefits remotely in the beginning of March 2020; WA WIC was able to continue remote benefit issuance throughout the transition to remote appointments. The other major programmatic change was a large-scale expansion of the approved WIC food list. The food list is updated annually, but typically 40 to 60 items are added; in response to COVID-19 pandemic–related food shortages and challenges finding WIC-allowed foods in the store, WA WIC staff approved approximately 600 new foods. WA WIC staff explained that they chose not to use the waivers related to food items so that added foods would not be revoked when the waivers expire.
      Figure 6Key federal waivers that the Washington State Special Supplemental Program for Women, Infants, and Children (WA WIC) adopted during the COVID-19 pandemic.
      AdaptationExplanation
      Physical Presence Waiver
      US Department of Agriculture
      Request for WIC flexibility in response to COVID-19.
      Participants are allowed to certify and recertify for WA WIC services remotely (via phone or video appointment) and defer height, weight, and hemoglobin measurements.
      Remote Benefit Issuance Waiver
      US Department of Agriculture
      Request for WIC flexibility in response to COVID-19.
      Staff are allowed to issue a participant’s benefits remotely.
      Separation of Duties Waiver
      US Department of Agriculture
      Request for WIC flexibility in response to COVID-19.
      A single employee is allowed to evaluate a participant’s eligibility for all certification criteria, and issue food benefits.

      Implementation

      Remote Services Implementation

      Phone appointments were mentioned by all staff as the most common way of reaching clients remotely. Although some local staff had attended state or agency-led training on using video conferencing, nearly all local staff said that video appointments had not yet been offered to WIC participants. Perceived challenges to offering video appointments included hesitancy from staff related to embarrassment interacting on camera, a lack of training, and limited access to video equipment. To help maintain contact with WIC participants, local staff, especially breastfeeding support staff, reported texting participants more frequently between appointments.
      To supplement remote services, many local staff reported mailing handouts to participants either before some appointments. so participants could follow along with the content while on the call, or after appointments when participants requested additional information. Staff also discussed implementing pick-up services for breast pumps and educational materials. A small subset of staff mentioned putting infant scales outside of homes so parents could measure their child if growth was concerning.

      Supports and Facilitators of the Implementation of Remote Services

      Transitioning to remote services required numerous supports from federal, state, and local agency staff. A federal waiver to defer measurements combined with state-level policy to defer submitting proofs and signatures on documents improved the flexibility of the certifications. As a result, staff reported that appointment efficiency increased; one said, “when you’re doing everything over the phone things are just a bit quicker. You’re talking to them, but you can kind of write your notes simultaneously . . . so that saves a lot of time” (staff 28). State staff also initially provided biweekly webinars for agency staff to share information and assess needs related to virtual appointments; these webinars tapered to monthly by December 2020. At the local level, common requests from staff included cell phones, laptops with cameras, and attachable webcams for desktops.

      Staff and Participant Satisfaction with Remote Services

      Both staff and participants expressed high satisfaction with remote service delivery. Figure 7 summarizes a subset of the most common advantages and disadvantages of remote services from staff and participant perspectives and presents a selection of illustrative quotes. The advantages of remote services were mentioned more often and outweighed the disadvantages presented.
      Figure 7Perceived advantages and disadvantages of conducting appointments remotely during the COVID-19 pandemic as reported by Washington State Special Supplemental Program for Women, Infants, and Children (WA WIC) staff focus group participants and WA WIC interview participants. This figure includes a subset of the most common advantages and disadvantages of remote services noted by both staff and participants; it is not an exhaustive list. Key quotes have been selected to represent the theme presented. Participants with S after their ID number were interviewed in Spanish, participants with an E after their ID number were interviewed in English.
      ThemeQuote
      WA WIC staff focus groups
      Advantages of remote services
       Increased convenience for working parents, large families, those with transportation barriers“I think the majority of our clients like it, because it’s quicker and they don’t have to come in.”—staff 28
       Decreased barriers to participation such as:
      • Having to take time off work for appointments
      • Having to bring multiple children to office for one child’s appointment
      • Having to secure access to reliable transportation
      • Having to travel a long way to reach the WIC office in remote/rural areas
      “All of the clients that I’ve talked to really appreciate that they don’t have to get in the car, load all the kids up, and go to the clinic.”—staff 33
       Increased ease of sharing resources virtually with participants“They find [links] very helpful because after they had the education with the peer counselors or the breastfeeding staff the mom is able to go back and review that video again for anything that she had missed.”—staff 02
       Increased amount of communication between staff and participants (ie, between appointments and for questions)“People used to save up their little questions for the end of an appointment and go ‘Oh, there was something else I wanted to ask’ that they couldn’t remember and now they’re texting me more than they used to with just little questions intermittently.”—staff 29
       Increased richness in conversations over the phone“I think in part of the certification they have richer discussions. Participants are willing to share a little bit more over the phone versus face to face. —staff 02
       Increased ability to adapt services to address participants needs in the moment“I was able to save that breastfeeding relationship by being able to meet that mom in that moment when she needed help right then with feeding. So that was really great for remote services.”—staff 29
      Disadvantages of remote services
       Difficulty obtaining consistent height, weight, and hemoglobin measurements for infants and children“It just feels like you’re flying without wings on not having accurate measurements and timely measurements.”—staff 25
       Missing in-person relationships and interactions with participants“Seeing clients in person you get to see how they’re feeling in their expressions when they’re talking to us and you don’t pick that up remotely when you’re just talking on the phone or even in a text and so we tend to miss out on a lot of information that we could pick up if we were with them in person.”—staff 29
       Increased difficulty of some appointment types (eg, new participants and participants using interpreters)“Those new certifications are harder. I find I take probably 15 to 20 minutes more than I used to.”—staff 40
       Difficulty engaging participants without handouts and materials to share and describe in-person“Not be able to have like visuals that I usually have when I’m explaining how to use the [WIC EBT] card . . . or while they’re in my office showing them how to download the app on their phone . . . not being able to help them with that I think some people struggle.”—staff 26
       Noticing at times participants seem distracted when taking appointments over the phone“Being on the phone it’s kind of hard to try to get like the client’s needs because they’re just kind of wanting to either just hurry up with the appointment because their kids are loud or they can’t hear us.”—staff 25
       Increased difficulty scheduling and dividing the caseload fairly for select agencies“Sometimes a certifier will get four of those five appointments at the same time versus, you know, somebody else who will get a nice spread-out schedule. When you have as many as 100-250 200 clients a day, you can’t go through and just individually piece it out.”—staff 11
      WA WIC interview participants
      Advantages of remote services
       Not having to secure childcare or bring children to appointments“It was more convenient for me to do it over the phone because, with my kids, that meant I didn’t have to bring them or find a babysitter for them. I was able to just be at home and have all the information ready for it.—participant 49E
       Saving on travel time and cost of transportation“It’s just convenient. You know, I don’t have to drive all the way to the actual office itself.”—participant 107E
       Not having to take time off from work“I can have my with appointments on calls from anywhere . . . I’m able to you know, be at work.”—participant 93E
       Accomplishing other tasks during appointments“I can have my with appointments on calls from anywhere . . . I’m able to, you know, be at work and take the call or you know doing whatever it is and I’m not having to get up to go somewhere and all of that so it’s pretty great.”—participant 93E
       Feeling safer during the pandemic“It’s actually better, it’s safer for people to be at home and especially because going to the WIC office, the office is really small, so they wouldn’t be able to keep the social distancing.”—participant 30S
       Appointments seeming more focused and personalized“Most of the time, it’s the same lady that calls me. So I mean . . . it kind of gets a little personal because we kind of get to know each other over the phone.”—participant 87E
      Disadvantages of remote services
       Missing information on height, weight, and iron for infants/children“Before, when we went in in person, we would know how the baby what was thriving and his weight and right now on the phone we wouldn’t be able to know.”—participant 25S
       Missing interactions between staff, other parents, and children“She misses the ‘Hello, how are you? How’s your daughter?’ Those interactions that she enjoyed while being in person.”—interpreter for participant 9S
       Feeling some appointments seemed rushed or brief“On the phone, it is kind of just quick like, ‘Well, we’re going to add this, and this is going to happen like this, and we’ll see you talk to you in like 3 months or something.’ So, I don’t know if we miss anything, but it’s just really different.”—participant 70E
       Feeling that some appointments might not allow enough space and time to observe and interact with children, especially for new parents“I feel like they give more information when the child is in the room, and then they can see, ‘Oh your child can do this. Your child can do that.’ So, they can provide more information about specifically my child.”—participant 36E
      Both local and state staff appreciated remote services as a way to reduce longstanding participation barriers. Phone calls worked well for most appointment types, according to most local staff. The appointment types that were mentioned as more difficult remotely included enrollment appointments (especially for first-time WIC participants) and some breastfeeding appointments addressing latch and positioning support. For example, one staff member shared that “it’s been very challenging offering these breastfeeding support services remotely. It’s hard to . . . convey the different positions or movements just, you know, over the phone” (staff 29). Although enrollments and breastfeeding support appointments were brought up by some staff as challenging, these opinions were not unanimous. For example, one breastfeeding educator said “I feel like even for breastfeeding they’re more relaxed when they are talking, because . . . we have the luxury to choose any time at their convenience and they can call me” (staff 39); in this way, remote services helped the participant feel comfortable and engage with breastfeeding staff when they needed it. The most reported element of remote services that detracted from staff satisfaction was the inability to consistently obtain height, weight, and hemoglobin measurements from participants. However, these challenges were mentioned less often than the success of increasing convenience for participants with remote services.
      Participants viewed phone appointments as easy to complete, convenient, and sufficient for both sharing and receiving information. Participants reported they could connect, hear, and complete the appointments over the phone and felt as, or more, comfortable receiving services on the phone compared with in-person. Approximately one-half of the participants interviewed indicated they occasionally took appointments away from home, including at work and while running errands. One participant noted that phone appointments “just made it really easy. It was convenient that as I’m doing my normal everyday things that I can still do my appointment” (participant 36E). None of the participants interviewed had conducted a video appointment and thus did not discuss satisfaction regarding video appointments.

      Satisfaction with the Expanded Food List

      When asked about their experience with the expanded food list, approximately one-half of participants were aware of the additional foods. Those participants were happy with the expansion and especially appreciated the increased variety of kid-friendly foods like string cheese, yogurt, cereal, milk, and juice. One participant said “Yes, I like them, because we have more and more of an option of how to get cheese and yogurt because that’s what [my son] likes” (participant 25S).
      The WICShopper app was mentioned by participants as helpful when planning for and actively shopping, especially alongside the expanded food list. The scanning feature made it easier for participants to confirm allowable foods. Although the WICShopper app was useful for a majority of the participants interviewed, some still noted difficulties around determining which foods were added to the food list and having items deemed not allowable at the register.

      Maintenance

      Interest in Maintaining Remote Services

      Both staff and participants expressed hope that remote services would continue beyond the COVID-19 pandemic. Almost all state and local staff advocated for a hybrid model, including both remote and in-person appointments. They recognized the increased convenience of remote appointments, and also noted that an in-person option could be useful for certain services and for parents needing to share sensitive information, including disclosures of domestic violence. One staff member shared that “moms sometimes like to come and share what’s happening in their homes and . . . the kind of abuse they were facing or things like that . . . so a hybrid fashion, I think, is a better idea” (staff 12). Although an in-person option is desired, many staff anticipated a decline in caseload if reverting back to all in-person appointments. WIC participants generally said they would be fine with returning to in-person services, although one-third of interviewees said they would want to know returning was safe in terms of COVID-19 pandemic protocols and another one-third said they would return if it was required.
      Nearly all participants interviewed hoped that remote services would continue after the COVID-19 pandemic. The most common reason given was the improved convenience of the phone appointments. Most participants supported an option for in-person appointments. For example, one participant suggested that “as far as reloading the card, then yeah, phones fine. But if it has anything to do with checkups or with growth information, then I would like those to be in-person” (participant 25S). Nearly one-quarter of WIC participants proposed that in-person appointments be provided for, or limited to, appointments with specific purposes like certifying, breastfeeding support, and/or discussing sensitive topics. Four participants—2 English-speaking participants and 2 Spanish-speaking participants—said that they would prefer all in-person services. Participants indicated that in-person appointments would allow more opportunities for socializing and observation of breastfeeding skills or child health. On this latter point, a participant said “To be honest, after COVID, [I] would like to go to the clinic because then they could assess the children. They could actually physically see the children and be able to evaluate them” (participant 57S).

      Staff Suggestions for Improving Remote Services

      The staff proposed many ideas for creating a hybrid service delivery model, including drop-in days for height and weight measurements and conducting the rest of the certification appointment remotely, and/or conducting biannual certifications in-person and all other appointments remotely. Staff also had ideas for capturing measurements without requiring physical presence, including improved data sharing among health providers and programs that capture measurements and technology for at-home measuring. Staff expressed strongly that consistent and accurate measurements are essential to assessing nutritional risk and, further, that WIC is unique in its ability to collect and use this information, especially among families that do not have insurance.

      Supports Needed to Maintain Remote Services

      Staff described federal- and state-level supports that would be necessary to continue providing remote services. All staff want to see flexibility in federal physical presence regulations. State staff also recognized the importance of further training on video conferencing, describing a significant learning curve for both staff and participants. Approximately one-quarter of staff expressed interest in conducting more video appointments, and participants had mixed and speculative views on video appointments because none had yet completed a video appointment. From the participant perspective, perceived advantages included seeing demonstrations, having staff interact with children, and feeling more comfortable seeing the staff member. The most common perceived disadvantage of video calls described by participants was that they would not be able to take calls “hands free,” which was an important convenience of phone appointments. Staff also suggested developing video orientations for using the WICShopper app and WIC EBT card translated into numerous languages to help supplement the education they give to new participants during their first appointment.

      Interest in and Improvements Recommended for a Maintained Expanded Food List

      Participants made clear that they would like the list of allowable food to remain expanded, with even more foods added. Participants said they would appreciate additional approved food brands, more organic options, and more flexibility in allowable food package sizes. Some WA WIC participants said that although the number of approved brands of certain foods increased, the allowable sizes of food items remained a constraint because food packaging sizes in stores differed. Participants also suggested improvements to the WICShopper app, including improved search features, increased scanning reliability, and additional meal planning features. Both staff and participants mentioned that participants should be able to order WIC groceries online and pick up curbside.

      Discussion

      Public health nutrition experts have called for evaluation of WIC service changes implemented in response to the COVID-19 pandemic to strengthen WIC and inform the upcoming Child Nutrition Reauthorization process.
      • Bleich S.
      • Dunn C.
      • Kenney E.
      • Fleischhacker S.
      Strengthening WIC’s impact during and after the COVID-19 pandemic.
      This study was designed to generate evidence-based suggestions for implementing, maintaining, and improving the changes adopted by WA WIC for future practice. Overall, these results illustrate that appointment show rates improved after the programmatic changes were implemented and that WIC participants and staff were highly satisfied. Staff and participants communicated strongly that they want to see both changes maintained in some form. The impact of these changes on WIC participation trends and food benefit redemption is less clear, given the COVID-19 pandemic context influencing WIC eligibility and food shopping experiences; increased unemployment,
      • Saloner B.
      • Gollust S.E.
      • Planalp C.
      • Blewett L.A.
      Access and enrollment in safety net programs in the wake of COVID-19: A national cross-sectional survey.
      increased food insecurity,
      • Saloner B.
      • Gollust S.E.
      • Planalp C.
      • Blewett L.A.
      Access and enrollment in safety net programs in the wake of COVID-19: A national cross-sectional survey.
      and decreased access to other food-related safety net programs,
      • Kinsey E.W.
      • Hecht A.A.
      • Dunn C.G.
      • et al.
      School closures during COVID-19: Opportunities for innovation in meal service.
      make it difficult to determine the extent to which increases in participation and food benefit redemption are attributable to WIC programmatic changes as opposed to increased need.
      The State of WIC
      WIC Research, Policy and Practice Hub.
      However, the changes demonstrate promise for optimizing WIC services beyond the COVID-19 pandemic.
      This study aligns with other research on WIC participants’ experiences with remote services during the COVID-19 pandemic and delves deeper into perceived advantages of remote appointments. Participants in Tennessee and now Washington State have indicated that phone appointments have been more convenient than in-person appointments.
      • McElrone M.
      • Zimmer M.C.
      • Anderson Steeves E.T.
      A qualitative exploration of predominantly White non-Hispanic Tennessee WIC participants’ food retail and WIC clinic experiences during COVID-19.
      Similar to participants in California, WA WIC participants felt comfortable with phone appointments and sharing documentation remotely.

      Ventura A, Martinez C, Whaley S. WIC participants’ perceptions of COVID-19-related changes to WIC recertification and service delivery. J Community Health. https://doi.org/10.1007/s10900-021-01026-8

      Uniquely, participants in this study highlighted specific advantages of the remote appointments that address longstanding barriers to WIC; the most notable benefit was not having to secure childcare for appointments or bring multiple children to appointments. Past research has demonstrated that bringing children to appointments is one of the most common barriers to recertifying.
      • Woelfel M.L.
      • Abusabha R.
      • Pruzek R.
      • Stratton H.
      • Chen S.G.
      • Edmunds L.S.
      Barriers to the use of WIC services.
      Other advantages mentioned by participants in this study that address barriers to using WIC services
      • Woelfel M.L.
      • Abusabha R.
      • Pruzek R.
      • Stratton H.
      • Chen S.G.
      • Edmunds L.S.
      Barriers to the use of WIC services.
      include saving on travel time and costs, taking appointments on breaks from work, and feeling like appointments were more personalized on the phone. Similar to findings from other states,

      Ventura A, Martinez C, Whaley S. WIC participants’ perceptions of COVID-19-related changes to WIC recertification and service delivery. J Community Health. https://doi.org/10.1007/s10900-021-01026-8

      ,
      • Mclean M.
      • Haynes L.
      • Tkachenko O.
      • Hennessey Z.
      Acceptability, preference, and no-show rates for in-person and phone-based consultations at nine WIC centers in New York City before and during COVID-19.
      WA WIC participants were highly satisfied with remote services and explicitly suggested their continuation.
      • McElrone M.
      • Zimmer M.C.
      • Anderson Steeves E.T.
      A qualitative exploration of predominantly White non-Hispanic Tennessee WIC participants’ food retail and WIC clinic experiences during COVID-19.

      Ventura A, Martinez C, Whaley S. WIC participants’ perceptions of COVID-19-related changes to WIC recertification and service delivery. J Community Health. https://doi.org/10.1007/s10900-021-01026-8

      • Mclean M.
      • Haynes L.
      • Tkachenko O.
      • Hennessey Z.
      Acceptability, preference, and no-show rates for in-person and phone-based consultations at nine WIC centers in New York City before and during COVID-19.
      Findings from this research also showcase satisfaction with the expanded food options during the COVID-19 pandemic, and suggest that the expanded food list likely offset a further decline in food benefit redemption rate due to COVID-19 pandemic–related food shortages.

      WIC Innovates to Support Maternal and Child Health During the Pandemic. Robert Wood Johnson Foundation. Published May 7, 2021. Accessed July 23, 2021. https://www.rwjf.org/en/blog/2021/05/wic-innovates-to-support-maternal-and-child-health-during-the-pandemic.html

      Food benefit redemption increased from April to May 2020, after the food list was expanded; while it did decrease again from May to June and July, this could have been related to the increase in Supplemental Nutrition Assistance Program benefits
      USDA increases monthly SNAP benefits by 40%. US Department of Agriculture.
      and the COVID-19 pandemic EBT disbursements that began in late June and early July in Washington,
      Washington families get help to buy food during school closures. Office of Superintendent of Public Instruction.
      which likely impacted the purchasing patterns of many WIC participants. This study also provides unique insight into participants’ opinions on expanded foods that were most appreciated during the COVID-19 pandemic, including string cheese, yogurt, milk, cereal, and juice. WA WIC expanded these categories without using waivers, so these additional items would not be revoked when waivers expire. To help determine which new items in the store were WIC-allowed, both California and WA WIC participants found the WICShopper app particularly helpful.
      • Au L.
      • Ritchie L.
      • Vital N.
      • et al.
      WIC is critical during the COVID-19 pandemic: Lessons learned from Los Angeles County participants.
      Despite the expanded food list and WICShopper app, participants from the current study noted that package size constraints were magnified during the COVID-19 pandemic. Package size availability has been a longstanding challenge reported by WIC participants,
      • Woelfel M.L.
      • Abusabha R.
      • Pruzek R.
      • Stratton H.
      • Chen S.G.
      • Edmunds L.S.
      Barriers to the use of WIC services.
      and this current study indicated it has continued to impede WIC benefit redemption even within the context of expanded food options. Current study findings add further support for policy change regarding package sizes of WIC approved foods.

      Enhancing the WIC Food Package: Impacts and Recommendations to Advance Nutrition Security. National WIC Association. Published June 2021. Accessed August 8, 2021. https://s3.amazonaws.com/aws.upl/nwica.org/nwa-wic-food-package-report.pdf

      This is the first study that presents the WIC staff perspective on how services changed during the COVID-19 pandemic and related suggestions for future practice. Staff perspectives were similar to WIC participant perspectives when describing the ease and convenience of remote services, aspects of remote services that decrease participation barriers, and the helpfulness of the WICShopper app. The main drawbacks to remote services outlined by staff also aligned with the participants’ perspectives; they wanted a way to capture consistent measurements remotely and they missed interacting with WIC participants in the clinic. Although there were similarities between staff and participant perspectives, they also diverged in several areas. First, some staff reported participants occasionally seemed distracted while taking phone appointments away from home. Participants did discuss taking appointments away from home, but within the context of convenience, highlighting the ability to multitask or maximize breaks from work as a benefit of remote services. Second, staff anticipated attrition once in-person services were re-established. Participants did report wanting to continue receiving WIC services remotely after the COVID-19 pandemic, but none said they would leave the program if in-person services were required. Staff and participants had similar suggestions for programmatic changes in the future, including transitioning to a hybrid model and including online ordering and pick-up options. Staff perspectives should be incorporated into future research, when possible, as it is valuable to know where WIC staff and participants’ priorities align in order to make recommendations for changes.
      Study findings add to the evidence supporting continued access to remote WIC services in addition to in-person services.
      The State of WIC
      WIC Research, Policy and Practice Hub.
      ,
      • McElrone M.
      • Zimmer M.C.
      • Anderson Steeves E.T.
      A qualitative exploration of predominantly White non-Hispanic Tennessee WIC participants’ food retail and WIC clinic experiences during COVID-19.
      ,
      One year of WIC during COVID-19: Waivers are vital to participation and benefit redemption.
      However, maintaining an option for remote certifications beyond the COVID-19 pandemic would require federal policy changes. For participants to be able to choose to certify and recertify remotely, either by video or phone, the federal physical presence requirement would need to be amended to allow either hybrid or full-remote options in addition to in-person. Some of the rationale for the physical presence requirement are to observe WIC participants,
      • Neuberger Z.
      Modernizing and Streamlining WIC Eligibility Determination and Enrollment Processes.
      identify health concerns,
      • Neuberger Z.
      Modernizing and Streamlining WIC Eligibility Determination and Enrollment Processes.
      personalize service delivery,
      • Neuberger Z.
      Modernizing and Streamlining WIC Eligibility Determination and Enrollment Processes.
      and protect against fraud.
      • Chaudhari D.
      • Shaffer V.
      • Logan C.
      Methods to Prevent fraud and abuse among staff and participants in the WIC Program.
      Video appointments could be a substitute for physical presence because they allow for participant observation and providers can successfully identify health concerns, as seen with the increasing use of videoconferencing in telemedicine.
      • Weinstein R.S.
      • Krupinski E.A.
      • Doarn C.R.
      Clinical examination component of telemedicine, telehealth, mHealth, and connected health medical practices.
      Furthermore, some participants in this current study felt that remote appointments were more personalized compared with in-person appointments and would like this option to continue in addition to in-person services.
      To facilitate the continued success of remote appointments, additional means of collecting height, weight, and hemoglobin measurements should be piloted and evaluated; one option to research is drop-in hours for collecting measurements. Staff also suggested implementing data-sharing protocols between WIC and health care providers to minimize the duplication of services among participants whose measurements are being captured elsewhere. In addition, training and technological resources to maximize remote service quality in the future should be strengthened.
      The State of WIC
      WIC Research, Policy and Practice Hub.
      ,

      WIC Innovates to Support Maternal and Child Health During the Pandemic. Robert Wood Johnson Foundation. Published May 7, 2021. Accessed July 23, 2021. https://www.rwjf.org/en/blog/2021/05/wic-innovates-to-support-maternal-and-child-health-during-the-pandemic.html

      ,
      • Whaley S.E.
      • Anderson C.E.
      The importance of federal waivers and technology in ensuring access to WIC during COVID-19.
      WA WIC staff advocated for additional video trainings for both staff and participants to feel comfortable providing and receiving services via video, as well as prerecorded video orientations in multiple languages about shopping with the WIC EBT card and WICShopper app. To improve the WIC shopping experience and food security, state programs should expand their approved food lists as much as possible within the current federal regulations. However, federal policy changes will be needed to address food package size discrepancies. Moreover, any change to WIC service delivery should be tracked and evaluated for both successes and unintended consequences.
      There are multiple strengths of the design, methods, and analysis of this study. Key strengths include the mixed-methods design; participation of a diverse participant sample, including staff perspectives; the iterative approach to coding qualitative data; and the member-checking process with WA WIC staff. Integration of programmatic data with qualitative data allowed for contextualization and explanation of quantitative trends
      • Holtrop J.S.
      • Rabin B.A.
      • Glasgow R.E.
      Qualitative approaches to use of the RE-AIM framework: Rationale and methods.
      ; use of both quantitative and qualitative data within the RE-AIM framework has been recommended and both were employed in this study for a rigorous exploration of the changes.
      • Glasgow R.E.
      • Harden S.M.
      • Gaglio B.
      • et al.
      RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review.
      Furthermore, having multiple coders code a subset of the transcripts, discuss discrepancies, and collaborate on themes enhanced the data triangulation process
      • Fusch P.
      • Fusch G.E.
      • Ness L.R.
      Denzin’s paradigm shift: Revisiting triangulation in qualitative research.
      ,
      • Tracy S.J.
      Qualitative quality: Eight “Big-tent” criteria for excellent qualitative research.
      and confirmability of these results.
      • Nowell L.S.
      • Norris J.M.
      • White D.E.
      • Moules N.J.
      Thematic analysis: Striving to meet the trustworthiness criteria.
      The participant sample also captured a diversity of perspectives about the programmatic changes.
      • McElrone M.
      • Zimmer M.C.
      • Anderson Steeves E.T.
      A qualitative exploration of predominantly White non-Hispanic Tennessee WIC participants’ food retail and WIC clinic experiences during COVID-19.
      Finally, in line with principles of rigor in qualitative research, the research team shared a preliminary synthesis of the results with WA WIC staff participants and discussed whether the findings resonated with their experiences.
      • Nowell L.S.
      • Norris J.M.
      • White D.E.
      • Moules N.J.
      Thematic analysis: Striving to meet the trustworthiness criteria.
      There are also limitations to this study. First, the study focused on one state’s WIC program, limiting the national generalization of the results. Second, interviews were only conducted in English and Spanish due to cost constraints; adding more languages could have increased the WIC participant sample diversity. Third, some trends, such as increases in participation, may have already been under way before COVID-19 pandemic shutdown orders; the team did not collect information that would elucidate whether these trends were due to growing awareness of the pandemic in late 2019/early 2020 or to simultaneous programmatic changes, such as changes in electronic enrollment systems. Fourth, the team engaged in member-checking with WA WIC staff; however, there was not time to conduct a member-checking process with the participant sample.
      • Tracy S.J.
      Qualitative quality: Eight “Big-tent” criteria for excellent qualitative research.
      Fifth, demographic data on race and ethnicity, gender, or age of WIC staff participants were not collected and WIC staff from diverse racial and ethnic groups were not purposively sampled. The interpretation of study findings may have been enhanced in the context of WIC staff participant race and ethnicity, gender, or age. Sixth, and finally, the WIC participant sample was recruited through the WICShopper app, which may have generated a sample that would be more comfortable with the technology and experience of remote service delivery; however, nearly all WIC participants in Washington State use the WICShopper app and all have experienced remote services, so the potential for a biased sample is low.

      Conclusions

      This study explored the reach, effectiveness, adoption, implementation, and maintenance of remote services and the expanded food list executed by the WA WIC program in response to the COVID-19 pandemic. After these changes were implemented, participation, appointment show rates, and food redemption increased. Both staff and participants were highly satisfied with remote services and envision a combination of remote and in-person services offered in the future. Participants appreciated the increased variety of kid-friendly food options. Some of the suggested WIC changes, especially the continuation of remote certifications, would require federal policy change, and others, including increased training for video appointments and additions to the app and food list, would not.

      Acknowledgements

      The authors would like to acknowledge the assistance of Cathy Franklin, MS, RD, Terri Trisler, MS, RDN, CD, and Anh Tran, PhD, MPH for their assistance in gathering data and designing this study, and Annie Vuong, BA for creating the figures seen in this manuscript. We have received permission to acknowledge these individuals.

      Author Contributions

      J. J. Otten, E. L. Quinn, and C. M. Rose collected the data. E. J. Morris, E. L. Quinn, JO, and C. M. Rose analyzed the data. E. J. Morris wrote the first draft with contributions from J. J. Otten, E. L. Quinn, C. M. Rose, M. Spiker, and J. O’Leary. All authors reviewed and commented on subsequent drafts of the manuscript.

      Supplementary Materials

      Figure 2Semi-structured focus group and key informant interview guide used with Washington Special Supplemental Nutrition Program for Woman, Infants, and Children (WA WIC) staff to determine their experiences and perceptions of programmatic changes during the COVID-19 pandemic. DOH = Department of Health; RD = registered dietitian.
      WIC STAFF INTERVIEW AND FOCUS GROUP QUESTIONS
      State Staff–Involved in Changes to Remote Certification and Services
      • 1)
        Please briefly introduce yourself and describe your role in the WIC program.
      • 2)
        Federal law currently requires certifications to be conducted in person, but a waiver during COVID allowed this to happen remotely or to postpone certain parts of the certification process. What does remote certification look like in Washington during the COVID pandemic? [Probe: mode (phone, video), and changes in frequency, duration, topics covered]
        • a.
          To what extent does remote certification look similar or different across the state? If there are differences, what are these and why?
      • 3)
        Think about how remote certification has been working in Washington during the COVID pandemic.
        • a.
          What has been working well?
        • b.
          What has been challenging? What additional supports are needed or logistical challenges need to be addressed?
      • 4)
        What would you like to see in federal policy about the option for remote certification after the COVID pandemic? Why is that?
        • a.
          What would DOH and local agencies need to address/clarify to offer remote certification?
      • 5)
        Next, I’ll walk us through a few WIC services, including nutrition education, health screenings and referrals, and breastfeeding support. I’m interested in hearing from you about how each service has changed since the start of the COVID pandemic, including how and the extent to which these services are now being offered remotely.
        • How have nutrition education services changed and to what extent are they being offered remotely? How consistent or different are these changes across the state?
        • How have health screenings and referrals changed and to what extent are they being offered remotely? How consistent or different are these changes across the state?
        • How has breastfeeding support/peer counseling changed and to what extent are they being offered remotely? How consistent or different are these changes across the state?
      • 6)
        What resources, training, guidance, or technology has been provided by DOH to support the adoption and implementation of remote services?
      • 7)
        Think about how remote services (nutrition education, referrals, breastfeeding support) have been working in Washington during the COVID pandemic.
        • a.
          What has been working well?
        • b.
          What has been challenging?
        • c.
          What additional supports are needed? What logistical challenges need to be addressed?
      • 8)
        Would you like to see remote services offered more, less, or about the same amount after the COVID pandemic? Why?
        • a.
          If interested in growing or maintaining remote services:
          • i.
            What supports would be needed to help that happen (eg, technology, training, communications/messaging, other)?
          • ii.
            What costs would be required?
          • iii.
            Would any policy changes be required?
          • iv.
            How feasible or likely does it seem that this could happen?
      • 9)
        How has offering WIC certification and services remotely impacted service quality, convenience, and outcomes?
      • 10)
        In addition to the ability to conduct certifications and services remotely, what other WIC changes or flexibilities would you like to see in Washington State to make it easier for clients to use the program during or after the COVID pandemic?
        • a.
          Are there any priorities that the agency is currently exploring or planning?
      • 11)
        How have changes to the certification, nutrition education, health screenings and referrals, and breastfeeding support/peer counseling been communicated to WIC clients?
      • 12)
        We plan to conduct focus groups with local staff and interviews with WIC clients about remote certification, remote services, and the recent expansion of Washington WIC’s list of allowable foods. What would you be most interested in hearing from local staff and clients about these topics?
      State Staff–Involved in Changes to Remote Certification and Services
      • 1)
        Please briefly introduce yourself and describe your role in the WIC program as it relates to the expanded food list.
      • 2)
        Why did DOH decide to expand the list of WIC allowed foods most recently in April?
        • a.
          Is the list reviewed and adjusted periodically according to a set timeline or was this a one-time effort based on response to stakeholder feedback or some other reason?
        • b.
          Since when/for how long had DOH been talking about expanding the list?
      • 3)
        Please tell me a bit about the process through which DOH decided which foods to add. For example, which stakeholders were involved and how did you decide which foods to add?
        • a.
          What were the most important considerations involved in deciding what foods to add (eg, costs, nutritional guidelines, retailer feedback)?
          • a.
            Of these, which were specific to the COVID-19 pandemic?
        • b.
          What were the sticking points, if any?
        • c.
          How long did it take to develop and adopt the expanded list?
      • 4)
        How was information about the expanded food list communicated to local WIC agencies, retailers, and WIC clients?
        • a.
          Other than these 3 groups, were there any other stakeholders that needed information about the list expansion?
        • b.
          How easy or hard was it to get information out about the expanded list to all necessary stakeholders?
        • c.
          Was information about the expanded food list highlighted through the WICShopper app in any way?
      • 5)
        What feedback have you received about the expanded food list from clients, WIC staff, and retailers? Has the feedback been generally positive, generally negative, or mixed from these different stakeholder groups?
        • a.
          Has any of the feedback led you think about additions or revisions that should be considered in the future? If so, what are some examples?
      • 6)
        How and to what extent do you think the expansion of the food list has impacted WIC clients?
        • a.
          To what extent do you think this may have been influenced by COVID-19 and changes in how people have been shopping for food or the availability of food items?
        • b.
          How might the impact of the expanded food list be different in more typical, non-COVID, times?
      • 7)
        Going forward, do you feel the food list should be expanded or refined further?
        • a.
          If so, in what ways?
        • b.
          If so, how important or urgent do you feel these changes are?
      • 8)
        We know other states also expanded their food list in response to COVID. What have your heard from them, and have you reconsidered or refined your approach based on this?
      Local WIC Staff
      • 1)
        To begin, please share how WIC [certification and services such as nutrition education, breastfeeding support, and referrals] have changed at your sites since the pandemic began.
        • a.
          [Program coordinators] How and where are appointments conducted? For example, are they happening in the office; curbside; or via phone, text, or chat platforms; GoToMeeting; or some other way?
        • b.
          [Program coordinators] What methods are used to send and sign documents?
        • c.
          [Certifiers and program coordinators] Is your agency receiving any bloodwork and height and weight information from participants? If so, how?
        • d.
          How are educational materials provided to clients?
      • 2)
        Think about the services your WIC site is currently offering remotely or differently in some way since the COVID pandemic.
        • a.
          [Program coordinators] What new technology, equipment, or training was needed?
        • b.
          [Program coordinators] To what extent have you been able to get this needed technology, equipment, or training?
        • c.
          [All other local staff] What new technology or equipment have you have needed to use, and how do you feel about this new technology or equipment?
        • d.
          [All other local staff] What training was provided? What has been most useful for you?
      • 3)
        Think about how the changes to WIC services [certification, nutrition education, breastfeeding support] have been working for you and other WIC staff.
        • a.
          What has been challenging and what has gone well?
        • b.
          How have the changes impacted staff workload?
        • c.
          How have the changes impacted the quality of WIC services, if at all? Are any tasks or services unable to happen or happening less consistently (eg, bloodwork, height/weight measurements, referrals to RDs, provision of educational materials)?
        • d.
          What additional support is needed, if any?
      • 4)
        Think about how these changes to WIC [certification, nutrition education, breastfeeding support] services have been working from the perspective of WIC clients.
        • a.
          What do clients find frustrating or confusing and what do clients seem to like?
        • b.
          What additional support do clients need?
        • c.
          How have these changes to the WIC services impacted their convenience for clients, or the effectiveness of the services?
        • d.
          Have you noticed any differences in how these changes have been experienced by specific groups of clients? For example, has the experience been different for tribal clients, clients that speak a language other than English, working parents, families of different sizes, or families who live in rural areas?
      • 5)
        What changes or supports would you recommend to improve the quality, convenience, or impact of remote WIC [certification, nutrition education, breastfeeding support] services?
      • 6)
        In general, would you like to see remote WIC [certification, nutrition education, breastfeeding support] services continue? If so, how important do you think this is?
        • a.
          If you feel there should be a mix of remote and in person services be offered, what might that look like?
      • 7)
        [Certifiers] As you know, the WIC program greatly expanded its list of eligible foods for purchase in April 2020. What feedback have you heard about this change from clients and retailers?
        • a.
          To what extent do you think the change has impacted benefit redemption or client food security? Do you think this may have been influenced by COVID-19 and changes in how people have been shopping for food or the availability of food items?
      • 8)
        We also plan to conduct interviews with WIC clients about changes made to WIC certification and services since COVID started. What questions you would suggest that we ask?
      • 9)
        Please think about 1 or 2 successful client interactions you have had. Did they happen remotely? If not, could they have happened if services were offered remotely? Why or why not?
      • 10)
        Those are all the questions we have today. Can you think of anything we did not talk about today that would be important to consider in relation to remote WIC services or the expanded food package?
      Figure 3Semi-structured interview guide used with Washington Special Supplemental Nutrition Program for Woman, Infants, and Children (WA WIC) participants to determine their experiences and perceptions of programmatic changes during the COVID-19 pandemic. EBT = electronic benefit transfer.
      WIC PARTICIPANT INTERVIEW QUESTIONS
      Note: When we ask questions like, “When and where did you most recently get signed up for WIC” we mean you or your children.
      Background:
      • 1)
        When and where did you most recently get signed up for WIC?
      • 2)
        [If applicable] Based on your answers to our online survey, it looks like you HAVE been on WIC before, is that correct? Was it from the same agency you are using now or a different one?
      Enrolling in WIC:
      • 3)
        When you signed up for WIC, was it in person, over the phone, or by video-chat?
      • 4)
        Tell me little about the process of getting onto WIC from what you remember. What about it was easy or convenient? What was confusing, difficult, or inconvenient?
      • 5)
        How did you learn about using your WIC card?
      • 6)
        Do you use the WICShopper app? If yes, how did you learn to use it?
      • 7)
        What is it like using the WICShopper app? What about using the WICShopper app was confusing, difficult, or inconvenient?
      • 8)
        Before COVID, getting on WIC always happened in-person at the clinic. Since last March, many clinics have handled some or all parts of getting on WIC over the phone or by video-chat. What do you think about getting signed up for WIC by phone or video-chat?
      Other WIC appointments:
      Since you got signed up, have you had other communication with WIC staff? If yes:
      • 9)
        Was the contact by phone, by text, by video-chat, in person, or a combination?
        • a.
          What about this worked well?
        • b.
          What about this was confusing, difficult, or inconvenient?
        • Further probes:
          • How easy or hard was it to connect with WIC staff over the phone/video and hear WIC staff during appointments?
          • How comfortable did you feel talking with WIC staff over the phone/video during your appointments? [If they were on WIC before COVID:] Did you feel any more or less comfortable talking with WIC staff over the phone/video as compared to in-person appointments?]
          • [If they were on WIC before COVID:] Do you feel like you get more, less, or different information and support from WIC when appointments are over the phone or video?
          • [If they were on WIC before COVID:] Is there anything you miss about the in-person appointments? If so, what do you miss and why?
          • Have any topics been hard to talk through with WIC staff over the phone?
      • 10)
        When appointments happen over the phone or video, do you make sure you’re at home at the scheduled time, or do you join the appointments wherever you happen to be?
      • 11)
        Were you offered the option to have a video-chat appointment? If so, did you do it? Why or why not?
        • a.
          What do you think about the idea of having appointments over video chat in the future? What might be helpful? What concerns do you have?
      • 12)
        [If breastfeeding:] Were you offered breastfeeding support? If yes, how did that work for you?
      • 13)
        [If primary language is non-English:] Was an interpreter used for any of your WIC appointments? If yes, how did that work for you?
      • 14)
        Would you like WIC appointments to happen over the phone or by video after COVID? Why?
      • 15)
        How will you feel if asked to come in-person for some or all WIC appointments after COVID?
      Getting WIC foods:
      • 16)
        During COVID, how easy or hard is it for you use your WIC card at the store? Why?
        • a.
          Which foods are the most difficult to find or shop for? Why?
        • b.
          Are there WIC foods you don’t buy every month? If so, which ones? Why is that?
      • 17)
        [If they were on WIC before COVID:] WIC expanded its list of eligible foods in April 2020. Were you aware of this? If yes, what did you think about the expansion of the food list?
      Other/Closing:
      • 18)
        As you know, WIC provides moms and children with food benefits, nutrition education, breastfeeding support, and other types of referrals. What’s most helpful to you about WIC?
      • 19)
        How concerned do you feel about the COVID pandemic? Would you say not at all, a little, moderately, or very concerned?
      • 20)
        Due to the COVID pandemic, have you or other people in your household had difficulties with any of the following things? I am going to list off several items. Please tell me yes or no for each.
        • a.
          Reduced wages, work hours, or lost job
          • i.
            If yes: Have you used paid annual leave, sick leave, unemployment benefits, or any other program to provide some wage replacement?
        • b.
          Childcare or schools being closed, or having less access to them.
          • i.
            If yes: How have you dealt with care for your children during this time?
        • c.
          Difficulty with transportation
          • i.
            If yes: What were the difficulties and how did you deal with them?
        • d.
          Difficulty with getting or having enough of the food your family needs
          • i.
            If yes: What were the difficulties and how did you deal with them? Has WIC helped you access food from other sources (beyond WIC)?
        • e.
          Difficulty with paying the rent or mortgage
        • f.
          Difficulty getting health care, including getting medications, or paying for medical costs
      • 21)
        Do you have any children who got meals from childcare or school before the COVID pandemic?
        • g.
          If yes: Have you continued to receive childcare or school or meals during the COVID pandemic?
        • h.
          Have you received pandemic EBT during the COVID pandemic? By pandemic EBT we mean extra EBT funds because your kids used to get food at school.
      • 22)
        Overall, do you have any suggestions for how WIC could work better for families in your community or families like yours?
      • 23)
        Can you think of anything we didn’t talk about today that would be good for us to know?

      References

        • The State of WIC
        WIC Research, Policy and Practice Hub.
        https://thewichub.org/the-state-of-wic/
        Date accessed: July 22, 2021
        • WIC racial-ethnic group enrollment data 2016
        US Department of Agriculture, Federal Nutrition Service.
        (Published July 18, 2019)
      1. Rural hunger in America: Special Supplemental Nutrition Program for Women, Infants, and Children.
        (Published online July 2018)
        • Carlson S.
        • Neuberger Z.
        WIC Works: Addressing the nutrition and health needs of low-income families for more than four decades. Center on Budget and Policy Priorities.
        (Published January 27, 2021)
        • Soneji S.
        • Beltrán-Sánchez H.
        Association of Special Supplemental Nutrition Program for Women, Infants, and Children with preterm birth and infant mortality.
        JAMA Netw Open. 2019; 2e1916722https://doi.org/10.1001/jamanetworkopen.2019.16722
      2. Gray K, Trippe C, Tadler C, Perry C, Johnson P, Betson D. National-and State-Level Estimates of WIC Eligibility and WIC Program Reach in 2017, Final Report: Volume I.; 2019:86. Accessed August 6, 2021. https://fns-prod.azureedge.net/sites/default/files/resource-files/WICEligibles2017-Volume1.pdf

        • Woelfel M.L.
        • Abusabha R.
        • Pruzek R.
        • Stratton H.
        • Chen S.G.
        • Edmunds L.S.
        Barriers to the use of WIC services.
        J Am Diet Assoc. 2004; 104: 736-743
      3. Review of WIC Food Packages: Improving Balance and Choice: Final Report. National Academies of Sciences, Engineering, and Medicine, 2017
        • McKinney S.
        Improving WIC usage: Office procedures, food package, and nutrition education.
        J Acad Nutr Diet. 2004; 104: 743-745
      4. WIC Outreach and Retention Survey Report. National WIC Association, 2018
      5. WIC Innovates to Support Maternal and Child Health During the Pandemic. Robert Wood Johnson Foundation. Published May 7, 2021. Accessed July 23, 2021. https://www.rwjf.org/en/blog/2021/05/wic-innovates-to-support-maternal-and-child-health-during-the-pandemic.html

        • Wolfson J.A.
        • Leung C.W.
        Food Insecurity and COVID-19: Disparities in early effects for US adults.
        Nutrients. 2020; 12: 1648
      6. FNS responds to COVID-19. US Department of Agriculture, Food and Nutrition Service.
        https://www.fns.usda.gov/coronavirus
        Date accessed: August 3, 2021
        • Glasgow R.E.
        • Harden S.M.
        • Gaglio B.
        • et al.
        RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review.
        Front Public Health. 2019; 7: 64
        • Rutherford Z.
        • Zwolinsky S.
        • Kime N.
        • Pringle A.
        A mixed-methods evaluation of CARE (cancer and rehabilitation exercise): A physical activity and health intervention, delivered in a community football trust.
        Int J Environ Res Public Health. 2021; 18: 3327
      7. USDA extends waivers: WIC continues to provide remote services. Washington State Department of Health. Published September 29, 2020. Accessed August 3, 2021. https://www.doh.wa.gov/Newsroom/Articles/ID/2397/USDA-extends-waivers-WIC-continues-to-provide-remote-services

        • Wiesman J.
        • Lofy K.
        Washington State Health Assessment.
        (Published March 2018)
      8. Report to the Legislature: Washington State Maternal Mortality Review Panel: Maternal Deaths 2014-2016. Washington State Department of Health, Prevention and Community Health Division. Accessed April 1, 2022. https://doh.wa.gov/sites/default/files/legacy/Documents/Pubs//141-010-MMRPMaternalDeathReport2014-2016.pdf

      9. Households on Cash, Food, and Medical Assistance Programs in Washington State by Primary Language. Washington State Department of Social and Health Services, 2021
        • Guest G.
        • Bunch A.
        • Johnson L.
        How many interviews are enough? An experiment with data saturation and variablity.
        Field Methods. 2016; 18: 59-82
      10. Microsoft Excel [computer program]. Microsoft Corporation, 2019
        Version: Version 16.30
      11. Tableau [computer program]. Tableau Software LLC, 2021
        Version: Version 2021.1
      12. Dedoose [computer program]. SocioCultural Research Consultants, LLC, 2021
        • Fusch P.I.
        • Ness L.R.
        Are we there yet? Data saturation in qualitative research.
        Qual Rep. 2015; 20 (1408-1406)
        • Berger R.
        Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative research.
        Qual Res. 2015; 15: 219-234
        • Nowell L.S.
        • Norris J.M.
        • White D.E.
        • Moules N.J.
        Thematic analysis: Striving to meet the trustworthiness criteria.
        Int J Qual Methods. 2017; 161609406917733847
      13. Population by race. Office of Financial Management. Published 2020. Accessed February 28, 2022. https://ofm.wa.gov/washington-data-research/statewide-data/washington-trends/population-changes/population-race

        • Population of Hispanic/Latino origin
        Office of Financial Management.
      14. Data dashboard: Demographics. Share-NW.
        (Published 2021)
        • US Department of Agriculture
        Request for WIC flexibility in response to COVID-19.
        (Published online March 20, 2020)
        • US Department of Agriculture
        Request for WIC flexibility in response to COVID-19.
        (Published online March 24, 2020)
        • US Department of Agriculture
        Request for WIC flexibility in response to COVID-19.
        (Published online March 30, 2020)
        • Bleich S.
        • Dunn C.
        • Kenney E.
        • Fleischhacker S.
        Strengthening WIC’s impact during and after the COVID-19 pandemic.
        (Published online July 2020)
        • Saloner B.
        • Gollust S.E.
        • Planalp C.
        • Blewett L.A.
        Access and enrollment in safety net programs in the wake of COVID-19: A national cross-sectional survey.
        PLoS One. 2020; 15e0240080https://doi.org/10.1371/journal.pone.0240080
        • Kinsey E.W.
        • Hecht A.A.
        • Dunn C.G.
        • et al.
        School closures during COVID-19: Opportunities for innovation in meal service.
        Am J Public Health. 2020; 110: 1635-1643
        • McElrone M.
        • Zimmer M.C.
        • Anderson Steeves E.T.
        A qualitative exploration of predominantly White non-Hispanic Tennessee WIC participants’ food retail and WIC clinic experiences during COVID-19.
        J Acad Nutr Diet. 2021; 121: 1454-1462
      15. Ventura A, Martinez C, Whaley S. WIC participants’ perceptions of COVID-19-related changes to WIC recertification and service delivery. J Community Health. https://doi.org/10.1007/s10900-021-01026-8

        • Mclean M.
        • Haynes L.
        • Tkachenko O.
        • Hennessey Z.
        Acceptability, preference, and no-show rates for in-person and phone-based consultations at nine WIC centers in New York City before and during COVID-19.
      16. USDA increases monthly SNAP benefits by 40%. US Department of Agriculture.
      17. Washington families get help to buy food during school closures. Office of Superintendent of Public Instruction.
        (Published June 10, 2020)
        https://waospi.medium.com/?p=d6a1535570a
        Date accessed: August 27, 2021
        • Au L.
        • Ritchie L.
        • Vital N.
        • et al.
        WIC is critical during the COVID-19 pandemic: Lessons learned from Los Angeles County participants.
        Curr Dev Nutr. 2021; 5 (205-205)
      18. Enhancing the WIC Food Package: Impacts and Recommendations to Advance Nutrition Security. National WIC Association. Published June 2021. Accessed August 8, 2021. https://s3.amazonaws.com/aws.upl/nwica.org/nwa-wic-food-package-report.pdf

      19. One year of WIC during COVID-19: Waivers are vital to participation and benefit redemption.
        (Published online June 2021)
        • Neuberger Z.
        Modernizing and Streamlining WIC Eligibility Determination and Enrollment Processes.
        Center on Budget and Policy Priorities, 2017
        • Chaudhari D.
        • Shaffer V.
        • Logan C.
        Methods to Prevent fraud and abuse among staff and participants in the WIC Program.
        US Department of Agriculture, Economic Research Service, 2001
        • Weinstein R.S.
        • Krupinski E.A.
        • Doarn C.R.
        Clinical examination component of telemedicine, telehealth, mHealth, and connected health medical practices.
        Med Clin North Am. 2018; 102: 533-544
        • Whaley S.E.
        • Anderson C.E.
        The importance of federal waivers and technology in ensuring access to WIC during COVID-19.
        Am J Public Health. 2021; 111: 1009-1012
        • Holtrop J.S.
        • Rabin B.A.
        • Glasgow R.E.
        Qualitative approaches to use of the RE-AIM framework: Rationale and methods.
        BMC Health Serv Res. 2018; 18: 177
        • Fusch P.
        • Fusch G.E.
        • Ness L.R.
        Denzin’s paradigm shift: Revisiting triangulation in qualitative research.
        J Soc Change. 2018; 10: 19-32https://doi.org/10.5590/JOSC.2018.10.1.02
        • Tracy S.J.
        Qualitative quality: Eight “Big-tent” criteria for excellent qualitative research.
        Qual Inq. 2010; 16: 837-851

      Biography

      E. J. Morris is a master’s student, Nutritional Sciences Program, School of Public Health, University of Washington, Seattle.
      J. J. Otten is an associate professor and food systems director, Nutritional Sciences Program, Center for Public Health Nutrition, and Department of Environmental and Occupational Health Sciences, School of Public Health, University of Washington, Seattle.
      E. L. Quinn is a research scientist, Department of Health Systems and Population Health, University of Washington, Seattle.
      C. M. Rose is a research scientist, Center for Public Health Nutrition and Department of Epidemiology, University of Washington School of Public Health, Seattle.
      M. Spiker is an assistant professor, Department of Epidemiology and Nutritional Sciences Program, University of Washington School of Public Health, Seattle.
      J. O’Leary is a WIC nutrition coordinator and a client services section manager, Washington State Department of Health, Olympia.