Advertisement

NOTICE: We are experiencing technical issues with Academy members trying to log into the JAND site using Academy member login credentials. We are working to resolve the issue as soon as possible. Alternatively, if you are an Academy member, you can access the JAND site by registering for an Elsevier account and claiming access using the links at the top of the JAND site. Email us at [email protected] for assistance. Thanks for your patience!

Administering a combination of online dietary assessment tools, the Automated Self-Administered 24-Hour Dietary Assessment Tool, and Diet History Questionnaire II, in a cohort of adults in Alberta's Tomorrow Project

Open AccessPublished:February 18, 2021DOI:https://doi.org/10.1016/j.jand.2021.01.014

      Abstract

      Background

      Evidence suggests that combining tools that gather short- and long-term dietary data may be the optimal approach for the assessment of diet–disease associations in epidemiologic studies. Online technology can reduce the associated burdens for researchers and participants, but feasibility must be demonstrated in real-world settings before wide-scale implementation.

      Objective

      The objective of this study was to determine the feasibility and acceptability of combining web-based tools (the Automated Self-Administered 24-hour Dietary Assessment Tool [ASA24-2016] and the past-year Diet History Questionnaire II [DHQ-II]) in a subset of participants in Alberta’s Tomorrow Project, a prospective cohort.

      Design

      For this feasibility study, invitations were mailed to 550 randomly selected individuals enrolled in Alberta’s Tomorrow Project. Consented participants (n = 331) were asked to complete a brief sociodemographic and health questionnaire, four ASA24-2016 recalls, the DHQ-II, and an evaluation survey.

      Participants/setting

      The study was conducted from March 2016 to December 2016 in Alberta, Canada. The majority of participants, mean age (SD) = 57.4 (9.8) years, were women (70.7%), urban residents (85.5%), and nonsmokers (95.7%).

      Main outcome measures

      Primary outcomes were number of ASA24-2016 recalls completed, response rate of DHQ-II completion, and time to complete each assessment.

      Statistical analyses

      The Wilcoxon signed rank sum test was used to assess differences in completion time.

      Results

      One-third (n = 102) of consenting participants did not complete any ASA24-2016 recalls. The primary reason to withdraw from the feasibility study was a lack of time. Among consenting participants, 51.9% (n = 172), 41.1% (n = 136), and 36.5% (n = 121) completed at least two ASA24-2016 recalls, the DHQ-II, and at least two ASA24-2016 recalls plus the DHQ-II, respectively. Median (25th to 75th percentile) completion times for participants who completed all recalls were 39 minutes (25 to 53 minutes) for the first ASA24-2016 recall and 60 minutes (40 to 90 minutes) for the DHQ-II.

      Conclusions

      Findings indicate combining multiple ASA24-2016 recalls and the DHQ-II is feasible in this subset of Alberta’s Tomorrow Project participants. However, optimal response rates may be contingent on providing participant support. Completion may also be sensitive to timing and frequency of recall administration.

      Keywords

      Research Question: What is the feasibility of administering two web-based dietary assessment tools in a subset of a longitudinal prospective cohort study?
      Key Findings: In a feasibility study of 331 consenting participants from Alberta’s Tomorrow Project, 52% (n = 172), 41% (n = 136), and 37% (n = 121) completed at least two Automated Self-Administered 24-hour Dietary Assessment Tool recalls, the Diet History Questionnaire II, and at least two Automated Self-Administered 24-hour Dietary Assessment Tool recalls plus the Diet History Questionnaire II, respectively. Optimal response rates may be contingent on providing participant support. Completion may also be sensitive to timing and frequency of recall administration.
      The influence of suboptimal dietary patterns on morbidity and mortality globally is increasing, contributing to the burden of chronic disease to a greater extent than well-known risks such as unsafe sex and alcohol, drug, and tobacco use.
      Global Panel on Agriculture and Food Systems for Nutrition
      Food systems and diets: facing the challenges of the 21st century.
      ,
      GBD 2017 Diet Collaborators
      Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017.
      As such, cancer prevention recommendations,
      World Cancer Research Fund/American Institute for Cancer Research
      Diet, nutrition, physical activity and cancer: a global perspective. continuous update project expert report.
      along with guidelines for other chronic diseases,
      • Eckel R.H.
      • Jakicic J.M.
      • Ard J.D.
      • et al.
      2013 AHA/ACC guideline on lifestyle management to reduce cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.
      ,
      • Ley S.H.
      • Hamdy O.
      • Mohan V.
      • Hu F.B.
      Prevention and management of type 2 diabetes: dietary components and nutritional strategies.
      continue to emphasize modification of dietary patterns to lower risk. An abundance of scientific evidence has identified associations between risk of chronic diseases and specific dietary components such as fruits and vegetables, dietary fiber, whole grains, nuts and seeds, and red/processed meats.
      World Cancer Research Fund/American Institute for Cancer Research
      Diet, nutrition, physical activity and cancer: a global perspective. continuous update project expert report.
      ,
      • Aune D.
      Plant foods, antioxidant biomarkers, and the risk of cardiovascular disease, cancer, and mortality: a review of the evidence.
      However, there is growing consensus that a focus on isolated foods or other dietary components may be misplaced given potential synergistic and antagonistic interactions among the range of constituents contributed by foods and beverages.
      World Cancer Research Fund/American Institute for Cancer Research
      Diet, nutrition, physical activity and cancer: a global perspective. continuous update project expert report.
      ,
      • Krebs-Smith S.M.
      • Subar A.F.
      • Reedy J.
      Examining dietary patterns in relation to chronic disease: matching measures and methods to questions of interest.
      • Reedy J.
      • Subar A.F.
      • George S.M.
      • Krebs-Smith S.M.
      Extending methods in dietary patterns research.
      US Depts of Health and Human Services and Agriculture
      2015-2020 Dietary Guidelines for Americans. 8th Edition.
      Indeed, despite decades of research, much remains to be learned about the influence of overall dietary patterns on health and disease risk.
      • Mayne S.T.
      • Playdon M.C.
      • Rock C.L.
      Diet, nutrition, and cancer: past, present and future.
      Although cohort studies can provide valuable opportunities to advance this area of inquiry, high-quality comprehensive dietary intake data are critical to characterizing dietary patterns.
      • Reedy J.
      • Subar A.F.
      • George S.M.
      • Krebs-Smith S.M.
      Extending methods in dietary patterns research.
      In large epidemiological studies, food frequency questionnaires (FFQ) are commonly used to capture habitual dietary intakes.
      • Prentice R.L.
      • Caan B.
      • Chlebowski R.T.
      • et al.
      Low-fat dietary pattern and risk of invasive breast cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial.
      • Thompson F.E.
      • Kipnis V.
      • Midthune D.
      • et al.
      Performance of a food-frequency questionnaire in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study.
      • Hu F.B.
      • Rimm E.
      • Smith-Warner S.A.
      • et al.
      Reproducibility and validity of dietary patterns assessed with a food-frequency questionnaire.
      FFQs typically use predefined questions on portion size and consumption frequency of specified food/drink items over a finite period of time to assess typical diet.
      • Thompson F.E.
      • Kirkpatrick S.I.
      • Subar A.F.
      • et al.
      The National Cancer Institute's Dietary Assessment Primer: a resource for diet research.
      FFQs can be self-administered by participants and are therefore relatively simple, cost-effective, and logistically feasible tools for studies with large sample sizes.
      • Kipnis V.
      • Subar A.F.
      • Midthune D.
      • et al.
      Structure of dietary measurement error: results of the OPEN biomarker study.
      ,
      • Boeing H.
      Nutritional epidemiology: new perspectives for understanding the diet-disease relationship?.
      However, it is now known that data from FFQs are affected by systematic error (bias) to a greater extent than data from short-term tools, such as 24-hour recalls (24HRs).
      • Freedman L.S.
      • Midthune D.
      • Carroll R.J.
      • et al.
      Adjustments to improve the estimation of usual dietary intake distributions in the population.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      Because bias attenuates risk estimates, the strength of observed associations between diet and health outcomes explored using FFQs alone may be biased toward the null.
      • Kipnis V.
      • Subar A.F.
      • Midthune D.
      • et al.
      Structure of dietary measurement error: results of the OPEN biomarker study.
      Bias also reduces statistical power, increasing the chance of false negative results,
      • Boeing H.
      Nutritional epidemiology: new perspectives for understanding the diet-disease relationship?.
      thereby necessitating very large cohort studies to assess relationships between diet and disease.
      Although traditionally not used in large-scale epidemiological studies because of burden associated with their administration, 24HRs have been shown to capture dietary intake with less bias than FFQs.
      • Olafsdottir A.S.
      • Thorsdottir I.
      • Gunnarsdottir I.
      • Thorgeirsdottir H.
      • Steingrimsdottir L.
      Comparison of women's diet assessed by FFQs and 24-hour recalls with and without underreporters: associations with biomarkers.
      • Schatzkin A.
      • Kipnis V.
      • Carroll R.J.
      • et al.
      A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      When compared with true intake assessed using unbiased recovery biomarkers, self-reported estimates of energy, protein, sodium, and potassium intakes from 24HR had higher correlations with true intake than those from FFQs.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      ,
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      ,
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for potassium and sodium intake.
      ,
      • Neuhouser M.L.
      • Tinker L.
      • Shaw P.A.
      • et al.
      Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women's Health Initiative.
      However, 24HRs do have limitations; a single 24HR is not representative of typical diet and multiple 24HRs are therefore needed to allow for adjustment for day-to-day variation in intake.
      • Zimmerman T.P.
      • Hull S.G.
      • McNutt S.
      • et al.
      Challenges in converting an interviewer-administered food probe database to self-administration in the National Cancer Institute Automated Self-administered 24-Hour Recall (ASA24).
      In addition, the number of 24HRs needed varies considerably depending on the dietary exposure of interest. For example, accurately estimating episodically consumed dietary components (such as beta carotene) requires a greater number of 24HRs than does estimating frequently consumed components (such as carbohydrates).
      • Stote K.S.
      • Radecki S.V.
      • Moshfegh A.J.
      • Ingwersen L.A.
      • Baer D.J.
      The number of 24 h dietary recalls using the US Department of Agriculture's automated multiple-pass method required to estimate nutrient intake in overweight and obese adults.
      Overall, with multiple 24HRs administrations, the strength of correlations with true intake is improved.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      This is true up to a certain point, at which correlations plateau and additional 24HRs have marginal added value.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      ,
      • Freedman L.S.
      • Midthune D.
      • Arab L.
      • et al.
      Combining a food frequency questionnaire with 24-hour recalls to increase the precision of estimation of usual dietary intakes-evidence from the Validation Studies Pooling Project.
      To overcome the limitations in each type of assessment tool, it has been suggested that the administration of multiple 24HRs in combination with FFQs may be an optimal approach in large observational cohorts.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      In studies that rely upon an FFQ as the main dietary assessment tool, measurement error can be reduced by calibrating the FFQ data to multiple 24HRs administered within a subsample.
      • Freedman L.S.
      • Schatzkin A.
      • Midthune D.
      • Kipnis V.
      Dealing with dietary measurement error in nutritional cohort studies.
      ,
      • Freedman L.S.
      • Commins J.M.
      • Willett W.
      • et al.
      Evaluation of the 24-hour recall as a reference instrument for calibrating other self-report instruments in nutritional cohort studies: evidence from the Validation Studies Pooling Project.
      Indeed, the attenuation of risk estimates observed from an FFQ has been shown to be improved by adjustment using 24HR data collected from a subset of participants.
      • Freedman L.S.
      • Commins J.M.
      • Willett W.
      • et al.
      Evaluation of the 24-hour recall as a reference instrument for calibrating other self-report instruments in nutritional cohort studies: evidence from the Validation Studies Pooling Project.
      Alternatively, multiple 24HRs can be administered to the entire cohort over a period such as a year, with an FFQ administered to provide insights into episodically consumed items.
      • Thompson F.E.
      • Kirkpatrick S.I.
      • Subar A.F.
      • et al.
      The National Cancer Institute's Dietary Assessment Primer: a resource for diet research.
      The combined dietary assessment approach is becoming increasingly viable for cohort studies due to technological innovations that have made it possible to administer 24HRs with greatly reduced burden and cost.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      ,
      • Kirkpatrick S.I.
      • Gilsing A.M.
      • Hobin E.
      • et al.
      Lessons from studies to evaluate an online 24-hour recall for use with children and adults in Canada.
      For example, the Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24) is a web-based system that enables the capture of dietary intake over the past 24 hours.
      • Subar A.F.
      • Kirkpatrick S.I.
      • Mittl B.
      • et al.
      The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute.
      ASA24 eliminates the need for interviewers and coders
      • Subar A.F.
      • Kirkpatrick S.I.
      • Mittl B.
      • et al.
      The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute.
      and has been shown to capture true intake among adults (ascertained through observation) with similar accuracy to interviewer-administered recalls.
      • Kirkpatrick S.I.
      • Subar A.F.
      • Douglass D.
      • et al.
      Performance of the Automated Self-Administered 24-hour Recall relative to a measure of true intakes and to an interviewer-administered 24-h recall.
      ,
      • Kirkpatrick S.I.
      • Guenther P.M.
      • Douglass D.
      • et al.
      The provision of assistance does not substantially impact the accuracy of 24-hour dietary recalls completed using the Automated Self-Administered 24-H Dietary Assessment Tool among women with low incomes.
      ASA24 has also been shown to be feasible for use with community-living adults,
      • Thompson F.E.
      • Dixit-Joshi S.
      • Potischman N.
      • et al.
      Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls in 3 diverse integrated health systems.
      suggesting that implementation in cohort studies should be feasible. However, although the premise of using multiple dietary tools has sound arguments from a theoretical perspective and evaluations of ASA24 on its own are promising, evaluating the administration of multiple ASA24 recalls along with an FFQ in real-world settings is required to examine feasibility and acceptability.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      In this study, the feasibility and acceptability of administering multiple ASA24 recalls along with a web-based past-year FFQ (Diet History Questionnaire II [DHQ-II]) in a subsample drawn from an established cohort of adults (Alberta’s Tomorrow Project [ATP]) was assessed. The specific objectives were to determine completion rates for up to four administrations of ASA24 and a single administration of the DHQ-II over 4 months, elucidate reasons for noncompletion, and explore sociodemographic and health characteristics of participants who complete multiple ASA24 recalls and the DHQ-II.

      Methods

      Study Population

      ATP is a longitudinal cohort of adults in Alberta, Canada, initiated in 2000 to provide a research platform to facilitate studies on the etiology of cancer and chronic disease.
      • Bryant H.
      • Robson P.J.
      • Ullman R.
      • Friedenreich C.
      • Dawe U.
      Population-based cohort development in Alberta, Canada: a feasibility study.
      ,
      • Robson P.J.
      • Solbak N.M.
      • Haig T.R.
      • et al.
      Design, methods and demographics from phase I of Alberta's Tomorrow Project cohort: a prospective cohort profile.
      From 2000 to 2015, ∼55,000 Albertans aged 35 to 69 years, with no previous history of cancer other than nonmelanoma skin cancer, were enrolled. Recruitment, enrollment, and data collection methods for the cohort are described in detail elsewhere.
      • Bryant H.
      • Robson P.J.
      • Ullman R.
      • Friedenreich C.
      • Dawe U.
      Population-based cohort development in Alberta, Canada: a feasibility study.
      ,
      • Robson P.J.
      • Solbak N.M.
      • Haig T.R.
      • et al.
      Design, methods and demographics from phase I of Alberta's Tomorrow Project cohort: a prospective cohort profile.
      ATP was approved by the former Alberta Cancer Board’s Research Ethics Committee and the University of Calgary Conjoint Health Research Ethics Board (recruitment and baseline data collection). The current feasibility study is a substudy of ATP and was approved by the Health Research Ethics Board of Alberta-Cancer Committee (current analysis).

      Study Design

      Recruitment and data collection for this feasibility study took place from March to December 2016. The feasibility study design is summarized in Figure 1.
      Figure thumbnail gr1
      Figure 1Study design to test the feasibility and acceptability of online dietary assessment tools in Alberta’s Tomorrow Project (ATP). aASA24-2016 = Automated Self-Administered 24-hour Dietary Assessment Tool. bCompletion rates were calculated using the number of invited participants at each step in the study. cDHQ-II = Diet History Questionnaire II.
      The sample size for the feasibility study was informed by the findings of the Validation Studies Pooling Project, a collaboration among researchers from five population-based studies with measures of self-reported dietary intake along with recovery biomarkers.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      Sample size considerations were based on energy intake estimated from 24HRs and FFQs. Compared with doubly labeled water, the intraclass correlation coefficients (ICCs) for energy intake, based on three 24HR recalls, were 0.28 and 0.34 for men and women, respectively.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      Therefore, it was predicted that the observed ICC derived from three 24HR administrations should be close to the midpoint of 0.31. Based on the method described by Bonett,
      • Bonett D.G.
      Sample size requirements for estimating intraclass correlations with desired precision.
      to observe an ICC of 0.31 with a 95% CI width of 0.20, 320 participants are required. However, some attrition or nonresponse was expected based on the results of a study comparing the ASA24 to the interviewer-administered Automated Multiple-Pass Method (AMPM).
      • Thompson F.E.
      • Dixit-Joshi S.
      • Potischman N.
      • et al.
      Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls in 3 diverse integrated health systems.
      Participants in that study completed two recalls (one each of ASA24 and AMPM in different order, two ASA24 recalls, or two AMPM recalls).
      • Thompson F.E.
      • Dixit-Joshi S.
      • Potischman N.
      • et al.
      Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls in 3 diverse integrated health systems.
      Among those who completed two ASA24 recalls, the attrition rate (ie, completed the first recall but not the second) was 5.7%. Those who completed an AMPM recall followed by ASA24 had an attrition rate of 14.7%,
      • Thompson F.E.
      • Dixit-Joshi S.
      • Potischman N.
      • et al.
      Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls in 3 diverse integrated health systems.
      which was the highest amongst the four groups. These findings suggested attrition rates could be higher in a study requiring four ASA24 recalls plus the DHQ-II. An estimated attrition rate of 20% was thus assumed, leading to a target sample size of 384 (320 × 1.2).
      A previous feasibility study
      • Lo Siou G.
      • Csizmadi I.
      • Boucher B.A.
      • et al.
      The comparative reliability and feasibility of the past-year Canadian Diet History Questionnaire II: comparison of the paper and web versions.
      that compared paper vs web-versions of the Canadian Diet History Questionnaire-II within ATP had an average invitation response rate of approximately 70%. Thus, in the current study, paper invitation packages were mailed to a random sample (selected by a random number generator) of 550 ATP participants (550 × 0.7 = 385) with a current E-mail address (used as a proxy for access to high-speed Internet, necessary to complete the web-based tools) and who had not been invited to the previous Canadian Diet History Questionnaire-II feasibility study.
      • Lo Siou G.
      • Csizmadi I.
      • Boucher B.A.
      • et al.
      The comparative reliability and feasibility of the past-year Canadian Diet History Questionnaire II: comparison of the paper and web versions.
      Invitation packages contained letters describing the study and consent forms, which were to be returned by mail in a postage-paid envelope. Participants were not provided with any monetary incentives to enroll in or complete the feasibility study.
      Participants were considered enrolled if they returned a signed consent form. Eligible participants who had not responded to the invitation package received reminders by E-mail at 3 and 6 weeks following the initial invitation, and one telephone call was made to nonresponders at 9 weeks to encourage study participation, after which no further recruitment attempts were made.
      Of 550 participants invited, 144 did not respond and 75 declined to participate (Figure 1). A final sample of 331 consented participants (60.2% response rate) participated in the study and were asked to complete a brief sociodemographic and health questionnaire, four ASA24 recalls, the DHQ-II, and an evaluation survey at the end of the study. A help desk was maintained and participants could call or E-mail for assistance with ASA24 and the DHQ-II during business hours (ie, Monday through Friday, 8 am to 4 pm Mountain time).

      Sociodemographic and Health Questionnaire

      A brief paper-based questionnaire to capture up-to-date information on sociodemographic characteristics (eg, marital status, education, and working status), tobacco use, and self-reported anthropometric measures (eg, standing height, body weight, and waist and hip circumferences) was mailed to consented participants. A 1-page summary of instructions for taking the anthropometric measures was included. Also included with the questionnaire were instructions and login credentials (study ID, participant ID, and passwords) for the online tools, ASA24, and DHQ-II. Requests to complete each dietary tool were emailed subsequently at fixed intervals during the study administration (login information was not resent with each request).

      ASA24-2016

      ASA24 is a web-based tool modeled on the AMPM and developed by the US National Cancer Institute (NCI).
      • Subar A.F.
      • Kirkpatrick S.I.
      • Mittl B.
      • et al.
      The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute.
      The interface guides participants through multiple passes to complete the recall, with prompts to report all foods and beverages consumed in the previous 24-hour period. Participants are first prompted to report all eating occasions and use a search tool to select foods and beverages consumed at each occasion. Next, participants are asked about preparation methods and portion sizes, with images used to enhance the accuracy of portion size estimation.
      • Kirkpatrick S.I.
      • Potischman N.
      • Dodd K.W.
      • et al.
      The use of digital images in 24-hour recalls may lead to less misestimation of portion size compared with traditional interviewer-administered recalls.
      At the time of the feasibility study, the 2016 Canadian version of ASA24 was not yet available and thus, the 2016 US version of ASA24 was used to reflect the interface that would be available in Canada in the near future. Although there are differences in nutrient databases between the countries, the interfaces are identical and thus this did not influence the capacity to assess feasibility for the purposes of informing the use of ASA24-Canada-2016 in ATP or other Canadian cohorts. The optional Supplements module was activated, but additional modules, including Location, Ate with, Source, and TV/Computer use during the meal/snack, were not activated to mitigate participant burden.
      Participants were requested to complete four ASA24 recalls over a 12-week time period. Each request was unannounced, such that although participants knew they would be asked to complete ASA24-2016 recalls and the DHQ-II, they did not know when. Once a recall was started, participants had 32 hours to complete it; otherwise, the recall was classified as incomplete. Participants who had not completed a recall (regardless of whether it was started) received up to three additional requests to complete each recall at 2, 3, and 5 business days after the initial request. Participants who did not start, or who started but did not complete a given recall continued to receive invites for subsequent recalls as well as for the DHQ-II.
      The ASA24 researcher website
      National Cancer Institute
      Automated Self-Administered 24 Hour Dietary Assessment Tool (ASA24) researcher website. Version 2.9.2.0.
      was used to track the status of participant recalls and download analysis files, which are coded using the Food and Nutrient Database for Dietary Surveys.
      US Dept of Agriculture
      USDA Food and Nutrient Database for Dietary Studies 2011-2012.
      Data on completion times for each ASA24-2016 recall were also downloaded from the researcher website.

      DHQ-II

      Participants were asked to complete the web-based DHQ-II 3 weeks following the final reminder to complete the final recall. The DHQ-II is a widely used past-year FFQ, also developed by the US NCI, that queries 134 foods and beverages and includes eight dietary supplement questions.
      National Institutes of Health, Epidemiology and Genomics Research Program
      Diet History Questionnaire - version 2.0.
      Identical paper- and web-based versions of DHQ-II are available from the NCI website.
      National Institutes of Health, Epidemiology and Genomics Research Program
      Diet History Questionnaire - version 2.0.
      The US version of the DHQ-II was chosen to match as closely as possible the 24HR data collected from ASA24-2016 in terms of the nutrient database. The web-based DHQ-II version integrates automated skip patterns that ensure each question is completed before advancing to the next. Two E-mail reminders to complete the DHQ-II were sent to participants at 2 and 3 weeks after the initial request. DHQ-II questionnaires that were started but not completed within 3 weeks were classified as incomplete.

      Evaluation Survey

      Three weeks after DHQ-II invitations were issued, participants were mailed a paper-based evaluation survey, which was completed and returned by 164 participants (59.0% response rate). Three participants completed the evaluation survey yet completed zero ASA24-2016 recalls and were excluded from analysis on evaluation survey findings. The survey queried participants’ self-reported time to complete the ASA24-2016 recalls and DHQ-II, ability to complete ASA24-2016 recalls on first request (yes/no), and factors that prevented completion. Questions from the System Usability Scale (SUS),
      • Bangor A.
      • Kortum P.
      • Miller J.
      Determining what individual SUS scores mean: adding an adjective rate scale.
      ,
      • Lewis J.
      • Sauro J.
      The factor structure of the system usability scale.
      were also included. The SUS is designed to evaluate the usability of a variety of interfaces such as websites, voice response systems, and television applications. Scores range from 0 to 100, with higher scores indicating greater usability. Usability questions focused on ASA24 because the DHQ-II was evaluated in a prior study of ATP participants.
      • Lo Siou G.
      • Csizmadi I.
      • Boucher B.A.
      • et al.
      The comparative reliability and feasibility of the past-year Canadian Diet History Questionnaire II: comparison of the paper and web versions.
      Further, the current study required participants to complete up to four ASA24-2016 recalls within 3 months. However, if the protocol were expanded to the full ATP cohort, it is likely the recalls would be administered over 1 calendar year, consistent with recommendations from a modeling study, which considered numerous scenarios.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      Therefore, participants were asked how many ASA24-2016 recalls they would be willing to complete over 1 year.
      Additional details collected included how participants accessed the online tools, including the device, operating system type and version, Internet browser, and speed of Internet connection. An open text field was provided for participants to list comments and suggestions related to their experience with the web-based dietary tools.

      Statistical Analysis

      Descriptive statistics (means [standard deviation], medians [25th to 75th percentiles]) were generated for continuous variables, and frequencies and percentages were generated for categorical variables. The Wilcoxon signed rank sum test was used to assess differences in completion time for the first and fourth ASA24-2016 recalls among participants who completed all four and to compare SUS scores among participants who completed only one ASA24-2016 recall and two or more ASA24-2016 recalls. Outliers for completion times were identified based on fifth (2 minutes) and 95th (141 minutes) percentiles and excluded from analysis; this was done because it was assumed those below the fifth percentile were incomplete and those above the 95th percentile may not have been completed in one sitting without breaks.
      The number and reason for contacts (telephone call or E-mail message) initiated by participants to the help desk were collected using an in-house tracking program and grouped into six mutually exclusive categories (ie, password issue, Internet/technical issue, away/busy and missed ASA24-2016/DHQ-II request, withdrawal from the substudy due to the need to complete ASA24-2016, withdrawal due to other reason, and other). All analyses were conducted using SAS Enterprise Guide,

      SAS Enterprise Guide [computer program]. Version 7.13. Cary, NC : SAS Institute Inc. 2018.

      and the criterion for statistical significance was set as α < .05 (two-tailed).

      Results

      The majority of respondents were women, urban residents, well educated, married or living with a partner, and current nonsmokers (Table 1). The characteristics of participants who completed the health questionnaire (N = 256) stratified by the number of dietary assessments completed are summarized in Table 1, whereas response rates for each tool and administration are shown in Figure 1.
      Table 1Sociodemographic characteristics collected from a health questionnaire of Alberta’s Tomorrow Project participants (N = 256) consented to a feasibility study of online dietary assessment tools, stratified by the number of Automated Self-Administered 24-hour Dietary Assessment Tool 2016 (ASA24-2016) recalls completed
      CharacteristicTotal No. of ASA24-2016 Recalls Completed≥ 2 ASA24-2016 and DHQ-II
      DHQ-II = Diet History Questionnaire II.
      Recalls Completed
      Overall
      Seventy-five participants consented to the study but did not complete a health questionnaire.
      01234
      (N = 256)(n = 51)(n = 44)(n = 52)(n = 54)(n = 55)(n = 115)
      n (%)
      Column percentages.
      Sex
      Men75 (29.3)21 (41.2)13 (29.6)10 (19.2)13 (24.1)18 (32.7)30 (26.1)
      Women181 (70.7)30 (58.8)31 (70.5)42 (80.8)41 (75.9)37 (67.3)85 (73.9)
      Age (y)
      35-4430 (11.7)4 (7.8)4 (9.1)10 (19.2)7 (13.0)5 (9.1)14 (12.2)
      45-5463 (24.6)13 (25.5)15 (34.1)15 (28.8)8 (14.8)12 (21.8)19 (16.5)
      55-6492 (35.9)13 (25.5)19 (43.2)18 (34.6)21 (38.9)21 (38.2)45 (39.1)
      ≥ 6571 (27.7)21 (41.2)6 (13.6)9 (17.3)18 (33.3)17 (30.9)37 (32.2)
      BMI
      BMI = body mass index; calculated based on self-reported standing height and body weight.
      Missing data (n = 1): 50 responses for this question were collected from the no recalls category.
      18.5-24.995 (37.1)15 (29.4)17 (38.6)22 (42.3)24 (44.4)17 (30.9)49 (42.6)
      25.0-29.9107 (41.8)20 (39.2)18 (40.9)21 (40.4)22 (40.7)26 (47.3)48 (41.7)
      ≥ 30.053 (20.7)15 (29.4)9 (20.4)9 (17.3)8 (14.8)12 (21.8)18 (15.7)
      Geographic location
      Geographic location was determined using postal codes (forward sortation area), where “0" as the second character of the first 3 digits of the postal code indicates rural residence.
      Urban219 (85.5)44 (86.3)37 (84.1)41 (78.8)49 (90.7)48 (87.2)99 (86.0)
      Rural37 (14.4)7 (13.7)7 (15.9)11 (21.2)5 (9.3)7 (12.7)16 (13.9)
      Education
      High school or less39 (15.2)10 (19.6)4 (9.1)8 (15.4)6 (11.1)11 (20.0)20 (17.4)
      Trade/technical school, university diploma/certificate101 (39.4)22 (43.1)16 (36.4)19 (36.5)23 (42.6)21 (38.2)44 (38.3)
      Bachelor’s or graduate degree116 (45.3)19 (37.3)24 (54.6)25 (48.1)25 (46.3)23 (41.8)51 (44.4)
      Marital status
      Married or living with partner204 (79.7)41 (80.4)37 (84.1)35 (67.3)46 (85.2)45 (81.8)94 (81.7)
      Divorced/separated/ widowed35 (13.7)7 (13.7)6 (13.6)10 (19.2)6 (11.1)6 (10.9)14 (12.2)
      Single/never married17 (6.6)3 (5.9)1 (2.3)7 (13.5)2 (3.7)4 (7.3)7 (6.1)
      Working status
      Working status question asked participants to choose all that apply.
      Full time113 (44.1)22 (43.1)27 (61.4)29 (55.8)16 (29.6)19 (34.6)37 (32.2)
      Part time46 (18.0)7 (13.7)5 (11.4)8 (15.4)15 (27.8)11 (20.0)29 (25.2)
      Retired81 (31.6)19 (37.3)8 (18.2)11 (21.2)20 (37.0)23 (41.8)43 (37.4)
      Other
      Other” includes participants who were looking after home and/or family, unable to work because of sickness or disability, unemployed, doing unpaid or voluntary work, and student.
      36 (14.1)7 (13.7)5 (11.4)5 (9.6)9 (16.7)10 (18.2)21 (18.3)
      Current smoking status
      Missing data (n = 1): 51 responses for this question were collected from the two recalls category.
      Daily/occasional smoker10 (4.0)3 (5.9)2 (4.5)4 (7.7)1 (1.8)0.01 (0.9)
      Nonsmoker245 (95.7)48 (94.1)42 (95.4)47 (90.4)53 (98.2)55 (100.0)114 (99.1)
      a Seventy-five participants consented to the study but did not complete a health questionnaire.
      b DHQ-II = Diet History Questionnaire II.
      c Column percentages.
      d BMI = body mass index; calculated based on self-reported standing height and body weight.
      e Missing data (n = 1): 50 responses for this question were collected from the no recalls category.
      f Geographic location was determined using postal codes (forward sortation area), where “0" as the second character of the first 3 digits of the postal code indicates rural residence.
      g Working status question asked participants to choose all that apply.
      h Other” includes participants who were looking after home and/or family, unable to work because of sickness or disability, unemployed, doing unpaid or voluntary work, and student.
      i Missing data (n = 1): 51 responses for this question were collected from the two recalls category.
      Forty percent (n = 132) completed at least one ASA24-2016 recall and the DHQ-II, whereas 36.5% (n = 121), 29.9% (n = 99), and 16.6% (n = 55) completed at least two, three, or all four ASA24-2016 recalls, respectively, in addition to the DHQ-II (Figure 2). Just more than half of consenting participants (n = 172) completed at least two ASA24-2016 recalls and 41.1% (n = 136) completed the DHQ-II (Table 2). One-third (n = 102) of consenting participants did not complete any ASA24-2016 recalls. Irrespective of DHQ-II completion, about one in five (60 of 331) completed four ASA24-2016 recalls. Overall, approximately 70% completed at least one administration of either of the two dietary assessment tools. Relative to completion of one or no ASA24-2016 recalls, larger proportions of participants who completed at least two recalls and the DHQ-II were women, working part-time or retired, current nonsmokers, and had a body mass index in the normal range (Table 1).
      Figure thumbnail gr2
      Figure 2The percentage of consented participants (n = 331) completing Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24-2016) recalls and Diet History Questionnaire II (DHQ-II) in a feasibility study from the Alberta’s Tomorrow Project. aWhite bars indicate the percentage of participants who completed a total of zero to all four ASA24-2016 recalls. bBlack bars indicate the percentage of participants within each ASA24-2016 recall group who also completed the DHQ-II. cA small number (n = 4) participants completed zero ASA24-2016 recalls, but completed the DHQ-II.
      Table 2Number of participant contacts to the help desk during a feasibility study of online dietary assessment tools in Alberta’s Tomorrow Project stratified by the number of Automated Self-Administered 24-hour Dietary Assessment Tool 2016 (ASA24-2016) recalls or Diet History Questionnaire II (DHQ-II) completed
      No. of contacts
      Contact from a participant was an incoming telephone call or E-mail to the study center.
      Total No. of ASA24-2016 Recalls CompletedDHQ-II Recalls Completed≥ 2 ASA24-2016 and DHQ-II Recalls Completed
      01234(n = 136)
      (n = 102)(n = 57)(n = 56)(n = 56)(n = 60)(n = 121)
      n (%)
      049 (48.0)17 (29.8)14 (25.0)25 (44.6)26 (43.3)114 (83.8)48 (39.7)
      127 (26.5)19 (33.3)17 (30.4)11 (19.6)18 (30.0)13 (9.6)29 (24.0)
      214 (13.7)10 (17.5)13 (23.2)9 (16.1)9 (15.0)8 (5.9)18 (14.9)
      ≥ 312 (11.8)11 (19.3)12 (21.4)11 (19.6)7 (11.7)1 (0.7)26 (21.5)
      a Contact from a participant was an incoming telephone call or E-mail to the study center.
      The majority of participants were not able to complete an ASA24-2016 recall on the first day the invitation was sent, as indicated by both researcher website data (Table 3) and the evaluation survey (Figure 3, Panel A). “Being too busy” was the most common reason given for not being able to complete a recall on the first request (Figure 3, Panel B). Likewise, lack of time was the primary reason to withdraw from the feasibility study. The proportion of participants who did not require a reminder to complete an ASA24-2016 recall increased from 8.2% at Recall 1 to 18.4%, 17.4%, and 18.8% for Recalls 2, 3, and 4, respectively, and was 24.6% for the DHQ-II. In addition, the percentage of incomplete ASA24-2016 recalls (ie, started but not completed) declined from Recall 1 (8.2%) to Recall 4 (4.5%) (Table 3). Among participants who completed all four recalls (n = 43, excluding 17 outliers) (Figure 4), ASA24-2016 completion times were median (25th to 75th percentile) of 39 minutes (25 to 53 minutes) and 36 minutes (27 to 61 minutes) for recalls 1 and 4, respectively (P = 0.28). DHQ-II completion times were median (25th to 75th percentile) of 60 minutes (40 to 90 minutes).
      Table 3Reminders required and completion rate of each Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24-2016) recall and Diet History Questionnaire II (DHQ-II) in a feasibility study from Alberta’s Tomorrow Project
      VariableASA24-2016 RecallDHQ-II (n= 284)
      Recall 1 (n = 331)Recall 2 (n = 315)Recall 3 (n = 304)Recall 4 (n = 292)
      n (%)
      No. of reminders
      E-mail reminders were sent to participants at 2, 3, and 5 business days after the initial E-mail request for ASA24-2016 recalls. E-mail reminders were sent to participants at 2 and 4 weeks after the initial request for the DHQ-II.
      027 (8.2)58 (18.4)53 (17.4)55 (18.8)70 (24.6)
      135 (10.6)33 (10.5)32 (10.5)24 (8.2)36 (12.7)
      21 (0.3)38 (12.1)31 (10.2)29 (9.9)30 (10.6)
      347 (14.2)37 (11.7)38 (12.5)39 (13.4)
      Completion rate
      Sum of participants who completed a recall with zero, one, two, or three reminders.
      110 (33.2)166 (52.7)154 (50.6)147 (50.3)136 (47.9)
      No response182 (55.0)
      Four participants did not have a valid E-mail address on their Alberta’s Tomorrow Project participant file and were classified under “no response.”
      111 (35.2)129 (42.4)124 (42.5)131 (46.1)
      Incomplete
      Participants who started an ASA24-2016 recall but did not submit before the 32-hour time limit had expired were categorized as “incomplete” for that recall. There was no time cutoff for completing the DHQ-II. From the researcher website, an “in-progress” DHQ-II that was started by the participant but not submitted by the time of evaluation survey mailout was considered incomplete.
      27 (8.2)27 (8.6)9 (3.0)13 (4.5)11 (3.9)
      Withdrawal12 (3.6)11 (3.5)12 (3.9)8 (2.7)6 (2.1)
      a E-mail reminders were sent to participants at 2, 3, and 5 business days after the initial E-mail request for ASA24-2016 recalls. E-mail reminders were sent to participants at 2 and 4 weeks after the initial request for the DHQ-II.
      b Sum of participants who completed a recall with zero, one, two, or three reminders.
      c Four participants did not have a valid E-mail address on their Alberta’s Tomorrow Project participant file and were classified under “no response.”
      d Participants who started an ASA24-2016 recall but did not submit before the 32-hour time limit had expired were categorized as “incomplete” for that recall. There was no time cutoff for completing the DHQ-II. From the researcher website, an “in-progress” DHQ-II that was started by the participant but not submitted by the time of evaluation survey mailout was considered incomplete.
      Figure thumbnail gr3
      Figure 3Panel A: The percentage of participants who completeda Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24-2016) recalls with only the initial invitation e-mail as a prompt (Initial request only) and participants who required at least one of 3 reminder e-mail messages to complete an ASA24-2016 recall (Needed ≥1 reminder e-mail) in a feasibility study from the Alberta’s Tomorrow Project. aOnly participants who completed at least one ASA24-2016 recall and an evaluation survey at the end of the study are reported (n = 161). bInitial request only: A participant completed an ASA24-2016 recall after the initial e-mail request and required no reminder e-mail messages to complete the recall. Reminder e-mails were sent at 2, 3, and 5 business days after initial e-mail request. cNeeded ≥1 reminder E-mail: Participant completed an ASA24-2016 recall but required a reminder e-mail. Panel B: Reasonsa provided by participantsb of a feasibility study from Alberta’s Tomorrow Project for the inability to complete an ASA24-2016 recall within 2 business days after the initial email request. aParticipants were asked to choose all that apply. bTotal number of reasons (n = 113) by 161 participants who completed an evaluation survey and at least one ASA24-2016 recall (n = 161).
      Figure thumbnail gr4
      Figure 4Boxplot of completion times for participants (n = 43) who completed all four Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24-2016) recalls in a feasibility study of online dietary assessment tools in Alberta’s Tomorrow Project. The middle line represents the median time for each ASA24-2016 recall, whereas upper and lower lines of the box represent the 75th (Q3) percentile and 25th (Q1) percentile completion times for each recall, respectively. Upper and lower whiskers represent the maximum and minimum completion times for each recall, respectively. Outliers were removed from the figure based on completion times outside the fifth percentile (2 minutes) and 95th (141 minutes) percentiles.
      The total number of contacts received from participants decreased as the study progressed, from 101 for the first recall to 23 for the fourth (Table 4). The most common reason for contacting the help desk was password issues, representing 44.0% of all contacts (Table 4). Smaller proportions of participants who completed three or four recalls (55.4% and 56.7%, respectively) contacted the help desk compared to participants who completed one or two recalls (70.2% and 75.0%, respectively) (Table 2).
      Table 4Number of contacts initiated by participants and reasons for contacting the help desk at each step of a feasibility study of online dietary assessment tools in Alberta’s Tomorrow Project participants
      Reason for contactTotal Contacts Initiated by Participants
      TotalASA24-2016
      ASA24 = Automated Self-Administered 24-hour Dietary Assessment Tool.
      Recall
      DHQ-II
      DHQ-II = Diet History Questionnaire II.
      Recall 1Recall 2Recall 3Recall 4
      (n = 268)(n = 101)(n = 56)(n = 40)(n = 23)(n = 48)
      n (%)
      Password issue118 (44.0)47 (46.5)22 (39.3)11 (27.5)9 (39.1)29 (60.4)
      Internet/technical issue
      Examples include Internet connection was lost during completion, login issues, Internet browser issue, getting an error message when trying to submit recall.
      41 (15.3)21 (20.8)8 (14.3)9 (22.5)2 (8.7)1 (2.1)
      Away/busy
      Participant informed the study center about being away, on vacation, or lack of time to complete a recall and asked for time extension.
      37 (13.8)10 (9.9)9 (16.1)9 (22.5)5 (21.7)4 (8.3)
      Withdrawal: ASA24
      Participant explicitly mentioned an aspect of the ASA24 as a reason for withdrawal such as website login issues or frustration finding items.
      11 (4.1)4 (4.0)3 (5.4)2 (5.0)1 (4.3)1 (2.1)
      Withdrawal: Other reason
      Participant mentioned lack of time, being busy, or did not provide a specific reason for withdrawal.
      16 (6.0)3 (3.0)3 (5.4)2 (5.0)3 (13.0)5 (10.4)
      Other
      Other includes contacts regarding the health questionnaire, general study questions, and updating of contact information.
      45 (16.8)16 (15.8)11 (19.6)7 (17.5)3 (13.0)8 (16.7)
      a ASA24 = Automated Self-Administered 24-hour Dietary Assessment Tool.
      b DHQ-II = Diet History Questionnaire II.
      c Examples include Internet connection was lost during completion, login issues, Internet browser issue, getting an error message when trying to submit recall.
      d Participant informed the study center about being away, on vacation, or lack of time to complete a recall and asked for time extension.
      e Participant explicitly mentioned an aspect of the ASA24 as a reason for withdrawal such as website login issues or frustration finding items.
      f Participant mentioned lack of time, being busy, or did not provide a specific reason for withdrawal.
      g Other includes contacts regarding the health questionnaire, general study questions, and updating of contact information.
      Participants who completed only one recall had significantly lower SUS scores (mean [SD] of 52.8 [22.4]) than participants who completed at least two recalls (mean [SD] of 62.7 [17.1]) (P = 0.03). Lower SUS scores were mainly driven by responses to questions pertaining to the complexity of ASA24, the ease of use, and the perception that ASA24 was cumbersome to complete (data not shown). The majority of participants (61%; n = 97) indicated willingness to complete three or fewer ASA24-2016 recalls over 1 year (Table 5). Average completion times for ASA24 recalls were similar between participants who were willing to complete 3 or fewer and 4 or more ASA24 recalls. Most participants used Internet Explorer or Google Chrome browsers. A higher proportion of participants willing to complete 3 or fewer ASA24 recalls used the MAC OS operating system (34.0%) compared with participants willing to complete 4 or more ASA24 recalls (14.8%) (Table 5).
      Table 5Online technology components of a feasibility study from Alberta’s Tomorrow Project in relation to the future willingness to complete Automated Self-Administered 24-hour Dietary Assessment Tool 2016 (ASA24-2016) recalls stratified by the number of recalls completed
      VariableTotalReported Future Willingness to Complete ASA24-2016 Recalls
      ≤3
      Participants were asked about their willingness to complete ASA24-2016 and the number of recalls they would be willing to complete in 1 year. They were given the options of 3, 4, 5, or 6 ASA24-2016 recalls in 1 year. “≤ 3” responses were derived from open text fields from participants who completed the evaluation survey at the end the feasibility study.
      ≥4
      No. of ASA24-2016 Recalls Completed
      1234Total1234
      (n = 97)(n = 21)(n = 22)(n = 25)(n = 29)(n = 61)(n = 6)(n = 13)(n = 20)(n = 22)
      mean (standard deviation)
      Average completion time (min)43.5 (23.2)42.2 (24.6)37.8 (15.1)51.8 (25.4)40.4 (23.9)44.1 (20.6)63.2 (32.2)30.0 (12.7)41.1 (12.0)49.3 (20.3)
      n (%)
      Internet browser
      Participants chose all that apply resulting in duplicate counts.
      Internet Explorer33 (34.0)10 (47.6)5 (22.7)8 (32.0)10 (34.5)25 (41.0)2 (33.3)3 (23.1)10 (50.0)10 (45.5)
      Firefox15 (15.0)4 (19.1)3 (13.6)4 (16.0)4 (13.8)10 (16.4)1 (16.7)2 (15.4)4 (20.0)3 (13.6)
      Chrome29 (30.0)2 (9.5)11 (50.0)8 (32.0)8 (27.6)25 (41.0)2 (33.3)7 (53.9)8 (40.0)8 (36.4)
      Safari24 (24.7)4 (19.1)3 (13.6)8 (32.0)9 (31.0)9 (14.8)1 (16.7)2 (15.4)3 (15.0)3 (13.6)
      Opera0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)
      Other1 (1.0)1 (4.8)0 (0.0)0 (0.0)0 (0.0)1 (1.6)0 (0.0)0 (0.0)1 (5.0)0 (0.0)
      Missing0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)1 (1.6)0 (0.0)1 (7.7)0 (0.0)0 (0.0)
      Connection speed
      Participants chose all that apply resulting in duplicate counts.
      Low
      Low-speed Internet connection includes dial-up, mobile 3G, and integrated services digital network.
      6 (6.0)3 (14.3)1 (4.6)1 (4.0)1 (3.5)6 (9.8)0 (0.0)4 (30.8)2 (10.0)0 (0.0)
      Medium
      Medium-speed Internet connection includes public Wi-Fi (eg, coffee shop or local library), mobile 4G Long-Term Evolution, and satellite.
      6 (6.0)1 (4.8)2 (9.1)2 (8.0)1 (3.5)6 (9.8)0 (0.0)4 (30.8)1 (5.0)1 (4.6)
      High
      High-speed Internet connection includes digital subscriber line and cable modem.
      67 (69.1)12 (57.1)14 (63.6)20 (80.0)21 (72.4)43 (70.5)5 (83.3)7 (53.9)15 (75.0)16 (72.7)
      Other7 (7.2)3 (14.3)1 (4.6)1 (4.0)2 (6.9)4 (6.6)0 (0.0)0 (0.0)3 (15.0)1 (4.6)
      Don't know11 (11.3)2 (9.5)4 (18.2)1 (4.0)4 (13.8)7 (11.5)1 (16.7)1 (7.7)0 (0.0)5 (22.7)
      Missing0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)
      Electronic device
      Participants chose all that apply resulting in duplicate counts.
      Personal computer78 (80.4)18 (85.7)18 (81.8)21 (84.0)21 (72.4)49 (80.3)4 (66.7)9 (69.2)16 (80.0)20 (90.9)
      Tablet11 (11.0)1 (4.8)2 (9.1)4 (16.0)4 (13.8)5 (8.2)1 (16.7)1 (7.7)1 (5.0)2 (9.1)
      Work computer8 (8.0)3 (14.3)2 (9.1)1 (4.0)2 (6.9)9 (14.8)1 (16.7)4 (30.8)3 (15.0)1 (4.6)
      Public computer0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)1 (1.6)0 (0.0)0 (0.0)1 (5.0)0 (0.0)
      Other3 (3.0)0 (0.0)1 (4.6)0 (0.0)2 (6.9)3 (4.9)0 (0.0)2 (15.4)1 (5.0)0 (0.0)
      Missing0 (0.0)0 (0.0)0 (0.0)0 (0.0)0 (0.0)1 (1.6)0 (0.0)1 (7.7)0 (0.0)0 (0.0)
      Operating system
      MAC OS33 (34.0)7 (33.3)8 (36.4)8 (32.0)10 (34.5)9 (14.8)1 (16.7)3 (23.1)3 (15.0)2 (9.1)
      Windows59 (60.8)14 (66.7)14 (63.6)15 (60.0)16 (55.2)48 (78.7)5 (83.3)8 (61.5)16 (80.0)19 (86.4)
      Other4 (4.0)0 (0.0)0 (0.0)2 (8.0)2 (6.9)1 (1.6)0 (0.0)0 (0.0)1 (5.0)0 (0.0)
      Missing1 (1.0)0 (0.0)0 (0.0)0 (0.0)1 (3.5)3 (4.9)0 (0.0)2 (15.4)0 (0.0)1 (4.6)
      a Participants were asked about their willingness to complete ASA24-2016 and the number of recalls they would be willing to complete in 1 year. They were given the options of 3, 4, 5, or 6 ASA24-2016 recalls in 1 year. “≤ 3” responses were derived from open text fields from participants who completed the evaluation survey at the end the feasibility study.
      b Participants chose all that apply resulting in duplicate counts.
      c Low-speed Internet connection includes dial-up, mobile 3G, and integrated services digital network.
      d Medium-speed Internet connection includes public Wi-Fi (eg, coffee shop or local library), mobile 4G Long-Term Evolution, and satellite.
      e High-speed Internet connection includes digital subscriber line and cable modem.

      Discussion

      The premise for combining dietary tools is to mitigate the errors inherent in each tool; therefore, more accurately capturing dietary intake,
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      ,
      • Subar A.F.
      • Freedman L.S.
      • Tooze J.A.
      • et al.
      Addressing current criticism regarding the value of self-report dietary data.
      which is beneficial to understanding the influence of total dietary patterns on disease risk. The development of web-based tools has reduced barriers previously posed by interviewer-administered 24HRs in terms of the costs and burden associated with collecting and coding data.
      • Subar A.F.
      • Kirkpatrick S.I.
      • Mittl B.
      • et al.
      The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute.
      ,
      • Kirkpatrick S.I.
      • Subar A.F.
      • Douglass D.
      • et al.
      Performance of the Automated Self-Administered 24-hour Recall relative to a measure of true intakes and to an interviewer-administered 24-h recall.
      However, technology is not a panacea and may introduce challenges, such as the need for participant computer literacy and the ability to navigate through multiple steps online with minimal to no assistance.
      • Kirkpatrick S.I.
      • Gilsing A.M.
      • Hobin E.
      • et al.
      Lessons from studies to evaluate an online 24-hour recall for use with children and adults in Canada.
      In addition, using multiple tools poses challenges due to the numerous steps and separate protocols that might be required to complete each tool on different platforms. In this study, almost half of participants completed each ASA24 and DHQ-II administration, but most did not complete all of the administrations. Thus, analytical methods are likely required to make use of intermittently missing data to provide a more realistic estimate of usual dietary intake.
      Recommendations for the number of 24HRs required to supplement FFQ data based on simulated data suggest that optimal gains in precision and power are obtained with four to six 24HR administrations.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      For most dietary components, predictive power in dietary data are readily apparent with two to four 24HRs.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      Further, data from the Validation Studies Pooling Project, which used recovery biomarkers to assess true intake, support the use of three or four 24HRs in addition to one FFQ to provide the most benefit for accurately assessing dietary intake.
      • Freedman L.S.
      • Midthune D.
      • Arab L.
      • et al.
      Combining a food frequency questionnaire with 24-hour recalls to increase the precision of estimation of usual dietary intakes-evidence from the Validation Studies Pooling Project.
      The results of the present feasibility study suggest it may not be realistic to expect a high proportion of participants to complete, at least within a compressed period of 4 months. However, it does appear feasible to collect at least two 24HRs and one FFQ over a relatively short period of time. Overall, seven in 10 participants completed at least one administration of one tool, similar to the response rate observed for prior follow-up data collections in this cohort.
      • Robson P.J.
      • Solbak N.M.
      • Haig T.R.
      • et al.
      Design, methods and demographics from phase I of Alberta's Tomorrow Project cohort: a prospective cohort profile.
      Other studies support the findings that the combined dietary tool approach recommended by Carroll and colleagues
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      can be implemented in large studies and cohorts. In the Interactive Diet and Activity Tracking in the American Association of Retired Persons study (n = 1,110 men and women aged 50 to 74 years), participants were asked to complete six ASA24 recalls and two FFQs over 1 calendar year and were compensated by financial incentives; 92% and 87% of men and women completed three or more recalls, respectively; and 81% and 85% of men and women completed the first FFQ.
      • Park Y.
      • Dodd K.W.
      • Kipnis V.
      • et al.
      Comparison of self-reported dietary intakes from the Automated Self-Administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers.
      The lower response rate observed in the current study could be due to the lack of monetary incentives and/or the request to complete four recalls over a shorter duration.
      Implementing a combined dietary assessment tool approach within a large population cohort requires careful planning and administration. This feasibility study has identified study protocol considerations that may improve participant and researcher experiences. Most of the participants who withdrew completed the sociodemographic and health questionnaire but no recalls. The key challenge in completing recalls was lack of time, which might relate to the implementation of the feasibility study during the summer months, a popular vacation time. Administering fewer recalls during these months in future studies might be a prudent way to ensure participants are available to complete dietary assessments. Although this may raise concerns related to capturing seasonal variation in dietary patterns, evidence from the United States suggests that such variation may be minimal.
      • Bernstein S.
      • Zambell K.
      • Amar M.J.
      • et al.
      Dietary intake patterns are consistent across seasons in a cohort of healthy adults in a metropolitan population.
      Also of note, many older Canadians are so-called snowbirds during the winter months, temporarily relocating to the United States or Mexico.
      • Coates K.S.
      • Healy R.
      • Morrison W.R.
      Tracking the snowbirds: seasonal migration from Canada to the U.S.A. and Mexico.
      Thus, there likely is no ideal time to implement dietary assessments. Spreading assessments over a 1-year time period, with multiple attempts allowed per administration, will likely provide the maximum opportunity for participants across a large sample to respond. Notably, complex and busy schedules throughout the year highlight an advantage of online dietary tools, which can be completed from any location in which the participant has high-speed Internet access, as the primary mode of data collection.
      For a study such as ATP that spans a wide geographical area (640,330 km2)
      Statistics Canada
      Alberta and Canada - Census profile. 2016 census.
      and has numerous participants outside of metropolitan areas, it is not feasible to implement in-person assistance. In this study, a help desk staffed by trained employees was available to participants by telephone and E-mail during office hours (8:00 am to 4:00 pm Mountain time on weekdays). The most common reason for contacting the help desk was password-related issues. Providing passwords to participants in close proximity to the requested completion of tools could help to mitigate this issue. In contrast, passwords were sent with the initial questionnaire but were not required until up to 4 months later in the case of the DHQ-II. Addressing other practical issues of this nature could help support the successful completion of online tools. Providing additional supports such as tutorials may lower attrition rates and encourage completion without reminders, increasing data completeness and lessening the burden on researchers.
      • Kirkpatrick S.I.
      • Gilsing A.M.
      • Hobin E.
      • et al.
      Lessons from studies to evaluate an online 24-hour recall for use with children and adults in Canada.
      ,
      • Kupis J.
      • Johnson S.
      • Hallihan G.
      • Olstad D.L.
      Assessing the usability of the Automated Self-Administered Dietary Assessment Tool (ASA24) among low-income adults.
      The NCI offers quick-start guides for ASA24, in addition to help guide documents on their website.
      National Cancer Institute
      Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24)-2016 help guide for respondents.
      Participants were provided with URLs to access these resources. The provision of freely accessible web-based tutorials on a study website could enable participants to obtain support at their convenience, viewing tutorials as many times as necessary to gain familiarity with the tasks involved in completing web-based tools. Conducting an introductory guided recall facilitated by study staff is another option, but in addition to the researcher burden, this may introduce reactivity bias to a greater extent than tutorials that can be viewed independently.
      Regardless of the supports offered, both dietary tools used required access to the Internet. Although several platforms such as computers, laptops, tablets, or mobile telephones can be used, some individuals may not have possessed adequate computer literacy to complete the assessments. Further, in rural or remote areas, access to high-speed Internet may be lacking, posing a barrier to completion.
      • Currie L.M.
      • Ronquillo C.
      • Dick T.
      Access to internet in rural and remote Canada.
      Thus, web-based administration may need to be supplemented with paper or other options, along with analytic strategies to combine data from different tools and modes of administration.
      The ability to accurately detect diet-disease associations is often limited by the accuracy of dietary data.
      • Boeing H.
      Nutritional epidemiology: new perspectives for understanding the diet-disease relationship?.
      ,
      • Schatzkin A.
      • Subar A.F.
      • Moore S.
      • et al.
      Observational epidemiologic studies of nutrition and cancer: the next generation (with better observation).
      Strategies to address this problem have included the use of regression calibration,
      • Freedman L.S.
      • Schatzkin A.
      • Midthune D.
      • Kipnis V.
      Dealing with dietary measurement error in nutritional cohort studies.
      which necessitates the collection of a reference measure, such as a recovery biomarker or 24HR data, in a subsample. The reference data are then used to reduce some of the error in the main instrument. In the current study, rather than using ASA24 as a reference tool to evaluate nutrient intake, the focus was on the feasibility of collecting 24HRs from as many participants as possible within the context of a cohort study such that the recalls could be used as the main instrument, with FFQs providing data on episodically-consumed dietary components. Complementing 24HRs with DHQ-II could leverage the strengths of short-term and long-term data, overcoming some challenges that have contributed to attenuation of observed diet-health relationships.
      • Carroll R.J.
      • Midthune D.
      • Subar A.F.
      • et al.
      Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
      ,
      • Subar A.F.
      • Freedman L.S.
      • Tooze J.A.
      • et al.
      Addressing current criticism regarding the value of self-report dietary data.
      ,
      • Conrad J.
      • Nothlings U.
      Innovative approaches to estimate individual usual dietary intake in large-scale epidemiological studies.
      To help support such approaches, there is a need for further development of analytic methods, particularly in cases in which missing data for some measures among some participants is almost a certainty.
      Several considerations should be borne in mind in interpreting the results of this study. The cohort is fairly well educated, thus, generalizability of response rates to other cohorts may be limited. A small mixed-method study has suggested some usability challenges with ASA24 among adults with low-incomes,
      • Kupis J.
      • Johnson S.
      • Hallihan G.
      • Olstad D.L.
      Assessing the usability of the Automated Self-Administered Dietary Assessment Tool (ASA24) among low-income adults.
      whereas a validation study showed ASA24 could be completed independently among a sample of 302 women with low incomes.
      • Kirkpatrick S.I.
      • Guenther P.M.
      • Douglass D.
      • et al.
      The provision of assistance does not substantially impact the accuracy of 24-hour dietary recalls completed using the Automated Self-Administered 24-H Dietary Assessment Tool among women with low incomes.
      Befire the feasibility study, ATP administered questionnaires almost exclusively by paper. A transition to web-based formats was deemed necessary to improve cost-effectiveness. This feasibility study thus not only assessed the specific dietary assessment tools but also general receptiveness to the use of web-based tools in the cohort. A learning effect was evident based on lower completion times and number of contacts made to the help desk as the study progressed. Moreover, some participants had completed the paper-based Canadian DHQ-I during enrollment into ATP, which might have facilitated improved completion rates of the DHQ-II in this study. However, a small percentage of participants withdrew from the feasibility study when the DHQ-II was administered that could have been due to fatigue from completing several dietary tools in short period of time or unfamiliarity with the online format of the dietary tool. Moving forward, as follow-up surveys are implemented via the Internet to the entire cohort, it may be reasonable to expect that familiarity with completing web-based surveys will improve. Thus, the approximate 50% response rate observed in this study may be higher for future assessments, particularly if they are spread out over a longer time period such as 1 year. Nonetheless, the overall response rate to the feasibility study was lower than intended, suggesting possible challenges to the recruitment of existing cohort members to complete additional measures, especially within a short time.
      Participants who had E-mail addresses associated with their participant files were randomly selected for invitation to the study. The assumption that an email address was a proxy for access to high-speed Internet and some computer literacy may not have been sound, as participants may have an E-mail address but do not use computers often or have inadequate Internet speed to complete the dietary tools. Another potential limitation of this study is the format and schedule of reminder emails sent to participants. A 32-hour completion period was chosen to provide participants with flexibility in completing an ASA24-2016 recall but the receipt of additional reminders within that 32-hour period may have been confusing.
      The version of ASA24 implemented in this study was US-based, because the Canadian version was under development. It is possible difficulties locating food and beverage items specific to Canada and not included the US database might have influenced response rates and led some participants to withdraw from the feasibility study. Finally, only one optional ASA24-2016 module (Supplements) was included. Other modules, such as “Location” and “Ate with,” provide important contextual information and also serve as cues to memory,
      • Lafrenière J.
      • Lamarche B.
      • Laramèe C.
      • Robitaille J.
      • Lemieux S.
      Validation of a newly automated web-based 24-hour dietary recall using fully controlled feeding studies.
      but may increase respondent burden. In particular, the “Source” module increases burden by adding a probe for nearly every food and beverage reported. The variations possible within ASA24 highlight the importance of carefully considering and piloting protocols before administration to large samples.

      Conclusions

      Technological advances such as web-based tools have greatly reduced the cost of collecting and coding comprehensive intake data, but resources to support participants are likely to be required for successful implementation. Future research should evaluate dietary patterns derived from each tool and from the tools combined in relation to disease outcomes; and explore statistical methods for combining the data from the two tools to optimize assessment of diet in the cohort, while accounting for the fact that not all participants will complete all dietary assessment administrations.

      Acknowledgements

      Alberta’s Tomorrow Project is possible due to the commitment of its research participants, its staff, and its funders.

      Author Contributions

      N. Solbak, G. Lo Siou, J. Vena, S. Kirkpatrick, P. Robson designed the research plan. N. Solbak and S. Paek facilitated participant recruitment, data entry, and collection. G. Lo Siou completed statistical analysis. N. Solbak, A. Al Rajabi, S. Kirkpatrick, and P. Robson interpreted study results. N. Solbak wrote the first draft of the manuscript.

      References

        • Global Panel on Agriculture and Food Systems for Nutrition
        Food systems and diets: facing the challenges of the 21st century.
        • GBD 2017 Diet Collaborators
        Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017.
        Lancet. 2019; 393: 1958-1972
        • World Cancer Research Fund/American Institute for Cancer Research
        Diet, nutrition, physical activity and cancer: a global perspective. continuous update project expert report.
        https://www.wcrf.org/dietandcancer
        Date accessed: August 21, 2019
        • Eckel R.H.
        • Jakicic J.M.
        • Ard J.D.
        • et al.
        2013 AHA/ACC guideline on lifestyle management to reduce cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.
        J Am Coll Cardiol. 2014; 63: 2960-2984
        • Ley S.H.
        • Hamdy O.
        • Mohan V.
        • Hu F.B.
        Prevention and management of type 2 diabetes: dietary components and nutritional strategies.
        Lancet. 2014; 383: 1999-2007
        • Aune D.
        Plant foods, antioxidant biomarkers, and the risk of cardiovascular disease, cancer, and mortality: a review of the evidence.
        Adv Nutr. 2019; 10: S404-S421
        • Krebs-Smith S.M.
        • Subar A.F.
        • Reedy J.
        Examining dietary patterns in relation to chronic disease: matching measures and methods to questions of interest.
        Circulation. 2015; 132: 790-793
        • Reedy J.
        • Subar A.F.
        • George S.M.
        • Krebs-Smith S.M.
        Extending methods in dietary patterns research.
        Nutrients. 2018; 10: 571
        • US Depts of Health and Human Services and Agriculture
        2015-2020 Dietary Guidelines for Americans. 8th Edition.
        • Mayne S.T.
        • Playdon M.C.
        • Rock C.L.
        Diet, nutrition, and cancer: past, present and future.
        Nat Rev Clin Oncol. 2016; 13: 504-515
        • Prentice R.L.
        • Caan B.
        • Chlebowski R.T.
        • et al.
        Low-fat dietary pattern and risk of invasive breast cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial.
        JAMA. 2006; 295: 629-642
        • Thompson F.E.
        • Kipnis V.
        • Midthune D.
        • et al.
        Performance of a food-frequency questionnaire in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study.
        Public Health Nutr. 2008; 11: 183-195
        • Hu F.B.
        • Rimm E.
        • Smith-Warner S.A.
        • et al.
        Reproducibility and validity of dietary patterns assessed with a food-frequency questionnaire.
        Am J Clin Nutr. 1999; 69: 243-249
        • Thompson F.E.
        • Kirkpatrick S.I.
        • Subar A.F.
        • et al.
        The National Cancer Institute's Dietary Assessment Primer: a resource for diet research.
        J Acad Nutr Diet. 2015; 115: 1986-1995
        • Kipnis V.
        • Subar A.F.
        • Midthune D.
        • et al.
        Structure of dietary measurement error: results of the OPEN biomarker study.
        Am J Epidemiol. 2003; 158: 14-26
        • Boeing H.
        Nutritional epidemiology: new perspectives for understanding the diet-disease relationship?.
        Eur J Clin Nutr. 2013; 67: 424-429
        • Freedman L.S.
        • Midthune D.
        • Carroll R.J.
        • et al.
        Adjustments to improve the estimation of usual dietary intake distributions in the population.
        J Nutr. 2004; 134: 1836-1843
        • Freedman L.S.
        • Commins J.M.
        • Moler J.E.
        • et al.
        Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
        Am J Epidemiol. 2014; 180: 172-188
        • Subar A.F.
        • Kipnis V.
        • Troiano R.P.
        • et al.
        Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
        Am J Epidemiol. 2003; 158: 1-13
        • Olafsdottir A.S.
        • Thorsdottir I.
        • Gunnarsdottir I.
        • Thorgeirsdottir H.
        • Steingrimsdottir L.
        Comparison of women's diet assessed by FFQs and 24-hour recalls with and without underreporters: associations with biomarkers.
        Ann Nutr Metab. 2006; 50: 450-460
        • Schatzkin A.
        • Kipnis V.
        • Carroll R.J.
        • et al.
        A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study.
        Int J Epidemiol. 2003; 32: 1054-1062
        • Carroll R.J.
        • Midthune D.
        • Subar A.F.
        • et al.
        Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology.
        Am J Epidemiol. 2012; 175: 340-347
        • Freedman L.S.
        • Commins J.M.
        • Moler J.E.
        • et al.
        Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for potassium and sodium intake.
        Am J Epidemiol. 2015; 181: 473-487
        • Neuhouser M.L.
        • Tinker L.
        • Shaw P.A.
        • et al.
        Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women's Health Initiative.
        Am J Epidemiol. 2008; 167: 1247-1259
        • Zimmerman T.P.
        • Hull S.G.
        • McNutt S.
        • et al.
        Challenges in converting an interviewer-administered food probe database to self-administration in the National Cancer Institute Automated Self-administered 24-Hour Recall (ASA24).
        J Food Compost Anal. 2009; 22: S48-S51
        • Stote K.S.
        • Radecki S.V.
        • Moshfegh A.J.
        • Ingwersen L.A.
        • Baer D.J.
        The number of 24 h dietary recalls using the US Department of Agriculture's automated multiple-pass method required to estimate nutrient intake in overweight and obese adults.
        Public Health Nutr. 2011; 14: 1736-1742
        • Freedman L.S.
        • Midthune D.
        • Arab L.
        • et al.
        Combining a food frequency questionnaire with 24-hour recalls to increase the precision of estimation of usual dietary intakes-evidence from the Validation Studies Pooling Project.
        Am J Epidemiol. 2018; 187: 2227-2232
        • Freedman L.S.
        • Schatzkin A.
        • Midthune D.
        • Kipnis V.
        Dealing with dietary measurement error in nutritional cohort studies.
        J Natl Cancer Inst. 2011; 103: 1086-1092
        • Freedman L.S.
        • Commins J.M.
        • Willett W.
        • et al.
        Evaluation of the 24-hour recall as a reference instrument for calibrating other self-report instruments in nutritional cohort studies: evidence from the Validation Studies Pooling Project.
        Am J Epidemiol. 2017; 186: 73-82
        • Kirkpatrick S.I.
        • Gilsing A.M.
        • Hobin E.
        • et al.
        Lessons from studies to evaluate an online 24-hour recall for use with children and adults in Canada.
        Nutrients. 2017; 9: 100
        • Subar A.F.
        • Kirkpatrick S.I.
        • Mittl B.
        • et al.
        The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute.
        J Acad Nutr Diet. 2012; 112: 1134-1137
        • Kirkpatrick S.I.
        • Subar A.F.
        • Douglass D.
        • et al.
        Performance of the Automated Self-Administered 24-hour Recall relative to a measure of true intakes and to an interviewer-administered 24-h recall.
        Am J Clin Nutr. 2014; 100: 233-240
        • Kirkpatrick S.I.
        • Guenther P.M.
        • Douglass D.
        • et al.
        The provision of assistance does not substantially impact the accuracy of 24-hour dietary recalls completed using the Automated Self-Administered 24-H Dietary Assessment Tool among women with low incomes.
        J Nutr. 2019; 149: 114-122
        • Thompson F.E.
        • Dixit-Joshi S.
        • Potischman N.
        • et al.
        Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls in 3 diverse integrated health systems.
        Am J Epidemiol. 2015; 181: 970-978
        • Bryant H.
        • Robson P.J.
        • Ullman R.
        • Friedenreich C.
        • Dawe U.
        Population-based cohort development in Alberta, Canada: a feasibility study.
        Chronic Dis Can. 2006; 27: 51-59
        • Robson P.J.
        • Solbak N.M.
        • Haig T.R.
        • et al.
        Design, methods and demographics from phase I of Alberta's Tomorrow Project cohort: a prospective cohort profile.
        CMAJ Open. 2016; 4: E515-E527
        • Bonett D.G.
        Sample size requirements for estimating intraclass correlations with desired precision.
        Stat Med. 2002; 21: 1331-1335
        • Lo Siou G.
        • Csizmadi I.
        • Boucher B.A.
        • et al.
        The comparative reliability and feasibility of the past-year Canadian Diet History Questionnaire II: comparison of the paper and web versions.
        Nutrients. 2017; 9: 133
        • Kirkpatrick S.I.
        • Potischman N.
        • Dodd K.W.
        • et al.
        The use of digital images in 24-hour recalls may lead to less misestimation of portion size compared with traditional interviewer-administered recalls.
        J Nutr. 2016; 146: 2567-2573
        • National Cancer Institute
        Automated Self-Administered 24 Hour Dietary Assessment Tool (ASA24) researcher website. Version 2.9.2.0.
        https://asa24.nci.nih.gov/researchersite/
        Date accessed: June 15, 2020
        • US Dept of Agriculture
        USDA Food and Nutrient Database for Dietary Studies 2011-2012.
        http://www.ars.usda.gov/ba/bhnrc/fsrg
        Date accessed: August 21, 2019
        • National Institutes of Health, Epidemiology and Genomics Research Program
        Diet History Questionnaire - version 2.0.
        https://epi.grants.cancer.gov/dhq2/
        Date accessed: August 21, 2019
        • Bangor A.
        • Kortum P.
        • Miller J.
        Determining what individual SUS scores mean: adding an adjective rate scale.
        J Usability Studies. 2009; 4: 114-123
        • Lewis J.
        • Sauro J.
        The factor structure of the system usability scale.
        in: Kurosu Human Centered Design. Springer, Berlin, Germany2009: 94-103
      1. SAS Enterprise Guide [computer program]. Version 7.13. Cary, NC : SAS Institute Inc. 2018.

        • Subar A.F.
        • Freedman L.S.
        • Tooze J.A.
        • et al.
        Addressing current criticism regarding the value of self-report dietary data.
        J Nutr. 2015; 145: 2639-2645
        • Park Y.
        • Dodd K.W.
        • Kipnis V.
        • et al.
        Comparison of self-reported dietary intakes from the Automated Self-Administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers.
        Am J Clin Nutr. 2018; 107: 80-93
        • Bernstein S.
        • Zambell K.
        • Amar M.J.
        • et al.
        Dietary intake patterns are consistent across seasons in a cohort of healthy adults in a metropolitan population.
        J Acad Nutr Diet. 2016; 116: 38-45
        • Coates K.S.
        • Healy R.
        • Morrison W.R.
        Tracking the snowbirds: seasonal migration from Canada to the U.S.A. and Mexico.
        Am Rev Canada Studies. 2002; 32: 433-450
        • Statistics Canada
        Alberta and Canada - Census profile. 2016 census.
        • Kupis J.
        • Johnson S.
        • Hallihan G.
        • Olstad D.L.
        Assessing the usability of the Automated Self-Administered Dietary Assessment Tool (ASA24) among low-income adults.
        Nutrients. 2019; 11: 132
        • National Cancer Institute
        Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24)-2016 help guide for respondents.
        • Currie L.M.
        • Ronquillo C.
        • Dick T.
        Access to internet in rural and remote Canada.
        Stud Health Technol Inform. 2014; 201: 407-412
        • Schatzkin A.
        • Subar A.F.
        • Moore S.
        • et al.
        Observational epidemiologic studies of nutrition and cancer: the next generation (with better observation).
        Cancer Epidemiol Biomarkers Prev. 2009; 18: 1026-1032
        • Conrad J.
        • Nothlings U.
        Innovative approaches to estimate individual usual dietary intake in large-scale epidemiological studies.
        Proc Nutr Soc. 2017; 76: 213-219
        • Lafrenière J.
        • Lamarche B.
        • Laramèe C.
        • Robitaille J.
        • Lemieux S.
        Validation of a newly automated web-based 24-hour dietary recall using fully controlled feeding studies.
        BMC Nutrition. 2017; 3: 34

      Biography

      N. Solbak is a project manager, Physician Learning Program, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; at the time of the study, he was a research assistant, Alberta’s Tomorrow Project, CancerCare Alberta, Calgary, Alberta, Canada.
      P. Robson is scientific director, CancerCare Alberta, Strategic Clinical Network, Alberta Health Services, Edmonton, Alberta, Canada.
      G. Lo Siou is a biostatistician, Alberta’s Tomorrow Project, CancerCare Alberta, Calgary, Alberta, Canada.
      A. Al Rajabi is a research associate, Alberta’s Tomorrow Project, CancerCare Alberta, Calgary, Alberta, Canada, and an assistant professor, Health Sciences Department, College of Natural and Health Sciences, Zayed University, Abu Dhabi, United Arab Emirates.
      S. Paek is a data analyst, Alberta’s Tomorrow Project, CancerCare Alberta, Calgary, Alberta, Canada.
      J. Vena is scientific director, Alberta’s Tomorrow Project, CancerCare Alberta, Calgary, Alberta, Canada.
      S. Kirkpatrick is an associate professor, School of Public Health and Health Systems, University of Waterloo, Waterloo, Ontario, Candada.