Evaluation of Web-Based, Self-Administered, Graphical Food Frequency Questionnaire

Published:January 24, 2014DOI:


      Computer-administered food frequency questionnaires (FFQs) can address limitations inherent in paper questionnaires by allowing very complex skip patterns, portion size estimation based on food pictures, and real-time error checking. We evaluated a web-based FFQ, the Graphical Food Frequency System (GraFFS). Participants completed the GraFFS, six telephone-administered 24-hour dietary recalls over the next 12 weeks, followed by a second GraFFS. Participants were 40 men and 34 women, aged 18 to 69 years, living in the Columbus, OH, area. Intakes of energy, macronutrients, and 17 micronutrients/food components were estimated from the GraFFS and the mean of all recalls. Bias (second GraFFS minus recalls) was −9%, −5%, +4%, and −4% for energy and percentages of energy from fat, carbohydrate, and protein, respectively. De-attenuated, energy-adjusted correlations (intermethod reliability) between the recalls and the second GraFFS for fat, carbohydrate, protein, and alcohol were 0.82, 0.79, 0.67, and 0.90, respectively; for micronutrients/food components the median was 0.61 and ranged from 0.40 for zinc to 0.92 for beta carotene. The correlations between the two administrations of the GraFFS (test–retest reliability) for fat, carbohydrate, protein, and alcohol were 0.60, 0.63, 0.73, and 0.87, respectively; among micronutrients/food components the median was 0.67 and ranged from 0.49 for vitamin B-12 to 0.82 for fiber. The measurement characteristics of the GraFFS were at least as good as those reported for most paper FFQs, and its high intermethod reliability suggests that further development of computer-administered FFQs is warranted.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of the Academy of Nutrition and Dietetics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Smith A.F.
        Cognitive processes in long-term dietary recall. National Center for Health Statistics.
        Vital Health Stat. 1991; 6: 1-34
        • Thompson F.E.
        • Subar A.F.
        • Brown C.C.
        • et al.
        Cognitive research enhances accuracy of food frequency questionnaire reports: Results of an experimental validation study.
        J Am Diet Assoc. 2002; 102: 212-225
        • Patterson R.E.
        • Kristal A.R.
        • Coates R.J.
        • et al.
        Low-fat diet practices of older women: Prevalence and implications for dietary assessment.
        J Am Diet Assoc. 1996; 96: 670-679
        • Schatzkin A.
        • Kipnis V.
        • Carroll R.J.
        • et al.
        A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: Results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) Study.
        Int J Epidemiol. 2003; 32: 1054-1062
        • Levander O.A.
        The need for measures of selenium status.
        J Am Coll Toxicol. 1986; 5: 37-44
        • Stryker W.S.
        • Kaplan L.A.
        • Stein E.A.
        • Stampfer M.J.
        • Sober A.
        • Willett W.C.
        The relation of diet, cigarette smoking, and alcohol consumption to plasma beta-carotene and alpha-tocopherol levels.
        Am J Epidemiol. 1988; 127: 283-296
        • Hunter D.J.
        • Morris J.S.
        • Chute C.G.
        • et al.
        Predictors of selenium concentration in human toenails.
        Am J Epidemiol. 1990; 132: 114-122
        • Brunner E.
        • Stallone D.
        • Janeja M.
        • Bingham S.
        • Marmot M.
        Dietary assessment in Whitehall II: Comparison of 7 d diet diary and food-frequency questionnaire and validity against biomarkers.
        Br J Nutr. 2001; 86: 405-414
        • Dixon L.B.
        • Subar A.F.
        • Wideroff L.
        • Thompson F.E.
        • Kahle L.L.
        • Potischman N.
        Carotenoid and tocopherol estimates from the NCI Diet History Questionnaire are valid compared with multiple recalls and serum biomarkers.
        J Nutr. 2006; 136: 3054-3061
        • Talegawkar S.A.
        • Johnson E.J.
        • Carithers T.
        • Taylor H.A.
        • Bogle M.L.
        • Tucker K.L.
        Total a-tocopherol intakes are associated with serum α-tocopherol concentrations in African American adults.
        J Nutr. 2007; 137: 2297-2303
        • Musgrave K.O.
        • Giambalvo L.
        • Leclerc H.L.
        • Cook R.A.
        • Rosen C.J.
        Validation of a quantitative food frequency questionnaire for rapid assessment of dietary calcium intake.
        J Am Diet Assoc. 1989; 89: 1484-1488
        • Kristal A.R.
        • Vizenor N.C.
        • Patterson R.E.
        • Neuhouser M.L.
        • Shattuck A.L.
        • McLerran D.
        Precision and bias of food frequency-based measures of fruit and vegetable intakes.
        Cancer Epidemiol Biomarkers Prev. 2000; 9: 939-944
        • Kristal A.R.
        • Curry S.J.
        • Shattuck A.L.
        • Feng Z.
        • Li S.
        A randomized trial of a tailored, self-help dietary intervention: The Puget Sound Eating Patterns Study.
        Prev Med. 2000; 31: 380-389
        • Maes L.
        • Cook T.L.
        • Ottovaere C.
        • et al.
        Pilot evaluation of the HELENA (Healthy Lifestyle in Europe by Nutrition in Adolescence) Food-O-Meter, a computer-tailored nutrition advice for adolescents: A study in six European cities.
        Public Health Nutr. 2011; 14: 1292-1302
        • Falomir Z.
        • Arregui M.
        • Madueno F.
        • Corella D.
        • Coltell O.
        Automation of food questionnaires in medical studies: A state-of-the-art review and future prospects.
        Comput Biol Med. 2012; 42: 964-974
        • Matthys C.
        • Pynaert I.
        • De Keyzer W.
        • De Henauw S.
        Validity and reproducibility of an adolescent web-based food frequency questionnaire.
        J Am Diet Assoc. 2007; 107: 605-610
      1. NutritionQuest. Block questionnaires. Accessed November 27, 2013.

      2. National Cancer Institute, Risk Factor Monitoring and Methods. Diet history questionnaire. Accessed November 27, 2013.

        • Labonte M.E.
        • Cyr A.
        • Baril-Gravel L.
        • Royer M.M.
        • Lamarche B.
        Validity and reproducibility of a web-based, self-administered food frequency questionnaire.
        Eur J Clin Nutr. 2012; 66: 166-173
        • Beasley J.M.
        • Davis A.
        • Riley W.T.
        Evaluation of a web-based, pictorial diet history questionnaire.
        Public Health Nutr. 2009; 12: 651-659
        • Vereecken C.A.
        • De Bourdeaudhuij I.
        • Maes L.
        The HELENA online food frequency questionnaire: Reproducibility and comparison with four 24-h recalls in Belgian-Flemish adolescents.
        Eur J Clin Nutr. 2010; 64: 541-548
      3. Viocare Technologies. VioScreen. Accessed November 27, 2013.

      4. Nutrition assessment. Accessed November 27, 2013.

        • Patterson R.E.
        • Kristal A.R.
        • Tinker L.F.
        • Carter R.A.
        • Bolton M.P.
        • Agurs-Collins T.
        Measurement characteristics of the Women's Health Initiative food frequency questionnaire.
        Ann Epidemiol. 1999; 9: 178-187
        • Thompson I.M.
        • Coltman C.A.
        • Crowley J.
        Chemoprevention of prostate cancer: The Prostate Cancer Prevention Trial.
        Prostate. 1997; 33: 217-221
        • Kristal A.R.
        • Arnold K.B.
        • Schenk J.M.
        • et al.
        Dietary patterns, supplement use and risk of symptomatic benign prostatic hyperplasia (BPH): Results from the Prostate Cancer Prevention Trial.
        Am J Epidemiol. 2008; 167: 925-934
        • Lippman S.M.
        • Klein E.A.
        • Goodman P.J.
        • et al.
        Effect of selenium and vitamin E on risk of prostate cancer and other cancers. The Selenium and Vitamin E Cancer Prevention Trial (SELECT).
        JAMA. 2009; 301: 39-51
        • White E.
        • Patterson R.E.
        • Kristal A.R.
        • et al.
        VITamins And Lifestyle Cohort Study: Study design and characteristics of supplement users.
        Am J Epidemiol. 2004; 159: 83-93
        • Neuhouser M.L.
        • Tinker L.
        • Shaw P.A.
        • et al.
        Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women's Health Intitiative.
        Am J Epidemiol. 2008; 167: 1247-1259
        • Kipnis V.
        • Midthune D.
        • Freedman L.S.
        • et al.
        Empirical evidence of correlated biases in dietary assessment instruments and its implications.
        Am J Epidemiol. 2001; 153: 394-403
        • Willett W.
        Reproducibility and validity of food-frequency questionnaires.
        Nutritional Epidemiology. 2nd ed. Oxford University Press, New York, NY1998: 101-147
        • White E.
        • Armstrong B.K.
        • Saracci R.
        Principles of Exposure Measurement in Epidemiology.
        2nd ed. Oxford University Press, New York, NY2008
        • Schakel S.F.
        Maintaining a nutrient database in a changing marketplace: Keeping pace with changing food products—A research perspective.
        J Food Comp Anal. 2001; 14: 315-322
        • Kristal A.R.
        • Shattuck A.L.
        • Williams A.E.
        Food frequency questionnaires for diet intervention research.
        in: Proceedings of the 17th National Nutrient Databank Conference. International Life Sciences Institute, Washington, DC1994: 110-125
        • Willett W.C.
        Correction of effects of measurement error.
        Nutritional Epidemiology. 2nd ed. Oxford University Press, New York, NY1998: 302-320
        • Beaton G.H.
        • Milner J.
        • Corey P.
        • et al.
        Sources of variance in 24-hour dietary recall data: Implications for nutrition study design and interpretation.
        Am J Clin Nutr. 1979; 32: 2546-2559
        • Thompson F.E.
        • Kipnis V.
        • Midthune D.
        • et al.
        Performance of a food-frequency questionnaire in the US NIH-AARP (National Institutes of Health-American Association of Retired Persons) Diet and Health Study.
        Public Health Nutr. 2008; 11: 183-195
        • Boucher B.
        • Cotterchio M.
        • Kreiger N.
        • Nadalin V.
        • Block T.
        • Block G.
        Validity and reliability of the Block98 food-frequency questionnaire in a sample of Canadian women.
        Public Health Nutr. 2006; 9: 84-93
        • Thompson F.E.
        • Midthune D.
        • Subar A.F.
        • Kipnis V.
        • Kahle L.L.
        • Schatzkin A.
        Development and evaluation of a short instrument to estimate usual dietary intake of percentage energy from fat.
        J Am Diet Assoc. 2007; 107: 760-767
        • Marks G.C.
        • Hughes M.C.
        • van der Pols J.C.
        Relative validity of food intake estimates using a food frequency questionnaire is associated with sex, age, and other personal characteristics.
        J Nutr. 2006; 136: 459-465
        • Subar A.
        • Thompson F.
        • Kipnis V.
        • et al.
        Comparative validation of the Block, Willett, and the National Cancer Institute food frequency questionnaires: The Eating at America's Table study.
        Am J Epidemiol. 2001; 154: 1089-1099
        • Kristal A.R.
        • Andrilla C.H.
        • Koepsell T.D.
        • Diehr P.H.
        • Cheadle A.
        Dietary assessment instruments are susceptible to intervention-associated response set bias.
        J Am Diet Assoc. 1998; 98: 40-43
        • Miller T.M.
        • Abdel-Maksound M.F.
        • Crane L.A.
        • Marcus A.C.
        • Byers T.E.
        Effects of social approval bias on self-reported fruit and vegetable consumption.
        Nutr J. 2008; 7 (article no. 18)


      A. R. Kristal is a member and associate head, Cancer Prevention Program, Fred Hutchinson Cancer Research Center, and a professor, Department of Epidemiology, University of Washington, Seattle, WA.


      A. S. Kolar is a project manager, Cancer Prevention Program, Fred Hutchinson Cancer Research Center, Seattle, WA.


      J. L. Fisher is a research scientist, Comprehensive Cancer Center and James Cancer Hospital, The Ohio State University, Columbus.


      J. J. Plascak is a statistical research associate, Comprehensive Cancer Center and James Cancer Hospital, The Ohio State University, Columbus.


      P. J. Stumbo is a research nutritionist emeritus, Institute for Clinical and Translational Sciences, University of Iowa, Iowa City.


      R. Weiss is president, Viocare, Inc, Princeton, NJ.


      E. D. Paskett is Marion N. Rowley Professor of Cancer Research; director, Division of Cancer Prevention and Control and Department of Internal Medicine, College of Medicine; professor, Division of Epidemiology, College of Public Health; and associate director for population sciences, Comprehensive Cancer Center, The Ohio State University, Columbus.