Reproducibility and Intermethod Reliability of a Calcium Food Frequency Questionnaire for Use in Hispanic, Non-Hispanic Black, and Non-Hispanic White Youth

Published:February 12, 2015DOI:



      A dietary assessment instrument designed for use in a nationally representative pediatric population was required to examine associations between calcium intake and bone mineral accrual in a large, multicenter study.


      To determine the reproducibility and intermethod reliability of a youth calcium food frequency questionnaire (FFQ) in a multiracial/ethnic sample of children and adolescents.


      Reproducibility (n=69) and intermethod reliability (n=393) studies were conducted by administering repeat FFQs and three unannounced 24-hour dietary recalls to stratified random samples of individuals participating in the Bone Mineral Density in Childhood Study.


      Children and adolescents ages 5 to 21 years.

      Main outcome measures

      Calcium intake estimated from the FFQ and 24-hour dietary recalls.

      Statistical analysis

      Reproducibility was assessed by the intraclass correlation coefficient (ICC). Intermethod reliability was assessed by deattenuated Pearson correlations between the FFQ and 24-hour recalls. Attenuation factors and calibration corrected effect estimates for bone density were calculated to determine the potential influence of measurement error on associations with health outcomes.


      The ICC (0.61) for repeat administrations and deattenuated Pearson correlation between the FFQ and 24-hour recalls (r=0.60) for all subjects indicated reproducibility and intermethod reliability (Pearson r=0.50 to 0.74 across sex and age groups). Attenuation factors were ≤0.50 for all sex and age groups and lower for non-Hispanic blacks (λ=0.20) and Hispanics (λ=0.26) than for non-Hispanic whites (λ=0.42).


      The Bone Mineral Density in Childhood Study calcium FFQ appears to provide a useful tool for assessing calcium intake in children and adolescents drawn from multiracial/ethnic populations and/or spanning a wide age range. However, similar to other FFQs, attenuation factors were substantially <1, indicating the potential for appreciable measurement error bias. Calibration correction should be performed and racial/ethnic differences in performance considered when analyzing and interpreting findings based on this instrument.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of the Academy of Nutrition and Dietetics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Feskanich D.
        • Willett W.C.
        • Colditz G.A.
        Calcium, vitamin D, milk consumption, and hip fractures: A prospective study among postmenopausal women.
        Am J Clin Nutr. 2003; 77: 504-511
        • Park S.Y.
        • Murphy S.P.
        • Wilkens L.R.
        • Stram D.O.
        • Henderson B.E.
        • Kolonel L.N.
        Calcium, vitamin D, and dairy product intake and prostate cancer risk: The Multiethnic Cohort Study.
        Am J Epidemiol. 2007; 166: 1259-1269
        • Xiao Q.
        • Murphy R.A.
        • Houston D.K.
        • Harris T.B.
        • Chow W.H.
        • Park Y.
        Dietary and supplemental calcium intake and cardiovascular disease mortality: The National Institutes of Health-AARP diet and health study.
        JAMA Intern Med. 2013; 173: 639-646
        • Black R.E.
        • Williams S.M.
        • Jones I.E.
        • Goulding A.
        Children who avoid drinking cow milk have low dietary calcium intakes and poor bone health.
        Am J Clin Nutr. 2002; 76: 675-680
        • Rockett H.R.
        • Wolf A.M.
        • Colditz G.A.
        Development and reproducibility of a food frequency questionnaire to assess diets of older children and adolescents.
        J Am Diet Assoc. 1995; 95: 336-340
        • Zemel B.S.
        • Carey L.B.
        • Paulhamus D.R.
        • Stallings V.A.
        • Ittenbach R.F.
        Quantifying calcium intake in school age children: Development and validation of the Calcium Counts! food frequency questionnaire.
        Am J Hum Biol. 2010; 22: 180-186
        • Willett W.
        Nutritional Epidemiology.
        3rd ed. Oxford University Press, New York, NY2013
        • White E.
        • Armstrong B.K.
        • Saracci R.
        • Armstrong B.K.
        Principles of Exposure Measurement in Epidemiology: Collecting, Evaluating, and Improving Measures of Disease Risk Factors.
        2nd ed. Oxford University Press, New York, NY2008
        • Subar A.F.
        • Thompson F.E.
        • Kipnis V.
        • et al.
        Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: The Eating at America's Table Study.
        Am J Epidemiol. 2001; 154: 1089-1099
        • Stram D.O.
        • Hankin J.H.
        • Wilkens L.R.
        • et al.
        Calibration of the dietary questionnaire for a multiethnic cohort in Hawaii and Los Angeles.
        Am J Epidemiol. 2000; 151: 358-370
        • Horn-Ross P.L.
        • Lee V.S.
        • Collins C.N.
        • et al.
        Dietary assessment in the California Teachers Study: Reproducibility and validity.
        Cancer Causes Control. 2008; 19: 595-603
        • Carithers T.C.
        • Talegawkar S.A.
        • Rowser M.L.
        • et al.
        Validity and calibration of food frequency questionnaires used with African-American adults in the Jackson Heart Study.
        J Am Diet Assoc. 2009; 109: 1184-1193
        • Kipnis V.
        • Carroll R.J.
        • Freedman L.S.
        • Li L.
        Implications of a new dietary measurement error model for estimation of relative risk: Application to four calibration studies.
        Am J Epidemiol. 1999; 150: 642-651
        • Kipnis V.
        • Midthune D.
        • Freedman L.S.
        • et al.
        Empirical evidence of correlated biases in dietary assessment instruments and its implications.
        Am J Epidemiol. 2001; 153: 394-403
        • Kalkwarf H.J.
        • Zemel B.S.
        • Gilsanz V.
        • et al.
        The bone mineral density in childhood study: Bone mineral content and density according to age, sex, and race.
        J Clin Endocrinol Metab. 2007; 92: 2087-2099
        • Zemel B.S.
        • Kalkwarf H.J.
        • Gilsanz V.
        • et al.
        Revised reference curves for bone mineral content and areal bone mineral density according to age and sex for black and non-black children: Results of the bone mineral density in childhood study.
        J Clin Endocrinol Metab. 2011; 96: 3160-3169
        • Lanou A.J.
        • Berkow S.E.
        • Barnard N.D.
        Calcium, dairy products, and bone health in children and young adults: A reevaluation of the evidence.
        Pediatrics. 2005; 115: 736-743
        • Winzenberg T.
        • Shaw K.
        • Fryer J.
        • Jones G.
        Effects of calcium supplementation on bone density in healthy children: Meta-analysis of randomised controlled trials.
        BMJ. 2006; 333: 775
        • Huncharek M.
        • Muscat J.
        • Kupelnick B.
        Impact of dairy products and dietary calcium on bone-mineral content in children: Results of a meta-analysis.
        Bone. 2008; 43: 312-321
        • Block G.
        • Hartman A.M.
        • Dresser C.M.
        • Carroll M.D.
        • Gannon J.
        • Gardner L.
        A data-based approach to diet questionnaire design and testing.
        Am J Epidemiol. 1986; 124: 453-469
        • Block G.
        • Coyle L.M.
        • Hartman A.M.
        • Scoppa S.M.
        Revision of dietary analysis software for the Health Habits and History Questionnaire.
        Am J Epidemiol. 1994; 139: 1190-1196
        • Block G.
        • Gillespie C.
        • Rosenbaum E.H.
        • Jenson C.
        A rapid food screener to assess fat and fruit and vegetable intake.
        Am J Prev Med. 2000; 18: 284-288
      1. Centers for Disease Control and Prevention, National Center for Health Statistics. National Health and Nutrition Examination Survey data. 1994. Accessed June 24, 2014.

        • Lappe J.M.
        • Watson P.
        • Gilsanz V.
        • et al.
        The longitudinal effects of physical activity and dietary calcium on bone mass accrual across stages of pubertal development.
        J Bone Miner Res. 2015; 30: 156-164
        • Gordon C.M.
        • Leonard M.B.
        • Zemel B.S.
        International Society for Clinical D. 2013 Pediatric Position Development Conference: Executive summary and reflections.
        J Clin Densitom. 2014; 17: 219-224
        • Slemenda C.W.
        • Miller J.Z.
        • Hui S.L.
        • Reister T.K.
        • Johnston Jr., C.C.
        Role of physical activity in the development of skeletal mass in children.
        J Bone Miner Res. 1991; 6: 1227-1233
        • Thompson F.E.
        • Byers T.
        Dietary assessment resource manual.
        J Nutr. 1994; 124: 2245S-2317S
        • Schatzkin A.
        • Kipnis V.
        • Carroll R.J.
        • et al.
        A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: Results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study.
        Int J Epidemiol. 2003; 32: 1054-1062
        • Spiegelman D.
        • McDermott A.
        • Rosner B.
        Regression calibration method for correcting measurement-error bias in nutritional epidemiology.
        Am J Clin Nutr. 1997; 65: 1179S-1186S
        • Jensen J.K.
        • Gustafson D.
        • Boushey C.J.
        • et al.
        Development of a food frequency questionnaire to estimate calcium intake of Asian, Hispanic, and white youth.
        J Am Diet Assoc. 2004; 104: 762-769
        • Harnack L.J.
        • Lytle L.A.
        • Story M.
        • et al.
        Reliability and validity of a brief questionnaire to assess calcium intake of middle-school-aged children.
        J Am Diet Assoc. 2006; 106: 1790-1795
        • Wong S.S.
        • Boushey C.J.
        • Novotny R.
        • Gustafson D.R.
        Evaluation of a computerized food frequency questionnaire to estimate calcium intake of Asian, Hispanic, and non-Hispanic white youth.
        J Am Diet Assoc. 2008; 108: 539-543
        • Rockett H.R.
        • Breitenbach M.
        • Frazier A.L.
        • et al.
        Validation of a youth/adolescent food frequency questionnaire.
        Prev Med. 1997; 26: 808-816
        • Taylor R.W.
        • Goulding A.
        Validation of a short food frequency questionnaire to assess calcium intake in children aged 3 to 6 years.
        Eur J Clin Nutr. 1998; 52: 464-465
        • Bertoli S.
        • Petroni M.L.
        • Pagliato E.
        • et al.
        Validation of food frequency questionnaire for assessing dietary macronutrients and calcium intake in Italian children and adolescents.
        J Pediatr Gastroenterol Nutr. 2005; 40: 555-560
        • Huybrechts I.
        • De Bacquer D.
        • Matthys C.
        • De Backer G.
        • De Henauw S.
        Validity and reproducibility of a semi-quantitative food-frequency questionnaire for estimating calcium intake in Belgian preschool children.
        Br J Nutr. 2006; 95: 802-816
        • Magkos F.
        • Manios Y.
        • Babaroutsi E.
        • Sidossis L.S.
        Differences in the quantitative and qualitative performance of a calcium-specific food frequency questionnaire across age and sex.
        J Hum Nutr Diet. 2006; 19: 331-342
        • Moore M.
        • Braid S.
        • Falk B.
        • Klentrou P.
        Daily calcium intake in male children and adolescents obtained from the rapid assessment method and the 24-hour recall method.
        Nutr J. 2007; 6: 24
        • Fuller W.A.
        Measurement Error Models.
        Wiley-Interscience, Hoboken, NJ2006
        • Carroll R.J.
        • Carroll R.J.
        Measurement Error in Nonlinear Models: A Modern Perspective.
        2nd ed. Chapman & Hall/CRC, Boca Raton, FL2006
        • Freedman L.S.
        • Schatzkin A.
        • Midthune D.
        • Kipnis V.
        Dealing with dietary measurement error in nutritional cohort studies.
        J Natl Cancer Inst. 2011; 103: 1086-1092


      N. J. Ollberding is an assistant professor, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.


      H. J. Kalkwarf is a professor, Division of General and Community Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH.


      V. Gilsanz is a professor, Division of Radiology, Children’s Hospital of Los Angeles, Los Angeles, CA.


      J. M. Lappe is a professor, Division of Endocrinology, Creighton University, Omaha, NE.


      S. E. Oberfield is a professor, Division of Pediatric Endocrinology Diabetes and Metabolism, Columbia University, New York, NY.


      J. A. Shepherd is an associate adjunct professor, Department of Radiology, University of California, San Francisco.


      K. K. Winer is director, Pediatric Endocrinology Program, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, MD.


      B. S. Zemel is a professor, Division of Gastroenterology, Hepatology, and Nutrition, Children's Hospital of Philadelphia, Philadelphia, PA.