Skip to main content

Advertisement

Log in

Risk prediction models of breast cancer: a systematic review of model performances

  • Review
  • Published:
Breast Cancer Research and Treatment Aims and scope Submit manuscript

Abstract

The number of risk prediction models has been increasingly developed, for estimating about breast cancer in individual women. However, those model performances are questionable. We therefore have conducted a study with the aim to systematically review previous risk prediction models. The results from this review help to identify the most reliable model and indicate the strengths and weaknesses of each model for guiding future model development. We searched MEDLINE (PubMed) from 1949 and EMBASE (Ovid) from 1974 until October 2010. Observational studies which constructed models using regression methods were selected. Information about model development and performance were extracted. Twenty-five out of 453 studies were eligible. Of these, 18 developed prediction models and 7 validated existing prediction models. Up to 13 variables were included in the models and sample sizes for each study ranged from 550 to 2,404,636. Internal validation was performed in four models, while five models had external validation. Gail and Rosner and Colditz models were the significant models which were subsequently modified by other scholars. Calibration performance of most models was fair to good (expected/observe ratio: 0.87–1.12), but discriminatory accuracy was poor to fair both in internal validation (concordance statistics: 0.53–0.66) and in external validation (concordance statistics: 0.56–0.63). Most models yielded relatively poor discrimination in both internal and external validation. This poor discriminatory accuracy of existing models might be because of a lack of knowledge about risk factors, heterogeneous subtypes of breast cancer, and different distributions of risk factors across populations. In addition the concordance statistic itself is insensitive to measure the improvement of discrimination. Therefore, the new method such as net reclassification index should be considered to evaluate the improvement of the performance of a new develop model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Parkin DM, Bray F, Ferlay J, Pisani P (2005) Global cancer statistics 2002. CA Cancer J Clinicians 55(2):74–108

    Article  Google Scholar 

  2. Ferlay J, Bray F, Parkin DM, Pisani P eds (2001) Globalcan 2000. IARC Press, Lyon

  3. VH BF (2002) IARC handbooks of cancer prevention. IARC Press, Lyon

    Google Scholar 

  4. Ries LAG EM, Kosary CL, et al. (2004) SEER cancer statistics review, 1975–2001. National Cancer Institute. http://seer.cancer.gov/csr/1975_2001/. Accessed Dec 22 2004

  5. Steyerberg EW (ed) (2009) Clinical prediction models a practical approach to development, validation, and updating. Springer Science + Business Media LLC, New York

    Google Scholar 

  6. Boyle P, Mezzetti M, La Vecchia C, Franceschi S, Decarli A, Robertson C (2004) Contribution of three components to individual cancer risk predicting breast cancer risk in Italy. Eur J Cancer Prev 13(3):183–191

    Article  PubMed  CAS  Google Scholar 

  7. Chlebowski RT, Anderson GL, Lane DS, Aragaki AK, Rohan T, Yasmeen S, Sarto G, Rosenberg CA, Hubbell FA (2007) Predicting risk of breast cancer in postmenopausal women by hormone receptor status. J Natl Cancer Inst 99(22):1695–1705

    Article  PubMed  CAS  Google Scholar 

  8. Colditz GA, Rosner B (2000) Cumulative risk of breast cancer to age 70 years according to risk factor status: data from the Nurses’ Health Study. Am J Epidemiol 152(10):950–964

    Article  PubMed  CAS  Google Scholar 

  9. Colditz GA, Rosner BA, Chen WY, Holmes MD, Hankinson SE (2004) Risk factors for breast cancer according to estrogen and progesterone receptor status. J Natl Cancer Inst 96(3):218–228

    Article  PubMed  CAS  Google Scholar 

  10. Decarli A, Calza S, Masala G, Specchia C, Palli D, Gail MH (2006) Gail model for prediction of absolute risk of invasive breast cancer: independent evaluation in the Florence-European prospective investigation into cancer and nutrition cohort. J Natl Cancer Inst 98(23):1686–1693

    Article  PubMed  Google Scholar 

  11. Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, Mulvihill JJ (1989) Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst 81(24):1879–1886

    Article  PubMed  CAS  Google Scholar 

  12. Lee EO, Ahn SH, You C, Lee DS, Han W, Choe KJ, Noh DY (2004) Determining the main risk factors and high-risk groups of breast cancer using a predictive model for breast cancer risk assessment in South Korea. Cancer Nurs 27(5):400–406

    Article  PubMed  Google Scholar 

  13. Rosner B, Colditz GA (1996) Nurses’ health study: log-incidence mathematical model of breast cancer incidence. J Natl Cancer Inst 88(6):359–364

    Article  PubMed  CAS  Google Scholar 

  14. Barlow WE, White E, Ballard-Barbash R, Vacek PM, Titus-Ernstoff L, Carney PA, Tice JA, Buist DS, Geller BM, Rosenberg R, Yankaskas BC, Kerlikowske K (2006) Prospective breast cancer risk prediction model for women undergoing screening mammography. J Natl Cancer Inst 98(17):1204–1214. doi:10.1093/jnci/djj331

    Article  PubMed  Google Scholar 

  15. Chen J, Pee D, Ayyagari R, Graubard B, Schairer C, Byrne C, Benichou J, Gail MH (2006) Projecting absolute invasive breast cancer risk in white women with a model that includes mammographic density. J Natl Cancer Inst 98(17):1215–1226

    Article  PubMed  Google Scholar 

  16. Gail MH, Costantino JP, Pee D, Bondy M, Newman L, Selvan M, Anderson GL, Malone KE, Marchbanks PA, McCaskill-Stevens W, Norman SA, Simon MS, Spirtas R, Ursin G, Bernstein L (2007) Projecting individualized absolute invasive breast cancer risk in African American women. J Natl Cancer Inst 99(23):1782–1792

    Article  PubMed  Google Scholar 

  17. Novotny J, Pecen L, Petruzelka L, Svobodnik A, Dusek L, Danes J, Skovajsova M (2006) Breast cancer risk assessment in the Czech female population—an adjustment of the original Gail model. Breast Cancer Res Treat 95(1):29–35

    Article  PubMed  Google Scholar 

  18. Rosner B, Colditz GA, Iglehart JD, Hankinson SE (2008) Risk prediction models with incomplete data with application to prediction of estrogen receptor-positive breast cancer: prospective data from the Nurses’ Health Study. Breast Cancer Res BCR 10(4):R55

    Article  Google Scholar 

  19. Tamimi RM, Rosner B, Colditz GA Evaluation of a breast cancer risk prediction model expanded to include category of prior benign breast disease lesion. Cancer. doi: 10.1002/cncr.25386

  20. Tice JA, Cummings SR, Smith-Bindman R, Ichikawa L, Barlow WE, Kerlikowske K (2008) Using clinical factors and mammographic breast density to estimate breast cancer risk: development and validation of a new predictive model. Ann Intern Med 148(5):337–347

    PubMed  Google Scholar 

  21. Tice JA, Cummings SR, Ziv E, Kerlikowske K (2005) Mammographic breast density and the Gail model for breast cancer risk prediction in a screening population. Breast Cancer Res Treat 94(2):115–122

    Article  PubMed  Google Scholar 

  22. Tice JA, Miike R, Adduci K, Petrakis NL, King E, Wrensch MR (2005) Nipple aspirate fluid cytology and the Gail model for breast cancer risk assessment in a screening population. Cancer Epidemiol Biomarkers Prev 14(2):324–328

    Article  PubMed  Google Scholar 

  23. Ueda K, Tsukuma H, Tanaka H, Ajiki W, Oshima A (2003) Estimation of individualized probabilities of developing breast cancer for Japanese women. Breast cancer (Tokyo, Japan) 10(1):54–62

    Article  Google Scholar 

  24. Bondy ML, Lustbader ED, Halabi S, Ross E, Vogel VG (1994) Validation of a breast cancer risk assessment model in women with a positive family history. J Natl Cancer Inst 86(8):620–625

    Article  PubMed  CAS  Google Scholar 

  25. Costantino JP, Gail MH, Pee D, Anderson S, Redmond CK, Benichou J, Wieand HS (1999) Validation studies for models projecting the risk of invasive and total breast cancer incidence. J Natl Cancer Inst 91(18):1541–1548

    Article  PubMed  CAS  Google Scholar 

  26. McKian KP, Reynolds CA, Visscher DW, Nassar A, Radisky DC, Vierkant RA, Degnim AC, Boughey JC, Ghosh K, Anderson SS, Minot D, Caudill JL, Vachon CM, Frost MH, Pankratz VS, Hartmann LC (2009) Novel breast tissue feature strongly associated with risk of breast cancer. J Clin Oncol 27(35):5893–5898

    Article  PubMed  Google Scholar 

  27. Rockhill B, Byrne C, Rosner B, Louie MM, Colditz G (2003) Breast cancer risk prediction with a log-incidence model: evaluation of accuracy. J Clin Epidemiol 56(9):856–861

    Article  PubMed  Google Scholar 

  28. Rockhill B, Spiegelman D, Byrne C, Hunter DJ, Colditz GA (2001) Validation of the Gail et al. Model of breast cancer risk prediction and implications for chemoprevention. J Natl Cancer Inst 93(5):358–366

    Article  PubMed  CAS  Google Scholar 

  29. Schonfeld SJ, Pee D, Greenlee RT, Hartge P, Lacey JV, Jr., Park Y, Schatzkin A, Visvanathan K, Pfeiffer RM (2010) Effect of changing breast cancer incidence rates on the calibration of the Gail model. J Clin Oncol 28(14):2411–2417. doi: 10.1200/JCO.2009.25.2767

    Google Scholar 

  30. Ulusoy C, Kepenekci I, Kose K, Aydintug S, Cam R (2010) Applicability of the Gail model for breast cancer risk assessment in Turkish female population and evaluation of breastfeeding as a risk factor. Breast Cancer Res Treat 120(2):419–424

    Article  PubMed  Google Scholar 

  31. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21(1):128–138. doi:10.1097/EDE.0b013e3181c30fb2

    Article  PubMed  Google Scholar 

  32. Vickers AJ, Cronin AM (2010) Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol 37(1):31–38. doi:10.1053/j.seminoncol.2009.12.004

    Article  PubMed  Google Scholar 

  33. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 27(2):157–172 discussion 207–112

    Article  PubMed  Google Scholar 

  34. Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19(4):453–473

    Article  PubMed  CAS  Google Scholar 

  35. Peduzzi P, Concato J, Feinstein AR, Holford TR (1995) Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48(12):1503–1510

    Article  PubMed  CAS  Google Scholar 

  36. Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV (2011) Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol 64(9):993–1000. doi:10.1016/j.jclinepi.2010.11.012

    Article  PubMed  Google Scholar 

  37. Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P (2004) Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 159(9):882–890

    Article  PubMed  Google Scholar 

  38. Haynes RB, Sackett DL, Guyatt GH (eds) (2006) Clinical epidemiology: how to do clinical practice research, 3rd edn. Lippincott Williams and Wilkins, Philadelphia

    Google Scholar 

  39. Breast cancer and hormone replacement therapy: collaborative reanalysis of data from 51 epidemiological studies of 52,705 women with breast cancer and 108,411 women without breast cancer. Collaborative Group on Hormonal Factors in Breast Cancer (1997). Lancet 350(9084):1047–1059

    Google Scholar 

  40. Ewertz M, Duffy SW, Adami HO, Kvale G, Lund E, Meirik O, Mellemgaard A, Soini I, Tulinius H (1990) Age at first birth, parity and risk of breast cancer: a meta-analysis of 8 studies from the Nordic countries. Int J Cancer 46(4):597–603

    Article  PubMed  CAS  Google Scholar 

  41. Pharoah PD, Day NE, Duffy S, Easton DF, Ponder BA (1997) Family history and the risk of breast cancer: a systematic review and meta-analysis. Int J Cancer 71(5):800–809

    Article  PubMed  CAS  Google Scholar 

  42. North RA, McCowan LME, Dekker GA, Poston L, Chan EHY, Stewart AW, Black MA, Taylor RS, Walker JJ, Baker PN, Kenny LC (2011) Clinical risk prediction for pre-eclampsia in nulliparous women: development of model in international prospective cohort. BMJ (Clinical research ed) 342:d1875s. doi: 10.1136/bmj.d1875

  43. Tyrer J, Duffy SW, Cuzick J (2004) A breast cancer prediction model incorporating familial and personal risk factors. Stat Med 23(7):1111–1130

    Article  PubMed  Google Scholar 

  44. Amir E, Evans DG, Shenton A, Lalloo F, Moran A, Boggis C, Wilson M, Howell A (2003) Evaluation of breast cancer risk assessment packages in the family history evaluation and screening programme. J Med Genet 40(11):807–814

    Article  PubMed  CAS  Google Scholar 

  45. Cook NR (2007) Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 115(7):928–935. doi:10.1161/CIRCULATIONAHA.106.672402

    Article  PubMed  Google Scholar 

  46. Cook NR, Buring JE, Ridker PM (2006) The effect of including C-reactive protein in cardiovascular risk prediction models for women. Ann Intern Med 145(1):21–29

    PubMed  CAS  Google Scholar 

  47. Janes H, Pepe MS, Gu W (2008) Assessing the value of risk predictions by using risk stratification tables. Ann Intern Med 149(10):751–760

    PubMed  Google Scholar 

  48. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM (2005) Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 97(16):1180–1184. doi:10.1093/jnci/dji237

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgment

This study was supported by the Health Intervention and Technology Assessment Program, the Thai Health Promotion Foundation, the Health Systems Research Institute, the Bureau of Policy and Strategy of the Ministry of Public Health, and Thai Health-Global Link Initiative Project.

Conflict of interest

TA: received honorarium from the Health Intervention and Technology Assessment Program, Ministry of Public Health, Thailand; received travel grants for conference from the Health Intervention and Technology Assessment Program and Faculty of Graduate Studies, Mahidol University, Thailand. YT, CW, VK, and AT have no conflicts of interest to declare.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ammarin Thakkinstian.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Anothaisintawee, T., Teerawattananon, Y., Wiratkapun, C. et al. Risk prediction models of breast cancer: a systematic review of model performances. Breast Cancer Res Treat 133, 1–10 (2012). https://doi.org/10.1007/s10549-011-1853-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10549-011-1853-z

Keywords

Navigation