Algorithm-Machine-Learning--photo-istockphoto-1224500457-2048x1365.jpgKlinrisk model shows strong performance in identifying high-risk CKD patients in the US

The Klinrisk chronic kidney disease progression model has been validated in US populations, offering potential for early intervention and cost savings, and high accuracy across healthcare systems.  

Domain(s): CKD, ESRD, machine learning


Summary

Background
Chronic kidney disease (CKD) awareness is low, and the disease is often caught late due to the asymptomatic nature of earlier stages[1],[2]. Early intervention of high-risk (CKD) can facilitate optimal medical management and improve outcomes[3],[4]. The previously validated Klinrisk chronic kidney disease progression model accurately predicts a 40% decline in estimated glomerular filtration rate (eGFR) or kidney failure using commonly available laboratory and demographic data in two Canadian samples[5],[6],[7]. We aimed to test the accuracy of this model in large US commercial, Medicare, and Medicaid populations. This validation is crucial for CKD patients as the model has the potential to identify high-risk individuals early, enabling timely intervention that could change the trajectory of CKD, prevent health complications and reduce associated costs.

Methods/Design
Three cohorts, consisting of insured adults enrolled in (a) commercial, (b) Medicare, and (c) Medicaid plans from the Carelon Research Healthcare Integrated Research Database (HIRD®) between 01/01/2007 and 12/31/2020 with at least 1 serum creatinine test, an eGFR between 15ml/min/1.73m2 and 180ml/min/1.73m2, and at least 7 of the 19 other laboratory analytes available were developed. From a payer perspective, we validated the Klinrisk model's ability to predict CKD progression over 2 years, focusing on the composite outcome of either a sustained 40% decline in eGFR or kidney failure. Model performance was evaluated using two elements: (1) how effectively to distinguish between different outcomes by area under the receiver operator characteristic curve (AUC), and (2) accuracy of a prediction by Brier scores. We also evaluated utilization and costs of care by category of risk.

Results
In all three cohorts, the Klinrisk model was effective at distinguishing between patients with worsening CKD and those without. This is shown by its high accuracy and precision score, which are summarized in Table 1. 

Table 1 Model performance for the prediction of 40% decline in eGFR or kidney failure at 2 years

Commercial

Medicare

Medicaid

n

4,410,131

341,666

92,956

AUC*
 (95% CI)

0.83
 (0.82 - 0.83)

0.80
 (0.79 - 0.80)

0.82
 (0.82 – 0.82)

Brier Score+
 (95% CI)

0.004
 (0.004 - 0.004)

0.023
 (0.023 – 0.023)

0.011
 (0.011 – 0.012)

*The AUC ranges from 0.5 to 1.0. An AUC of 0.5 signifies that the test is equivalent to random chance in differentiating between individuals with and without the disease. In contrast, an AUC of 1.0 represents perfect discrimination. Generally, AUC values above 0.80 are considered clinically valuable, whereas those below 0.80 have limited clinical significance.[8]
+ The Brier Score ranges from 0 to 1 and measures the accuracy of a predictive model. A lower score indicates higher accuracy, with 0 representing perfect accuracy.


When we grouped patients based on their risk levels, we found a clear difference between high-risk (top 10%) and low-risk (bottom 50%) individuals.

  • Increasing 8-16 times more in CKD-related inpatient admissions and ER visits for high-risk patients (Figure 1)
  • At least triple all-cause costs per patient per month (PPPM) and fifteen times more CKD-related PPPM costs for high-risk patients (Figure 2)

Figure 1                                                                                                         Figure 2 

1Klinrisk.png                                      2Klinrisk.png


Key takeaways

  • The algorithm performed well across time horizons among insured members. Our analysis focused on a 2-year period, aligning with a payer perspective to optimize risk management and resource allocation.
  • There was a significant healthcare utilization and cost disparity between high and low-risk patient groups. This indicates that the model effectively distinguishes individual-level risk, allowing for targeted interventions.
  • This study, with its extensive sample size, confirms the model's robustness in predicting CKD progression across varied healthcare systems. Robust validation with routinely collected data is necessary and feasible before clinical implementation.
  • Algorithm users can choose different levels of risk based on their needs, allowing them to customize their approach for specific medical situation. This is similar to recent developments where multiple prediction models have been created to improve the accuracy of earl-stage CKD predictions, going beyond what exiting tools like Kidney failure Risk Equation (KFRE) tool9],[10].


Publications

  • "Validation of the Klinrisk machine learning model for CKD progression in a large representative US population" submitted to Kidney International (under review).
  • Tangri, Navdeep1; Ferguson, Thomas W.1; Bamforth, Ryan J.1; Teng, Chia-Chen2; Smith, Joseph L.2*; Guzman, Maria2; Goss, Ashley3. External Validation of the Klinrisk Model in US Commercial, Medicare Advantage, and Medicaid Populations: SA-OR37. Journal of the American Society of Nephrology 34(11S):p 71, November 2023. | DOI: 10.1681/ASN.20233411S171a


Carelon Research Project team: Chia-Chen Teng, Joseph L. Smith*, Maria Guzman 
*Carelon Research associate at the time of the study.


For more information on a specific study or to connect with the Actionable Insights Committee,
contact us at [email protected].

Sponsor: This study was conducted by Carelon Research, Inc. in collaboration with Elevance Health and Boehringer Ingelheim Pharmaceuticals, Inc.

Dissemination and sharing of the Newsletter is limited to Elevance Health and its subsidiaries, and included findings and implications are for Elevance Health and its affiliates’ internal use only.

[1] Tuot, D. S. et al. Chronic kidney disease awareness among individuals with clinical markers of kidney dysfunction. Clin J Am Soc Nephrol 6, 1838-1844 (2011). https://doi.org/10.2215/CJN.00730111
[2] Shlipak, M. G. et al. The case for early identification and intervention of chronic kidney disease: conclusions from a Kidney Disease: Improving Global Outcomes (KDIGO) Controversies Conference. Kidney Int 99, 34-47 (2021). https://doi.org/10.1016/j.kint.2020.10.012 
[3] Heerspink, H. J. L. et al. Dapagliflozin in Patients with Chronic Kidney Disease. N Engl J Med 383, 1436-1446 (2020). https://doi.org/10.1056/NEJMoa2024816
[4] Neal, B. et al. Canagliflozin and Cardiovascular and Renal Events in Type 2 Diabetes. N Engl J Med 377, 644-657 (2017). https://doi.org/10.1056/NEJMoa1611925 
[5] Ferguson, T. et al. Development and External Validation of a Machine Learning Model for Progression of CKD. Kidney Int Rep 7, 1772-1781 (2022). https://doi.org/10.1016/j.ekir.2022.05.004 
[6] Tangri, N. et al. Validation of the Klinrisk chronic kidney disease progression model in the FIDELITY population. Clin Kidney J 17, sfae052 (2024). https://doi.org/10.1093/ckj/sfae052 
[7] Tangri, N. et al. Machine learning for prediction of chronic kidney disease progression: Validation of the Klinrisk model in the CANVAS Program and CREDENCE trial. Diabetes Obes Metab (2024). https://doi.org/10.1111/dom.15678
[8] Çorbacıoğlu ŞK, Aksel G. Receiver operating characteristic curve analysis in diagnostic accuracy studies: A guide to interpreting the area under the curve value. Turk J Emerg Med. 2023 Oct 3;23(4):195-198. doi: 10.4103/tjem.tjem_182_23. PMID: 38024184; PMCID: PMC10664195.
[9] Tangri, N. et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA 305, 1553-1559 (2011). https://doi.org/10.1001/jama.2011.451
[10] Kidney Disease: Improving Global Outcomes, C. K. D. W. G. KDIGO 2024 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease. Kidney Int 105, S117-S314 (2024). https://doi.org/10.1016/j.kint.2023.10.018 

Ready to get started? Sign up now!

Lorem ipsum dolor sit amet

2-Col, Left Image

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut.

  • Many addon features
  • Fully responsive & adaptive
  • SEO optimized
  • Attractive with a modern touch
  • Full Support

Consectetur adipiscing elit...

Joanna C.

"Et harum quidem rerum facilis est et expedita distinctio!"

Stanley T.

"Nam libero tempore, cum soluta nobis est eligendi."

Danielle W.

"Temporibus autem quibusdam et aut officiis debitis!"

Teams at Carelon Research