1 Institute of Laboratory Medicine, Clinical Chemistry and Molecular Diagnostics, University Hospital Leipzig, Paul-List-Str. 13-15, D-04103 Leipzig, Germany.
2 Anesthesiology and Intensive Care Medicine, University Hospital Greifswald, Ferdinand-Sauerbruch-Straße, D-17475 Greifswald, Germany.

Correspondence: Sebastian Gibb <>

Last updated: 2021-08-19

Checks: 6 1

Knit directory: ampel-leipzig-meld/

This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.


The R Markdown file has unstaged changes. To know which version of the R Markdown file created these results, you’ll want to first commit it to the Git repo. If you’re still working on the analysis, you can ignore this warning. When you’re finished, you can run wflow_publish to commit the R Markdown file and build the HTML.

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

The command set.seed(20210604) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 8f25899. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:


Ignored files:
    Ignored:    _targets/
    Ignored:    container/
    Ignored:    logs/

Untracked files:
    Untracked:  METHODS
    Untracked:  TODO
    Untracked:  coxtime-mwe.R
    Untracked:  coxtime.R
    Untracked:  deepsurv-mwe.R
    Untracked:  deepsurv.R
    Untracked:  scripts/R.sh

Unstaged changes:
    Modified:   Makefile
    Modified:   _targets.R
    Modified:   analysis/article.Rmd
    Modified:   analysis/benchmarks.Rmd
    Modified:   code/01-data.R
    Modified:   code/02-mlr-tasks.R
    Modified:   code/03-mlr-learners.R
    Modified:   code/04-mlr-benchmarking.R
    Modified:   guix/channel/ampel/packages/rpackages.scm
    Modified:   guix/manifest.scm
    Modified:   scripts/Rscript.sh

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.


These are the previous versions of the repository in which changes were made to the R Markdown (analysis/article.Rmd) and HTML (docs/article.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
html afa48d9 Sebastian Gibb 2021-08-07 chore: rebuild site
Rmd d32f7b3 Sebastian Gibb 2021-08-07 fix: gtsummary compact theme
Rmd ebf878b Sebastian Gibb 2021-08-07 fix: observed vs expected mortality table
Rmd 4ec3d40 Sebastian Gibb 2021-08-04 feat: add table 1 and table observed vs expected mortality
Rmd e41dd8c Sebastian Gibb 2021-08-03 feat: add flowchart and discussion notes
Rmd 523c286 Sebastian Gibb 2021-08-02 refactor: use bookdown::html_document2 instead of workflowr::wflow_html to allow crossreferences
html 3aab3e1 Sebastian Gibb 2021-08-01 chore: rebuild site
Rmd 91bfbd2 Sebastian Gibb 2021-08-01 feat: add first article draft with pandoc filters

1 Introduction

Liver cirrhosis is the terminal result of fibrotic remodeling of liver tissue due to chronic damage. An organ failure is often irreversible and the only available therapy is liver transplantation. However, the shortage in grafts for transplantation from deceased donors requires risk stratification and precise allocation rules. The allocation of liver transplantation in most countries is based on disease severity determined by the model of end-stage liver disease (MELD) (Malinchoc et al. 2000; Wiesner et al. 2003; Organ Procurement and Transplantation Network 2021). The MELD score estimates the patients’ 3-month mortality risk based on laboratory results, namely bilirubin, creatinine and international normalized ratio (INR). In general the MELD score is extended by the sodium level (MELD-Na score) as this was found to be an important additional risk factor in liver cirrhosis (Kim et al. 2008; Organ Procurement and Transplantation Network 2021). The MELD was initially developed for predicting the survival of patients undergoing transjugular intrahepatic portosystemic shunts. Afterwards it was revalidated for predicting mortality risk in patients awaiting a liver transplantation. Although the MELD score should be an objective allocation score especially the creatinine and INR are highly dependent on the utilized laboratory methods (Trotter et al. 2004; Cholongitas et al. 2007). Patients with identical disease state could have very different MELD scores and thus get different priority on the liver transplantation waiting list. Furthermore often, e.g. for acute-on-chronic liver failure, the MELD score underestimates the mortality risk (Hernaez et al. 2020).

There have been some attempts to use the data extracted from more than 300.000 electronic medical records from two hospitals in the United States to improve the MELD score (Kartoun et al. 2017). The derived MELD-Plus7 and MELD-Plus9 risk score add albumin, white blood cell count, age and total cholesterol and length of stay to the MELD-Na variables. Despite its published prediction improvement the MELD-Plus scores are not used for transplant allocation yet.

As depicted by MELD-Plus risk scores better predictive scores often need more variables and are more complicated. To reduce the risk of overlooking or incorrectly calculating and interpreting the results, clinical decision support systems may be used and could improve patient safety. The research project on digital laboratory medicine (AMPEL) develops a clinical decision support system based on laboratory diagnostics that should support clinical practitioners in interpreting the laboratory results and taking the necessary medical interventions (Eckelt et al. 2020).

This study aims to find clinical and laboratory values that improve the risk stratification for liver transplantation over classical MELD, MELD-Na and MELD-Plus scores and could be implemented as part of the AMPEL clinical decision support system.

2 Material and methods

2.1 Study population

In a retrospective cohort study we followed 778 consecutive patients, who were recruited during the evaluation process for liver transplantation at the University Hospital of Leipzig from November 2012 to Juni 2015. For each patient we recorded 44 variables. Among them are age, sex, etiology of liver disease, complications as listed in Table 3.1 and 25 laboratory measurements.

Figure 2.1: Flowchart Inclusion/Exclusion.

We excluded 124 patients from our analysis (Figure 2.1). One was younger than 18 years. 24 had a liver transplantation before and in 99 cases the follow up data were missing or invalid.

The Ethics Committee at the Leipzig University Faculty of Medicine approved the retrospective usage of the data for our study (reference number: 039/14ff).

2.2 MELD scores

MELD and MELD-Na were calculated as described in Organ Procurement and Transplantation Network (2021) using the following formulas: \(MELD = 10 * (0.957 \ln(creatinine [mg/dl]) + 0.378 \ln(bilirubin [mg/dl]) + 1.120 * \ln(INR) + 0.643)\), Creatinine, bilirubin and INR values lower than 1.0 mg/dl were set to 1.0 mg/dl. The maximum allowed creatinine was 4.0 mg/dl. If patients received dialysis, creatinine was set to 4.0 mg/dl. For MELD-Na MELD was calculated as above and recalculated if greater than 11 using: \(MELD\text{-}Na = MELD_i + 1.32 * (137 - Na [mmol/l]) - (0.033 * MELD_i * (137 - Na [mmol/l]))\)

Serum sodium values lower than 125 mmol/l or higher than 137 mmol/l were set to 125 mmol/l and 137 mmol/l, respectively.

The MELD-Plus7 risk score was calculated as described in Kartoun et al. (2017): \[ \begin{aligned} L = 8.53499496 &+ \\ 2.59679650 &* \log_{10}(1 + creatinine [mg/dl]) + \\ 2.06503238 &* \log_{10}(1 + bilirubin [mg/dl]) + \\ 2.99724802 &* \log_{10}(1 + INR) - \\ 6.47834101 &* \log_{10}(1 + sodium [mmol/l]) - \\ 6.34990436 &* \log_{10}(1 + albumin [g/l]) + \\ 1.92811726 &* \log_{10}(1 + wbc [th/cumm]) + \\ 0.04070442 &* age [years] \end{aligned} \]

\[MELD\text{-}Plus = \frac{\exp(L)}{1 + \exp(L)}\]

The MELD-Plus9 score was ignored because the length of stay information was not available for our cohort and the model itself wasn’t superior to MELD-Plus7 in its original publication (Kartoun et al. 2017).

2.3 Machine learning

3 Results

3.1 Baseline characteristics

Table 3.1: Baseline characteristics. AIH, autoimmune hepatitis; ALAT, alanine aminotransferase; ALF, acute liver failure; ASAT, aspartate aminotransferase; CLI, chronic liver insufficiency; GIB, gastrointestinal bleeding; HBV, hepatitis B virus; HCV, hepatitis C virus; IL-6, interleukin 6; INR, international normalized ratio; LTx, liver transplantation; NASH, non-alcoholic steatohepatitis; PBC, primary biliary cirrhosis; PSC, primary sclerosing cholangitis; SBP, spontaneous bacterial peritonitis
Characteristic Female, N = 240 Male, N = 414 Overall, N = 654
General
Follow-up time [days] 181 (17, 411) 198 (54, 378) 191 (38, 384)
Age 56 (51, 64) 58 (52, 63) 58 (52, 64)
LTx 20 (8.3%) 40 (9.7%) 60 (9.2%)
Laboratory measurements
Total bilirubin [µmol/l] 21 (10, 49) 26 (14, 51) 24 (13, 50)
Cystatin C [mg/l] 1.31 (1.04, 1.80) 1.32 (1.08, 1.76) 1.32 (1.07, 1.79)
(Missing) 2 5 7
Creatinine [µmol/l] 73 (58, 91) 85 (71, 110) 80 (65, 104)
INR 1.20 (1.06, 1.50) 1.22 (1.10, 1.44) 1.21 (1.09, 1.45)
(Missing) 1 1 2
Sodium [mmol/l] 138.8 (135.4, 141.0) 138.0 (135.0, 140.1) 138.1 (135.2, 140.4)
(Missing) 5 4 9
WBC [exp 9/l] 6.7 (4.7, 8.8) 6.2 (4.6, 7.8) 6.3 (4.7, 8.3)
(Missing) 10 13 23
IL-6 [pg/ml] 10 (5, 36) 14 (6, 47) 13 (6, 42)
(Missing) 27 29 56
Albumin [g/l] 41 (35, 46) 38 (33, 43) 39 (34, 45)
(Missing) 2 4 6
Total protein [g/l] 71 (63, 76) 71 (65, 76) 71 (65, 76)
(Missing) 2 4 6
Cholesterine [mmol/l] 4.64 (3.41, 5.56) 4.28 (3.34, 5.15) 4.40 (3.34, 5.31)
ALAT [µkat/l] 0.29 (0.21, 0.42) 0.36 (0.23, 0.56) 0.33 (0.22, 0.48)
(Missing) 2 8 10
ASAT [µkat/l] 0.68 (0.49, 1.05) 0.81 (0.61, 1.23) 0.77 (0.55, 1.16)
(Missing) 3 5 8
MELD
MELD score 10 (7, 18) 12 (9, 17) 11 (8, 17)
(Missing) 2 1 3
MELD Category
[6,9] 111 (47%) 146 (35%) 257 (39%)
[10,20) 80 (34%) 196 (47%) 276 (42%)
[20,30) 30 (13%) 50 (12%) 80 (12%)
[30,40) 12 (5.0%) 14 (3.4%) 26 (4.0%)
[40,52) 5 (2.1%) 7 (1.7%) 12 (1.8%)
(Missing) 2 1 3
MELD-Na score 10 (7, 19) 12 (9, 20) 11 (8, 20)
(Missing) 5 4 9
MELD-Plus7 risk score 0.04 (0.02, 0.09) 0.05 (0.03, 0.11) 0.04 (0.02, 0.11)
(Missing) 17 18 35
Entities
Cirrhosis 203 (85%) 391 (94%) 594 (91%)
(Missing) 1 0 1
ALF 6 (2.5%) 2 (0.5%) 8 (1.2%)
CLI 30 (12%) 21 (5.1%) 51 (7.8%)
Etiology (more than 1 per patient possible)
Ethyltoxic 114 (48%) 297 (72%) 411 (63%)
HBV 4 (1.7%) 16 (3.9%) 20 (3.1%)
HCV 17 (7.1%) 28 (6.8%) 45 (6.9%)
AIH 22 (9.2%) 9 (2.2%) 31 (4.7%)
PBC 17 (7.1%) 0 (0%) 17 (2.6%)
PSC 5 (2.1%) 11 (2.7%) 16 (2.4%)
NASH 17 (7.1%) 31 (7.5%) 48 (7.3%)
Cryptogenic 36 (15%) 32 (7.7%) 68 (10%)
Complications (more than 1 per patient possible)
Dialysis 14 (5.9%) 19 (4.6%) 33 (5.1%)
(Missing) 1 1 2
GIB 58 (24%) 104 (25%) 162 (25%)
HCC 18 (7.5%) 103 (25%) 121 (19%)
SBP 31 (13%) 60 (15%) 91 (14%)
(Missing) 0 1 1
Mortality
Within 7 days 16 (7.2) 18 (4.5) 34 (5.5)
Within 30 days 25 (12) 31 (8.1) 56 (9.4)
Within 90 days 32 (16) 49 (13) 81 (14)
Within 365 days 39 (20) 61 (18) 100 (18)

3.2 MELD scores

According to Wiesner et al. (2003) we divided our cohort into 5 MELD-score risk categories, with 257, 276, 80, 26, and 12 patients, respectively. As depicted in Figure ?? and Table 3.2 our observed 90 day mortality was 1.4 to 2.9 times higher than the predicted one. Just in the group with the lowest MELD scores the mortality was much lower.

Table 3.2: Observed vs MELD-expected 90 day mortality. MELD mortality values are taken from Wiesner et al. (2003). All patients censored before day 90 are ignored for the calculation of the MELD-expected deaths. SMR, Standardized mortality ratio = observed deaths/expected deaths.
MELD category Observed deaths (n) Expected deaths (n) Standardized mortality ratio (SMR) Observed mortality (%) Expected mortality (%)
[6,9] 1 4.0 0.3 0.5 1.9
[10,20) 21 13.0 1.6 8.9 6.0
[20,30) 34 11.6 2.9 51.0 19.6
[30,40) 18 10.0 1.8 92.8 52.6
[40,52) 6 4.3 1.4 100.0 71.3

4 Discussion

In this retrospective analysis of 654 consecutive patients, who were recruited during the evaluation process for liver transplantation at the University Hospital of Leipzig we compared 11 different statistical and machine learning algorithms.

  • higher mortality than predicted, compare Hernaez et al. (2020)
  • adding sodium doesn’t add much/any discrimination information
    • as already seen in Kartoun et al. (2017)
  • Kartoun et al. (2017) uses > 60 variables mostly clinical stuff, just few laboratory values, none of the clinical stuff remains
  • general low sample size thus simpler models/fewer predictors are preferred.
  • comparison of different predictors of MELD-Plus7 (+MELD-Plus9) vs our features
    • 90 day mortality post discharge
    • poorer performance in own validation set (lower and more uniform disease severity, also lower MELD)
  • machine learning algorithms worse than statistical learning (lower c-index, higher variance/instability)
    • additive effects (favour statistical models)
    • low sample size, for ML more than 200 events per predictor variable necessary (Ploeg, Austin, and Steyerberg 2014)
  • national register required

4.1 Limitations

  • comparable mortality?
  • retrospective analysis
  • single center
  • unknown cause of death (death with vs of end-stage liver disease)
  • bias caused by LTx censorship?

5 References

Cholongitas, E., L. Marelli, A. Kerry, M. Senzolo, D. W. Goodier, D. Nair, M. Thomas, D. Patch, and A. K. Burroughs. 2007. “Different Methods of Creatinine Measurement Significantly Affect MELD Scores.” Liver Transplantation 13 (4): 523–29. https://doi.org/10.1002/lt.20994.

Eckelt, F., J. Remmler, T. Kister, M. Wernsdorfer, H. Richter, M. Federbusch, M. Adler, et al. 2020. “Verbesserte Patientensicherheit Durch “Clinical Decision Support Systems” in Der Labormedizin.” Der Internist 61 (5): 452–59. https://doi.org/10.1007/s00108-020-00775-3.

Hernaez, R., Y. Liu, J. R. Kramer, A. Rana, H. B. El-Serag, and F. Kanwal. 2020. “Model for End-Stage Liver Disease-Sodium Underestimates 90-Day Mortality Risk in Patients with Acute-on-Chronic Liver Failure.” Journal of Hepatology, June. https://doi.org/10.1016/j.jhep.2020.06.005.

Kartoun, U., K. E. Corey, T. G. Simon, H. Zheng, R. Aggarwal, K. Ng, and S. Y. Shaw. 2017. “The MELD-Plus: A Generalizable Prediction Risk Score in Cirrhosis.” PLOS ONE 12 (10): e0186301. https://doi.org/10.1371/journal.pone.0186301.

Kim, W. R., S. W. Biggins, W. K. Kremers, R. H. Wiesner, P. S. Kamath, J. T. Benson, E. Edwards, and T. M. Therneau. 2008. “Hyponatremia and Mortality Among Patients on the Liver-Transplant Waiting List.” New England Journal of Medicine 359 (10): 1018–26. https://doi.org/10.1056/NEJMoa0801209.

Malinchoc, M., P. S. Kamath, F. D. Gordon, C. J. Peine, J. Rank, and P. C. J. ter Borg. 2000. “A Model to Predict Poor Survival in Patients Undergoing Transjugular Intrahepatic Portosystemic Shunts.” Hepatology 31 (4): 864–71. https://doi.org/10.1053/he.2000.5852.

Organ Procurement and Transplantation Network. 2021. “Policies.” 2021. https://optn.transplant.hrsa.gov/media/1200/optn_policies.pdf.

Ploeg, T. van der, P. C. Austin, and E. W. Steyerberg. 2014. “Modern Modelling Techniques Are Data Hungry: A Simulation Study for Predicting Dichotomous Endpoints.” BMC Medical Research Methodology 14 (1). https://doi.org/10.1186/1471-2288-14-137.

Trotter, J. F., B. Brimhall, R. Arjal, and C. Phillips. 2004. “Specific Laboratory Methodologies Achieve Higher Model for Endstage Liver Disease (MELD) Scores for Patients Listed for Liver Transplantation.” Liver Transplantation 10 (8): 995–1000. https://doi.org/10.1002/lt.20195.

Wiesner, R., E. Edwards, R. Freeman, A. Harper, R. Kim, P. Kamath, W. Kremers, et al. 2003. “Model for End-Stage Liver Disease (MELD) and Allocation of Donor Livers.” Gastroenterology 124 (1): 91–96. https://doi.org/10.1053/gast.2003.50016.


sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-unknown-linux-gnu (64-bit)

Matrix products: default
BLAS/LAPACK: /gnu/store/bs9pl1f805ins80xaf4s3n35a0x2lyq3-openblas-0.3.9/lib/libopenblasp-r0.3.9.so

locale:
 [1] LC_CTYPE=de_DE.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=de_DE.UTF-8        LC_COLLATE=de_DE.UTF-8    
 [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=de_DE.UTF-8   
 [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] gtsummary_1.4.2 ameld_0.0.12    survival_3.2-11 targets_0.4.2  

loaded via a namespace (and not attached):
 [1] sass_0.4.0              tidyr_1.1.3             jsonlite_1.7.2         
 [4] splines_4.1.0           lgr_0.4.2               highr_0.9              
 [7] set6_0.2.1              yaml_2.2.1              mlr3misc_0.9.3         
[10] bbotk_0.3.2             pillar_1.6.2            backports_1.2.1        
[13] lattice_0.20-44         glue_1.4.2              uuid_0.1-4             
[16] digest_0.6.27           RColorBrewer_1.1-2      promises_1.2.0.1       
[19] checkmate_2.0.0         colorspace_2.0-2        htmltools_0.5.1.1      
[22] httpuv_1.6.1            Matrix_1.3-4            pkgconfig_2.0.3        
[25] bookdown_0.22           DiagrammeR_1.0.6.1      purrr_0.3.4            
[28] scales_1.1.1            processx_3.5.2          mlr3_0.11.0            
[31] whisker_0.4             later_1.2.0             git2r_0.28.0           
[34] tibble_3.1.3            generics_0.1.0          distr6_1.5.2           
[37] ggplot2_3.3.5           ellipsis_0.3.2          mlr3proba_0.4.0        
[40] withr_2.4.2             cli_3.0.1               magrittr_2.0.1         
[43] crayon_1.4.1            paradox_0.7.1           evaluate_0.14          
[46] ps_1.6.0                fs_1.5.0                fansi_0.5.0            
[49] parallelly_1.27.0       broom.helpers_1.3.0     palmerpenguins_0.1.0   
[52] tools_4.1.0             data.table_1.14.0       lifecycle_1.0.0        
[55] stringr_1.4.0           munsell_0.5.0           callr_3.7.0            
[58] compiler_4.1.0          rlang_0.4.11            grid_4.1.0             
[61] rstudioapi_0.13         gt_0.3.0                htmlwidgets_1.5.3      
[64] visNetwork_2.0.9        R62S3_1.4.1             igraph_1.2.6           
[67] rmarkdown_2.9           gtable_0.3.0            codetools_0.2-18       
[70] mlr3learners_0.4.5      R6_2.5.0                knitr_1.33             
[73] dplyr_1.0.7             utf8_1.2.2              commonmark_1.7         
[76] mlr3extralearners_0.5.0 workflowr_1.6.2         rprojroot_2.0.2        
[79] mlr3tuning_0.8.0        stringi_1.7.3           parallel_4.1.0         
[82] Rcpp_1.0.7              vctrs_0.3.8             tidyselect_1.1.1       
[85] xfun_0.24