Benchmarking variational AutoEncoders on cancer transcriptomics data

Mostafa Eltager, Tamim Abdelaal, Mohammed Charrout, Ahmed Mahfouz, Marcel J.T. Reinders, Stavros Makrodimitris*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)
15 Downloads (Pure)


Deep generative models, such as variational autoencoders (VAE), have gained increasing attention in computational biology due to their ability to capture complex data manifolds which subsequently can be used to achieve better performance in downstream tasks, such as cancer type prediction or subtyping of cancer. However, these models are difficult to train due to the large number of hyperparameters that need to be tuned. To get a better understanding of the importance of the different hyperparameters, we examined six different VAE models when trained on TCGA transcriptomics data and evaluated on the downstream tasks of cluster agreement with cancer subtypes and survival analysis. We studied the effect of the latent space dimensionality, learning rate, optimizer, initialization and activation function on the quality of subsequent downstream tasks on the TCGA samples. We found βTCVAE and DIP-VAE to have a good performance, on average, despite being more sensitive to hyperparameters selection. Based on these experiments, we derived recommendations for selecting the different hyperparameters settings. To ensure generalization, we tested all hyperparameter configurations on the GTEx dataset. We found a significant correlation (ρ = 0.7) between the hyperparameter effects on clustering performance in the TCGA and GTEx datasets. This highlights the robustness and generalizability of our recommendations. In addition, we examined whether the learned latent spaces capture biologically relevant information. Hereto, we measured the correlation and mutual information of the different representations with various data characteristics such as gender, age, days to metastasis, immune infiltration, and mutation signatures. We found that for all models the latent factors, in general, do not uniquely correlate with one of the data characteristics nor capture separable information in the latent factors even for models specifically designed for disentanglement.

Original languageEnglish
Article numbere0292126
JournalPLoS ONE
Issue number10 October
Publication statusPublished - 5 Oct 2023

Bibliographical note

Publisher Copyright:
Copyright: © 2023 Eltager et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: ME,MR: European Union’ H2020
research and innovation program under the MSCA
grant agreement [861190 (PAVE)]: ( TA,AM,MR: NWO Gravitation project:
BRAINSCAPES: A Roadmap from Neurogenetics to
Neurobiology (NWO: 024.004.012) SM&MR:have
received funding from the Convergence Health &
Technology program of the Delft University of
Technology and Erasmus Medical Center. The
funders had no role in study design, data collection
and analysis, decision to publish, or preparation of
the manuscript.


Dive into the research topics of 'Benchmarking variational AutoEncoders on cancer transcriptomics data'. Together they form a unique fingerprint.

Cite this