bullet Sensors & Transducers Journal

    (ISSN: 2306-8515, e-ISSN 1726-5479)


    About this Journal

   Information for Authors

   Editorial Board

   Editorial Calendar

   Current Issue

   Browse Journal

S&T journal's cover

Submit Press Release

Submit White Paper

25 Top Downloaded Articles (2007-2012)


Contact Us





Vol. 234, Issue 6, June 2019, pp. 1-8




Variational Autoencoders and the Variable Collapse Phenomenon

Andrea Asperti

University of Bologna, Department of Informatics: Science and Engineering (DISI) Via Zamboni, 33, 40126 Bologna BO, Italy


Received: 15 May 2019 /Accepted: 15 June 2019 /Published: 30 June 2019

Digital Sensors and Sensor Sysstems


Abstract: In Variational Autoencoders, when working in high-dimensional latent spaces, there is a natural collapse of latent variables with minor significance, that get altogether neglected by the generator. We discuss this known but controversial phenomenon, sometimes referred to as overpruning, to emphasize the under-use of the model capacity. In fact, it is an important form of self-regularization, with all the typical benefits associated with sparsity: it forces the model to focus on the really important features, enhancing their disentanglement and reducing the risk of overfitting. In this article, we discuss the issue, surveying past works, and particularly focusing on the exploitation of the variable collapse phenomenon as a methodological guideline for the correct tuning of the model capacity, and of the loss function parameters.


Keywords: Variational autoencoders, Variable collapse, Overpruning, Sparsity, Kullback-Leibler divergence, Generative models, Gaussian mixture models.


Acrobat reader logo Click <here> or title of paper to download the full pages article in pdf format



This work is licensed under a Creative Commons 4.0 International License


 Creative Commons License





1999 - 2019 Copyright , International Frequency Sensor Association (IFSA) Publishing, S.L. All Rights Reserved.

Home - News - Links - Archives - Tools - Voltage-to-Frequency Converters - Standardization - Patents - Marketplace - Projects - Wish List - e-Shop - Sensor Jobs - Membership - Videos - Publishing - Site Map - Subscribe - Search

 Members Area -Sensors Portal -Training Courses - S&T Digest - For advertisers - Bookstore - Forums - Polls - Submit Press Release - Submit White Paper - Testimonies - Twitter - Facebook - LinkedIn