bullet Sensors & Transducers Journal

    (ISSN: 2306-8515, e-ISSN 1726-5479)

   

    About this Journal

   Information for Authors

   Editorial Board

   Editorial Calendar

   Current Issue

   Browse Journal

S&T journal's cover

Submit Press Release

Submit White Paper

25 Top Downloaded Articles (2007-2012)

Search

Contact Us

 

 

 

 

Vol. 234, Issue 6, June 2019, pp. 1-8

 

Bullet

 

Variational Autoencoders and the Variable Collapse Phenomenon
 

Andrea Asperti

University of Bologna, Department of Informatics: Science and Engineering (DISI) Via Zamboni, 33, 40126 Bologna BO, Italy

 

Received: 15 May 2019 /Accepted: 15 June 2019 /Published: 30 June 2019

Digital Sensors and Sensor Sysstems

 

Abstract: In Variational Autoencoders, when working in high-dimensional latent spaces, there is a natural collapse of latent variables with minor significance, that get altogether neglected by the generator. We discuss this known but controversial phenomenon, sometimes referred to as overpruning, to emphasize the under-use of the model capacity. In fact, it is an important form of self-regularization, with all the typical benefits associated with sparsity: it forces the model to focus on the really important features, enhancing their disentanglement and reducing the risk of overfitting. In this article, we discuss the issue, surveying past works, and particularly focusing on the exploitation of the variable collapse phenomenon as a methodological guideline for the correct tuning of the model capacity, and of the loss function parameters.

 

Keywords: Variational autoencoders, Variable collapse, Overpruning, Sparsity, Kullback-Leibler divergence, Generative models, Gaussian mixture models.

 

Acrobat reader logo Click <here> or title of paper to download the full pages article in pdf format

 

 

This work is licensed under a Creative Commons 4.0 International License

 

 Creative Commons License
 

 

 

 

 


1999 - 2019 Copyright , International Frequency Sensor Association (IFSA) Publishing, S.L. All Rights Reserved.


Home - News - Links - Archives - Tools - Voltage-to-Frequency Converters - Standardization - Patents - Marketplace - Projects - Wish List - e-Shop - Sensor Jobs - Membership - Videos - Publishing - Site Map - Subscribe - Search

 Members Area -Sensors Portal -Training Courses - S&T Digest - For advertisers - Bookstore - Forums - Polls - Submit Press Release - Submit White Paper - Testimonies - Twitter - Facebook - LinkedIn