Marginalized Denoising Autoencoders For Domain Adaptation

Abstract

Glorot et al. (2011) have successfully used Stacked Denoising Autoencoders (SDAs) (Vincent et al., 2008) to learn new representations for domain adaptation, resulting in record accuracy levels on well-known benchmark datasets for sentiment analysis. The representations are learned by reconstructing input data from partial corruption. In this paper, we introduce a variation, marginalized SDA (mSDA). In contrast to the original SDA, mSDA requires no training through back-propagation as we explicitly marginalize out the feature corruption and solve for the parameters in closed form. mSDA learns representations that lead to comparable accuracy levels, but can be implemented in only 20 lines of MATLAB and reduces the computation time on large data sets from several days to mere minutes.