Deep Learning Via Semi-supervised Embedding

Abstract

We show how nonlinear embedding algorithms popular for use with <i>shallow</i> semi-supervised learning techniques such as kernel methods can be applied to deep multilayer architectures, either as a regularizer at the output layer, or on each layer of the architecture. This provides a simple alternative to existing approaches to <i>deep</i> learning whilst yielding competitive error rates compared to those methods, and existing <i>shallow</i> semi-supervised techniques.