Rectifier Nonlinearities Improve Neural Network Acoustic Models

LeakyReLU：使用修正的非线性提高神经网络声学模型.

多任务学习(Multi-Task Learning)

Multi Task Learning.

Training Deeper Convolutional Networks with Deep Supervision

使用深度监督训练更深的卷积网络.

Deeply-Supervised Nets

深度监督网络.

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

ELU：消除偏差偏置的指数线性单元.

Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear Units

GELU：随机正则化的高斯误差线性单元.