This post has moved to Knowledge Distillation - Can a student exceed their teacher performance.