What Makes a Reward Model a Good Teacher? An Optimization Perspective - View it on GitHub