Code for Deep RL from Human Preferences [Christiano et al]. Plus a webapp for collecting human feedback - View it on GitHub