A workaround to issues with Rllib, given it does not work for your current gym environment. CarRacing-v0 is one of these. - View it on GitHub