Sample code for parallelizing across multiple CPU/GPUs on a single machine to speed up deep learning inference - View it on GitHub