Deploy TensorFlow object detection model in less than 5 minutes ?!
Follow me on Twitter for more AI / Cloud / Tech related stuff!
As we all know, starting a Proof of Concept around object detection is not that easy. Which framework should you use ? TensorFlow object detect ? Gluon CV ? PyTorch Vision ? And how to deploy them quickly ? After all I just want to get quick result for my demo !
Stay zen, and admire this beautiful picture of this beautiful moutain.
In the portfolio of Google’s projet around AI, there are two thing that could be combined to provide an answer to the previously stated problem. Let me introduce TensorFlow Object Detection, and TensorFlow Serving. The first one is a combination of script to easily train and export object detection models. It also contain a model zoo, with pretrained models. The second one is a production oriented tool to deploy model for inference with the best available TensorFlow guidelines.
It starts to be very interesting when you discover that the pretrained models in the model zoo are given in two format. One more oriented toward further training, the method is called transfer learning in the domain. And the other one, the saved model, is directly deployable by TensorFlow Serving.
So, in the end, we could use the official Docker image of TF Serving, add a little bit of scripting and get a container wrapping our model.
First, let’s choose a model to deploy in the model zoo. For our example, we will use “ssd_resnet_50_fpn_coco”. It will provide us a very good baseline, which is both fast and accurate.
Note the url of the model in zoo (you don’t have to download this file): http://download.tensorflow.org/models/object_detection/ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz
Let’s build our custom image !
You now have a REST API on localhost:8080 serving you model. But how to get inference ?
TensorFlow Serving has it’s very special definition of api endpoint and payload structure that you can check on official documentation. In our case, we will use the predict API.
I wrote a client to test the API, it will only print the response (containing detections) but it should be simple enough to let you hack with it.
Note that it require to install Pillow, numpy and request (pip install pillow numpy requests) and Python >3.6.
By removing the line 14, you could get a script compatible with 3.4 (and maybe 2.7, but I didn’t tested it).
Happy Hacking around object detection !
