TensorBoard Embedding Projector with Precomputed Embeddings

This is a short practical guide for using TensorBoard Embedding Projector with precomputed embeddings. For those who never used Embedding Projector, it is awesome. Google Machine Learning team continues to impress. With that being said, it comes with minimal documentation. Google has put out an example notebook yet it is tightly coupled with TensorFlow and Keras. It is not obvious how the integration works if you already have your embeddings computed and just want to load those into the tool. To make matters even more amusing or frustrating depending on your state of mind, you can easily upload embeddings as a tsv file into the web-based version of the tool yet this option doesn’t exist if you want to integrate the Projector with the rest of your code using the TensorBoard module.

The below integration example is based on Tensorflow 2.0 mostly for the sake of implementation speed. It should also be doable via TensorBoard PyTorch API. The code below assumes the implementation in the Google CoLab notebook. I further assume that you have your embeddings as a NumPy array with dimensions of [“number of embedding vectors” x “embeddings dimensionality” ] and the labels for your embeddings as a python list of strings with the length of [“number of embedding vectors”]

As always, we first need to import a bunch of stuff

Now we need to create a directory where TensorBoard will look for data

Metadata (labels) can be ingested as a tsv file

Embedding itself first needs to be transformed into the tf.variable and then stored as a tf.model checkpoint. It doesn’t matter that we don’t run any TF model, we still can have a checkpoint!

All that is left is to add a few lines of code to configure the Projector. Since I couldn’t find any documentation explaining what exactly this code is doing, it remains a mystery. My guess is that it is a tool configuration

With that, you have everything you need to run your very own Embedding Projector

After the TensorBoard loads, just pick the Projector option from the dropdown and enjoy!