Skip to content

Integrate TSNE and Facenet to faces engine

Minh Nghia Duong requested to merge tsne-streaming-processing into gsoc21-faces-engine

Hi,

I did some tests with TSNE in order to check its performance with continuous data usecase. Unfortunately, if we separate data into smaller subsets and use sliding window to update as new data came in, the projected data will be all messed up since it falls into a local structure instead of having a global view of the dataset.

Currently, if we run TSNE with 4CPU on the dataset of around 13000 entries; it will take around 3 minutes to finish the computation. The face classification for each face on the projected data takes less than 1ms. However, with the situation, we need to reproject the entire face data every time new faces are added. It can be expensive, depend on how often new data is added and the size of the existing data.

I have a proposition to push the Face embedding extraction and TSNE projection as background processing, right after face detection. This way, the processing can take a few minutes but users shall merely notice it.

Merge request reports