Metric Learning/Contrastive Framework
FaceNet, used verbatim, may not be optimal. However the idea of class-level contrastive learning sounds like quite a promising way to promoting clustering. For problems where we expect more latent space structures from the data than the simple “in-the-same-class-or-not” clustering, this method may not suffice. It only promotes interclass distance maximization and intraclass distance minimization, but in cases where, for example, some classes are closer than others, such as cell types along the
Another issue with FaceNet is that the clustering is still label-dependent — so the latent space structure is post hoc and not clearly inherent to the input data. This might not be important for a classification problem — but is certainly important in cases when we care about interpretability (specifically in the sense of caring about scientifically meaningful representations).
Self-Supervised Learning (SSL)
Although these SSL methods do not proactively promote clustering. They allow you to leverage unlabelled data (which often exists in greater abundance compared to labelled data) to learn better and lower dimensional latent representations of the input data. Clustering methods such as
If we see promising result from UMAP on SSL-learned features, it is in some sense significantly more valuable than those learned from supervised problems or derivatives of supervised problems (like ). The SSL-learned features are trained without labels, so the emergent structure should be indicative of an inherent structure that is independent of the classification task, rather than post hoc structure that is a shadow of the classification task.
The intention of
Graphical Neural Networks
For introductory information about Graphical Neural Networks, see
It seems like GNN is not exactly directly relevant to the clustering task we are interested in.