This is a note for image-matching things in the deep learning field.
Started this in 2020’s summer and it is a part of an SRTP project regarding an AR app with Haojie He and Lingming Zhang.
SuperPoint & SuperGlue
Overview
SuperPoint is a great feature extractor.
SuperGlue is a good image-matching network based on features extracted from images.
It is a really nice project by Magic Leap.
I did a little simplification and put the code here.
Experience and thoughts
For SuperPoint, I think it is a mature network and can be used as a helpful feature extracting tool. (And there are some wonderful tricks in the code! )
For SuperGlue, I didn’t quite get the key to it, such as the Attention Network and the Encoding Method.
In addition, it is quite hard to transfer models from SuperGlue. I have already transferred SuperPoint into Libtorch code, which can be used in C++. It is also easy to transfer to ONNX or other frameworks. However, I suffered greatly in transferring SuperGlue into libtorch code or ONNX. Hence the SuperGlue can work fine on Python but has some difficulty ( for me ) to be deployed.
So I think, later on, I will find a more simple ( hopefully, also more powerful ) matching network so that it can be easily deployed. Or maybe I can create my own networks and do some training.
