If you’ve had a chance to play around with OpenCV 3 (and do a lot of work with keypoint detectors and feature descriptors) you may have noticed that the SIFT and SURF implementations are no longer included in the OpenCV 3 library by default.
Unfortunately, you probably learned this lesson the hard way by opening up a terminal, importing OpenCV, and then trying to instantiate your favorite keypoint detector, perhaps using code like the following:
$ python
>>> import cv2
>>> detector = cv2.FeatureDetector_create("SIFT")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'FeatureDetector_create'
Oh no! There is no longer a cv2.FeatureDetector_create method!
The same is true for our cv2.DescriptorExtractor_create function as well:
>>> extractor = cv2.DescriptorExtractor_create("SIFT")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'DescriptorExtractor_create'
Furthermore, cv2.SIFT_create and cv2.SURF_create will fail as well:
>>> cv2.SIFT_create() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'module' object has no attribute 'SIFT_create' >>> cv2.SURF_create() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'module' object has no attribute 'SURF_create'
I’ll be honest — this had me scratching my head at first. How am I supposed to access SIFT, SURF, and my other favorite keypoint detectors and local invariant descriptors if cv2.FeatureDetector_create and cv2.DescriptorExtractor_create have been removed?
The cv2.FeatureDetector_create and cv2.DescriptorExtractor_create were (and still are) methods I used all the time. And personally, I really liked the OpenCV 2.4.X implementation. All you needed to do was pass in a string and the factory method would build the instantiation for you. You could then tune the parameters using the getter and setter methods of the keypoint detector or feature descriptor.
Furthermore, these methods have been part of OpenCV 2.4.X for many years. Why in the world were they removed from the default install? And where were they moved to?
In the remainder of this blog post, I’ll detail why certain keypoint detectors and local invariant descriptors were removed from OpenCV 3.0 by default. And I’ll also show you where you can find SIFT, SURF, and other detectors and descriptors in the new version of OpenCV.
Why were SIFT and SURF removed from the default install of OpenCV 3.0?
SIFT and SURF are examples of algorithms that OpenCV calls “non-free” modules. These algorithms are patented by their respective creators, and while they are free to use in academic and research settings, you should technically be obtaining a license/permission from the creators if you are using them in a commercial (i.e. for-profit) application.
With OpenCV 3 came a big push to move many of these “non-free” modules out of the default OpenCV install and into the opencv_contrib package. The opencv_contrib packages contains implementations of algorithms that are either patented or in experimental development.
The algorithms and associated implementations in opencv_contrib are not installed by default and you need to explicitly enable them when compiling and installing OpenCV to obtain access to them.
Personally, I’m not too crazy about this move.
Yes, I understand including patented algorithms inside an open source library may raise a few eyebrows. But algorithms such as SIFT and SURF are pervasive across much of computer vision. And more importantly, the OpenCV implementations of SIFT and SURF are used by academics and researchers daily to evaluate new image classification, Content-Based Image Retrieval, etc. algorithms. By not including these algorithms by default, more harm than good is done (at least in my opinion).
How do I get access to SIFT and SURF in OpenCV 3?
To get access to the original SIFT and SURF implementations found in OpenCV 2.4.X, you’ll need to pull down both the opencv and opencv_contrib repositories from GitHub and then compile and install OpenCV 3 from source.
Luckily, compiling OpenCV from source is easier than it used to be. I have gathered install instructions for Python and OpenCV for many popular operating systems over on the OpenCV 3 Tutorials, Resources, and Guides page — just scroll down the Install OpenCV 3 and Python section and find the appropriate Python version (either Python 2.7+ or Python 3+) for your operating system.
How do I use SIFT and SURF with OpenCV 3?
So now that you have installed OpenCV 3 with the opencv_contrib package, you should have access to the original SIFT and SURF implementations from OpenCV 2.4.X, only this time they’ll be in the xfeatures2d sub-module through the cv2.SIFT_create and cv2.SURF_create functions.
To confirm this, open up a shell, import OpenCV, and execute the following commands (assuming you have an image named test_image.jpg in your current directory, of course):
$ python
>>> import cv2
>>> image = cv2.imread("test_image.jpg")
>>> gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
>>> sift = cv2.xfeatures2d.SIFT_create()
>>> (kps, descs) = sift.detectAndCompute(gray, None)
>>> print("# kps: {}, descriptors: {}".format(len(kps), descs.shape))
# kps: 274, descriptors: (274, 128)
>>> surf = cv2.xfeatures2d.SURF_create()
>>> (kps, descs) = surf.detectAndCompute(gray, None)
>>> print("# kps: {}, descriptors: {}".format(len(kps), descs.shape))
# kps: 393, descriptors: (393, 64)
If all goes well, you should be able to instantiate the SIFT and SURF keypoint detectors and local invariant descriptors without error.
It’s also important to note that by using opencv_contrib you will not be interfering with any of the other keypoint detectors and local invariant descriptors included in OpenCV 3. You’ll still be able to access KAZE, AKAZE, BRISK, etc. without an issue:
>>> kaze = cv2.KAZE_create()
>>> (kps, descs) = kaze.detectAndCompute(gray, None)
>>> print("# kps: {}, descriptors: {}".format(len(kps), descs.shape))
# kps: 359, descriptors: (359, 64)
>>> akaze = cv2.AKAZE_create()
>>> (kps, descs) = akaze.detectAndCompute(gray, None)
>>> print("# kps: {}, descriptors: {}".format(len(kps), descs.shape))
# kps: 192, descriptors: (192, 61)
>>> brisk = cv2.BRISK_create()
>>> (kps, descs) = brisk.detectAndCompute(gray, None)
>>> print("# kps: {}, descriptors: {}".format(len(kps), descs.shape))
# kps: 361, descriptors: (361, 64)
What's next? I recommend PyImageSearch University.
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 30+ Certificates of Completion
- ✓ 39h 44m on-demand video
- ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
In this blog post we learned that OpenCV has removed the cv2.FeatureDetector_create and cv2.DescriptorExtractor_create functions from the library. Furthermore, the SIFT and SURF implementations have also been removed from the default OpenCV 3 install.
The reason for SIFT and SURF removal is due to what OpenCV calls “non-free” algorithms. Both SIFT and SURF are patented algorithms, meaning that you should technically be getting permission to use them in commercial algorithms (they are free to use for academic and research purposes though).
Because of this, OpenCV has made the decision to move patented algorithms (along with experimental implementations) to the opencv_contrib package. This means that to obtain access to SIFT and SURF, you’ll need to compile and install OpenCV 3 from source with opencv_contrib support enabled. Luckily, this isn’t too challenging with the help of my OpenCV 3 install guides.
Once you have installed OpenCV 3 with opencv_contrib support you’ll be able to find your favorite SIFT and SURF implementations in the xfeatures2d package through the cv2.xfeatures2d.SIFT_create() and cv2.xfeatures2d.SURF_create() functions.
Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF
Enter your email address below to join the PyImageSearch Newsletter and download my FREE 17-page Resource Guide PDF on Computer Vision, OpenCV, and Deep Learning.
