So what type of shape descriptors does OpenCV provide?
The most notable are Hu Moments which are can be used to describe, characterize, and quantify the shape of an object in an image.
Hu Moments are normally extracted from the silhouette or outline of an object in an image. By describing the silhouette or outline of an object, we are able to extract a shape feature vector (i.e. a list of numbers) to represent the shape of the object.
We can then compare two feature vectors using a similarity metric or distance function to determine how “similar” the shapes are.
In this blog post I’ll show you how to extract the Hu Moments shape descriptor using Python and OpenCV.
OpenCV and Python versions:
This example will run on Python 2.7/Python 3.4+ and OpenCV 2.4.X/OpenCV 3.0+.
OpenCV Shape Descriptor: Hu Moments Example
As I mentioned, Hu Moments are used to characterize the outline or “silhouette” of an object in an image.
Normally, we obtain this shape after applying some sort of segmentation (i.e. setting the background pixels to black and the foreground pixels to white). Thresholding is the most common approach to obtain our segmentation.
After we have performed thresholding we have the silhouette of the object in the image.
We could also find the contours of the silhouette and draw them, thus creating an outline of the object.
Regardless of which method we choose, we can still apply the Hu Moments shape descriptors provided that we obtain consistent representations across all images.
For example, it wouldn’t make sense to extract Hu Moments shape features from the silhouette of one set of images and then extract Hu Moments shape descriptors from the outline of another set of images if our intention is to compare the shape features in some way.
Anyway, let’s get started and extract our OpenCV shape descriptors.
First, we’ll need an image, diamond.png:
This image is of a diamond, where the black pixels correspond to the background of the image and the white pixels correspond to the foreground. This is an example of a silhouette of an object in an image. If we had just the border of the diamond, it would be the outline of the object.
Regardless, it is important to note that our Hu Moments shape descriptor will only be computed over the white pixels.
Now, let’s extract our shape descriptors:
>>> import cv2
>>> image = cv2.imread("diamond.png")
>>> image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
The first thing we need to do is import our cv2 package which provides us with our OpenCV bindings.
Then, we load our diamond image off disk using the cv2.imread method and convert it to grayscale.
We convert our image to grayscale because Hu Moments requires a single channel image — the shape quantification is only carried out among the white pixels.
From here, we can compute our Hu Moments shape descriptor using OpenCV:
>>> cv2.HuMoments(cv2.moments(image)).flatten()
array([ 6.53608067e-04, 6.07480284e-16, 9.67218398e-18,
1.40311655e-19, -1.18450102e-37, 8.60883492e-28,
-1.12639633e-37])
In order to compute our Hu Moments, we first need to compute the original 24 moments associated with the image using cv2.moments.
From there, we pass these moments into cv2.HuMoments, which calculates Hu’s seven invariant moments.
Finally, we flatten our array to form our shape feature vector.
This feature vector can be used to quantify and represent the shape of an object in an image.
What's next? I recommend PyImageSearch University.
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 30+ Certificates of Completion
- ✓ 39h 44m on-demand video
- ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
In this blog post I showed you how to use the Hu Moments OpenCV shape descriptor.
In future blog posts I will show you how to compare Hu Moments feature vectors for similarity.
Be sure to enter your email address in the form below to be informed when I post new awesome content!
Download the Source Code and FREE 17-page Resource Guide
Enter your email address below to get a .zip of the code and a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!
Hi Adrian,
Thank you for this great post.
I have one question. Shouldn’t the Hu’s moment invariants be log transformed?
I’ve seen in some forums that this is better for shapes feature extraction and pattern classification. Couldn’t confirm it though.
hu = cv2.HuMoments(cv2.moments(image)).flatten()
print -np.sign(hu) * np.log10(np.abs(hu))
Hi Marcio, that’s a really, really great question, thanks for asking! So yes, when comparing Hu moments it can be helpful to log transform the Hu moments features. This helps prevent noise and “spikiness” of features. Is it absolutely necessary? It depends on your application. I always start with the normal Hu moments and then log transform if necessary.
Hi Adrian,
My name’s Bustomi, i’ve developing apps to translate ASL to text on android and using OpenCV. I’ve done with background segmentaion, sparate foreground, find contour. I want to get Hu Momments value from object and then train those using LibSVM on OpenCV android. Could uexplain to me how to do that or even better give me example, please. Thanks in advance.
Best,
Bustomi
Hi Bustomi, I don’t have any experience using OpenCV on Android. But the example in this post demonstrates how to extract Hu Moments. From there all you need to do is pass them into your favorite machine learning library.
Hello, I’m doing a work about text recognition using Opencv 2.4.11 and Python 2.7.
My teacher told that HU moments were a good basis for my future code to work, and I find this code really interesting.
I was using png pictures with Arial Text and I think that this code is capable of finding HU moments on each letter.
My questions are:
– Can I use this code to recognize more than one letter each time?
– Can this code be used algong with webcams to recognize hand written letters?
– Do you have any ideia how could I continue my work using this code?
Any help would be aprecciated. Waiting for your reply.
Thank you
You can certainly use Hu Moments and Zernike moments for character recognition; however, the Histogram of Oriented Gradients descriptor is much better suited for this type of problem.
In order to recognize letters, you’ll need to extract features from characters then train a machine learning classifier. From there, you can use the classifier to recognize new characters.
I would suggest taking a look at Practical Python and OpenCV. Inside the book I have an entire chapter dedicated to building a handwritten digit recognizer using HOG and Linear SVM. This code can be easily adjusted to use Hu Moments if you would like.
Can I use HuMoments to find a descriptor for every pixel of the image (considering a neighbourhood)?
I’m not sure I understand your question, but the Hu Moments descriptor takes into account every pixel in the input image (or ROI) when you compute it.
Hi Adrian, out of these two (Hu Moments and Zernike moments) which is better for object shapes. Like i want to detect guns, so out of these two which will give better results and why ? I an using HOG, color and shape as features.
Out of the two, Zernike Moments tends to give better results for 2D shape recognition. But if you’re looking to detect objects (such as guns) in various lighting conditions, poses, etc., you would be better off training your own custom HOG + Linear SVM detector. And depending on the complexity of your images, you may even need deep learning.
Hi, Can you add the Humemnts comparison part also. Thanks
Do you mean comparing Hu Moments to determine the similarity between shapes? If so, I detail Hu Moments (and many other shape descriptors) inside the PyImageSearch Gurus course.
Otherwise, you may find this blog post on comparing Zernike Moments helpful.
Hi Adrian
You mentioned in your other post that Zernlke moments are invariant to rotation, which enables us to do perspective transform even and compare it safely. Butif we look for a more precision match between two contour sets , should we consider Hu moments in that case? Your thoughts please…..
Regards
Shankar
Simple. I would test both and compare the results, then go with the method that provided the best results. Hu Moments will work just fine in very controlled conditions but you’ll often get more mileage from Zernike Moments.
Hi,is it possible to recognize shape of different cars uniquely by using OpenCv?
Hi Koushik — you wouldn’t want to use basic image descriptors to recognize cars. Instead, I would recommend a machine learning or deep learning approach. I’ll be demonstrating how to recognize the make and model of cars in my upcoming book.
Hi Adrian,
Thanks for all your codes!.I use many of them.
I have a Q- Which method do you think is best for shape matching of hand gestures.
I have a large vocabulary of 10,000 different hand gestures.
I tried many methods like HOG, SIFT, SURF, BRIEF, moments, convexhull. But none is giving me good accuracy
It really depends. If your method doesn’t need to be invariant to rotation, HOG is a good choice. SIFT/SURF/etc. require an additional step of keypoint detection. In low contrast situations you may not be able to detect enough keypoints to build a reliable model. Instead you might want to consider looking at stereo/depth cameras for this project.
Hi Adrian.
Great Post. I applied following Hu descriptor on images in my dataset which consists of images of elephants, horses, buildings, Buses etc (Wang Dataset) with some backgrounds in them. so I am trying to classify them on basis of shape feature and key objects in images should match by shape descriptor of Hu moments. I am comparing Hu moments of two images to calculate the similarity between them. In case of dinosaurs(with no background) I am getting almost 100% accuracy with this descriptor but for others, I am nearly getting 10% accuracy with this. So I think that background is making things worse. So What type of segmentation I can apply to the image before passing it to Hu moments descriptor for minimizing the effect of the background. I have tried applying adaptive thresholding and auto_canny edge detection on the image but it’s not improving the results. what type of preprocessing I can do on images for such case?
You need an extremely good segmentation of your object to apply Hu moments or Zernike moments. If you cannot segment the foreground from the background the resulting feature vector will not be very discriminative. You may want to consider using an object detection method such as HOG + Linear SVM.
How to compare two images of shapes, to find similarly between them?
Can we use hue moments for it? And if so, how to compare hue values?
You can use Hu moments to compare shapes. You would:
1. Extract Hu moments from the shapes
2. Compute the Euclidean distance between them
Shapes with smaller Euclidean distances are more similar.
If you’re interested in learning more about quantifying images/objects and comparing them I would suggest you take a look at the PyImageSearch Gurus course where I have over 60+ lessons on feature extraction and comparing images.
Hi, I’m working on research using the HU Moment Invariants. I want to ask, do you know the meaning of each seven characteristics of HU Moment Invariants? Please provide an explanation and reference. Thank you very much.
Hey Ayu, I would suggest referring to the original Hu moments paper for those details.
Actually i have already read many books about Seven Hu Moment Invariants… But still i don’t understand about the characteristics of the 7 Hu Moment… Can you help me to explain each characteristics of the 7 Hu Moment ?
I am extracting features using Hu Moments from segmented license plate characters? The 7 moments are calculated and I can see almost similar values for all the numbers and characters. Is it possible to detect the desired characters from these extracted features?
Technically yes, but it the character recognition won’t be that accurate, especially in real-world images. I would suggest referring to the PyImageSearch Gurus course where I cover ANPR in detail.
Hi, I am trying to extract features of the image based on the objects of the indoor scene image. I am trying to slice the image into several sub-images and then adaptive thresholding on them and finally, apply moments (hu and zenike). Is it a good option for objects shape feature extraction which will be used to extract the feature of the whole image? Thanks
That can work but the problem is that you’ll need to reliably segment the objects from the image. Secondly, these descriptors are not viewpoint invariant. You may need a stronger feature extraction method and model. I would suggest going through the PyImageSearch Gurus course to learn more about these advanced methods.
Can I use this features for image recognition having multiple shapes?
Yes, but that really depends on your actual shapes themselves. If you can describe what you are trying to achieve I can try to provide a suggestion to you.
I am trying to extract features from fruits for fruits recognition. The dataset I use is Fruits-360.
https://github.com/Horea94/Fruit-Images-Dataset
Is it better to use Moment Invariants (Hu, Zernike) or HOG and why? And, which works better for multiple fruits recognition in a single image.
I would instead suggest you use fine-tuning via deep learning.