Last updated on July 7, 2021.
In a previous PyImageSearch blog post, I detailed how to compare two images with Python using the Structural Similarity Index (SSIM).
Using this method, we were able to easily determine if two images were identical or had differences due to slight image manipulations, compression artifacts, or purposeful tampering.
Today we are going to extend the SSIM approach so that we can visualize the differences between images using OpenCV and Python. Specifically, we’ll be drawing bounding boxes around regions in the two input images that differ.
To learn more about computing and visualizing image differences with Python and OpenCV, just keep reading.
- Update July 2021: Added a section on the methods to compare images for differences and resources for additional reading about how to use siamese networks.
Image Difference with OpenCV and Python
In order to compute the difference between two images we’ll be utilizing the Structural Similarity Index, first introduced by Wang et al. in their 2004 paper, Image Quality Assessment: From Error Visibility to Structural Similarity. This method is already implemented in the scikit-image library for image processing.
The trick is to learn how we can determine exactly where, in terms of (x, y)-coordinate location, the image differences are.
To accomplish this, we’ll first need to make sure our system has Python, OpenCV, scikit-image, and imutils.
You can learn how to configure and install Python and OpenCV on your system using one of my OpenCV install tutorials.
If you don’t already have scikit-image installed/upgraded, upgrade via:
$ pip install --upgrade scikit-image
While you’re at it, go ahead and install/upgrade imutils as well:
$ pip install --upgrade imutils
Now that our system is ready with the prerequisites, let’s continue.
Computing image difference
Can you spot the difference between these two images?
If you take a second to study the two credit cards, you’ll notice that the MasterCard logo is present on the left image but has been Photoshopped out from the right image.
You may have noticed this difference immediately, or it may have taken you a few seconds. Either way, this demonstrates an important aspect of comparing image differences — sometimes image differences are subtle — so subtle that the naked eye struggles to immediately comprehend the difference (we’ll see an example of such an image later in this blog post).
So why is computing image differences so important?
One example is phishing. Attackers can manipulate images ever-so-slightly to trick unsuspecting users who don’t validate the URL into thinking they are logging into their banking website — only to later find out that it was a scam.
Comparing logos and known User Interface (UI) elements on a webpage to an existing dataset could help reduce phishing attacks (a big thanks to Chris Cleveland for passing along PhishZoo: Detecting Phishing Websites By Looking at Them as an example of applying computer vision to prevent phishing).
Developing a phishing detection system is obviously much more complicated than simple image differences, but we can still apply these techniques to determine if a given image has been manipulated.
Now, let’s compute the difference between two images, and view the differences side by side using OpenCV, scikit-image, and Python.
Open up a new file and name it image_diff.py , and insert the following code:
# import the necessary packages
from skimage.metrics import structural_similarity as compare_ssim
import argparse
import imutils
import cv2
# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-f", "--first", required=True,
help="first input image")
ap.add_argument("-s", "--second", required=True,
help="second")
args = vars(ap.parse_args())
Lines 2-5 show our imports. We’ll be using compare_ssim (from scikit-image), argparse , imutils , and cv2 (OpenCV).
We establish two command line arguments, --first and --second , which are the paths to the two respective input images we wish to compare (Lines 8-13).
Next we’ll load each image from disk and convert them to grayscale:
# load the two input images imageA = cv2.imread(args["first"]) imageB = cv2.imread(args["second"]) # convert the images to grayscale grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY) grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
We load our first and second images, --first and --second , on Lines 16 and 17, storing them as imageA and imageB , respectively.
Then we convert each to grayscale on Lines 20 and 21.
Next, let’s compute the Structural Similarity Index (SSIM) between our two grayscale images.
# compute the Structural Similarity Index (SSIM) between the two
# images, ensuring that the difference image is returned
(score, diff) = compare_ssim(grayA, grayB, full=True)
diff = (diff * 255).astype("uint8")
print("SSIM: {}".format(score))
Using the compare_ssim function from scikit-image, we calculate a score and difference image, diff (Line 25).
The score represents the structural similarity index between the two input images. This value can fall into the range [-1, 1] with a value of one being a “perfect match”.
The diff image contains the actual image differences between the two input images that we wish to visualize. The difference image is currently represented as a floating point data type in the range [0, 1] so we first convert the array to 8-bit unsigned integers in the range [0, 255] (Line 26) before we can further process it using OpenCV.
Now, let’s find the contours so that we can place rectangles around the regions identified as “different”:
# threshold the difference image, followed by finding contours to # obtain the regions of the two input images that differ thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1] cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) cnts = imutils.grab_contours(cnts)
On Lines 31 and 32 we threshold our diff image using both cv2.THRESH_BINARY_INV and cv2.THRESH_OTSU — both of these settings are applied at the same time using the vertical bar ‘or’ symbol, | . For details on Otsu’s bimodal thresholding setting, see this OpenCV documentation.
Subsequently we find the contours of thresh on Lines 33-35. The ternary operator on Line 35 simply accommodates difference between the cv2.findContours return signature in various versions of OpenCV.
The image in Figure 4 below clearly reveals the ROIs of the image that have been manipulated:
Now that we have the contours stored in a list, let’s draw rectangles around the different regions on each image:
# loop over the contours
for c in cnts:
# compute the bounding box of the contour and then draw the
# bounding box on both input images to represent where the two
# images differ
(x, y, w, h) = cv2.boundingRect(c)
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
# show the output images
cv2.imshow("Original", imageA)
cv2.imshow("Modified", imageB)
cv2.imshow("Diff", diff)
cv2.imshow("Thresh", thresh)
cv2.waitKey(0)
Beginning on Line 38, we loop over our contours, cnts . First, we compute the bounding box around the contour using the cv2.boundingRect function. We store relevant (x, y)-coordinates as x and y as well as the width/height of the rectangle as w and h .
Then we use the values to draw a red rectangle on each image with cv2.rectangle (Lines 43 and 44).
Finally, we show the comparison images with boxes around differences, the difference image, and the thresholded image (Lines 47-50).
We make a call to cv2.waitKey on Line 50 which makes the program wait until a key is pressed (at which point the script will exit).
Next, let’s run the script and visualize a few more image differences.
Visualizing image differences
Using this script and the following command, we can quickly and easily highlight differences between two images:
$ python image_diff.py --first images/original_02.png --second images/modified_02.png
As you can see in Figure 6, the security chip and name of the account holder have both been removed:
Let’s try another example of computing image differences, this time of a check written by President Gerald R. Ford (source).
By running the command below and supplying the relevant images, we can see that the differences here are more subtle:
$ python image_diff.py --first images/original_03.png --second images/modified_03.png
Notice the following changes in Figure 7:
- Betty Ford’s name is removed.
- The check number is removed.
- The symbol next to the date is removed.
- The last name is removed.
On a complex image like a check it is often difficult to find all the differences with the naked eye. Luckily for us, we can now easily compute the differences and visualize the results with this handy script made with Python, OpenCV, and scikit-image.
Other methods to compare images for differences
This tutorial covered how to use the Structural Similarity Index (SSIM) to compare two images and spot differences between the two.
SSIM is a traditional computer vision approach to image comparison; however, there are other image difference algorithms that can be utilized, specifically deep learning ones.
To start, one could apply transfer learning by:
- Selecting a pre-trained CNN (ResNet, VGGNet, etc.) on a large, diverse image dataset (e.g., ImageNet)
- Removing the fully connected layer head from the network
- Passing all images in your dataset through the CNN and extracting the features from the final layer
- Computing a k-Nearest Neighbor comparison between the features and then finding the most similar images to the input query
However, keep in mind that the k-NN approach does not actually “learn” any underlying patterns/similarity scores in images in your dataset.
To actually learn image differences and make robust comparisons between them, you should utilize siamese networks.
The following tutorials will teach you about siamese networks:
- Building image pairs for siamese networks with Python
- Siamese networks with Keras, TensorFlow, and Deep Learning
- Comparing images for similarity using siamese networks, Keras, and TensorFlow
- Contrastive Loss for Siamese Networks with Keras and TensorFlow
Additionally, siamese networks are covered in detail inside PyImageSearch University.
What's next? I recommend PyImageSearch University.
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 30+ Certificates of Completion
- ✓ 39h 44m on-demand video
- ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
In today’s blog post, we learned how to compute image differences using OpenCV, Python, and scikit-image’s Structural Similarity Index (SSIM). Based on the image difference we also learned how to mark and visualize the different regions in two images.
To learn more about SSIM, be sure to refer to this post and the scikit-image documentation.
I hope you enjoyed today’s blog post!
And before you go, be sure to enter your email address in the form below to be notified when future PyImageSearch blog posts are published!
Download the Source Code and FREE 17-page Resource Guide
Enter your email address below to get a .zip of the code and a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!
This is so cool (even though I have no specific use-case for now…) 😀
Thanks for writing this post!
Thanks Linus, I’m glad you enjoyed the post 🙂
can we get two persons image comparision source code..and result should be in percentage format….if two images are same it should be show 100%,,, in case images are different then 0%
What parts of the person are you trying to compare? Are you doing facial recognition?
im try facial recognition and live traacking system plz help me
If you’re interested in facial recognition definitely take a look at the PyImageSearch Gurus course where I cover facial recognition in detail.
Nice job ! Thank’x again Adrian, you are helping nad inspiring me every day.
Can you please make a tutorial on how to detect soccer players on a pitch ? This technique is used to compute satistics of players performences using Computer Vision.
Hi Mourad — detecting (and tracking) soccer players on a pitch is a non-trivial problem. It’s not “easy” by any means as it involves both object detection and tracking (due to players being occluded). This is still an open-ended area of research.
Is it python 2 or 3?
This project is compatible with both Python 2.7 and Python 3.
thanks for this I will use this in my recent project fault finding on production batch
this is very important for me—PI!
Glad to hear it John!
Nice post but what about changes of colour that are intensity neutral, e.g. if a colour image has been changed to a grey scale image the above approach will see no difference, likewise if, in a part of the image r & g values have been swapped.
Image difference algorithms do exactly that — detect image differences. If you convert an image to grayscale or swap RGB values you’ll by definition be changing the image. Perhaps I’m misunderstanding your question?
Let us take your credit card example, if rather than deleting the logo you were to replace the red circle with an equal intensity of blue, then the yellow with an equal intensity of red and finally the blue with an equal intensity of yellow you would, of course, have changed the image __but__ the method shown in your code above would state that there was no difference between the two images, (as they are both reduced to grey scale before comparing.
Well, to start, when converting RGB images to grayscale, each of the channels are not weighted equally. Secondly, if you are concerned with per-channel differences, simply run the image difference algorithm on each channel of your input images.
Nice writeup would any of these work on two identical images but different scales? Trying to find a reliable method to compare two images that may be a different scale.
Thanks for sharing this.
For images that have different scales, but the same aspect ratio, simply resize both image so that they are the same dimensions. From there, this method will work.
i tired of install import cv2 in my mac book plz help out. im from india,im always follow ur web, projects . thanks for helping for all.
Are you having trouble compiling and installing OpenCV? If so, I would suggest talking a look at the Quickstart Bundle and Hardcopy Bundle of my book, Practical Python and OpenCV. Both of these bundles include a pre-configured Ubuntu VirtualBox virtual machine that has OpenCV + Python pre-installed. Best of all, this VM will run on Linux, macOS, and Windows. It’s by far the fastest way to get up and running with OpenCV.
I really like your walk-through of the code with examples. It makes it is easy to follow and understand. Great work.
Thanks Andreas, I’m glad you found the tutorial helpful! 🙂
I was having an issue differentiating between two very similar images (with my own eyes), and wanted to write a little program to do it for me. Thanks for explaining this!
Adrian,
When trying to install scikit-image I ran into a memory error when pip was installing matplotlib. I think it’s because pip’s caching mechanism is “trying to read the entire file into memory before caching it…which poses a problem in a limited memory environment” (https://stackoverflow.com/questions/29466663/memory-error-while-using-pip-install-matplotlib).
The way to avoid caching is the following:
pip --no-cache-dir install scikit-imageHopefully this saves some people from scratching their heads.
Thanks for sharing Bill!
Thanks for the post..very useful..
I’m glad you found it useful, Vijeta!
Hi Adrian,
would you use this technique to identify if an object popped up in an image?
Specifically, I have a wildcam that takes pictures a soon as movement is detected. Most of the time it is birds and squirrels. What i would like to do, is check if a deer or a wild boar is in the image. This helps reduce flipping 700 images just to find those few with the relevant animal. Of course deer and boar may show up during day and night ;-).
For your particular project, I would treat this as a motion detection problem. The tutorial I linked to will help you build your wildlife camera surveillance system. From there, you’ll want to train a custom image classifier to recognize any animals you are interested in. Exactly which machine learning method you should use depends on your application, but I would recommend starting with PyImageSearch Gurus which has over 40+ lessons on image classification and object detection.
Hey Adrian, thanks so much for the tutorial! Are there other techniques to do this process for the same object but taken in two different photos. For example, if I wanted to compare a stop sign in two different photos, even if the photo is cropped the images will differ slightly by nature (due to lighting and other variables). Thanks!
To start, you would want to detect and extract the stop signs from the two input images. Exactly how you do this depends on your image processing pipeline. Once you have the stop signs there are a number of ways to compare them. For this method to work best, you would need to align the stop signs, which likely isn’t ideal. I would instead treat this is as an image similarity problem and apply image descriptors and feature extraction to quantify the stop signs.
Thanks for the response! So would better techniques be things like zernike moments and color histogram comparisons?
Again, I think this depends on how you define “similarity”. What specifically are you trying to detect that is difference between road signs?
My primary suggestion would be to take a look at the PyImageSearch Gurus course where I discuss object detection, feature extraction, and image similarity in detail.
I am trying to detect differences in color and shape. Essentially trying to determine if a street sign is misprinted by comparing it to a correctly printed one.
It’s hard to give an exact suggestion without seeing example images you are working with. Again, I would refer you to the PyImageSearch Gurus course I mentioned in the previous comment for more details on image similarity and comparison.
Hi! The difference between the images that you have used is that there is a feature missing. However, what would happen if the feature exists but is slightly modified? For example, instead of CU MEMBER in the example image, its written as CW MEMBER. Or suppose the spacing between CU and MEMBER gets increased. Or maybe the font gets changed. Will the SSI show a significant difference? If not, is there a better way to capture such differences?
The SSI will capture such differences. I suggest you use your favorite image editor to create images with the text as you mentioned above and give it a try.
i solved that, now i have another error–sys error:
error: the following arguments are required: -f/–first, -s/–second
usage: image_diff.py [-h] -f FIRST -s SECOND
image_diff.py: error: the following arguments are required: -f/–first, -s/–second
Please read up on how to use command line arguments.
I was able to follow these instructions only by using sudo on my Linux mint system
Nice work there, much appreciate it ! Is there a way to compare the images using the SSIM approach without converting it to greyscale? I realised there are some slight color changes in my images when viewing it with naked eyes but there were no difference after converting it to greyscale.
If you wanted to compute SSIM for an RGB image you would simply separate the image into its respective Red, Green, and Blue components, compute SSIM for each channel, and average the values together.
If 2 images are in different point of view, contrast, noise.. ? How can I know they are the same or not?
Typically if you have objects that are captured at different viewing angles you would detect keypoints, extract local invariant descriptors, and then apply keypoint matching using RANSAC. If too few keypoints match, then the objects are not the same. If there is a reasonable percentage of overlap in the match, then the objects can be considered the same.
I teach this in detail (with code) inside Practical Python and OpenCV where we learn how to recognize the covers of books.
brilliant! thank you for sharing
Thanks Prabhakar 🙂
Hi Adrian, how to make output in terminal like boolean, 1 for no different and 0 for different?
For that I would also read this blog post as well. Basically, you need to supply a threshold on the SSIM or MSE value. If the threshold is small enough, there is no difference.
thanks, great job !!
Hi Adrian!
Great Tutorial for image difference detection. I just wanted to know if instead of drawing rectangle around the difference region, can we mask that area out? So that the whole image is visible and the part which is different is white. How can we do it? Can you please help.
You could mask the area out as well. You could draw a (filled in) rectangle around the contour area. You could compute the bounding box and then draw the rectangle. You could use the
cv2.bitwise_andfunction. There are a number of different ways to accomplish this task.It is great! Thank you.
But I wondered the value of it in reality because we always have to get the original image for comparison. Usually, it is not that kind of easy job.
If you are trying to compare two images for similarity you must have an original image of some sort in your dataset — otherwise you would not have anything to compare against.
hi adryan i found this error
pi@raspberrypi:~ $ python diffrence.py –first 3.jpg –second 4.jpg
Traceback (most recent call last):
File “diffrence.py”, line 23, in
(score, diff) = structural_similarity (grayA, grayB,full= True)
TypeError: structural_similarity() got an unexpected keyword argument ‘full’
Can you run
pip freezeand confirm which version of scikit-image you are using?Hi I am improving my knowledge from your tutorials.
I have question like at road intersection we have combination of different signs like traffic light is with green colour and priority sign how can recognition this combination . Display a text like vehicle has priority to move?? Like if I detect both signs together . Or priority and straight ahead sign is there how can recognize the combination of 2 signs saying that vehicle having priority in this road??
I would need to see an example image of what you’re working with, but if I understand your question correctly you would need to train a multi-class object detector that can recognize each of the traffic signs + priority indicators.
hi adryan, i am having this version
scikit-image==0.10.1
how can i solve this problem. please help.
actually we want this for detecting the errors in PCB board. we are using a 8 mp camera which gives less clarity image. can you suggest a better option for this
That is a pretty old version of scikit-image so that’s likely the issue. Make sure you upgrade it:
$ pip install --upgrade scikit-imageThank you for your share
Very useful for me
It works on my Raspberry Pi.
But this message appears, what’s mean?
* (Original:10126): WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
Hi Pochao — try this package to get rid of that warning:
sudo apt-get install libcanberra-gtk*Hi Adrian,
Nice article about the comparison since I don’t want to compare the complete image will it be possible to compare a part of the reference image with the current image and then wanted to decide on the correctness. Is there an option for this.
There are multiple ways to accomplish this, most are dependent on the exact images you are trying to compare. If you know the region you want to extract I would suggest using NumPy array slices to extract the ROI and then compare it. If the region location can vary in the images, use keypoint detection + local invariant descriptors + keypoint matching as I do in the “Book cover recognition” chapter of Practical Python and OpenCV.
hi adryan
can i use this method for detecting the errors in a pcb board (for example soldering, improper connection etc.).i have a 8mp pi cam. but when i used this method it showing lot and lot of errors which is actually not needed. how can i fix it. please help.why it is showing lot of errors when comparing two images taken using pi camera?.please help me to fix it. waiting for your suggestions.
If you are able to align both images, then yes, this method would work for simple image differences.
Hi Adrian, thanks for the code!
I just can’t manage to run it properly, I keep getting the error:
image_diff.py: error: argument -f/–first is required
I can’t really trace where this comes from, I am quite new to this. Do you know what I have to change or install for this error to disappear? Would be great!! Thanks in advance, cheers!
Hi Ilja — please read up on command line arguments. You need to open up a terminal and execute the script via the command line, exactly as I do in this blog post.
Hi Adrian,
Thanks for the tutorial.
I will send you two images which are almost the same. If you calculate the difference via ImageJ, you will see a black image but by using you algorithm it just cause chaos.
I think cv2.THRESH_OTSU is the culprit here.
Thanks and Regards,
Bilal
It’s hard to say what the exact issue is without seeing the images. ImageJ might be using a different implementation or it could be nuance of your two particular images.
Sorry for the delayed response. How can I send the images to you? I have put my email address as well.
Please send me a message and from there we can chat over email. You can send the images as attachments or upload to a cloud-based service such as Dropbox.
Hi Adrian,
I’m trying to solve a problem of checking if 2 given handwritten signatures are taken from the same person. Can you show me how to do it?
Hi Adrian,
Thanks for the very useful post. In one of my use case, I gotta compare two images to figure out the dimension differences between the two as one of the image is a reference master image. Any pointer on how to approach this use case?
Hey Gandhirajan — what do you mean by “dimension differences”? Are you referring to computing the width or height of an object?
Hey ..
Thank u for this code !
I installed dependencies , scikit image ..still there is error showing can’t import compare.ssim
Please help with this !
Thank u !
Hey Pranali — just to confirm, which version of scikit-image do you have installed? You can verify via “pip freeze”.
I’m also facing the same error.
My version : scikit-image==0.16.2
Hello Adrian,
Is there any way to automatically decide which filter will be applied on image by analyzing image quality.
While searching for the above query i reach to your this post, so I ask here.
Thanks a lot..!!
I’m not sure what you mean by “automatically decide which filter”. Could you elaborate?
This guide was very helpful. I could understand everything.Thankyou for putting this. But when I run the program i get this :
ImportError: cannot import name compare_ssim
Hey Rinsha — what version of scikit-image are you using?
Hey Adrian,
This was a very nice tutorial. Although i wanted to know if there is a way to show any difference in intensity of the photos. Eg. Green and DarkGreen.
Once you know where in the images the difference occurs you can extract the ROI and then do a simple subtraction on the arrays to obtain the difference in intensities.
Hello Adrian,
Thank u for this code
still there is error showing can’t import compare.ssim
version of scikit-image:
Python 3.5.2
>>> import skimage
>>> print(skimage.__version__)
0.10.1
You need to upgrade your scikit-image install:
$ pip install --upgrade scikit-imageI want to know this function compare_ssim the param data_range how to work. Because I know the function is compare every px‘s ssim score.Actually two picture is so similar but their score is so low.
I try to change the data_range the score will improve.But I don’t know how it work.Hope you give me a answer.
The data_range here would be the distance between the minimal pixel value (0) and max pixel value (255). This value is estimated from the data point (unsigned 8-bit integers).
Hi! Great code, but I encountered an issue. It works fine when there is a difference, finding and drawing the contours. However, when there is no evident difference between the two images, it draws thousands of contours across the image, and that affects the code I’m using this for. What causes this and how can I fix it?
I haven’t encountered this problem before. To be honest I’m not sure what would cause it. A quick fix would be to check the returned SSIM score and if it’s above a (user defined) threshold you can simply mark the images as “similar enough/identical” and continue on with the program.
Hi Adrian,
I want to compare 2 images that have a subtle difference :
1st image has a nut and bolt without grease.
2nd image has a nut and bolt with grease.
I want the “grease” difference to be the output.
Could you please let me know which would be the most accurate way to achieve this?
Would the code mentioned for the above example be useful or is there a better way to handle this?
Also, please let me know is Open CV is the only way to achieve this? I checked the AWS, Azure APIs but could not find any service that would solve this.
I need this for handling a Business use case, so please let me know the best option.
Thanks in advance.
Regards,
Sudheendra
It’s very likely that you will need to implement this algorithm by hand, again, most likely using OpenCV. There will not be an “off the shelf” solution for you to solve this project.
Do you have any example images of what the two images you need to compare look like? It would be best to see them before providing any advice.
Hi Adrian,
Thanks for the reply.
I have uploaded 2 sample images in pasteboard:
Without grease image – https://pasteboard.co/HiObAUb.jpg
With grease image – https://pasteboard.co/HiObPBj.jpg
When these 2 images are compared , I want the difference to be highlighted – main difference being the grease. This is a use case from an automobile industry.
Could you please let me know how to implement and get very good accuracy.
I want this to be implemented on AWS. I saw one more link of yours in which you had done all the pre-setup in AWS for python and CV.
Please guide me. Eagerly waiting for your reply.
Regards,
Sudheendra
The first step here would be to detect the grease. You could try color thresholding but that wouldn’t work well with varying lighting conditions. Instead, I would suggest trying to train a deep learning-based segmentation network, such as Mask R-CNN or UNet. It will be a very challenging project to say the least (just wanted to give you a warning).
Hi Adrian,
Thanks for the reply. I can sense that it will not be easy. 🙂 Neverthless, thanks for the advice. I will try it out. I have another immediate business requirement and I need your guidance for sure 🙂
We need a customized image processing algorithm that would provide a specific set of output parameters after an image is processed.
If a manufacturing part like fastener is the image, then after processing this image, we need output parameters like dent, line, scratch, ring etc and their accuracy percentage. This would be a quality check of the part. This would enhance the visual inspection and result in a better qualify of the end product by identifying defective parts in a early phase before assembly.
For this, we need an algorithm built on a specific image classifier and a trained model. Please let me know which platform should I use and how should I proceed.
Regards,
Sudheendra
While you could use basic image processing here I do not think it would work well for a robust solution. Deep learning-based methods would likely achieve the highest accuracy provided you have enough training data for each defect/problem you’re trying to detect. I personally prefer Keras for training deep neural networks. I discuss how to work with Keras and train your own networks inside my book, Deep Learning for Computer Vision with Python.
Thank you for your reply. I will check the details and get back to you.
Regards,
Sudheendra
Hi Adrian, many thanks for the tutorial. A question regarding thresholding: “On Lines 31 and 32 we threshold our diff image using both cv2.THRESH_BINARY_INV and cv2.THRESH_OTSU-both of these settings are applied at the same time using the vertical bar”. I applied the same technique in my project (where I am detecting/following a black line in front of the robot) and it improves the number of frames in the video where line is detected correctly, which is good! But I do not understand how it works! I read the documentation you referred to, and it says “OTSU algorithm finds the optimal threshold value automatically”. So when using the following line of code for thresholding, which threshold value is finally used? The one I define or the one determined by the algorithm behind OTSU? I am just trying to understand why it is improving the performance of my code. Thanks 🙂
cv2.threshold(blurred, 100, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)
The “cv2.threshold” function will return two values: the threshold value T and the thresholded image itself. The value “T” is what the Otsu method automatically completed.
Hi,
I really love your article, but I have few questions. I’m thinking about developing the testing website framework for my company to detect the bug from the new version vs old. This test cases include color matching, positioning, etc. My first question is what is the main reason that you use gray scale for your comparison, I see in compare_ssim we can add multichannel=True then the color image will be comparable. Second, if the first question did work, so could I use it for the testing framework, or do you have any suggestion?
Thanks,
For this particular example we were examining the “structural components” of the input object. The actual color did not matter. If you are monitoring a website then color would certainly matter as a bug in the CSS could cause a different color text, background, etc. You would want to experiment with both. You should also take a look at template matching to help you locate specific components on the website.
Hi!
Did you have some progress with the testing framework with images?
Thanks
Question: Does this library perform well when there are differences in rotation, translation, and scaling? I deal with scanned images where there can be a slight skew when scanning that introduces an angle of rotation. When comparing two scanned images they also might not have the same X-Y scan origin and hence have a translation. And the pixel densities and related scaling may also differ due to scanner and output differences.
No, this method does not work for differences in rotation, translation, scaling, etc. For that you would use to use keypoint detectors, local invariant descriptors, and keypoint matching to locate objects in your two images.
Hi! I am having trouble running this program. I have installed and updated scikit-image on my computer, but it is still saying that there is no module named skimage. Here is the exact error:
from skimage.measure import compare_ssim as ssim
ModuleNotFoundError: No module named ‘skimage’
I am confused why it is not recognizing skimage even though I have downloaded it on my computer.
Thanks.
Are you using Python virtual environments? My guess is that you may have installed scikit-image globally but not into your Python virtual environment.
Hi Adrian,
Great article. I am new to python. I am trying to capture characteristics of 2 different image shapes. Characteristics such as color, shape, size, location. I need to compare the differences between the two images. For example imageA may consist of a small circle and imageB may have a larger circle. The difference between imageA and imageB in this case would be that ImageB’s circle grew in size.
May I ask what python library(s) would be the best to handle this type of task? Thanks
OpenCV would be ideal for this task along with (potentially) a machine learning library such as scikit-learn. I would suggest you work through Practical Python and OpenCV where I teach the fundamentals of the OpenCV library and even include some case studies on how to compare images. One chapter even covers how to recognize the covers of books in images via keypoint matching — this algorithm could be adapted for your own problems.
A more in-depth study of feature extraction and object detection/recognition can be found inside the PyImageSearch Gurus course which includes 160+ lessons and approximately 70 lessons specifically related to your problem.
I hope that helps point you in the right direction!
This is perfect! Thanks so much. It works perfectly for my image comparison automation.
Awesome, I’m glad to hear it Deepali!
Hello Adrian. Ty for really good job. Could you tell me: can i add arguments(path to images) not like keys in console or parameters in IDE, but just like parameters?
I have some Selenium test what producing screenshots, so i want compare screenshots with
references.
Hey Viktor — have you considered creating a configuration file, perhaps in JSON format, and then updating the script to load the configuration/parameters when you execute the script? That would probably be the easiest.
Hi Adrain,
This works perfect. But How can I implement this same concept with two different dimension-ed images. Is that possible.? Thank you.
If the images are similar in aspect ratio I would suggest resizing them so that each image has the same dimensions. If they do not have similar dimensions you should apply more advanced techniques, such as keypoint matching.
Sir while using vars(line 13) I m getting an exception from system . What should I do ??
What is the exact exception? My guess is that you did not provide the command line arguments properly. I would suggest you read this post on command line arguments to see if that resolves your error.
Hi Adrian,
I read your article very well. After then, I have a question.
How to measure the percentage of images between same images but different ratio?
for example, if I make a 10% enlarged origin image, instead of resizing, I cut the side part of an image and make same size with the origin image, After then, How can I make an algorithm to measure high percentage of that image?
Building an algorithm that can still measure image differences even after distortions can be incredibly challenging. You might want to take a look at “perceptual hashing” papers for inspiration, including the work from TinEye. I do not have a solution offhand for this project.
hi dear Adrian
could you please help me and tell me how can I filter an image so that only the pixels close to a certain color are left ?
I think you’re referring to color thresholding.
tnx alot
Hi Adrian,
With the RANSAC algorithm as you commented earlier, can we looks for similarities in images with slightly differing features? For example, two images of American Express Gold Card taken from slightly different angles, the cards having different numbers, names, expiry dates, but rest of the cards are the exact same?
I wouldn’t recommend using keypoint matching to help detect differences. You would normally use keypoints and keypoint matching to verify correspondences. If you’re interested in the actual text on the card (name, expiration date, etc.) you should just OCR the image.
Thanks Adrian. What I meant is, I am not interested in the card details, but just to verify whether two cards are of the same type (say both are American Express Gold cards, but belonging to different persons). Will this algorithm help? To detect that the two cards are of the same type even though taken from slightly different angles, and some content being different (names, card numbers, expiry dates, etc.) ?
The best way to detect differences in name, number, expiration dates, and card types would be to OCR the card and compare directly. You could use this method to a degree but it wouldn’t be as accurate as OCR’ing directly.
Thanks for sharing with us the top class tips. I love this post so much. Your image differences with open cv and pathon help me a lot. This post just makes me crazy.
Thanks David, I’m glad you enjoyed the post 🙂
Hi Adrian,
thresh = cv2.threshold(diff,0,255,cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU) [1]
cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
what does this [1] refer to? without [1] I get an error telling _’tuple’ object has no attribute ‘copy’_ can you please brief it .
—– Your tutorials are awesome and makes me learn more. And I really appreciate you for helping out even for older posts —-
The “1” is the index of the returned value. The
cv2.thresholdfunction will return two values: The threshold value T and the thresholded image itself. Without knowing your exact full error I’m not sure what the error may be.Hi Adrian,
Is there any way to detect changes in gradient and colour, as well as the visual differences used in the credit card example above together?
I’m having a hard time identifying these differences in grayscale as the threshold algorithm used to draw the bounding box isn’t picking these up.
Absolutely. I actually cover how to detect changes in gradient for barcode detection in this post. You can adapt it for your own purposes 🙂
Hi Adrian
Can you give me some input on applying this method on video files?
I have a photo of the garage from inside looking out and video from the same angle filming car attempting to park in. My idea was to make some sort of subtraction to remove everything but the car and then draw contours of result. In perfect world this would mean that only car would be that contour and I would draw rectangle around it and show that rectangle on original video frame.
I managed to do it almost using gauss blur, addWeighted and adaptiveTreshold on both frames and then subtract them, but problem is that car contour is too small when car is outside the garage and it is not detected until is to close.
Your method gives me better results when car is far, but problem occurs when car get closer and car lights hit the wall and “difference” between frames is detected. Also, because car moves, everywhere where car was is also detected as difference.
My attempts to filter out those lights and reflections were in vain because compare_ssim works even worse then.
My question is: Is there any way to apply some treshold on frames before comparing them with compare_ssim so I can avoid shadows and reflections? Also can it work only to show differences from video file (like cv2.subtract) so I can avoid false difference detection behind the car?
Sorry for long post 🙂
Hi , If the text difference is recognised and if it is printed it will be even better
You can implement that functionality yourself if you so wish by following my text recognition tutorial.
Thanks Adrian for the post, I am looking forward to use your tutorial(s) as a springboard into computer visioning.
Sorry for my noobs question: how can I get the compared images?
I’m not sure what you mean by “get the compared images”? Could you clarify?
What I meant is the sample of the two image used for the comparism. The picture of the “Credit Card”.
Nevertheless Dr. Adrian Thanks for your response. I have run the code with difference images and it run successfully
Awesome, I’m glad it worked!
is this approach is also applicable for motion detection using surveillance camera .
I would suggest using this tutorial for motion detection.
Hi.. I was wondering if you might have a suggestion for looking at the same image but with a different illumination. Say a picture that is of the exact same area just at a different time of day?
Thanks.
What specifically would you be trying to compare in those two images?
Awesome tutorial by the way!! Right now, the contours are based on mean structural similarity but what difference function should I use for contouring based on color? Thanks
Hey there, Shreyans. I’m not sure what you mean by “contouring based on color” — could you elaborate?
Hey Adrian, I meant two scenes, same object, different color. I only consider the contour with the maximum area, so I wanna look for the difference in the scene based on color and not structure.
There are a few ways to approach that but I think color histograms would be the easiest approach. Extract the object, compute color histograms, and then compare.
Have a good day Mr. Author,
My English is not very good so I can not say much.
Thank you for writing this tutorial. Please take me as your student.
Thank you very much.
Thank you Nang, I appreciate that!
you are awesome, it works for me.
i wrote it in google collab and it works with tiny customized.
many thanks Adrian, i will follow your blog after this example.
Awesome, I’m glad it worked for you!
Hey I have two images, I prepossessed one to make it ready for comparing with second image. because both had different sizes But this introduced many extra differences. How to make this algorithm ignore the nominal pixel differences and just spot the main differences? Please help me
Hm, that really depends on your example images themselves and what differences you are trying to detect. Could you elaborate on your project a bit more?
Hi.
The provided code works very well for me.
If I have a string I get a nice box around the entire string, instead of a square around each letter.
But I have a dotted line, where each dot get marked with a square. How do I change this threshold?
I have been trying to make this work with RGB Jpegs for the last 3 hours and can’t figure it out… and it’s driving me nuts! 😀
Is there any complete documentation on compare_ssim somewhere online? I can’t fing anything better than this ( https://kite.com/python/docs/skimage.measure._structural_similarity.compare_ssim ), which doesn’t really explain how to use RGB images other than using the “multichannel” flag. Maybe I should just convert the images to three gray channels and compare these but that seems unnecessarily computationally intensive and I have a strong feeling that there must be a better way… any tips? Thanks!
It looks like that website is just a copy of the scikit-image documentation on SSIM:
http://scikit-image.org/docs/dev/api/skimage.measure.html?highlight=compare_ssim#skimage.measure.compare_ssim
If you set
multichannel=Trueit should work for you.Hi Adrian
Thank u for the blog post.
Sorry, my English is not good.
Follow your tutorial,it throw a “not find scipy module” erro when run to “from skimage.measure import compare_ssim”,then i use pip install scipy,after that the code is running very well.I guess the scipy module is referenced inside the skimage module, but it is not bundled when it is installed.
Correct, the scikit-image library requires SciPy. Congrats on resolving the issue!
Hi Adrian ,
Thanks for all your tutorials CHAMP. You are the BEST.
Very clear explanations.
Thank you for the kind words 🙂
Hi! Another awesome tutorial! Many thanks. Could you provide me some links or helpful tips how we can do similar task using deep neural networks ?
Can u post a C++ version of this?
Sorry, I only cover Python on the PyImageSearch blog.
Hey,
Thank you for your face recognition tutorials .
I’m planning to do a project on accident detection via image processing by taking the cctv images.Do you think this image comparison method can be used for doing the same?
Can you suggest a method to compare a normal car and one which has undergone a crash via feature extraction?
kindly suggest an less time consuming method
I would recommend you use Instance Segmentation and Mask R-CNN instead — that will give you a better method to compute a pixel-wise mask for the cars. You can then check to see if any pixels of the mask “overlap”.
Glad! I found this blog…Lot of content to go through..Thank Q 😉
Thanks Anirudh, I hope you enjoy the tutorials!
Can we use this to find difference between forged and real signature ? If not, it would be glad if you post a tutorial of that over here
Sorry, I do not have any tutorials on signature verification. I will consider it for a future tutorial though.
How can I do this for mutiple images, where I want a cumulative score of how similar mutiple images are?
You would loop over all images in your dataset, compare them, and then average the scores to obtain your cumulative score.
I am working finding out the similarity score of multiple images (approx. 2000). I am looping over the images and creating a dictionary to save the data i want for the final report. It also involves showing the images using matplotlib.
The problem is the whole process is taking 3 hours to complete.
Can you suggest something to reduce the execution time ??
Are you using the similarity method covered in this tutorial? Or are you using a different algorithm?
Hi. I tried to use this program to find differences between two images. But instead of finding individual differences, it just marked huge areas, where things were changed. How can I improve this?
The code will be very, very sensitive to changes in the images. If it has marked large areas as different then they are most certainly different.
I have the same problem. Due to the noise, this algorithm marks a huge area. How can I solve this?
Hi Adrian
Congratulations on your tutorial.
I need to check in the photo if the vehicle has a crash, can I use the image difference?
thank you
You should consider using “object detection” and “instance segmentation” — I’ve authored tutorials on both topics. Use the search bar to look for them 🙂
Dear Author,
How may I configure the sensetivity of this algorythm. At the moment it detects 1000 changes even smallest that has to be ignored. I want to detect only significant changes to make result not 1000 but 3-4 for example. To make the detection only in case when something really changed.
Thank you.
Hi Adrian,
I have the same problem as Andrei. I look for the way to decrease the sensitivity of the algorithm. I have two photos takes one after another (a few seconds difference between photo take) with only one different item on it, but as a result, I get about 100 elements which changed.
Hi Adrian,
Greetings!
As always, your posts are life saving revelations! Never thought something like this can be done. I was trying Imagick to perform such changes.
Would you please tell me, how would this handle images with different resolutions?
Lets say the card A is 72Dpi and card B image is 300 Dpi. Definitely the pixel values would change.
How can we handle this scenario?
I have a task to compare 2 logos of the same brand and check if the logo under test is stretched or skewed. In short, need to test if logo is perfect.
Thank you very much!
Hi Adrian,
how can we develop the same with deep learning??
There really isn’t a reason to. Don’t use a jackhammer when a simple hammer will suffice.
Hi Adrian, thank you for the good work and your code. It really helped me to understand the image search concept. Anyway, I am working on a project to compare two PDFs (or you can say scanned images of the document, theY may be difference in scale, rotation etc as they manually scanned). Is it possible with this concept?
If not, which concept would you suggest me to adopt?
I would suggest performing image registration first via keypoint detection, local invariant descriptors, and keypoint matching.
Hi Adrian,
Greetings!
It’s a nice tutorial.
Had a query on image comparison.
Is it possible to have a region based comparison on two images. Since the image has Time Display it will be varying, so if i compare with above method i will be getting a mismatch. So i want to go for region based comparison.
Regards
Saran
If you know the ROIs of both areas you should simply use NumPy array slicing to extract them and then perform the comparison. Otherwise, take a look at Practical Python and OpenCV where you can match images based on keypoint correspondences.
I need an algorithm to compare picture taken at intervals against a reference. I also want a compare only a window. If possible I would like it to auto align the picture with the reference before comparing. The picture is an appliance with LEDs. I want it to notify me if the LEDs are lit.
thanks
Hi Adrian, Great tutorial. I am looking something similar to this.
Do you have any tutorial/blog about image comparison with OpenCV Java library?
Raj
Sorry, no, I primarily cover Python here.
Hi,
Great tutorial as always. Do you have a recommendation on how to count how many more or less pixels there are between the two images?
Congratulations on the blog. Do you do paid services?
You mean like consulting/contracting work? If so, refer to my FAQ.
Hello adrian,
Thank you so much, It perfectly works.
But what if the two images I would like to compare, was captured from 2 different angle, have different noise and not the same dimensions ( I need a better solution than manual dimensions modification).
Waiting for your reply.
If you need to compare two images for similarity from different viewing angles I would recommend you apply keypoint detection, feature extraction, and keypoint matching — all of which are covered in Practical Python and OpenCV.
Thanks Adrain I have been enjoying your post for quite sometime now they are helpful. I want to know if the Structural Similarity method can be used for template matching and if possible how to go about it
Hi! I was wondering if you could help me with a project of mine. I have to find the difference between to photos but there are specific differences between each. Such as the disappearance of an area, a color change, etc. If you could tell me where to start that would be great!! Great Article and thank you!
Can i compare to objects using raspberry Pi?
Means I want to make a software that
Distinguish between objects
Using their outlines as objects may vary in color’s
In general, yes, you can, but basic image processing techniques may not be sufficient for high accuracy. I would recommend you read Raspberry Pi for Computer Vision if you’re interested in learning more.
Hi! Thanks for writing this tutorial, it has been useful for my research. Easy explained and with clarity.
I don’t know if you update your tutorials in order to reuse them for the future. I know that this could be difficult because you have lots of them. I hope that this could be helpful for you or any that reads this comment.
While I was testing your code in my machine (on March, 2020), I saw that compare_ssim has been deprecated and skimage recommends the use of structural_similarity in skimage.metrics. In the warning message they also establish that in a future release, compare_ssim will be deleted from the API, that is why I think it is important for me to warning you all.
Again, thanks for your tutorials and help. Regards!
Thanks for the heads up, Javier! I’ll ensure this post is updated once scikit-image is updated.
Thanks a lot for this tutorial, it was very helpful. In addition, I would like to ask a question: what is the difference between this method and the direct subtraction of two pictures?
The SSIM method is more complex and more robust. You can read more about it here.