If you haven’t noticed, the term “feature vector” is used quite often in this blog. And while we’ve seen it a lot, I wanted to dedicate an entire post to defining what exactly a feature vector is.

What is an Image Feature Vector?

Image Feature Vector: An abstraction of an image used to characterize and numerically quantify the contents of an image. Normally real, integer, or binary valued. Simply put, a feature vector is a list of numbers used to represent an image.

As you know, the first step of building any image search engine is to define what type of image descriptor you are going to use. Are you trying to characterize the color of an image and extracting color features? The texture? Or the shape of an object in an image?

Once you have selected an image descriptor, you need to apply your image descriptor to an image. This image descriptor handles the logic necessary to quantify an image and represent it as a list of numbers.

The output of your image descriptor is a feature vector: the list of numbers used to characterize your image. Make sense?

Two Questions to Ask Yourself

Here is a general template you can follow when defining your image descriptors and expected output. This template will help ensure you always know what you are describing as well as what the output of your descriptor represents. In order to apply this template, you simply need to ask yourself two questions:

1. What image descriptor am I using?
2. What is the expected output of my image descriptor?

Let’s make this explanation a little more concrete and go through some examples.

If you’re a frequent reader of this blog, you know that I have an obsession with both Jurassic Park and Lord of the Rings. Let’s introduce my third obsession: Pokemon. Below is our example image that we will use throughout this blog post — a Charizard.

Now, fire up a Python shell and follow along:

>>> import cv2
>>> image = cv2.imread("charizard.png")
>>> image.shape
(198, 254, 3)

Here we are just importing cv2, our Python package that interfaces with OpenCV. We then load our Charizard image off of disk and examine the dimensions of the image.

Looking at the dimensions of the image we see that it has a height of 198 pixels, a width of 254 pixels, and 3 channels — one for each of the Red, Green, and Blue channels, respectively.

Raw Pixel Feature Vectors

Arguably, the the most basic color feature vector you can use is the raw pixel intensities themselves. While we don’t normally use this representation in image search engines, it is sometimes used in machine learning and classification contexts, and is worth mentioning.

Let’s ask ourselves the two questions mentioned in the template above:

1. What image descriptor am I using? I am using a raw pixel descriptor.
2. What is the excepted output of my descriptor? A list of numbers corresponding to the raw RGB pixel intensities of my image.

Since an image is represented as NumPy array, it’s quite simple to compute the raw pixel representation of an image:

>>> raw = image.flatten()
>>> raw.shape
(150876,)
>>> raw
array([255, 255, 255, ..., 255, 255, 255], dtype=uint8)

We can now see that our image has been “flattened” via NumPy’s flatten method. The Red, Green, and Blue components of the image have been flattened into a single list (rather than a multi-dimensional array) to represent the image. Our flattened array has a shape of 150,876 because there exists 198 x 254 = 50,292 pixels in the image with 3 values per pixel, thus 50,292 x 3 = 150,876.

Color Mean

Our previous example wasn’t very interesting.

What if we wanted to quantify the color of our Charizard, without having to use the entire image of raw pixel intensities?

A simple method to quantify the color of an image is to compute the mean of each of the color channels.

Again, let’s fill out our template:

1. What image descriptor am I using? A color mean descriptor.
2. What is the expected output of my image descriptor? The mean value of each channel of the image.

And now let’s look at the code:

>>> means = cv2.mean(image)
>>> means
(181.12238527002307, 199.18315040165433, 206.514296508391, 0.0)

We can compute the mean of each of the color channels by using the cv2.mean method. This method returns a tuple with four values, our color features. The first value is the mean of the blue channel, the second value the mean of the green channel, and the third value is the mean of red channel. Remember, OpenCV stores RGB images as a NumPy array, but in reverse order. We actually read them backwards in BGR order, hence the blue value comes first, then the green, and finally the red.

The fourth value can be ignored and exists only so that OpenCV’s built-in Scalar class can be used internally. This value can be ignored as such:

>>> means = means[:3]
>>> means
(181.12238527002307, 199.18315040165433, 206.514296508391)

Now we can see that the output of our image descriptor (the cv2.mean function) is a feature vector with a list of three numbers: the means of the blue, green, and red channels, respectively.

Color Mean and Standard Deviation

Let’s compute both the mean and standard deviation of each channel as well.

Again, here is our template:

1. What image descriptor am I using? A color mean and standard deviation descriptor.
2. What is the expected output of my image descriptor? The mean and standard deviation of each channel of the image.

And now the code:

>>> (means, stds) = cv2.meanStdDev(image)
>>> means, stds
(array([[ 181.12238527],
       [ 199.1831504 ],
       [ 206.51429651]]), array([[ 80.67819854],
       [ 65.41130384],
       [ 77.77899992]]))

In order to grab both the mean and standard deviation of each channel, we use the cv2.meanStdDev function, which not surprisingly, returns a tuple — one for the means and one for the standard deviations, respectively. Again, this list of numbers serves as our color features.

Let’s combine the means and standard deviations into a single color feature vector:

>>> import numpy as np
>>> stats = np.concatenate([means, stds]).flatten()
>>> stats
array([ 181.12238527,  199.1831504 ,  206.51429651,   80.67819854,
         65.41130384,   77.77899992])

Now our feature vector stats has six entries rather than three. We are now representing the mean of each channel as well as the standard deviation of each channel in the image.

Color Histograms

Going back to the Clever Girl: A Guide to Utilizing Color Histograms for Computer Vision and Image Search Engines and Hobbits and Histograms, we could also use a 3D color histogram to describe our Charizard.

1. What image descriptor am I using? A 3D color histogram.
2. What is the expected output of my image descriptor? A list of numbers used to characterize the color distribution of the image.

>>> hist = cv2.calcHist([image], [0, 1, 2], None, [8, 8, 8], [0, 256, 0, 256, 0, 256])

Here we have a 3D histogram with 8 bins per channel. Let’s examine the shape of our histogram:

>>> hist.shape
(8, 8, 8)

Our histogram has a shape of (8, 8, 8). How can we use this as a feature vector if it’s multi-dimensional?

We simply flatten it:

>>> hist = hist.flatten()
>>> hist.shape
(512,)

By defining our image descriptor as a 3D color histogram we can extract a list of numbers (i.e. our feature vector) to represent the distribution of colors in the image.

Summary

In this blog post we have provided a formal definition for an image feature vector. A feature vector is an abstraction of the image itself and at the most basic level, is simply a list of numbers used to represent the image. We have also reviewed some examples on how to extract color features.

The first step of building any image search engine is to define your image descriptor. Once we have defined our image descriptor we can apply our descriptor to an image. The output of the image descriptor is our feature vector.

We then defined a two step template that you can use when defining your image descriptor. You simply need to ask yourself two questions:

1. What image descriptor am I using?
2. What is the expected output of my image descriptor?

The first question defines what aspect of the image you are describing, whether it’s color, shape, or texture. And the second question defines what the output of the descriptor is going to be after it has been applied to the image.

Using this template you can ensure you always know what you are describing and how you are describing it.

Finally, we provided three examples of simple image descriptors and feature vectors to make our discussion more concrete.

What's next? I recommend PyImageSearch University.

Course information:
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled

I strongly believe that if you had the right teacher you could master computer vision and deep learning.

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s not the case.

All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.

Inside PyImageSearch University you'll find:

• ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
• ✓ 30+ Certificates of Completion
• ✓ 39h 44m on-demand video
• ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
• ✓ Pre-configured Jupyter Notebooks in Google Colab
• ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
• ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
• ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
• ✓ Access on mobile, laptop, desktop, etc.

Click here to join PyImageSearch University

Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to join the PyImageSearch Newsletter and download my FREE 17-page Resource Guide PDF on Computer Vision, OpenCV, and Deep Learning.

Join the Newsletter!

Website