In last week’s blog post I demonstrated how to count the number of frames in a video file.
Today we are going to use this knowledge to help us with a computer vision and image processing task — visualizing movie barcodes, similar to the one at the top of this post.
I first became aware of movie barcodes a few years back from this piece of software which was used to generate posters and trailers for the 2013 Brooklyn Film Festival.
Since I started PyImageSearch I’ve received a handful of emails regarding generating movie barcodes, and awhile I normally don’t cover visualization methods, I ended up deciding to write a blog post on it. It is a pretty neat technique after all!
In the remainder of this tutorial I’ll be demonstrating how to write your own Python + OpenCV application to generate movie barcodes of your own.
Looking for the source code to this post?
Jump Right To The Downloads SectionGenerating movie barcodes with OpenCV and Python
In order to construct movie barcodes we need to accomplish three tasks:
- Task #1: Determine the number of frames in a video file. Computing the total number of frames in a movie allows us to get a sense of how many frames we should be including in the movie barcode visualization. Too many frames and our barcode will be gigantic; too little frames and the movie barcode will be aesthetically unpleasing.
- Task #2: Generating the movie barcode data. Once we know the total number of video frames we want to include in the movie barcode, we can loop over every N-th frame and compute the RGB average, maintaining a list of averages as we go. This serves as our actual movie barcode data.
- Task #3: Displaying the movie barcode. Given the list of RGB averages for a set of frames, we can take this data and create the actual movie barcode visualization that is displayed to our screen.
The rest of this post will demonstrate how to accomplish each of these tasks.
Movie barcode project structure
Before we get too far in this tutorial, let’s first discuss our project/directory structure detailed below:
|--- output/ |--- videos/ |--- count_frames.py |--- generate_barcode.py |--- visualize_barcode.py
The output
directory will store our actual movie barcodes (both the generated movie barcode images and the serialized RGB averages).
We then have the videos
folder where our input video files reside on disk.
Finally, we need three helper scripts: count_frames.py
, generate_barcode.py
, and visualize_barcode.py
. We’ll be discussing each of these Python files in the following sections.
Installing prerequisites
I’ll assume you already have OpenCV installed on your system (if not, please refer to this page where I provide tutorials for installing OpenCV on a variety of different platforms).
Besides OpenCV you’ll also need scikit-image and imutils. You can install both using pip
:
$ pip install --upgrade scikit-image imutils
Take the time to install/upgrade these packages now as we’ll need them later in this tutorial.
Counting the number of frames in a video
In last week’s blog post I discussed how to (efficiently) determine the number of frames in a video file. Since I’ve already discussed the topic in-depth, I’m not going to provide a complete overview of the code today.
That said, you can find the source code for count_frames.py
below:
# import the necessary packages from imutils.video import count_frames import argparse import os # construct the argument parse and parse the arguments ap = argparse.ArgumentParser() ap.add_argument("-v", "--video", required=True, help="path to input video file") ap.add_argument("-o", "--override", type=int, default=-1, help="whether to force manual frame count") args = vars(ap.parse_args()) # count the total number of frames in the video file override = False if args["override"] < 0 else True total = count_frames(args["video"], override=override) # display the frame count to the terminal print("[INFO] {:,} total frames read from {}".format(total, args["video"][args["video"].rfind(os.path.sep) + 1:]))
As the name suggests, this script simply counts the number of frames in a video file.
As an example, let’s take the trailer to my favorite movie, Jurassic Park:
After downloading the .mp4 file of this trailer (included in the “Downloads” section at the bottom of this tutorial), I can execute count_frames.py
on the video:
$ python count_frames.py --video videos/jurassic_park_trailer.mp4 [INFO] 4,790 total frames read from jurassic_park_trailer.mp4
As my output demonstrates, there are 4,790 frames in the video file.
Why does the frame count matter?
I’ll discuss the answer in the following section.
Generating a movie barcode with OpenCV
At this point we know how to determine the total number of frames in a video file — although the exact reasoning as to why we need to know this information is unclear.
To understand why it’s important to know the total number of frames in a video file before generating your movie barcode, let’s dive into generate_barcodes.py
:
# import the necessary packages import argparse import json import cv2 # construct the argument parse and parse the arguments ap = argparse.ArgumentParser() ap.add_argument("-v", "--video", required=True, help="path to input video") ap.add_argument("-o", "--output", required=True, help="path to output JSON file containing frame averages") ap.add_argument("-s", "--skip", type=int, default=0, help="# of frames to skip (to create shorter barcodes)") args = vars(ap.parse_args())
Lines 2-4 import our required Python packages while Lines 7-14 parse our command line arguments.
We’ll require two command line arguments along with an optional switch, each of which are detailed below:
--video
: This is the path to our input video file that we are going to generate the movie barcode for.--output
: We’ll be looping over the frames in the input video file and computing the RGB average for every N-th frame. These RGB averages will be serialized to a JSON file so we can use this data for the actual movie barcode visualization in the next section.--skip
: This parameter controls the number of frames to skip when processing the video. Why might we want to skip frames? Consider the Jurassic Park trailer above: There are over 4,700 frames in a movie clip under 3m30s. If we used only one pixel to visualize the RGB average for each frame, our movie barcode would be over 4,700 pixels wide! Thus, it’s important that we skip every N-th barcode to reduce the output visualization size.
Our next code block handles initializing our list of frame averages and opening a pointer to our video file via the cv2.VideoCapture
method:
# initialize the list of frame averages along with the total # number of frames read avgs = [] total = 0 # grab a pointer to the video file print("[INFO] looping over frames in video (this will take awhile)...") video = cv2.VideoCapture(args["video"])
Now that our variables are initialized, we can loop over the frames and compute our averages:
# loop over the frames of the video while True: # grab the current frame (grabbed, frame) = video.read() # check to see if we have reached the end of the # video if not grabbed: break # increment the total number of frames read total += 1 # check to see if we should compute the average RGB value # of the current frame if args["skip"] == 0 or total % args["skip"] == 0: avg = cv2.mean(frame)[:3] avgs.append(avg) # release the video pointer video.release()
We use the .read
method to grab the next frame from the video file (Line 28) and increment the total number of frames processed (Line 36).
We then apply the --skip
command line argument to determine if the current frame should be included in the avgs
list or not (Line 40).
Provided the frame should be kept, we compute the RGB average of the frame
and update the avgs
list (Lines 41 and 42).
After processing all frames from the video file, we can serialize the RGB averages to disk:
# dump the frame averages to file print("[INFO] saving frame averages...") f = open(args["output"], "w") f.write(json.dumps(avgs)) f.close() print("[INFO] {:,} total frame averages saved".format(len(avgs)))
Executing this script on our Jurassic Park trailer, you’ll see the following output:
Notice here that 199 frames have been saved to disk using a frame skip of 25.
Going back to the output of count_frames.py
we can see that 4,790 / 25 = ~199 (the calculation actually equals ~192, but that’s due to a discrepancy in count_frames.py
— to learn more about this behavior, please see last week’s blog post).
You should also now see a file named jurassic_park_trailer.json
in your output
directory:
$ ls -l output/ total 968 -rw-r--r-- 1 adrianrosebrock staff 11696 Jan 10 07:35 jurassic_park_trailer.json
Visualizing the movie barcode with OpenCV
Now that we have our RGB averages for the frames in the movie, we can actually visualize them.
Open up the visualize_barcode.py
file and insert the following code:
# import the necessary packages import numpy as np import argparse import json import cv2 # construct the argument parse and parse the arguments ap = argparse.ArgumentParser() ap.add_argument("-a", "--avgs", required=True, help="path to averages JSON file") ap.add_argument("-b", "--barcode", required=True, help="path to output barcode visualization image") ap.add_argument("-t", "--height", type=int, default=250, help="height of output barcode image") ap.add_argument("-w", "--barcode-width", type=int, default=1, help="width of each bar in output image") args = vars(ap.parse_args())
This particular script requires two command line arguments followed by two optional ones:
--avgs
: This switch is the path to our serialized JSON file that contains the average RGB values for every N-th frame in our video.--barcode
: Here we supply the path to our output movie barcode visualization image.--height
: This parameter controls the height of the movie barcode visualization. We’ll default this value to a height of 250 pixels.--barcode-width
: Each individual bar (i.e., RGB average) in the movie barcode needs to have a width in pixels. We set a default value of 1 pixel per bar, but we can change the width by supplying a different value for this command line argument.
We are now ready to visualize the barcode:
# load the averages file and convert it to a NumPy array avgs = json.loads(open(args["avgs"]).read()) avgs = np.array(avgs, dtype="int") # grab the individual bar width and allocate memory for # the barcode visualization bw = args["barcode_width"] barcode = np.zeros((args["height"], len(avgs) * bw, 3), dtype="uint8") # loop over the averages and create a single 'bar' for # each frame average in the list for (i, avg) in enumerate(avgs): cv2.rectangle(barcode, (i * bw, 0), ((i + 1) * bw, args["height"]), avg, -1) # write the video barcode visualization to file and then # display it to our screen cv2.imwrite(args["barcode"], barcode) cv2.imshow("Barcode", barcode) cv2.waitKey(0)
Lines 20 and 21 load the serialized RGB means from disk.
Lines 26 and 27 utilize the --height
switch along with the number of entries in the avgs
list and the --barcode-width
to allocate memory for a NumPy array large enough to store the movie barcode.
For each of the RGB averages we loop over them individually (Line 31) and use the cv2.rectangle
function to draw the each bar in the movie barcode (Lines 32 and 33).
Finally, Lines 37-39 write the movie barcode
to disk and display the visualization to our screen.
Movie barcode visualizations
To see the movie barcode visualization for the Jurassic Park trailer, make sure you have downloaded the source code + example videos using the “Downloads” section of this post. From there, execute the following commands:
$ python generate_barcode.py --video videos/jurassic_park_trailer.mp4 \ --output output/jurassic_park_trailer.json --skip 25 $ python visualize_barcode.py --avgs output/jurassic_park_trailer.json \ --barcode output/jurassic_park_trailer.png --barcode-width 5
The large green bars at the beginning of the barcode correspond to the green preview screen required by the Motion Picture Association of America, Inc.
The blues in the middle of the movie barcode refer to:
- The heavy downpours and the bluish tint cast by the strong thunderstorm when things start to go downhill for our park visitors.
- Tim’s blue shirt while in the jeep.
- Denny Nedry in the blue-tinted embryo cold storage room.
Let’s now visualize the movie trailer to The Matrix, another one of my all-time favorite movies:
Below you can see the output from running count_frames.py
and generate_barcode.py
on the movie clip:
And here follows the actual movie barcode:
Perhaps unsurprisingly, this movie barcode is heavily dominated by the The Matrix-style greens and blacks.
Finally, here is one last example from the movie trailer for The Dark Knight:
Here are the commands I used to generate the barcode:
$ python count_frames.py --video videos/the_dark_knight_trailer.mp4 $ python generate_barcode.py --video videos/the_dark_knight_trailer.mp4 \ --output output/the_dark_knight_trailer.json --skip 25 $ python visualize_barcode.py --avgs output/the_dark_knight_trailer.json \ --barcode output/the_dark_knight_trailer.png --barcode-width 5
Along with the final visualization:
Note: I used the website keepvid.com to download the trailers to the movies mentioned above. I do not own the copyrights to these videos nor do I claim to — this blog post is for educational and example purposes only. Please use wisely.
What's next? I recommend PyImageSearch University.
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 30+ Certificates of Completion
- ✓ 39h 44m on-demand video
- ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
In today’s blog post I demonstrated how to generate video barcodes using OpenCV and Python.
Generating video barcodes is normally used for design aesthetics and admittedly doesn’t have a far-reaching computer vision/image processing purpose (other than visualization itself).
That said, you could treat the movie barcodes as a form of “feature vector” and use them to compare other movie barcodes/movie clips for similarity or even determine where in a video a given clip appears.
I hope you enjoyed this blog post!
Be sure to enter your email address in the form below to be notified when future blog posts are published.
Download the Source Code and FREE 17-page Resource Guide
Enter your email address below to get a .zip of the code and a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!