Writing to video with OpenCV

Let me just start this blog post by saying that writing to video with OpenCV can be a huge pain in the ass.

My intention with this tutorial is to help you get started writing videos to file with OpenCV 3, provide (and explain) some boilerplate code, and detail how I got video writing to work on my own system.

However, if you are trying to write videos to file with OpenCV in your own applications, be prepared to:

1. Do research on the video codecs installed on your system.
2. Play around with various codec + file extensions until the video writes to disk successfully.
3. Make sure you’re in a secluded place away from children — there will be a lot of swearing and cursing.

You see, while the functions used to create video files with OpenCV such as cv2.VideoWriter , cv2.VideoWriter_fourcc , and cv2.cv.FOURCC are quite well documented, what isn’t nearly as documented is the combination of codec + file extension required to successfully write the video file.

It’s been a long time since I needed to create an application to write videos to file with OpenCV, so when I sat down to compose the code for this blog post, I was very surprised (and super frustrated) with how long it took me to put together the example.

In fact, I was only able to get the code working with OpenCV 3! The code detailed in this post is not compatible with OpenCV 2.4.X (although I have highlighted the code changes required to run on OpenCV 2.4 if you want to give it a try).

Note: If you need help installing OpenCV on your system, please consult this page for a list of installation instructions for various platforms. Also be sure to take a look at the Quickstart Bundle and Hardcopy Bundle of Practical Python and OpenCV which include a downloadable Ubuntu VirtualBox virtual machine with Open 3 pre-configured and pre-installed.

Anyway, in the remainder of this post, I’ll demonstrate how to write video to file using OpenCV. And hopefully you’ll be able to use this code in your own applications without tearing out too much hair (I’m already so bald, so I don’t have that problem).

Writing to video with OpenCV

The purpose of this tutorial is to learn how to write frames to video using OpenCV and Python. We’ll discuss how to:

1. Access a video stream, using either a builtin/USB webcam or the Raspberry Pi camera module.
2. Read frames from the stream.
3. Construct a new frame that visualizes the original image, plus the Red, Green, and Blue channel components individually.
4. Write our newly constructed frame out to video file using OpenCV.

The results will look similar to the screenshot below:

Here we can see the output video being played in QuickTime, with the original image in the top-left corner, the Red channel visualization in the top-right, the Blue channel in the bottom-left, and finally the Green channel in the bottom-right corner.

Again, the code for this post is meant to be ran with OpenCV 3 or 4, so make sure you have the right version installed on your system before executing the code.

Creating our OpenCV video writer

Let’s go ahead and get started writing to video with OpenCV.

Open up a new file, name it write_to_video.py , and insert the following code:

# import the necessary packages
from __future__ import print_function
from imutils.video import VideoStream
import numpy as np
import argparse
import imutils
import time
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-o", "--output", required=True,
	help="path to output video file")
ap.add_argument("-p", "--picamera", type=int, default=-1,
	help="whether or not the Raspberry Pi camera should be used")
ap.add_argument("-f", "--fps", type=int, default=20,
	help="FPS of output video")
ap.add_argument("-c", "--codec", type=str, default="MJPG",
	help="codec of output video")
args = vars(ap.parse_args())

We start off on Lines 2-8 by importing our required Python packages. We’ll be using the (highly efficient and threaded) VideoStream class which gives us unified access to both builtin/USB webcams along with the Raspberry Pi camera module. The VideoStream class is implemented inside the imutils Python package. You can read more about the VideoStream class, how it can access multiple camera inputs, and efficiently read frames in a threaded manner in this tutorial.

Also, if you do not already have imutils installed on your system, you’ll want to do that now:

$ pip install imutils

From there, we parse our command line arguments on Lines 11-20. Our write_to_video.py script requires one command line argument, followed by three optional ones:

• --output : This is the path to where our output video file will be stored on disk.
• --picamera : Here we can specify if we want to use the Raspberry Pi camera module instead of a builtin/USB camera. Supply a value > 0 at runtime to access the Pi camera module.
• --fps : This switch controls the desired FPS of the output video. Ideally, the FPS of the output video should be similar to the FPS of your video processing pipeline.
• --codec : Here we supply the FourCC, or four character code, an identifier for the video codec, compression format, and color/pixel format in video files.

As I mentioned at the top of this post, you’ll likely be spending a lot of time tweaking the --output video file extension (e.x., .avi , .mp4 , .mov , etc.) and the --codec value.

As the name suggests, the FourCC is always four characters. Popular examples include MJPG , DIVX , and H264 . You can see the entire list of possible FourCC codes here.

When I originally wrote the code for this blog post, I spent hours trying to figure out the right combination of both file extension and FourCC. I tried dozens of combinations, but none of them worked.

Furthermore, these combinations are also a bit temperamental. When I tried using a MJPG codec with a .mp4 or .mpg file extension, the video wouldn’t write to file. But when I used MJPG with a .avi file extension, the frames were magically written to video file.

Baffling, right?

My point here is that you’ll need to spend time playing around with these values. Depending on your setup and video codecs installed on your system, different combinations may work while others may not.

That said, I found the combination of MJPG and .avi worked on both my OSX machine and my Raspberry Pi out of the box, so if you’re having issues getting video to write to file, be sure to try these combinations first!

Let’s continue working through the write_to_video.py script:

# initialize the video stream and allow the camera
# sensor to warmup
print("[INFO] warming up camera...")
vs = VideoStream(usePiCamera=args["picamera"] > 0).start()
time.sleep(2.0)

# initialize the FourCC, video writer, dimensions of the frame, and
# zeros array
fourcc = cv2.VideoWriter_fourcc(*args["codec"])
writer = None
(h, w) = (None, None)
zeros = None

Here we initialize our VideoStream and allow the camera sensor to warmup (Lines 25 and 26).

We then initialize our fourcc codec using the cv2.VideoWriter_fourcc function and the --codec value supplied as a command line argument.

Note: For OpenCV 2.4.X, you’ll need to change cv2.VideoWriter_fourcc(*args["codec"]) function to cv2.cv.FOURCC(*args["codec"]) .

From there, initialize a few more variables, including our video writer, width and height of our frame, and finally a NumPy array that will be comprised entirely of zeros (more on this later).

Now comes the main loop of our script:

# loop over frames from the video stream
while True:
	# grab the frame from the video stream and resize it to have a
	# maximum width of 300 pixels
	frame = vs.read()
	frame = imutils.resize(frame, width=300)

	# check if the writer is None
	if writer is None:
		# store the image dimensions, initialize the video writer,
		# and construct the zeros array
		(h, w) = frame.shape[:2]
		writer = cv2.VideoWriter(args["output"], fourcc, args["fps"],
			(w * 2, h * 2), True)
		zeros = np.zeros((h, w), dtype="uint8")

On line 36 we start looping over frames from our video stream, reading them, and then resizing them to have a width of 300 pixels (Lines 39 and 40).

We then make a check to see if the writer is None (Line 43), and if it is, we need to initialize it. First, we grab the spatial dimensions (i.e., width and height) of the frame (Line 46) followed by instantiating the cv2.VideoWriter (Lines 47 and 48).

The cv2.VideoWriter requires five parameters:

• The first parameter is the path to the output video file. In this case, we’ll supply the value of the --output switch, which is the path to where our video file will live on disk.
• Secondly, we need to supply the fourcc codec.
• The third argument to cv2.VideoWriter is the desired FPS of the output video file.
• We then have the width and height of output video. It’s important that you set these values correctly, otherwise OpenCV will throw an error if you try to write a frame to file that has different dimensions than the ones supplied to cv2.VideoWriter .
• Finally, the last parameter controls whether or not we are writing color frames to file. A value of True indicates that we are writing color frames. Supplying False indicates we are not writing color frames.

You’ll notice that I am using a width and height that are double that of the original frame — why is that?

The reason is because our output video frame will have two rows and two columns — storing a total of four “images”. We thus need double spatial dimensions of the original frame.

Finally, Line 49 constructs an array of zeros with the same shape as the original frame.

We are now ready to construct or output frame and write it to file:

	# break the image into its RGB components, then construct the
	# RGB representation of each frame individually
	(B, G, R) = cv2.split(frame)
	R = cv2.merge([zeros, zeros, R])
	G = cv2.merge([zeros, G, zeros])
	B = cv2.merge([B, zeros, zeros])

	# construct the final output frame, storing the original frame
	# at the top-left, the red channel in the top-right, the green
	# channel in the bottom-right, and the blue channel in the
	# bottom-left
	output = np.zeros((h * 2, w * 2, 3), dtype="uint8")
	output[0:h, 0:w] = frame
	output[0:h, w:w * 2] = R
	output[h:h * 2, w:w * 2] = G
	output[h:h * 2, 0:w] = B

	# write the output frame to file
	writer.write(output)

First, we split the frame into its Red, Green, and Blue components, respectively (Line 53). We then construct a representation of each channel using zeros to fill in the appropriate dimensions (Lines 54-56).

We are now ready to create our output video frame, where the dimensions will be exactly double the width and height of the resized frame (Line 62).

Lines 63-66 handle storing the original frame in the top-left corner of the output image, the Red channel in the top-right, the Green channel in the bottom-right, and finally the Blue frame in the bottom-left.

The output frame is written to file using the write method of the cv2.VideoWriter .

Finally, our last code block handles displaying the output frames to our screen and performing a bit of cleanup:

	# show the frames
	cv2.imshow("Frame", frame)
	cv2.imshow("Output", output)
	key = cv2.waitKey(1) & 0xFF

	# if the `q` key was pressed, break from the loop
	if key == ord("q"):
		break

# do a bit of cleanup
print("[INFO] cleaning up...")
cv2.destroyAllWindows()
vs.stop()
writer.release()

Be sure to make note of Line 84 where we call the release method of the writer — this ensures that the output video file pointer is released.

Running our OpenCV video writer

To execute our OpenCV video writer using a builtin/USB webcam, use the following command:

$ python write_to_video.py --output example.avi

If you instead want to use the Raspberry Pi camera module, use this command:

$ python write_to_video.py --output example.avi --picamera 1

In either case, your output should look similar to my screenshot below:

Here you can see the original frame on the left, followed by the modified output frame that visualizes the RGB channels individually on the right.

After pressing the q  key and terminating the Python script, we can see that example.avi  has been written to my disk:

I can then open up this file in QuickTime or VLC and view it:

A full video demonstration of the OpenCV video writer can be seen below:

Help! My video isn’t writing to file with OpenCV.

As I’ve mentioned multiple times earlier in this post, the main reason your video is not writing to file is likely the combination of video codec and file extension.

I found that by using a codec of MJPG  and file extension .avi , I was able to successfully write to video using OpenCV on both my OSX machine and Raspberry Pi. I would suggest starting with these values and working from there.

But in all honesty, you will have to spend time banging your head against a wall to resolve this problem. You’ll need to try different combinations of FourCC and file extensions. Furthermore, OpenCV does not return any helpful error messages regarding this problem, so it’s pretty much a trial and error situation.

While I haven’t attempted this myself, I have heard that installing FFMPEG (and even re-compiling OpenCV with FFMPEG support) can help enable more video codecs.

Finally, if you find a combination of FourCC and file extension that works, be sure to post in the comments section, including the:

• FourCC that you used.
• File extension.
• Operating system.
• And any other relevant information.

This way we can compile a set of combinations that routinely work and (hopefully) avoid the troublesome situation of frames not being written to video file.

Summary

In today’s blog post, we learned how to write frames to video using OpenCV and Python. Specifically, we used OpenCV 3 and the cv2.VideoWriter  method to handle writing frames to file.

The key to creating a working video writer is to determine the correct combination of (1) FourCC and (2) file extension.

Furthermore, you need to ensure you have the proper video codec installed on your system –otherwise, your frames will not be written to file (and OpenCV won’t provide you with any helpful error messages or warnings).

If you find a combination of FourCC and file extension that works for you, be sure to post in the comments section, detailing which FourCC you used, the video file extension that worked, your operating system, and other other relevant information on your setup. Ideally, we can create a list of FourCC and file extension combinations that routinely work for people.

In next week’s post, we’ll create a more practical application of video writing were we save key event video clips, allowing us to parse an entire video file and save only the most interesting clips.

Before you go, be sure to signup for the PyImageSearch Newsletter using the form below — you won’t want to miss the next post on key event video clips!