Over the past year the PyImageSearch blog has had a lot of popular blog posts. Using k-means clustering to find the dominant colors in an image was (and still is) hugely popular. One of my personal favorites, building a kick-ass mobile document scanner has been the most popular PyImageSearch article for months. And the first (big) tutorial I ever wrote, Hobbits and Histograms, an article on building a simple image search engine, still gets a lot of hits today.
But by far, the most popular post on the PyImageSearch blog is my tutorial on installing OpenCV and Python on your Raspberry Pi 2 and B+. It’s really, really awesome to see the love you and the PyImageSearch readers have for the Raspberry Pi community — and I plan to continue writing more articles about OpenCV + the Raspberry Pi in the future.
Anyway, after I published the Raspberry Pi + OpenCV installation tutorial, many of the comments asked that I continue on and discuss how to access the Raspberry Pi camera using Python and OpenCV.
In this tutorial we’ll be using picamera, which provides a pure Python interface to the camera module. And best of all, I’ll be showing you how to use picamera to capture images in OpenCV format.
Read on to find out how…
IMPORTANT: Be sure to follow one of my Raspberry Pi OpenCV installation guides prior to following the steps in this tutorial.
OpenCV and Python versions:
This example will run on Python 2.7/Python 3.4+ and OpenCV 2.4.X/OpenCV 3.0+.
Step 1: What do I need?
To get started, you’ll need a Raspberry Pi camera board module.
I got my 5MP Raspberry Pi camera board module from Amazon for under $30, with shipping. It’s hard to believe that the camera board module is almost as expensive as the Raspberry Pi itself — but it just goes to show how much hardware has progressed over the past 5 years. I also picked up a camera housing to keep the camera safe, because why not?
Assuming you already have your camera module, you’ll need to install it. Installation is very simple and instead of creating my own tutorial on installing the camera board, I’ll just refer you to the official Raspberry Pi camera installation guide:
Assuming your camera board and properly installed and setup, it should look something like this:
Step 2: Enable your camera module.
Now that you have your Raspberry Pi camera module installed, you need to enable it. Open up a terminal and execute the following command:
$ sudo raspi-config
This will bring up a screen that looks like this:
Use your arrow keys to scroll down to Option 5: Enable camera, hit your enter key to enable the camera, and then arrow down to the Finish button and hit enter again. Lastly, you’ll need to reboot your Raspberry Pi for the configuration to take affect.
Step 3: Test out the camera module.
Before we dive into the code, let’s run a quick sanity check to ensure that our Raspberry Pi camera is working properly.
Note: Trust me, you’ll want to run this sanity check before you start working with the code. It’s always good to ensure that your camera is working prior to diving into OpenCV code, otherwise you could easily waste time wondering when your code isn’t working correctly when it’s simply the camera module itself that is causing you problems.
Anyway, to run my sanity check I connected my Raspberry Pi to my TV and positioned it such that it was pointing at my couch:
And from there, I opened up a terminal and executed the following command:
$ raspistill -o output.jpg
This command activates your Raspberry Pi camera module, displays a preview of the image, and then after a few seconds, snaps a picture, and saves it to your current working directory as output.jpg .
Here’s an example of me taking a photo of my TV monitor (so I could document the process for this tutorial) as the Raspberry Pi snaps a photo of me:
And here’s what output.jpg looks like:
Clearly my Raspberry Pi camera module is working correctly! Now we can move on to the some more exciting stuff.
Step 4: Installing picamera.
So at this point we know that our Raspberry Pi camera is working properly. But how do we interface with the Raspberry Pi camera module using Python?
The answer is the picamera module.
Remember from the previous tutorial how we utilized virtualenv and virtualenvwrapper to cleanly install and segment our Python packages from the the system Python and packages?
Well, we’re going to do the same thing here.
Before installing picamera , be sure to activate our cv virtual environment:
$ workon cv
Note: If you are installing the the picamera module system wide, you can skip the previous commands. However, if you are following along from the previous tutorial, you’ll want to make sure you are in the cv virtual environment before continuing to the next command.
And from there, we can install picamera by utilizing pip:
$ pip install "picamera[array]"
IMPORTANT: Notice how I specified picamera[array] and not just picamera .
Why is this so important?
While the standard picamera module provides methods to interface with the camera, we need the (optional) array sub-module so that we can utilize OpenCV. Remember, when using Python bindings, OpenCV represents images as NumPy arrays — and the array sub-module allows us to obtain NumPy arrays from the Raspberry Pi camera module.
Assuming that your install finished without error, you now have the picamera module (with NumPy array support) installed.
Step 5: Accessing a single image of your Raspberry Pi using Python and OpenCV.
Alright, now we can finally start writing some code!
Open up a new file, name it test_image.py , and insert the following code:
# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
rawCapture = PiRGBArray(camera)
# allow the camera to warmup
time.sleep(0.1)
# grab an image from the camera
camera.capture(rawCapture, format="bgr")
image = rawCapture.array
# display the image on screen and wait for a keypress
cv2.imshow("Image", image)
cv2.waitKey(0)
We’ll start by importing our necessary packages on Lines 2-5.
From there, we initialize our PiCamera object on Line 8 and grab a reference to the raw capture component on Line 9. This rawCapture object is especially useful since it (1) gives us direct access to the camera stream and (2) avoids the expensive compression to JPEG format, which we would then have to take and decode to OpenCV format anyway. I highly recommend that you use PiRGBArray whenever you need to access the Raspberry Pi camera — the performance gains are well worth it.
From there, we sleep for a tenth of a second on Line 12 — this allows the camera sensor to warm up.
Finally, we grab the actual photo from the rawCapture object on Line 15 where we take special care to ensure our image is in BGR format rather than RGB. OpenCV represents images as NumPy arrays in BGR order rather than RGB — this little nuisance is subtle, but very important to remember as it can lead to some confusing bugs in your code down the line.
Finally, we display our image to screen on Lines 19 and 20.
To execute this example, open up a terminal, navigate to your test_image.py file, and issue the following command:
$ python test_image.py
If all goes as expected you should have an image displayed on your screen:
Note: I decided to add this section of the blog post after I had finished up the rest of the article, so I did not have my camera setup facing the couch (I was actually playing with some custom home surveillance software I was working on). Sorry for any confusion, but rest assured, everything will work as advertised provided you have followed the instructions in the article!
Step 6: Accessing the video stream of your Raspberry Pi using Python and OpenCV.
Alright, so we’ve learned how to grab a single image from the Raspberry Pi camera. But what about a video stream?
You might guess that we are going to use the cv2.VideoCapture function here — but I actually recommend against this. Getting cv2.VideoCapture to play nice with your Raspberry Pi is not a nice experience (you’ll need to install extra drivers) and something you should generally avoid.
And besides, why would we use the cv2.VideoCapture function when we can easily access the raw video stream using the picamera module?
Let’s go ahead and take a look on how we can access the video stream. Open up a new file, name it test_video.py , and insert the following code:
# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 32
rawCapture = PiRGBArray(camera, size=(640, 480))
# allow the camera to warmup
time.sleep(0.1)
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
# grab the raw NumPy array representing the image, then initialize the timestamp
# and occupied/unoccupied text
image = frame.array
# show the frame
cv2.imshow("Frame", image)
key = cv2.waitKey(1) & 0xFF
# clear the stream in preparation for the next frame
rawCapture.truncate(0)
# if the `q` key was pressed, break from the loop
if key == ord("q"):
break
This example starts off similarly to the previous one. We start off by importing our necessary packages on Lines 2-5.
And from there we construct our camera object on Line 8 which allows us to interface with the Raspberry Pi camera. However, we also take the time to set the resolution of our camera (640 x 480 pixels) on Line 9 and the frame rate (i.e. frames per second, or simply FPS) on Line 10. We also initialize our PiRGBArray object on Line 11, but we also take care to specify the same resolution as on Line 9.
Accessing the actual video stream is handled on Line 17 by making a call to the capture_continuous method of our camera object.
This method returns a frame from the video stream. The frame then has an array property, which corresponds to the frame in NumPy array format — all the hard work is done for us on Lines 17 and 20!
We then take the frame of the video and display on screen on Lines 23 and 24.
An important line to pay attention to is Line 27: You must clear the current frame before you move on to the next one!
If you fail to clear the frame, your Python script will throw an error — so be sure to pay close attention to this when implementing your own applications!
Finally, if the user presses the q key, we break form the loop and exit the program.
To execute our script, just open a terminal (making sure you are in the cv virtual environment, of course) and issue the following command:
$ python test_video.py
Below follows an example of me executing the above command:
As you can see, the Raspberry Pi camera’s video stream is being read by OpenCV and then displayed on screen! Furthermore, the Raspberry Pi camera shows no lag when accessing frames at 32 FPS. Granted, we are not doing any processing on the individual frames, but as I’ll show in future blog posts, the Pi 2 can easily keep up 24-32 FPS even when processing each frame.
What's next? I recommend PyImageSearch University.
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 30+ Certificates of Completion
- ✓ 39h 44m on-demand video
- ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
This article extended our previous tutorial on installing OpenCV and Python on your Raspberry Pi 2 and B+ and covered how to access the Raspberry Pi camera module using Python and OpenCV.
We reviewed two methods to access the camera. The first method allowed us to access a single photo. And the second method allowed us to access the raw video stream from the Raspberry Pi camera module.
In reality, there are many ways to access the Raspberry Pi camera module, as the picamera documentation details. However, the methods detailed in this blog post are used because (1) they are easily compatible with OpenCV and (2) they are quite speedy. There are certainly more than one way to skin this cat, but if you intend on using OpenCV + Python, I would suggest using the code in this article as “boilerplate” for your own applications.
In future blog posts we’ll take these examples and use it to build computer vision systems to detect motion in videos and recognize faces in images.
Be sure to sign up for the PyImageSearch Newsletter to receive updates when new Raspberry Pi and computer vision posts go live, you definitely don’t want to miss them!
Download the Source Code and FREE 17-page Resource Guide
Enter your email address below to get a .zip of the code and a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!
Hello Adrian,
Thank you for this new demonstration, which works very well!
I wait for the following episode with impatience, to understand how to capture a detecting motion and tracking a person.
Thank you very much to share with us your experiment.
Christian
Awesome Adrian!
Thanks a lot for this! (and for how cleanly everything is explained).
I’ll make sure to stay tuned.
Fabio Guarini
Hello Adrian,
Thanks for the documentation. But I have a question to Frame rate. In the example above we have a frame rate of 32, but I get only 10 Images per second. Would it be faster to use c/C++ instead?
Dieter
Are you using the Raspberry Pi B/B+ or the Pi 2? You can easily get 32 FPS on the Pi 2. Using C/C++ will almost certainly be faster, but I would check ensure that your camera is working properly. It could be possible that the frames are not being read from the camera fast enough.
Hello Adrian:
I thank you for opening the door for many of us beginners to run OpenCV on raspberry.
I just ran “test_video.py” example above, but I am getting about 10 frames per second instead of 32. I am using raspberry Pi 2.
Could I be missing something obvious?
Very best regards,
Razmik Karabed
You can try increasing the FPS by using threading. Also, reducing the frame dimensions from 640 x 480 to 320 x 240 will also dramatically increase your frame rate.
is picamera capable for live video streaming to do face recognition
Face recognition algorithms don’t “care” where the video stream is from, as long as you can read the frames from a stream. Inside the PyImageSearch Gurus course I demonstrate how to do facial recognition using the Raspberry Pi picamera module.
for raspberry pi 3 people should know that “Enable/disable connection to the Raspberry Pi Camera”. is the options you want to click on after you have done sudo raspi-config. Then you will have to enable the camera from there. Different then raspberry pi 2 =]
How did you get X11 to work with OpenCV? Are you using the same `cv2.imshow(“image”, image)` call? or something special? I can’t seem to get it to work with my windows system.
Are you using X11 forwarding when ssh’ing into your Pi? I’m not sure about windows systems, but the command on Unix systems is
ssh -X pi@ipaddr. Your code will not have to change at all and you’ll still be able to usecv2.imshow.I finally got it to work. It was a combination of things I believe. I’ve come to realize that the RasPi sometimes gets into a bad state and needs to be rebooted. Also, I don’t think that I was capturing the `waitkey` correctly and the script was ending too soon.
Another question though. I’ve noticed that using OpenCV doesn’t always get me the native resolution of the camera. Do you know how to get this? This is true when using the built-in Surface Pro 2 camera and my current web cam. Both should be 720p, but I’m getting something in the range of 480. Do you know how to get this to the native resolution?
I’m simply using the `cam = cv2.VideoCapture()` to obtain the device and `(grabbed, image) = cam.read()` to grab the image.
When accessing the camera through the Raspberry Pi, I actually prefer to use the
picameramodule rather thancv2.VideoCapture. It gives you much more flexibility, including obtaining native resolution. Please see the rest of this blog post for more information on manually setting the resolution of the camera.`PiCamera` isn’t limited to the native camera? Meaning that I can use that module with any webcam?
It depends. If it’s USB based you might be out of luck. But if your “native” camera can be plugged into the slot in the Pi, then you’re in business (I’m not familiar with the cameras you mentioned previously).
Ok that makes sense. I found this as well. I’ll check it out and see if it works:
http://stackoverflow.com/a/20120262/447015
Thank you very much! This blog is very helpful ! I ‘m looking forward to your new blogs!
Hi Adrian!!
First of all, THANKS A BUNCH MAN!! I understood almost all the proccesses of this tutorial and the previous one, and i had no errors.
But i have a question, I am using a GoPiGo Robot from Dexter industries, and i would like to make the robot follow an object. Using this code, i’ve seen that the capturing of the images goes a little bit slow, so in real time i don’t know if i could get the same faster in order to do the tracking.
Do you have an idea how it could be faster?
And, Do you have any post related with the tracking objects?
Thanks again Adrian!
Hi Dave, I’m not familiar with the GoPiGo, but if the image capturing is slow then you probably want to reduce the FPS and the resolution. I also cover tracking objects inside my book, Practical Python and OpenCV.
Hello Adrian:
First, thanks for the wonderful blogs.
I have a similar problem like Dave. but I’ve been using the Pi camera module. I used your method (test_image.py) to capture the image before processing it, I figured out that the major timing approximately 1.3 sec (for Pi 2) is spending on taking an image. I used the time() function to calculate the time. Also, when I tried to remove the line 12 (time.sleep(0.1)), the quality of a taken image wasn’t good enough for my application. Would you describe more details how to reduce the FPS and resolution in order to decrease the timing for taking an image?
Thank you so much!
Hey John, the resolution of the camera is controlled by Lines 9 and 11. Lower the resolution to 320 x 240, and you should see a substantial pickup. Secondly, I’m not sure if you noticed or not, but I just did a series of blog posts on how to increase the FPS of your Pi using Python and OpenCV. Click here to read more.
Hello Adrian,
Amazing tutorial! Everything works great, except that I’m able to get only 2-3 fps with the code above. I am using the model B+. I have recorded videos with >30 fps using raspivid, so I know that the camera module works just fine. What is it that I am missing?
Thanks,
Max
Hi Max, that is definitely pretty strange, I’m not sure why you would only be getting 2-3 FPS. Maybe try reducing the image resolution and see if that helps?
Adrian,
thank you for these tutorials. i have succesfully completed the first tutorial about the installations, and steps 1-4 of this tutorial. The code in step 5 however, returns: “gtk warning ** cannot open display: :0.0”.
i am have the rpi b+ connected to my television with an hdmi cable. have you, or anyone else, seen and fixed this problem before? when i google this error i only get the suggestion to export the display variable (as i did) but that does not seem to work.
Thank you!
Hey Vincent, are you running the command from the X GUI? It sounds like you’re simply executing the command from the command line (which you absolutely should do), but you need to have your GUI launched. Run
startxto load the X interface, then open up a terminal and execute the script.Adrian,
that fixed it. thank you!
No problem!
Hi,
I understand that is impossible to run a py program with uv4l driver without a GUI ?
I have the same problem ok Gtk-warning ….. My script works fine under X interface.
Thanks
How are you accessing your Raspberry Pi? Via SSH? VNC?
Hi, my problem is same as Vincent, and after I run startx, I got server error, then how to fix it? Anyone can help me? Thank you very much!!!
Hi Adrian,
First of all, a wonderful post. I followed your previous post to install OpenCV and Python in Raspbian without any problem. I’ve just gone through this post and tried both test_image.py and test_video.py without ‘real’ problem as well.
The only issue I have is it seems that I’m not quite getting 32 FPS for my RPi2 at 640×480. I compared the video with raspivid -d and feel that at 640×480, it wasn’t as smooth. On the other hand, if I change the resolution to 480×320, they seems comparable. So, I’m wondering how to get a more precise FPS info for realtime video. Any suggestions?
Thanks
Are you running any other applications on the Pi while trying capturing the video? You should definitely be able to get 20+ FPS at 640 x 480 without a real issue. Also make sure that the camera is connected properly. It’s rare, but I’ve seen situations where the camera connection is a bit loose and while the Pi can see that the camera is there, the actual frame rate drops.
Hi Adrian,
No, just running X and a terminal. The connection to the camera should be fine as I simply change the resolution to 480×320 and it works. Also running raspivid -d without problem. Anyway, in order to quantify it, I need a way to measure the frame rate. May be record it and see from there.
Hi Adrian,
i was using exactly your code with the same peripherals; the only difference is that I am using Raspberry Pi and not Raspberry Pi 2.
As there are more users having problems with the framerate:
Might it be possible that the normal Raspberry Pi is that much worse when it comes to framerate than the new Raspberry Pi 2?
I have a program running where I do some blob detection and get maybe 2-3 frames with a resolution of 640&480.
Also with your code I do not get much more than maybe 5 fps @640&480.
Absolutely — the original Raspberry Pi is much, much slower than the P 2. I would recommend upgrading to the Pi 2 if you can, it’s definitely worth it.
Wow – that was an unexpected quick reply – thanks for that!
Since I already have the hardware would it make sense to re-write the code to C in order to receive at least 15 frames?
What would you say?
You’ll likely get some performance gains by dropping down into C, but in reality the previous Pi’s only had one core so there’s only so much performance that you can really gain. If you really want to obtain faster performance, pick up a Pi 2.
Nice blog entry, as always, Adrian!
I was also seeing slow and sluggish frame rate when I was forwarding the video back to my PC via SSH. The frame rate was much better when I setup a VNC server on the Pi2, connected to it from the PC, and then started test_video.py on the VNC’s desktop.
I’m prettty sure I wouldn’t have any issues if I just hooked a monitor to the Pi, but I like the monitor-less configuration better.
FYI, link for VNC setup on Pi that I used: http://elinux.org/RPi_VNC_Server
Absolutely, VNC will increase lag dramatically. The Pi 2 is actually running at a pretty high framerate, but the problem arises when you try to stream the results back to your VNC client.
Hi,
I’ve managed to put your example working. I just need now to flip the image (to then convert the image to HSV and track the brightest spot on the image) and I can’t do that with cv.flip() and cv2.flip() functions.
Can you give me any clue?
Hey Pedro, what do you mean by “flipping” the image? Do you mean flipping the image horizontally or vertically? Or are you trying to convert directly to the HSV color space? If you want to find the brightest spot in an image, you’ll also want to take a look at this post.
Thanks for the reply Adrian.
I want to rotate the image 180º.
About that post, I’ll try that method using the camera to do the processing in live stream.
If you want to rotate the image 180 degrees, you’ll need the
cv2.rotatefunction. I cover the very basic image processing functions in this post as well as in my book, Practical Python and OpenCV.Thank you very much Adrian.
I found on that post what I needed! And I’ve also managed to found the way to make it spot a laser!
Thank you for all the help, that’s all I needed (I hope!)!
Very nice! What was your approach to spot the laser?
As I found on some articles, the laser spot must be the brightest point on the screen.
But this isn’t so linear to apply on the raspberry pi camera. I have to be careful with the ambient birghtness. If the ambient is too bright, it detect false positives.
Now I’m with another problem: processing speed. My program can only process a frame every 150ms. This is much slow for my application 🙁 I need now to find another faster solution and when I find the resolution for my problems I post them here!
Thank you once again!
Very nice solution Pedro, congrats! And a quick way to obtain faster FPS is to simply downscale the image. Less data to process == faster runtime.
Hello Adrian,
Thanks for this awesome tutorial!!! Everything went smoothly!!!! Now, I just need to figure out how to read letters and numbers from images, taken by the pi camera….do you have a tutorial on reading text from images???
Regards,
M
Hi Marcellus, I actually cover the basics of recognizing digits inside my book, Practical Python and OpenCV + Case Studies. There is a chapter dedicated entirely to recognizing digits that you could absolutely use and modify to your needs.
Thanks!
Hey Adrian!
When i do:
sudo python myfile.py i get this error
ImportError:No module named cv2
and when i do python myfile.py it works correctly. But i need to use opencv with sudo, due to i need to communicate with another extension wich requires super user permission. Is there some way i can do that ?
Thanks again!
Hey Dave, I’ve actually never done this before, but here’s my best guess: If you want to use OpenCV inside a virtual environment for the root user, then you’ll need to switch over to the root user account and repeat steps 7 and 10 for root. I’m not sure if the sudo command will be able to access the root virtual environment once it’s created, you may actually have to switch over to the root account to run the script and ensure it accesses the virtual environment.
Sorry about my ignorance Adrian, but what do you mean by “steps 7 an 10 for root?
Thanks a bunch!
You’ll need to launch a root shell:
$ sudo /bin/bashand then create your virtual environment and sym-link OpenCV as the root user, which should be in the directory /root.Thanks Adrian! I’ll try it.
I had the same issue.
You can fix it by changing wich python you use.
like this: sudo /home/pi/.virtualenvs/opencv/bin/python
and change opencv with your environment. This did the trick for me
Hello Adrian!
First of all I’d like to say that you are one of the greatest computer-vision tutors I have seen 🙂
I have a question related to this article: If I want to capture two consecutive frames from a video stream (at every iteration), what is the correct method? I would like to compute some differences between frames for detecting motion.
Best regards!
Thanks for the kind compliment Andrew! 😀
If you want to compare the difference between two frames, you will need two variables: the
previousframe and thecurrentframe. Right after Line 20 I would check to see if thepreviousframe has been initialized or not. If not, initialize it as the current frame. Otherwise, you’ll have thecurrentandpreviousframe together and you’ll be able to compute the differences between them.I do have some pretty epic plans to cover motion detection with the Raspberry Pi, so definitely stay tuned!
Hi Adrian, I’m new with raspberry so I have some questions. After finish your tutorial I can’t import cv2, but if i work out of virtualenv I can import cv2. Do you know what happens?
Please help me.
Hi Pablito, this tutorial actually built on my previous tutorial on installing OpenCV on your Raspberry Pi. Take a look at Step 10 where the OpenCV library is sym-linked into the virtual environment.
You do not have to use virtual environments if you do not want, it’s just good practice.
Hello Adrian! Firstly, excellent tutorial. I enjoyed the use of the virtual environment in the first tutorial and this one did a beautiful job of following on. I have got my RaspiPiCam working using the code you have provided and can take still and motion video. However, I am now lost at the next step in the OpenCV testing process.
When I go to the available OpenCV Samples under the “/opencv-2.4.10/samples/python2” folder and attempt to run them, they do not recognize the RaspiPiCam stream. In particular, they do not like statements like: “try: video_src = video_src[0]” (as found in facedetect.py).
I believe that there is a method to get OpenCV to directly play video from a RaspiPiCam using Python (as found here: http://raspberrypi.stackexchange.com/questions/17068/using-opencv-with-raspicam-and-python), but I can’t get it to work and was wondering if you had a more direct / elegant solution.
I am trying to avoid re-writing all of the existing OpenCV samples simply so that they work with my RaspiPiCam instead of an actual USB cam. Thanks!!
Hi Stephen, I definitely appreciate not wanting to rewrite all of the OpenCV examples as that can be quite time consuming and tedious. If you want to work directly with the Raspberry Pi camera module, you can try installing the uv4l drivers. However, they can also be a pain to install. And more importantly, those drivers are not kernel level drivers — they will run as users threads. This means that they will be a bit slow.
In general, I think you have two options:
1. Update the OpenCV examples to use the Raspberry Pi code that I have detailed above.
2. Purchase a USB camera, like the Logitech C210. All you need to do is plug the camera into the Pi and it should be automatically recognized. And from there you won’t have to change any code in the OpenCV examples.
I know that’s probably not the answer you were hoping for, but there isn’t exactly a clean and elegant solution to this particular problem.
I thought that this might be the case. Thank you very much for the quick response and your thoughts!
No problem, glad to help! Let me know which route you end up going.
Step 5, running headless with putty, fails for me with the message “Gtk-WARNING **: cannot open display”, at line 19. I got it to work using the RasPi desktop over VNC, but only after I had rerun my profile in that environment. Enabling X11 forwarding in the putty configuration did not work in my case.
Hi Joe — I’m sorry to hear that X11 forwarding did not work, that’s very strange. I don’t have a Windows system, and thus no access to Putty, so I can’t give it a shot to replicate the error. But whenever I ssh into my Pi with X11 forwarding from my command line, I can tell you that my command looks like this:
$ ssh pi@my_ip_addressI hope some fellow Windows users on the blog can help out!
Hi Adrian Rosebrock
I am working in a project which i have to scan qr code and barcode by using python.Does it existe a tutoriel for doing that .
thx
Best regards
Hey, thanks for the comment. I don’t have any tutorials related to scanning the actual barcode, but I do have a tutorial on detecting barcodes in images which you may find useful.
Hi Adrian,
I’m having a problem with step 5. I get an error when running ‘python test_image.py’:
—
OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script) in cvShowImage, file /home/pi/opencv-2.4.10/modules/highgui/src/window.cpp, line 501
Traceback (most recent call last):
File “test_image.py”, line 19, in
cv2.imshow(“Image”, image)
cv2.error: /home/pi/opencv-2.4.10/modules/highgui/src/window.cpp:501: error: (-2) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function cvShowImage
—
All the installation steps worked, so I don’t think this is a problem with that. I think this is an issue with X11 forwarding. I use this command to log in to the pi: ‘ssh -X pi@piaddr’ I’m using a mac and RPi B+. I tried running ‘startx &’ and ‘/etc/X11/Xsession’ but neither worked. How do you get images to display over your ssh connection? Is there any other setup required?
Thanks,
Rafi
Hey Rafi, it looks like you didn’t perform Step 3 of the Raspberry Pi + OpenCV install tutorial. Step 3 involves installing
libgtk2.0-dev. Go back to Step 3, installlibgtk2.0-dev, and then re-compile and install OpenCV and this should take care of the problem.I have the same problem.
And I installed libgtk2.0-dev before build and make. So this doesn’t fix the problem.
Try installing libgtk-3-dev as well. Make sure you delete your “build” directory, re-create it, and re-run CMake + make. This has fixed the issue anytime myself and most other readers have encountered your error. In fact, the error posted is the hallmark of OpenCV not being able to build GUI support. Unfortunately without physical access to your Pi I’m not sure what the particular error is, but I would double-check the GUI libraries on your machine.
hello guys.. may i ask something .. is it possible when the camera recognize an object then it convert it into speech. bcause we have a project design that is related with this post. Our project is for the blind impaired person, entitled ” Audio navigator for blind impaired”. the concept of this, it is a device wearable by a blind then we planned to use PI cam to detects motions and object that can gives awareness to a blind person of whats happening in his environment. for ex. the camera detects traffic light, the camera will automatically recognize it that it is a traffic light, then color sign green,orange and red that can tells to a blind when he/she can cross in certain road to avoid accident. in this case, we hope that it would help to them to make life easier by using this invention. pls anyone who can give ideas .. honestly my knowledge of making this is very limited so thats why i decided to approach this site. i am a student only.. pls forgive my grammar… thank you.. Godbless you all
Converting an image to text (and then to speech) is a pretty challenging project and is still under active research. Both Stanford and Google are currently researching methods for automatic image captioning which captions images via text strings. From there, those text strings can be passed on to speech algorithms. But it’s still an incredibly challenging problem and very much in its infancy.
Very interesting; thanks for this RPi-OpenCV tutorial! I have started doing something similar as you can see here: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=114550
although I am using OpenCV 3.0.0 instead of 2.4 as I gather you are using. As long as you’re compiling the library yourself, it is just as easy to install 3.0 and it was actually faster to compile than the Pi2 timings you listed for your install.
Doing foreground extraction and blob detection I see only 8 fps at 320×240 resolution on a RPi2, but I have not tried to optimize my algorithms as yet, and that includes reading the h264 input file instead of taking a live feed.
Very nice, thanks for sharing! I’ll actually have OpenCV 3.0 install instructions for the Raspberry Pi 2 online within the next 2 weeks (pretty excited to get them pushed online). Your project sounds great so far, congrats! And 8 FPS for background subtraction using the MOG methods sounds about right.
Hello Adrian,
First off, I want to thank you for these awesome tutorials.
My problem is that when I write test_image.py, and run it I get an error “picamera.exc.PiCameraMMALError: Camera component couldn’t be enabled: Out of resources (other than memory)”. I was wondering if that is an error due to the RPi or with my camera, or with something else?
Hey Neilesh — I have honestly never encountered that error before. That definitely sounds like an error related to your Raspberry Pi or the camera, not OpenCV.
Experienced the same error when I mistakenly used python3 to create the file (after setting up python2 environment)
Hi Movaid — I would suggest checking the common picamera errors page. My guess is that either (1) your Raspberry Pi camera module is not installed properly, (2) you did not enable it via
rapsi-config, or (3) you did not runrpi-updateto download the latest firmware.same problem happen.help me.
I had similar problem. I found the error is caused since my pi camera was not released by previously executed program. For example, if you run this program and close the image window using mouse and clicking close on menu instead of pressing key, I received this error.
Hi Adrian
Do you have a sample code to access multiple camera in a single raspberry pi, I mean attach two to three camera to the same Raspberry pi and read frames from each of them process it and save it ?
Hey Girish — I don’t have any complete examples, but all you need is the
cv2.VideoCapture(0)function where the 0 is the first camera, 1 would be the second camera, 2 would be the third, etc. Just maintain a list of capture objects and you’ll be able to access each of the cameras.Hello Adrian,
Step 5, i get
“Traceback (most recent call last):
File “”, line 1, in
import cv2
ImportError: No module named cv2″
When running import cv2.
Any ideas as to what might be happening and how to overcome this? I have completed your “installing openCV and Python on raspberry pi” tutorial. Thanks for your help.
Josh
It sounds like you are not in the
cvvirtual environment. Use theworkoncommand to access it before executing any code that accesses OpenCV:$ workon cv$ python
>>> import cv2
...
Hey Adrian !
thaks for this tutorial !
I have a RP 2 with the camera module installed correctly I think and performed all the steps from opencv installation to this tuto and everything seems to work fine but I can’t acheive a fps above 15 in 640×480 ( a simple calculation displayed every 2 sec). nothing else is working in the same time. I access the Pi through VNC, using direct access ( hmdi cable, keyboard ..) don’t seems to provide the desired performance ( fps << 32).
I have tried to remove the imshow thinking that displaying images with opencv would lead to performance drop but I've not noticed a significant gain in performance…
I don't think this is about the exposure, but I don't have tested it in a very luminous condition.
should I deactivate automatic exposure ? ( is it even possible ?)
I know you have answered several questions like this but I can't understand why I can't acheived the 32 fps as you do 🙁
do you have any idea ?
thanks again for your time and tutorials which are excellent !
The first way you can increase FPS is to simply reduce your image size. One way you might be able to boost performance is to take a look at the V4L2 drivers for the Pi. I personally haven’t had much luck with them, but I know others that have. The V4L2 drivers can (theoretically) improve your frame rate and let you use the
cv2.VideoCapturefunction rather than thepicameramodule which should improve the FPS a little bit.Hi again,
if opencv can only work with images at around 15 fps ( I don’t know why but let say)
Do you think it is possible to have a high frame rate display , an other thread processing the frame that give back results as an overlay ?
thanks in advance !! 🙂
Absolutely! It’s very common to dedicate one thread to grabbing frames from the camera device and then handing them off to the thread that is doing the actual processing of the images. This ensures that the main thread is not delayed by the polling of the camera.
Hi,
Compliments for your article.
Do you think is it possible to build up an head counter with raspberry pi?
I have a shop and would like to count people coming in and going out from the shop door placing a camera over the door.
Tks
Hey Pippo, thanks for the comment. And yes, it’s absolutely possible to build a head counter to count people with the Raspberry Pi. I think you might like this particular post on motion detection and tracking to get you started.
So far so good. I have tried other ways to get opencv on my pi using the pi cam. And have waisted some serious time. Dude you totally hooked it up. Thank you so much. Have you thought of doing a automatic pan and tilt face follow with an arduino, and a couple of servos?
Hey Tyrone, thanks for the awesome suggestion, I’ll definitely look into it!
hai adrian, thank you for your tutorial
why if me use syntax cv2.absdiff with the picamera array inside in syntax, i always have error, the error say
“size of input do not match (the operation is neither ‘array op array’ (where arrays have same size and the same numberof channels), nor ‘array op scalar’, nor ‘scalar op array’) in arithm_op….”
i have same resolution with the parameter in syntax cv2.absdiff, but why i have problem like that ? can you help me ?
When you take the difference between two images they need to be the same size (in terms of width and height) and in terms of channels. Either the two images you are trying to compare do not have the same width and height and/or one is grayscale and the other is RGB. Make sure all the dimensions match before using the
cv2.absdifffunction.This is failing to run with an import error “No module named cv2”
Any ideas?
This is a on a fresh install of Rasperian, Open CV / Python 2.74 (using your instructions), and I am in the virtual environment CV when running the code.
thanks
FIXED: Really strange… when I left the machine last night, I had completed all steps of the install CV and Python including the last step which tested it. I noticed earlier today that the .profile change made last night was no longer there (which I fixed) by repeating step 7. I then repeated step 10 to fix this problem of cv2 not being defined. If there is anything I am missing about the virtual environment, and things I need to add elsewhere in Unix, please let me know. Thanks
You’re the second person in the past 72 hours who has mentioned that their changes to the
.profilefile disappeared (in the other persons case, it was after a reboot. That is really strange behavior, I’m honestly not sure about that one. If you don’t mind, could you post on the official Raspberry Pi forums and see if they have any suggestions? I would love to know why the updates are being overwritten.Hi, how can i get time for each frame using the cv.GetCaptureProperty() in the code mentioned above for capturing the video stream.
Since we’re using the
picameramodule, you won’t be able to access any other properties associated with the camera like you would in OpenCV. You can dump an image to file directly with the timestamp included; otherwise, just use thedatetimemodule to grab the current time as the frame is read.Hi Adrian
Thank you very much! for This blog is very good.
Thanks Rafael! 😀
Hi adrian,
I am ganesh from india, I am going to making an secuirity system based on raspberry pi .In that interfacing camera and storing data in memory card, please suggest me how i start this project? which procedure i follow?.
Hey Ganesh, you might want to try my tutorial on building a home surveillance and motion detection with the Raspberry Pi, Python, OpenCV, and Dropbox.
Hi, Adrian. Thanks for your guidance over Raspberry PI. But I have a question here. After the Raspberry Pi camera module has been installed, how could I actually open up the terminal as shown in Step 2? Is that necessary to use Linux platform? Thanks for your prompt reply, much appreciated.
There are many ways to open up a terminal using the Pi, but I would suggest going through the official Raspberry Pi documentation for more info on launching a terminal.
Hi Adrian, Before I purchased the course i wanted to know if it included a way that i can always have the camera live without recording, when a hang gesture is detect take a picture? is this possible? Thank you so much I’m starting to become a big fan of the website.
The course itself does not include a method to perform hand gesture recognition, but that is something that I hope to cover in the future. In the meantime, it is covered inside the PyImageSearch Gurus course.
I just claimed my spot. So I’m guessing i would have to wait for the next course to start in order for me to learn how to keep the camera feed live all the time ?
You certainly don’t have to join in the course if you don’t want to, I was simply saying that I haven’t had a chance to cover hand guest recognition on the PyImageSearch blog yet, but it will be covered inside the PyImageSearch Gurus course.
Leaving the your webcam feed running all the time is pretty easy. Just SSH into your Pi. Launch
screen. Execute your script. Close your screen session. Then log out of your Pi. Your script will run without you having to be attached to the Pi.Hi Adrian, I have an error when I try to run the code of test_video.py which is
TypeError: ‘float” object is not iterable
Can you help me and thank for this post
What line of code is throwing that error?
Hey, Adrian! Thank you for this post, it’s wonderful.
I got an error while trying to execute test_image.py.
File "test_image.py", line 18, in cv2.imshow("Image", image) NameError: name 'image' is not definedCan you help me with this problem?
Thanks
If you’re getting an error related to the image not being defined, then I would go back to the test script and examine the output of the
image = rawCapture.arrayis a valid. It could be that the Raspberry Pi camera itself is not configured properly.Hi
I am new with the raspberry pi and i installed opencv and python on my raspberry pi.. Now I want to take an image by my pi camera using opencv. Already I followed your steps carefully but finally when i write $python test_image.py .. I saw that the light of the camera is working, it wants to take a picture but it dosn’t take!! and this is the warning error that is displayed for me :
(Image:2312):Gtk-WARNING**:cannot open display:
How can I solve this error ??? plz help me as soon as possible 🙁
Please see my response to Kronos and Joe Landau above.
Dear Adrian,
First of all thank you very much for your great support on either how to install opencv into RP and how to use picamera. On the past I tried some other methods on other web sites for installing opencv, but I was not succesfull, but with your support finally iI did it.
Anyway, Now I am able to run python opencv exmples with a usb camera, but after installing pi camera using pip install picamera “array”
I am having issue. When I triy your example , the python compiler says there is no array method in picamera. What is wrong, could you please help, thank you
I can see two things that might have went wrong here. First, make sure you are in your virtual environment when installing the
picamera[array]:Lastly, make sure you have quotes around “picamera[array]” when you install it:
$ pip install "picamera[array]"A+ on a Pi 2. thanks Adrian
Nice, I’m glad it worked for you Michael! 🙂
Hi Adrian, this is some great work to get started with the picamera. However, I would like to clarify on the use of this particular line (#22) on video capture
> key = cv2.waitKey(1) & 0xFF
What’s the importance of this part?
If you use the
cv2.imshowwithout usingcv2.waitKeythen your window will show up and then disappear immediately. Thecv2.waitKeyfunction allows the window to be stayed open and optionally grabs the key that is pressed.Hi Adrian,
This is a detailed, well written and nicely explained tutorial. Thank you very much for sharing it.
The only suggestion I would have is to add something on X11 forwarding. It may even be a link to your favorite blog that goes over how to do it. I had to take a little detour to get this working to see the images and the stream. I used Xming and Putty following the instructions from here: http://laptops.eng.uci.edu/instructional-computing/incoming-students/using-linux/how-to-configure-xming-putty
I am looking forward to all the possibilities and interesting projects using CV.
-Sidd
Hey Sidd — Thanks for the note on X11 forwarding. Once you have X11 installed (whether or OSX or Linux), it can be done using a simple command:
$ ssh -X pi@pi_ip_addressOn OSX, you’ll need to download and install Quartz first.
The only issue with X11 forwarding is that it can be a bit slow for streaming the results back from a webcam/video device.
I have a raspberry pi and HDMI monitor but do not have the keyboard. Is it possible for me to log in to Pi using SSH and forward display to HDMI monitor?
Currently, I am able to forward X11 on my Windows laptop using XMing and Putty but video streaming is very slow and wants to use raspberry pi HDMI output on a monitor.
Streaming frames over a network will be slower than natively displaying them to an attached monitor. There are ways to speed up the process (i.e., gstreamer) but overall, if you want minimal lag you should be viewing the frames on a monitor attached to the Pi.
Just the one a newbie needs. Perfect. Keep it up.
I enjoyed following the steps and for a change something from the net works as it is described.
I’m glad the tutorial helped! 🙂
Hi everybody,
i installed Opencv 3.0, Python 3.2.3 on my Raspi2 (followed Adrians nice tutorial…).
When I start test_image.py I get an error as soon I move around the mouse over the Image Window:
GLib-GObject-WARNING**: Attempt to add property GtkSettings::gtk-label-select-on-focus after class was initialised.
I don´t ssh to my raspi…I use the HDMI Output.
When I google the error it looks like I´m not the only one – but I´m definetly one of the Noobs who don´t know how to solve it 😉
Can anybody help me with this?
@Adrian..really nice work you are doing here. Nice tutorials and blog post. Keep it up!
Stephan the Kraut
Hey Stephan — I’ve ran into that GTK warning myself. I’ve installed OpenCV on hundreds of systems, but it only seems to happen on the Raspberry Pi. I’m honestly not what causes it, but it’s clearly from the GTK library. It doesn’t affect OpenCV at all, other than the warning message is displayed to the terminal which can be a bit annoying.
Had the same exact problem. Did you find a solution? I want to believe it could be a bad connection, but upon trying the test_image.py the ole pi cam is working for sure just not the video stream.
Hi
I would like to know if the face recognition method can be able to identify a person through comparing it with another photo taken before??
Thanks
Absolutely. In fact, that’s how most face recognition algorithms such as Eigenfaces and LBPs for face recognition are trained. Both are covered inside the PyImageSearch Gurus course.
Hello Adrian,
I’m developing a UAV (rover) , using also your code for item recognition.
It works fine. THanks for your great job.
I’d like now to live streaming the results ( captured images + the circle ).
How can I pass those result to my web server?
(I’m using tornado)
BR
Oscar
Hey Oscar — there are a number of ways to pass the results to your web server. Inside this post I show how to develop a computer vision web API that images can be uploaded to. I use Django for this project (not Tornado), but the principles are still the same.
Thanks for the lesson sir, now I can access my camera on raspberry pi. hehe….
Thank you so much for this! I am using for my design project and we fried our pi and had to start over. It worked perfectly the first time so Now we must do it again. But now we are getting a gtk warning and I have read about to install gtk, re compile and install open cv. thank YOU!!!!
Are you getting an error or a warning related to GTK? If you’re getting an error regarding unable to open the display, then I assume you’re SSH’ing into your Pi. You’ll need to enable X11 forwarding:
$ ssh -X pi@ip_addressMany thanks for such a great article. I started to work with the PI v1 some time back but found that it was too slow for my processing needs. However seeing that the camera has been better integrated into Python makes it a lot easier.
One thing that has always been a issue re OpenCV and SimpleCV is Blobs. Hopefully the newer OpenCV has made it easier.
Many thanks
Marc
Hey Marc, are you referring to the blob detector that comes with OpenCV? If so, I can try to write a blog post on it in the future to help clarify things.
Hi Adrian, Thanks for the article. Its so well explained. Followed the steps and was able to get the image and video working with Opencv.. Keep it up. 🙂
Thanks Ragu! 🙂
Hi, one week ago there was no eror, now its happening like this
pi@raspberrypi ~ $ raspistill -o output.jpg
mmal: No data received from sensor. Check all connections, including the Sunny one on the camera board
I’m not sure about that one. I would (1) make sure that the camera is enabled via
raspi-config(just in case it somehow got turned off) and (2) double and triple check the connections on your board. If you’re still getting an error, you might need to post on the official Raspberry Pi forums.Thanks for the tutorial Sir..I kindly wish to know if it will be possible to get the camera running automatically when my raspberry starts…any script file which could execute the profile, work on and subsequently the camera working. I will really be grateful if you could help me out.
2) How could i get dropbox to send me a notification anytime a picture is added into it
3) what do i have to change if im to upload the images on some server, later to be retrieved via an app?
To get a Python script automatically running when your Raspberry Pi starts, I would suggest using crontab. You can specify to run a shell script on reboot. Inside this shell script you should put the
source,workon, andpythoncommands to run your script.As for Dropbox sending you a notification, I’m not sure what you mean by that. Total disclaimer: I am not a Dropbox developer and this was the first time I used their API. I would suggest posting on the Dropbox Forum.
Finally, you should look into using the pysftp package for uploading to a server.
Thank you for the tutorial, very clear.
But I’m facing an issue while running the command: python test_image.py (Step 5):
(Image:1448): Gtk-WARNING **: cannot open display:
Any help, please?
Please see my reply to Joe Landau above.
Can you explain me how i create a QR-Code reader with this?
I don’t cover how to read a QR code on this blog, but I demonstrate how to detect barcodes in video streams. I’ve never personally tried it, but I’ve heard that zbar is a good library for reading barcodes.
Hi Adrian,
GOOD WORK MAN!! Worked with me perfect.
Now I want to change the settings of the camera programmatically. Settings like brightness, saturation, contrast, exposure, etc. Can you help me with this?
Thanks
I would suggest taking a look at the picamera documentation. The page linked to demonstrates how to adjust brightness. Similar examples can be found throughout the docs.
Hello Adrian,
First I want to thank you for all your great tutorials, they have been a great resource. Next, I have installed OpenCV and Python on my raspberry pi 2 successfully. I then followed this tutorial and was able to run all the way through both step 5 and step 6 successfully the first time around. Because they worked as planned I set my work aside for about a week. When I returned and tried to run the same scripts in the virtual environment without making any changes I get the same GTK warning as those previous comments. The window opened the first time and now the window will not open. I am not running my pi through ssh but rather the HDMI port. My question is then, are there any updates or additional libraries needed to fix this warning? Your help is greatly appreciated.
All the best
Which GTK warning are you getting? An error related to the display being unable to open? Or the one related to the gtk-label-select? If it’s the latter, you can ignore this warning. I’m not sure why it happens, but it seems to be Raspberry Pi specific and it will not impact your usage of OpenCV. If it’s the latter, then make sure you have launched the Raspberry Pi desktop and are not trying to execute it via the command line at the Pi boots into.
As for the image opening a first time, but not a second, that is very, very strange behavior and not something I have encountered before.
Hello!
First of all Adrian Rosebrock thank you very much for a great tutorial.
I have tried the live display code above and it works fine, but if I increase the resolution to like 1296×972 the fps drop allot. There is a big difference if I compare it with raspivid live display.
For now I’m not doing any image processing.
Is there any solution to increase the fps to like 25 (resolution > 640×480) so that the stream will be smooth? I need these for kind of live magnification. And maybe I will have to draw a line on every image so that’s why I would like to use OpenCV and Python instead of raspivid.
Realistically, if you want to obtain ~25 FPS, your images will need to be smaller than 640 x 480. The larger your resolution becomes, the more data there is, and hence the processing rate will drop. I personally haven’t tried this, but you might want to install the V4L2 drivers so you can access the Raspberry Pi camera module via the
cv2.VideoCapturefunction and see if FPS rates improve.Thank you very much for your answer.
Yesterday I found that I can make an overlay if I use picamera preview. So I can easily draw a line on a live stream and it works great. That’s all I need for now.
I will take a look at v4l2 driver if I ever need to make some processing on a stream.
Interesting. How do you draw on the picamera preview? I haven’t seen that done before.
Hey Adrian,
I have a Raspberry Pi Model B+ and I was successful setting up the picamera in order to get a image up to step #5, but when it’s time to get a videostream in step#6, the python code will go through and I can clearly see that the LED n the camera is on but I am not seeing the window come up with a videostream of myself. What am I doing wrong? Maybe my FPS shouldn’t be the same as the one in your tutorial but I already tried to lessen it and it does the same thing.
How are you accessing your Pi? Are you ssh’ing into your Pi or using VNC?
i need to detect circles in video with c++ using raspicam and hough transform
Hey Maria, I actually cover circle detection in this blog post, but I only have Python code, not C++. I hope that helps point you in the right direction at least.
Hi Adrian
first of all i want to thank you for this very useful tutorial.
I want to ask about step 4, which im trying to install pi camera.
how long should i wait for the installation? because when the installation reach “Running setup.py bdist_wheel for numpy . . . -” it stops very long there and nothing in progress any further… (sry for my bad english) thank you!
For a Raspberry Pi 2, the installation can take 15-20 minutes. For a model B, it can take anywhere from 45-60 minutes In either case, you’ll likely want to go make a cup of coffee or go for a long walk while NumPy installs 🙂
Dear Adrian,
first off all, thanks for the great turtorial.
Is it possible to show the frame in fullscreen without any border at the top?
Or to show the frame in a specified coordination. It always shown up in the left bottom corner.
Thank you.
Nico
I don’t think it’s possible to show the frame “fullscreen” with OpenCV. The GUI functions included with OpenCV are meant to be barebones and used for debugging and building simple GUI-based projects. For more advanced GUI operations, I suggest using either Tkinter or Qt.
As for placing the frame in a specified coordinate, yes, you can actually accomplish that using the
cv2.moveWindowfunction:cv2.moveWindow("WindowName", x, y)Hi,i need to incresase fps to 60 i’m using camera board with raspicam
my code in c++
i tried using raspiCamCvSetCaptureProperty(capture,CV_CAP_PROP8_FPS,60)
but no effect 🙁
I don’t have any C++ code on this blog, but I would encourage you to read this blog post on increasing the FPS processing rate of your video pipeline.
Hello Adrian,
Should you recommend a way to save the video stream to the working directory in order to play it later?
Thanks in advance!
Sure, there are two blog posts I recommend for writing video to file:
Hi Adrian,
I did your previous tutorial to install OpenCV and did everything here up to step 5 to do try to display the image with the test_image.py script.
I’m in cv environment and when I type:
python test_image.py
I get this error message:
(Image:31875): Gtk-WARNING **: cannot open display:
And I obviously don’t see the image. Any idea about my problem?
Thanks. JP
It sounds like you’re SSH’ing into your Pi. Please see my reply to Kronos above — you need to enable X11 forwarding in your SSH command:
$ ssh -X pi@your_ip_addressHi,Thankyou for openning a new door for me to know pi,i am a student in China,i had a question that when i were tring do”sudo apt-get install libgtk2.0-dev”, it can’t work. And i am sure using the new “source.list”,how can i use the pi B+ to install the environment of Opencv?
Please figure it out! Very Thanks!
PS:Is it because these “software” out of date ?
PS2:I can’t buy your bool in China! PITY!
Hey Neal — I’m sorry to hear about the issue with the book in China. Send me an email, perhaps we can figure out a workaround. As for the libgtk-2.0-dev issue, what is the error message you are getting?
Hey Adrian,
I’m measuring the read rates for capture_continuous and it looks like every three or four frames, it takes significantly longer to generate a frame.
The read times look like this.
Frame 1: .02 (Seconds)
Frame 2: .03 (Seconds)
Frame 3: .02 (Seconds)
Frame 4: .11 (Seconds)
It’s happening in a somewhat regular pattern (every third/fourth frame takes 3/4 times as long ).
Any idea what’s going on or is that the expected behavior of capture_continuous.
Kav
Very interesting, I can’t say I’ve ever encountered that before (or measured it). You might want to try posting on the picamera GitHub to see if they know anything about it. I would also encourage you to try using threading to facilitate faster frame reads as well.
Hi Adrian .Thanks for the tutorial.I will ask how can I make a real time face recognition on raspberry pi over this tutorial or do you have another tutorial for this ?I will be very happy if you help me ,thanks.
I don’t have any real-time face recognition tutorials publicly available; however, I do cover it in detail inside PyImageSearch Gurus.
Was wondering if it is possible to run 2x Pi Cameras from the same Raspberry Pi?
I know there is only a single camera connector, but could a second be added via the GPIO pins or could the camera be chained together?
If a second Pi is required, what is the maximum length of the cable? Does it have to be a ribbon cable or are round cables available?
Thanks for your time.
I’ve seen various hacks online that have chained together up to 4 Pi Cameras, but in general, I don’t recommend this. Instead, I would just connect multiple USB cameras. It’s much easier this way 🙂
Hi Adrian
I am trying to install to raspberry pi camera module since last year
What I did is;
– source ~/.profile
– workon cv
– pip install “picamera[array]”
Finally I wrote the test_image.py code to test, but there is no any luck ever to run the code
It gives me the error
ImportError: No module named array
I guess I am the only one who is able to use raspberry pi camera
Any help will pe approximated
Thanks
Murat Gozu
It definitely sounds like
picameradid not install correctly. Try manually typing in thepip install "picamera[array]"command to ensure there are no formatting issues during the copy and paste.Hi Adrian,
Everything has been great and is work up to step 5. I have copied the code for capturing an image and saved it etc.
Running ‘python testimage.py’ from the terminal executes the code correctly.
However, when I run it from the python shell in IDLE3, i get the error:
Traceback (most recent call last):
File “/home/pi/.testimage.py”, line 6, in
import cv2
ImportError: No module named ‘cv2’
I have run the source ~/.profile and workon cv commands in the terminal before opening IDLE3.
I guess i’m lacking a little understanding about environments.
Is this behaving correctly?
If so, what is the advantage to running the program from the terminal, as opposed to from the IDLE3 shell? Because to me, it just feels more natural to press F5 from the actual program.
Thanks!
Michael
I presume you’re using the GUI version of IDLE? If so, that’s the problem. Unfortunately, IDLE does not respect Python virtual environments like the command line does. I would suggest either using the command line version or IDLE, or, better yet, use something like IPython Notebooks.
Hi Adrian,
When I run my code I’m getting error as
from picamera.array import PiRGBArray
ImportError : No module named picamera.array
What would be the reason?
Thanks.
Angel Jenifer
Make sure you have installed the
picamera[array]library:Hi Adrian,
I have the same issue above, though definitely installed “picamera[array]”, and now says already satisfied, but still same error when running in virt_env.
“no module named picamera.array”?
It looks to me that the issue is that picamera[array] is automatically installing to python3 when I have opencv3 installed for python2.
I either get no picamer, or no cv2 module? I can’t get the 2 to work together on python2. Even running:
$ sudo pip2 install “picamera[array]” says already satisfied, though I know it is not?
It sounds like you are not installing the
picamera[array]library into your Python 2.7 virtual environment where OpenCV 3 is installed. Can you runpip freezefrom your virtual environment and ensure that picamera is listed?Yeah so i only had to run it with workon cv. Now it works.
Congrats on resolving the issue Stefan!
Hey Adrian!
First of all, thank you for all your tutorials, I have managed to resolve almost all my issues using your guides. However, I am running into an error when trying to install picamera[array].
Within the virtual environment, when I try to install with the above command, I run into a “BadZipFile” error. When I try installing then with “sudo pip”, it says it is successfully installed but when I try to import the library it says that there is no module named picamera.array.
I have uninstalled and installed several times and after installation even if I only do it within the virtual environment, I can only import the picamera module outside of the virtualenv. Do you (or does anyone) know how to resolve this issue?
Thank you
It sounds like when you tried to install the picamera module inside your virtual environment there was a problem with your internet connection. Uninstall picamera from your virtual environment and then use this command to install:
$ pip install "picamera[array]" --no-cache-dirThis should ensure that a new .zip file of the module is downloaded.
Thanks a lot for nice guidance and suggestions.
My question is; in recording video, is it possible to change the “frame rate” and the “frame size”? I mean, for example, first I choose the frame rate =32 and size=(640,480), then after some time I change the frame rate =45 and size (320,240), but all in one program. is it possible?
If it is possible, then can I store the output video in one file? I mean, some portion of output file has different frame rate and size with other portion. is it possible?
If I understand your question correctly, you want to store one frame size at a given FPS for a period of time and then later store a different frame size at a different FPS? If so, no, that’s not possible. You can adjust the frame size simply by making your frame larger/smaller to fit the original output dimensions — but you cannot adjust the FPS. You would need to create two separate output files.
thank you.
you mean for changing FPS, I have to make another output file. But I can use only one code. In that code, if I change the FPS, then i store the images in another output file.
When I change the FPS or frame size, No need to reboot the raspberry, or no need to use another code, am I right?
Correct, you can use the same code. But if you want to change FPS or frame dimensions, you should create a separate output file.
Hello Adrian, Thank you for the lesson, very helpful.
I can successfully run the test_image.py script but when I run the test_video.py script I get no error but the image is black. I am using the Raspberry Pi 3 board and the 8 MP Ver2.1 camera.
I’m not sure what you mean by “image is black” — but the code should still work with the newer v2 camera module.
The video image output to the monitor (via HDMI) is solid black. However, if I test the camera using a raspvid command, like: “raspivid -t 5000 -o” the video image is as expected.
I also tested with the V1 pi camera with same results. I tried a better power supply (2 Amp) with same results. I reduced fps and image size with same results.
That is very strange. Can you confirm which version of picamera you are using? I’ve heard there are some issues with the latest 1.11 release.
I found the fix to my Black Video issue and I hope you will add this simple step to your lesson. After setting up RP3/OpenCV using Install guide: Raspberry Pi 3 + Raspbian Jessie + OpenCV 3, simply reboot the RP before trying to execute any CV related code. Following reboot, the black video image issue goes away. I repeated the complete install twice with the same results, on the third try I rebooted before executing opencv related code and all is well.
Hey Jim – Congrats on resolving the issue, but to be honest, I’ve never encountered it before. It’s a good tip for anyone else who encounters this issue, but to be honest, this sounds like something very specific to your setup.
Hi Adrian, thanks for all the lessons.
I also have the same Issue as the other Jim. When I run the test_video.py script I get a small window appear called Frame but the window is just black, no video appearing. Im running on a Pi 3 using the first gen camera. The LED on the front of the camera turns red as normal. any ideas? no error messages are output to the terminal window and the script exits when the Q key is pressed.
Im using a Raspian installation as detailed in your lesson for installing PI 3 with open CV, and im in the CV virtual env.
thanks
Jim
Hi Jim — please see my comment to “red” below to see the resolution to this problem. Also, I assume you’re using Python 2.7? Or are you using Python 3?
perfect! that fixed it, many thanks Adrian
Now for the cool stuff
Nice, glad to hear it 🙂
Hey Adrian,
I’m running a Raspberry Pi 3 B with the new 8 MP PiCamera. I have installed and am successfully running the (cv) environment. I got the document scanner to work with your example images and with images I have previously saved, but would now like to integrate my PiCamera into the scanner and skin detection programs (and others).
I’ve downloaded the source files, but am running into trouble at “Step 5: Accessing a single image of your Raspberry Pi using Python and OpenCV.” When I run test_example.py, it gets hung up on line 18 camera.caputre(raw.Capture, format = “bgr”). It gives me the following error- “TyperError: startswith first arg must be bytes or a tuple of bytes, not str”. Please let me know if you’ve got any insight into what could be going wrong or if you need more information.
Thanks in advance, I really appreciate all of these tutorials.
Man, I’ve been getting a lot of emails regarding this. Can you please confirm what version of picamera you’re running? I have a bad feeling that it might be the latest 1.11 version which if you look at the GitHub issues, is having a ton of problems. Luckily, I think the fix is simply to downgrade to a previous version. Please let me know which
picameraversion you’re running so I can confirm this (I’m traveling right now and don’t have physical access to my Pi).Hi Adrian,
Thanks so much for the response. I ran pip list while in the (cv) virtual environment, and it appears you were right. My picamera is version (1.11). What is the easiest way for me to downgrade to 1.10?
Please see my reply to “red” above.
Adrian, thank you for sharing your knowledge. I also had the aforementioned issue and wanted to let you know that it worked fine once I downgraded picamera to 1.10 as suggested.
Thank you!
Thanks for letting me know Felipe, I appreciate it!
It might help to uninstall picamera “sudo apt-get remove ” (python-picamera or python3-picamera [or both]) and then do a pip install… http://picamera.readthedocs.io/en/release-1.11/install.html#alternate-distro-installation
Away from my pi unfortunately but the “error” I get is something like gttk: lense focus was initialized and my frame is pitch black no picture for python test_video.py and I am in the source~/.profile
workon cv
python
I have a feeling that you’re using picamera v1.11 and Python 2.7. Try downgrading to picamera v1.10 and this should resolve the blank/black frame issue:
There are some issues with the most recent version of picamera that are causing a bunch of problems for Python 2.7 and Python 3 users.
Same problem here. I found that if I set the format to ‘rgb’, the sample will display the stream, but in the wrong colors. However, I did modify the sample so that a camera.capture occurs on the exit, and this file is saved as RGB with no issues on displaying the correct colors when I do an imshow on the resulting file..
Hey Paul — thanks for sharing. I’ll write an updated blog post that details some of the common errors in this comment thread.
Downgrading picamera worked for me!
Congrats on resolving the issue, Ben!
I was using picamera v1.13 and had the same black screen problem. This fixed it thanks!
Congrats on resolving the issue, Jake!
Hello Adrian,
Thanks for all the tutorials. It’s been really helpful. This tutorial worked great through vnc, and everything run fine. However, when I try to run test_image.py through terminal I get the error, “(Image:2063): Gtk-WARNING **: cannot open display:”. I tried to ssh in using -x like you recommended to a previous commenter, but it didn’t fix the error. Do you know any way to resolve this issue?
Thanks 🙂
The “X” should be capitalized, like this:
$ ssh -X pi@your_ip_addressFrom there, try running the
test_image.pyscript.It still pulls up a Gtk warning. I tried going into ssh_config and changing ssh Forward X11 to yes, but it still doesn’t work. Then I tried using xhost, but it always pulls up an error.
Hey Marina — I’m sorry to hear about the continued issues with X11 forwarding. But in this case, I’m honestly not sure what the particular problem may be.
Ok, well thanks anyway 🙂
Hi,
I think I know what you were doing wrong, because I have been encountering the same thing here and ran out of solutions until I bumped into a video on youtube. Basically you need to install XQuartz on Mac and use its terminal to try to run what Adrian was advising us to run. At first I was running doing the same exact stuff on the terminal I usually use but when I tried the XQuartz terminal it worked.
Hope that helps
Thanks for sharing, Abanoub!
I also had the same error, and solved by installing Xquartz!! After I install Xquartz, even when I run the code in my default terminal, a new window of image or video pops up with Xquartz. Thank you very much guys.
Thanks for sharing, Sang!
Hey Adrian
I would like too first say a huge thanks for the tutorials you offer here and take the same opportunity to drop a quick question.
I was able to run the image test, but the video test is giving me a black screen. I am not sure if you have any idea of what is happening.
But if I use raspivid -o video.h264 it 50000, it streams and record the video with no issue.
It sounds like you are using
picamera=1.11which is known to have some issues. I would suggest upgrading yourpicamerainstallation:$ pip install --upgrade "picamera[array]"Hi Adrian, great tutorial!
I installed OpenCV 3.1 with python 3 on my Raspi3 from your tutorial: https://www.pyimagesearch.com/2016/04/18/install-guide-raspberry-pi-3-raspbian-jessie-opencv-3/
Everything works, but the video feed is slow and laggy! Even using a resolution of 320×320. Isn’t this strange on a raspi3?
It’s actually not surprising that the video feed feels laggy. The reason for this is because you’re performing frame polling in the same loop that displays it to your screen — your I/O latency is huge. You can actually increase the FPS of your processing pipeline by utilizing threading.
Hi Adrian,
I got these errors when i executed “test_image.py ” code
File "/home/pi/.virtualenvs/cv/local/lib/python3.4/site-packages/picamera/encoders.py", line 534, in __init__ if not resize and format != 'yuv' and input_port.name.startswith('vc.ril.video_splitter'): TypeError: startswith first arg must be bytes or a tuple of bytes, not strhow to solve this error ??
It sounds like you’re using
picamera=1.11. There are some know issues with picamera=1.11. I would suggest upgrading to the latest release which will resolve the error:$ pip install --upgrade picameraHello Adrian,
I apologize for not sticking to the topic entirely (I don’t have OpenCV installed, yet). I captured the images with picamera (1.10 for now) in different formats, but I don’t know (I’m a beginner and am overwhelmed by the volume of Python documentation) how to view YUV, RGB and Bayer data stored in, e. g., ‘image.data’ file. Can it be done with some Python module or do I need to install some application like OpenCV? I also would like to compress the captured image – would you recommend gzip and other Python modules or something else?
Thank you and I’m sorry if I’m bothering you.
Without knowing what exactly you’re trying to do with the different color spaces, I’m not sure what your end goal is. If you would like to capture a frame from a video stream, convert them to a bunch of different color spaces, and then write them to file, then yes, using OpenCV would be better for this. You can specify which color space you would like a frame returned as in the picamera module — but you’ll only get one color space. For all the others, you should use OpenCV to perform the color space conversion for you. After your script runs, it’s likely easy enough to create a tar or gzip file from the terminal.
For now I captured a still image with raspistill and with the use of picamera library. In the latter case I captured in YUV, RGB, and Bayer data, each in the files, e. g. image.data. (I followed Advanced recipes from picamera 1.10 manual.) Yesterday I installed Gimp and opened them but I’m not sure if that was OK (I pasted one on https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=153414&p=1007734#p1007734).
My goal (it’s for diploma) is to compress the captured image with different tools (don’t know what, yet, probably zlib, gzip, jpeg, pngmaybe even some isolated classical method), lossy and lossless – in Python and in Matlab.
I would compare the results (by file size and visually) so I guess I need some unencoded, uncompressed image to begin with – that’s why I wanted YUV and/or RGB and raw Bayer. If that is how it’s done in practice (I’m a beginner …).
If your goal is to compare various compression algorithms, then you’ll want to store the raw bitmaps of the images to start. Bitmap images perform no compression and are quite large since they store every RGB pixel at every RGB location. I would start with bitmap files and then explore various compression algorithms.
Hi Adrian,
Please Help me
I ran “test_video.py” but i get a black window that its name is “Frame”
if i delete “use_video_port=True” from the program the window shows the camera pictures but it is very slow.
what is wrong with my program?
thanks
It sounds like your Raspberry Pi camera drivers may be out of date. Try running:
$ sudo raspi-updateTo update your Pi, restart the Pi, and then give it another try.
Hello! First of all, thank you very much for your blog, it is super interesting and I’ve learned a lot in the last months about Python + OpenCV (I’d been using OpenCV in C++ during college). I have a problem, when I finish the program, the kernel dies and the camere does not shutdown, is there something (like freeing memory) I’m missing? I’m working in Spyder. Hope you can answer my question. Greeting from Mexico!
I personally haven’t used Spyder — can you confirm the existence of this bug when executing via the terminal as well? Either way, it sounds like a threading error. I would suggest inserting
printstatements into the code to determine exactly where the threading error is happening.Hi! First off – thanks for some amazing tutorials!
I’m a total rookie with rpi and programming, but I’ve got this far now. When I try to use the output and test the camera, everything works fine. But when I run your test_image.py i get a image that is very under-exposed, and almost dark. And when I run test_video.py, the frame is completely black. Have I done something wrong, or am I missing something?
Its worth to mention – I strugglede quite a bit with the OpenCV installation, and although the installatino of 3.0 went smooth perhaps some remains of the first try is corrupting the process?
If your frame is completely black, it sounds like you might be using outdated Raspberry Pi drivers. I would suggest running
rpi-updateand updating the Raspberry Pi firmware.Hi Adrian,
Thanks for the teaching.
I followed every step on “install openCV 3 on a Raspberry Pi 3 runing Raspbian Jessie” successfully, and then followed every step on this post successfully, until that we I did: $pip install “picamera[array]”, the computer said : picamera 1.12 does not provide the extra ‘array’ .
What is the problem, and how to fix it?
Hey Tom — I’m not particularly sure about that error message. I just ran:
$ pip install "picamera[array]"And it worked just fine. Just to make sure, type your commands into the command line instead of copying and pasting, just in case that is causing any problems.
Hi Adrian,
Thanks for the quick reply.
No error message is seen this time when I run $ pip install “picamera[array]” .
But then I have following error message when I run $ python test_image.py:
(Image:2229): Gtk-WARNING **: cannot open display:
And I do not see any image on the TV on which my Raspberry Pi 3 Model B is connected, nor on the computer terminal which connected to the Raspberry Pi through SSH.
I did all those steps through SSH connection. I see that you already mentioned above that using $ ssh –X to overcome this problem. But I am using Window. I do not know how have the similar setting for Window.
I was trying to avoid the problem with SSH, therefore, I tried to run $ python test_image.py locally (i.e. open a terminal on the TV with the keyboard and mouse that are on the Raspberry Pi 3 Model B. I first tried to run $ source ~/.profile, but I got the following error message:
Bash: ~/.profile: No such file or directory
Is this normal? I can run what you teach only through SSH, not locally?
You can still execute the code locally, that’s not a problem. As for your particular error message, it seems like your
~/.profilefile is not being found. Are you sure it exists? And are you executing the code as any other user than the pi user?Finally, regarding the -X switch for X11 forwarding, your Windows SSH client should have an option for X11 forwarding — be sure to search through the options for it.
Hi Adrian,
I see the image on my PC terminal now, after having installed the Xming on the PC and selected the proper setting for x11 on the putty on the PC. But only a portion of the image is displayed.
I have ran the test_video.py. It works fine. I have got the image, though the frame rate was likely lower than 30.
I still cannot run $ source ~/.profile locally. How can I tell what user I am when I am executing the code?
I did not do anything extra locally. It was just whatever the state or user when the raspberry Pi was turn on. And I did not do anything extra remotely either. It was just whatever the state or user when I log-in through the SSH.
If you’re running the
test_video.pyscript over the network, then yes, the frame rate will appear quite low due to I/O latency. This is because it takes considerable time to transmit each individual frame over the network from your Pi and to your machine.Finally, keep in mind that
source ~/.profileshould only be executed on your Pi (whether via a standard terminal or SSH). You do not need to execute it on your local machine.when is write pip install “picamera[array]”
its open” requirement already satisfied (use –upgrade to upgrade) : picamera [array] in/us”
It sounds like you already have the
picamera[array]module installed on your system. To upgrade to the latest version you can use:$ pip install --upgrade "picamera[array]"how to capture image manually by pressing any key?
for that how we get camera stream on screen?
Your video stream should be automatically displayed to your screen using the
cv2.imshowfunction as detailed in this image. You can then save any frame usingcv2.imwrite.I’d like to thank you so much for your help. I’m using your documents to make my first steps with Raspberry Pi. I’m from Brazil, and it’s complicate to find out documents in portuguese so rich of details like yours. Thank you again.
No problem Rossi, I’m happy I could have been of help.
Hi
I want to know how to access mulitple in one rasberrypi,I mean C++ and shell access VideoCapture(0) at the same time.
In the C++, I use People Counting. link: https://www.youtube.com/watch?v=0Q31goWSDgA
In the shell, I also use web streaming.
Can I use one pi cam to use multiple access?
Thank you…..
I don’t cover how to access multiple video streams using OpenCV in C++, but I do cover it using Python. You can read the full tutorial here.
Hi. i have a problem. my camera works fine for Image capturing. but when i run video capturing code it wont display anything. only black screen get appear. whats the reason for this???no error shows out though
Hey Umesh — take a look at my latest blog post where I demonstrate how to resolve this issue.
Hi Adrian,
Thank you for this and other tutorials that helped me a lot.
I run into this Gtk-WARNING **: cannot open display:
But my situation is slightly different and then the solutions from the comments above does not work for me.
The first is that I’m using the Raspberry Pi Compute module, and it does not have enough space to install full Raspbian, so I have Raspbian lite, just command line, no desktop, and I cant install xorg, as the device does not have enough space for both opencv and xorg.
Without xorg I can not forward x11 through SSH.
Do you know what I can do to fix it? It has to be possible as raspistill is able to display the image in the screen without problem, just imshow() I have this problem.
Hmm, this is an interesting question. If you can’t install the X window manager can you see if there is any other (smaller, lighter) window manager you can install? I honestly haven’t tried this without a window manager so I don’t know the solution off the top of my head.
Ok, thank you.
I though that maybe the lighter solution could be to install something lighter than raspbian-lite, I’m trying with minibian-wifi and use a nodejs server to output the video. I’ll write if I’m successful with it and upload code to github. By now I’m fighting with the wifi for some reason is getting disconnected from time to time.
I will try using node-python, I’m not sure if there is a better way to do it.
hey Adrian,thanks for another great post.As I followed along your post I got the test_image.py running perfectly like magic.However ,when I wrote (And the copy-pasted) the code of test_video.py it doesn’t work.A window comes up ,where I recon the video would be streamed ,but it’s just a black window.NO video streaming.If you read this comment then please do walk me through this problem.
Hey Ekhwan — Please see this blog post where I address common Raspberry Pi camera issues, including the solution to this issue.
Thanks.But I’ll make sure to be back if I run into any other problems cause the way you explain stuff is super easy to understand.
Hi Adrian! I meet some problems. I am connecting to the raspberry pi through SSH, using Putty on the windows and I use a VNC viewer. It all works nice until the step of “python test_image.py”, and it says that (Image:1554) Gtk-WARNING **: cannot open display. I have read the comments of Joe Landau but I still can not understand how to fix it. If I run “python test_image.py” on the SSH(Putty), the above error message occurs. If I run “python test_image.py” on the LXTerminal in the VNC viewer, it says that “no module named cv2”. And I try to type in the command “workon cv” in the LXTerminal in the VNC viewer, it says that workon:command not found.
Do you have any clue?
Thank you!
The GTK warning can be resolved by X11 forwarding. I’m not a Windows user and I don’t use Putty so I’m not sure how to enable X11 forwarding from Putty, but it is 100% possible — you’ll just need to do a little research from there.
As for the LXTerminal from VNC try using the following commands:
hi adrian. im doing a project on Raspberry Pi based face recognition for the blind person. i need to connect two cameras with raspbery pi 3. but the problem is how to the video stitching . And i aslo need a wearable device to notify the person about the video,in audio format. any help how to connect the wearable device with raspberry pi?
I can’t provide any insight regarding wearable devices with the Raspberry Pi, but I cover video stitching with the Pi here.
Hi Adrian;
Thanks a lot for your documentation and demonstration. I have a little problem here. I used same version of raspbian with raspberry 3. The image worked well but i didn`t get video. I got only small window which was all black. Could please help me about that? than you very much !
That sounds like an issue with your firmware version. this blog post details how to resolve the issue.
Hello Adrian,
Thanks so much! Your instructions (here and others I’ve read through) have been extremely insightful.
However, in this instance, I’ve run into a difficulty with Step 5. I’ve got my virtual environment up and going, but I keep getting an “ImportError: No module named ‘picamera’ ” message. I’ve double-checked that picamera and picamera.array are installed in the virtual environment:
pip-3.2 install picamera
Attempting to upgrade is either a requirement already up-to-date (picamera), or has a major error and cancels itself (picamera[array]).
So, then, picamera is installed in my virtual environment, if I understand correctly, but I keep getting the ImportError message whenever I run the test_video code.
What’s going on? (And is there anything else you need to know?)
Hey David, I had to edit your comment to remove some of the output from the terminal — the output was destroying the formatting of the terminal.
That said, it sounds like you’re explicitly using pip-3.2 to install the picamera module which is incorrect. Once you enter your virtual environment you only need to “pip”. The virtual environment automatically determines the correct version of pip to use.
For example, here is the correct way to install picamera into your virtual environment:
Hi the single image works, but on the video stream I get only a black screen (without error message). This is the case for 640×480 and 320×240 on 29-32fps.. system: raspi3, opencv3.1,
UPDATE: problem solved with RPI update:
sudo rpi-update
fantastic 🙂
Sorry, there is one point block me -_-|| How to build up a new file and insert the code when I use Putty to connect my Raspberry pi?
You should use a terminal-based text editor. The easiest terminal text editor to use for beginners is
nano:$ nano /path/to/your/fileHi Adrian
Could you please do a tutorial on how to stream using netcat, or gstreamer..? I am trying to make a pan and tilt object tracking device on raspberry pi using python and opencv. My confusion is how to use python bindings of gstreamer within the code so that the video can be monitored on a pc.
Thank you..!
I will certainly consider doing a tutorial on gstreamer. It’s been on my “idea list” for awhile, but I’ve been focusing on neural network topics lately. I’ll try to get back to doing a tutorial on gstreamer, but I’m not sure when that might be.
Hi Adrian, I appreciate your tutorials and will be buying your book as soon as my budget is opened for the new year. In the meantime, I am wanting to use the Raspi camera with your ball tracking tutorial. Can you tell me what I need to do to use the pi camera instead of a usb cam?
Hey Leigh — I would use this blog post as your starting point. Use the template I have provided and ensure you can read frames from the Pi camera video stream. From there, any code within the
whileloop of the ball tracking example needs to be included inside theforloop of this example. Keep in mind that you’ll have to replace.readwith the appropriatepicameramodule calls.Hello Adrian,
I have started doing a project with raspberry pi and image processing and the robot is supposed to track a ball. What camera module will be better to use for speedy applications? Is that RPi camera? Or a normal webcam?
Awaiting quick reply 🙂
Either one will work for simple ball tracking. If you go with a USB webcam I would recommend the Logitech C920.
Hi Adrain, the blog was very helpful in setting up raspberry pi. Also the tips and tricks were very useful. Thanks a lot.
Fantastic, I’m happy to hear it! Congrats on getting OpenCV installed on your system.
Another great tutorial! Thank you!
Thanks Adam, I’m happy you enjoyed it!
Hello Adrian,
Firstly, thanks for this tutorial, but I have a question
I am a beginner in raspberry pi, and I want to make shapes recognition like cube or cylinder shape, so what is the steps to do it ?
thank you 🙂
I would suggest starting with this tutorial on shape detection.
Hello, this is an awesome post. This will be my first Pi project. What Pi do you use? what about the housing it is in? lastly what is CV? and how do you get it. Thanks!
I would recommend the Raspberry Pi 3 for computer vision. The Pi 2 is also suitable but since the Pi 3 is a little faster, I would use that. Regarding the “cv” virtual environment, you create it when following my OpenCV install tutorials. You could also purchase a copy of my Practical Python and OpenCV book and get a pre-configured Raspbian .img file with OpenCV pre-installed.
Hello Adrian,
i write my Bachelor Thesis now. I have a Robot with a Raspberry Pi and an Android Application. My job is to shorten the time of the transfer process between RasPberry and Android, when the camera of RasPBerry makes a video.
Previously, this process takes about 2 seconds.
I wrote the code “test_video.py” from you into the RasPberry, but when I lately connect the robot to the Android application, “MediaPlayer error (1, -2147483648)” comes out. So my question is this: what should I do about it?
There were previously the written codes, but not of me. I dont know where is the problem.
Hey Solveig — what method are you currently using to send frames from the Pi to your Android app? I would suggest using a message passing library such as PyZMQ or RabbitMQ.
Hello Adrian,
I followed your tutorials but I came to a problem, the frame is full black when I started the test_video.py. A bought a pi camera module V2 which is 8MP. I installed opencv 3.0 and python 2.7. Could you recomand a solution?
thank you
It sounds like you need to upgrade the firmware on your Raspberry Pi. See this post for more information.
Thank you!
I updated the firmware and it worked.
Fantastic, I’m happy to hear that was the solution 🙂
Hi adrian,
I’m trying to get the image frame of vehicle number plate using my camera module. can you help me out with the opencv code?
I discuss Automatic License Plate Recognition (ANPR) and provide code for ANPR inside the PyImageSearch Gurus course.
Another black image here, but I had just “upgraded” picamera to 1.13 from 1.10. After running “pip install picamera==1.10” to roll back to 1.10, the program works fine!
THanks for sharing Ken — and congrats on resolving the issue! For what it’s worth, I also discuss common Raspberry Pi camera module errors in this post.
Dear Mr. Adrian i am currently using raspberry pi3 with jessie installed for my project and i am trying to Accessing the Raspberry Pi Camera with OpenCV and Python. after all five steps done including creating file test_image.py, i am using $python test_image.py command following error occurs:-
/home/pi/.virtualenvs/cv/local/lib/python3.4/site-packages/picamera/encoders.py:545: PiCameraResolutionRounded: frame size rounded up from 1366×768 to 1376×768
width, height, fwidth, fheight)))
(Image:1235): Gtk-WARNING **: cannot open display:
please help me in this at the earliest
thanks
It sounds like you might be SSH’ing into your Pi, in which case you should enable X11 forwarding to see the output image:
$ ssh -X pi@your_ip_addresshello dearest Mr.Adrian apparently issue is fixed and i must thanking you tons and tons, thank you so much to your real support cheers !!! thing is i were using an windows machine and i use to take the raspberry pi 3 in remote over SSH by using putty, now after you clarified to use above suggested command, i disabled SSH and worked on Raspi itself and tried the command python test_image /video.py and it worked well and i enabled SSH again tried taking raspi machine in remote by using windows machine through putty and tried your suggested command $ ssh -X pi@10.1.105.37 and again tried to execute command python test_image.py and still the following different error occurs
Traceback (most recent call last):
File “test_video.py”, line 5, in
import cv2
ImportError: No module named cv2
Many Thanks,
Once you SSH into your Raspberry Pi you need to access your Python virtual environment which would have your OpenCV bindings installed:
Hello Adrian,
Thanks for your for your great tutorial! I followed your introductions in this blog and it works after several problems. When I run your program test_video.py on my raspberry pi 2 I got the following warning:
(cv) pi@raspberrypi ~ $ python test_video.py
(Frame:2586): GLib-GObject-WARNING **: Attempt to add property GtkSettings::gtk-label-select-on-focus after class was initialised
q(cv) pi@raspberrypi ~ $ q
What does this warning mean? What is going wrong?
Nevertheless the video stream is shown in the window. (sometimes the pi needs a restart)
Would be great to get help.
Thanks a lot,
Regards Ulrich
This is not an error, just a warning. It comes from the GTK library. It can be ignored as it is not related to OpenCV.
hello Adrian,
so a while ago i followed your tutorial on how to install opencv3 on raspbian jessie and i successfully installed the opencv and i installed
Python 3.4.2 successfully as well now i am following your tutorial on how to Access the Raspberry Pi Camera with OpenCV and Python and i stumbled across an error when i try to execute the test_image.py file please what can i do to fix this and thank you in advance 🙂
Error found below:
(python test_image.py
File “test_image.py”, line 1
Python 3.4.2 (default, Oct 19 2014, 13:31:11)
^
SyntaxError: invalid syntax)
It sounds like you are trying to execute the Python script inside IDLE — don’t do this. Execute the script via the command line (no IDLE).
I had such a hard time figuring this out. All you have to do is delete the top of the test_video.py text (Everything above the first lines of Code. ex Python 3.5.4) and it opens right up.
Hi Adrian,
Great tutorials. I am doing a final year project of an autonomous car with vision processing. I’ve followed the openCV 3 install tutorial and now this one and all works fine until the test_video.py. I just get a black screen when i run this script. Any ideas ???
Regards,
Paddy
Refer to this post to resolve the issue.
Hello
I was faced to the same problem of slow frame rate (2 or 3 frames per second ) when using putty with X11 forwarding , but when switching to VNC all things works fine
Thanks for sharing Dakna!
I did the same and it worked. Much significant difference.
i go error like
ImportError: cannot import name ‘PiRGBArray’
how to solve it…
It sounds like you haven’t installed the
picameralibrary. Go back and ensurepicamerahas been installed.Typo in test_video.py
format=”bgr”
should be “rgb”
It messed with me for a bit – was getting black screen
Great resource anyway!
That’s not a typo — it should be “bgr”. OpenCV represents images in BGR order rather than RGB. If you were getting a blank screen, you likely need to update your Raspberry Pi firmware, as discussed in this post.
xlib etension RANDR is missing on display 1:0 !! what this mean ?
i’m using tighvncserver in raspberry and remmina ubuntu in laptop
using sudo python test.py i get : client is not authorised to access to server and no image is showen
Can you try SSH’ing into your Pi with X11 forwarding and seeing if that resolves your error?
Hi,Mehdi. Have you fixed this problem. I have the same problem. can you share with me your solution? Thanks
Hi Adrian,
I followed your steps into executing the code and also downloading opencv. Everything was installed successfully. However when I try to run the code using Putty and Xming in my laptop through SSH I get an error message saying:
…
PuTTY X11 proxy: Unsupported authorisation protocol
(Image:927): Gtk-WARNING **: cannot open display: localhost:10.0
However, when I connect my raspberry pi 3+ using an HDMI cable to my TV I don’t get an error. I just want it to work through my laptop as it would be much more convenient rather than me having to connect and disconnect to my TV to check if it is working. I hope you can still try to help me.
You need to enable X11 forwarding from your SSH connection to see the image on your screen. As for the error message from PuTTY, I’m not sure what is causing that. I would check the PuTTY documentation.
Hi Adrian,
Can we do face detection through pi camera which should be live?
I have followed yours tutorials on basic motion detection & home surveillance , It worked for me but can we do face detection through pi camera?If yes can u guide me what to do.
Thanks in advance
Hi Sufi — I actually cover real-time face detection for the Raspberry Pi inside Practical Python and OpenCV. Face recognition on the Raspberry Pi is then covered inside the PyImageSearch Gurus course.
Thanks for the great step-by-step guide. I’ve follow all your steps until I encountered an error upon running “python test_image.py” (or python test_video.py” for that matter). It says “cannot connect to X server”. Would you happen to know what might the cause be? I’m using a Pi Camera module as well, running on a Raspberry Pi 3 Model B.
How are you accessing your Raspberry Pi? Over SSH? VNC?
Hi,
I got to the step:
python test_image.py
and got the following error: I called the above file picamera.py
Traceback (most recent call last):
…
ImportError: No module named array
Tried this from a python 3 virtual env as well.
Will appreciate any input. Thanks.
Make sure you install NumPy + picamera[array] into your Python virtual environment:
both the packages are present. still get the error.
Hi djoseph — I’m sorry to hear about the continued error. Unfortunately, without physical access to your machine I’m not sure what the exact error is. I would validate that both packages have installed into your Python virtual environment via
pip freeze. Also double-check that you are inside your Python virtual environment when executing the script.I get the same error, even after reinstalling numpy and picamera (both return Requirement already satisfied )
Thanks for your help>> please I need a python code for video streaming using Webcam rather rpi camera.
Please refer to this blog post to help accessing either the webcam or Raspberry Pi camera module using the same class.
Dear Mr. Adrian,
first i have to say, you do a very great job!
I have quite the same problem like:
Giridharan Ravichandhran March 6, 2017 at 12:02 am #
I also using raspberry pi3 with jessie and i am trying to Accessing the Raspberry Pi Camera with OpenCV and Python2.7.
But after $python test_image.py command following error occurs:-
…
(Image:807): Gtk-WARNING **: cannot open display:
please help me with this problem
(and i’m sorry for any mistakes of my english, i’m german 🙂
thanks
It sounds like you’re SSH’ing into your Raspberry Pi rather than executing the script using a keyboard + HDMI monitor. This isn’t a problem, but you do need to supply the
-Xflag to SSH to utilize X11 forwarding:$ ssh -X pi@your_ip_addressI’m direct on the Raspberry (Keyboard, Mouse(with USB-Bluetooth-connector), Monitor (HDMI))
Is there an other possibility for this error?
Thanks a lot!
That is very strange if you’re using a local setup. I would suggest (re)installing X11, then re-compiling OpenCV.
You are right. That’s strange 🙂
But that’s could be the core of my problem ==> what is X11?
I’m so sorry. I realy not an expert.
Thanks!
X11 is a window manager typically used by Unix user interfaces.
hello Adrian
i ran your code and it worked but i had some issues :
1) i need to detect faces and on the other hand i need to recognize these faces so i need a high res. pictures, so i try to keep around 768*480 most of the time but i get a very high delay when i run the code, I blinked a LED in front of the raspberry pi cam and could see the blinking on the screen after 1 sec, and thats kinda bad for my application.
is theres a way to reduce this delay ??
one more thing, how can i measure this delay ?
Thanks for your time, I appreciate it .
To start, make sure you are using threading to access your video stream. Secondly, keep a large copy of the frame, but only apply face detection to a smaller, resized version. Face detection can be an expensive operation, especially for the Raspberry Pi. The less data you have to process, the better. If a face is detected, extract it from the higher resolution frame (multiply bounding box coordinates by ratio of new image size to old image size), and then pass into your face recognizer. I would also inspect your code to determine what the bottleneck is in case it is not the detection phase.
Great, if i got you correctly i reshaped the code to the following
and indeed i got a better results, but one more thing left, i wish to try different resolution and measure the corresponding delay over the camera, how could this be accomplished, were should i search to do this, i have GooGled all the OpenCV forums and Python Blogs without any clue on how to calculate this delay, i wish you could guide me for the first dew steps too.
Thanks a lot again, your making me a huge favor with your generous help.
I’m not sure what you mean by “delay”, but if you want to measure the frame processing rate of your pipeline, I would follow the instructions in this blog post by using the
FPSclass.Hi Adrian,
Thanks a lot for the great tutorials.
I have (almost) everything working … I am using the python2.7 with the open CV.
The problem is that I can’t figure out how to install the “picamera[array]” to be synced with the python2.7, it is automatically installed into the python3.4 which is not working with the opencv from some reason… thanks a lot.
Hey Alex — are you using Python virtual environments? Or are you trying to install “picamera[array]” globally on your system?
Hi! I followed your earlier tutorial to install OpenCV 3.0.0 on Jessie. I installed in python 3.4.2. I created the test_image.py file but when i try to execute in the terminal it shows:
Traceback (most recent call last):
File “test_image.py”, line 5, in
import cv2
ImportError: No module named cv2
Any reason for this?
Hi Govini — if you are having trouble importing the
cv2library you’ll want to refer to the “Troubleshooting” section of in this tutorial. Most of the time this error is caused by you not being in your Python virtual environment before executing the script.Hi thanks for all this. Compiling OpenCV worked flawless (RPI3, all fresh, including fast sd card) The RPI processor (4 cores) became quite hot.
Hi Adrian, I would like to know if the Raspberry Pi camera module needs to be calibrated to have images without distortion and to know the intrinsic and extrinsic parameters of the camera, if so could you please tell me how I can calibrate it, thank you very much.
You would need to calibrate the Raspberry Pi camera yourself. I would suggest starting here to learn how to do so.
Hi, I downloaded your program and tried to execute it, but nothing happens, just after two seconds, (cv)pi@raspberrypi: ~ $ reappears, can you help me out? I installed open cv with the help of your tutorials
It sounds like your Raspberry Pi cannot access your camera module. Make sure the camera module is enabled via
raspi-configand then runraspistillto see if the camera can save an image to disk. You should also double-check the camera connection on the Pi as you may have installed it incorrectly.Hi Adrian,
I am just starting out on this, and your instructions are very good. I appreciate the dedication you have to this. I got all of my install done correctly (I believe) as I achieved all of the checks you showed. However when I try to run the test_image.py I get the error:
“can’t find ‘__main__’ module in ‘test_image.py’ ”
I’m sure it’s something simple, but do you have any idea what I did wrong?
thank you,
-Matt
Hi Matt — I would suggest that you use the “Downloads” section of this blog post to download the source code and then try executing it. It might be the case that there was as error during your copy and paste of the code into a new file.
Hi Adrian,
I was wondering if there is any way to use usb cameras with the pi with opencv python. I’m trying to use two of them to get 3d images. Is there a better alternative?
Absolutely. See this post on accessing USB and/or Raspberry Pi camera module with the Pi. You should then read this post on multiple cameras with the Raspberry Pi.
can we do motion detection using webcam instead of picam?
Yes, absolutely. I would suggest following this tutorial and then updating it to use the VideoStream class.
Adrian
I know this post has been up for a couple years now, but it’s still great. Thanks a lot. Now I’m going to start on your tutorial using Django and try to stream video on to my Raspberry web server so I can view the video on a web browser. Any suggestions are welcomed.
Hi Daniel — thank you for the kind words, it’s much appreciated. I don’t have any tutorials on streaming frames directly from the camera to a web server, but I’ll add it to my queue of ideas. Thank you for the suggestion!
hi Adrian,
thank you for the wonderful tutorial , but when i run test_video.py the frame is just blank even though my camera is working fine
This is an issue with your Raspberry Pi camera module. Follow this blog post to resolve the issue.
As others have posted previously using the test_video.py code with 640×480 resolution I only get about 10 fps. I am using a Pi 3 and Pi Camera V2.1. Threading provides essentially no improvement as the bottleneck appears to be in the capture_continuous method not in the display portion of the code. I’ve tried changing cameras and camera cables also with no improvement.
I would be grateful to hear from anyone who has figured this out or can at least confirm the same results with the same hardware.
As best I can tell there seems to be some issue within PiCamera (I’m using v1.13) – maybe PiRGBArray or capture_continuous? Following the method described by Amrosik here: https://github.com/waveform80/picamera/issues/327 (near the bottom of the page) produced dramatic improvements in image capture speed for me.
Thanks for sharing Michael. I will look into Amrosik’s solution further.
Hi ,
Thank you for the useful tutorial.
I’m using multi camera adapter Module to capture 2 different images with two cameras. The problem is that I can just capture one image with one of the cameras.I do not know how capture image with the second one. I would be appreciative if someone help me.
Thank you.
Just to clarify — are you using two Raspberry Pi camera modules? Or one Raspberry Pi camera and one USB camera?
Adrian,
Wonderful set of articles you have put together! With them I have gotten the requisite software installed and the Pi camera doing useful things in just a single evening, up to and including motion detection. Thanks for the effort. Bravo.
Thank you for the kind words, I’m happy you found the tutorials helpful 🙂
hello Adrian, thank you for your useful guides 🙂
i have a problem with opening images by command ( cv2.imshow) ! it gives me this error when i use the command :
traceback (most recent call last):
…
cv2.imshow(“image”,image)
AttributeError: ‘module’ object has no attribute ‘imshow’
How did you install OpenCV on your system?
can I use webcam??
Yes, please see this tutorial for using a USB webcam instead of the Raspberry Pi camera module.
Hi Adrian,
I wanted to thank you first of all for the great tutorial!
However, I am experiencing a lot of latency with video streaming. I believe it is an ssh problem and was wondering if there were some workaround you could recommend.
Using X11 over ssh creates so much latency that the video becomes unusable. I’ve tried VNC as well and can definitely say there is quite an improvement regarding the latency, however, it still feels laggy…
Is there any better way to stream video remotely that you could guide me towards? I was thinking about trying Gstream next…
Anyways, thanks again !
Correct, if you are using X11 to view the output of your Pi are are introducing a HUGE amount of I/O latency. Keep in mind that the latency you are seeing is purely network overhead. If your goal is to minimize the latency, use gstream.
HEY!!! Thank you for the tutorial!!
My other question is… If I want to do the project just using the OpenCV module and not the PiCamera module… would that work fine still?
I would suggest you use the “VideoStream” class detailed in this blog post.
Hi Adrian
My Camera works when I run the command raspistill -o output.jpg
but the script doesn’t work for me i get an error saying with the cv2 import
saying No module named cv2
Hi Brian — if you cannot import the “cv2” library then either (1) OpenCV is not installed on your system or (2) you are not accessing the Python virtual environment before importing it. Please consult my OpenCV install tutorials for your particular system.
Instead of using Picamera can we use our mobile phone as a camera for RaspberryPi ?
if yes,can you explain how to capture an image and stream live video using the android apps
I’m sure it’s possible by hacking together some scripts but I have never tried this. Sorry!
How to interface raspberry pi with PC?
That’s a great tutorial.
Actually I’m working on data retrieval system where I need to capture the image/video ,send it to cloud and then access the data in my PC through Wifi. Can u help me?
There are a number of ways to do this. If you can elaborate on your project I can try to provide some suggestions to you.
HI Adrian,
Thank you for this tutorial, I have a question – How can I use this to stream the frames through a socket server to my computer. I will like to capture the video and livestream to my computer. But before I send the frame, I need to append it to a list because i am making a Machine Learning Project where I need every frame to be saved. I have looked at this tutorial by PiCamera Documentation – http://picamera.readthedocs.io/en/release-1.13/recipes2.html#rapid-capture-and-streaming – but I don’t know how to take these frames and convert it into a numpy array and append it to a list. Can you please help me or post a blog post on this. I need these frames to be saved as a numpy array so opencv can access them later. Thank you for all your help!
Hi Amaan — thank you for the suggestion. I will definitely consider this for a blog post but I cannot guarantee if/when that may be. Thank you again!
Hi Amaan
Have you find a code to do what you want ?
I whant to do the same thong, but impossible to find a code that functionning.
Ludo
Hi Adrian,
Got a problem with Step 5.
Getting:
WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
Any ideas?
Hey Sala — that’s actually not an error, it’s just a warning. It’s safe to ignore.
If you want to remove the warning you would need to first execute:
$ sudo apt-get install libcanberra-gtk*And then recompile + reinstall OpenCV on your Raspberry Pi.
Hello Adrian,
Great tutorial!
One problem I am having is that once I run the test_video.py program, all I’m getting is a black screen. Hitting ‘q’ on the keyboard breaks from the loop, so I know that the program is making it so far.
Any suggestions on how to fix this?
Thanks,
Aditya C
It sounds like you may need to update your Raspberry Pi kernel. Please see see blog post for more information.
Hey Adrian,
Great tutorial. Huge fan of your site/blog style.
Question for you: Is there any way to [periodically] start saving the frames to a video (instead of a series of images)?
Background: I’m doing some processing on each frame, and under certain criteria I would like to start recording/capturing a video. Maybe there is some method to push each frame to a video file-like object then save to an MP4 (or whatever format)?? I understand you can only have one connection to the camera, so I couldn’t simply spawn another camera video recording while continuing to process each frame..
Any advice would be greatly appreciated!
Cheers,
Ian
Hey Ian — I cover how to use OpenCV to write video as “key event clips” here. That might be enough for your project.
Otherwise, why not use list/queue/another data structure to save the frames in memory? If you aren’t storing many frames this is perfectly reasonable.
A more “hacked” solution would be just to write the frames to disk via
cv2.imwrite(with Unix timestamps as their filenames). When you have enough frames, have a separate thread start that builds your video.Adrian,
Thanks for this great tutorial. I’m one step closer to putting a system together to measure the motion of Polaris with the hopes of making a real time “Seeing” measurement for Astro-photography.
I’m using a Pi2 and both test_image and test_video scripts are running but I can’t stop them. Keyboard inputs are reflected in the terminal screen, but nothing happens. I have to close the terminal screen to stop the python script. Any clues?
I’m also getting the
WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
Thanks!
Brian
Hi Brian, so two things:
1. You need to actually “click” the window OpenCV opens first and then press a key on your keyboard. The window has to be open and active for the key presses to be registered.
2. That is a warning from GTK. It can be safely ignored.
What to do if I dont have a HDMI display and I connected my raspberry pi 3 through ethernet to laptop using vncserver, will this code still works ?
Yes, you can use X11 forwarding by SSH’ing into the Pi from your local machine:
$ ssh -X pi@your_ip_addressKeep in mind that the code will appear to run much slower due to the network latency of transferring the frames from the Pi to your local machine.
Hello Adrian,
I am getting this error.
$ python detect.py –output output
Traceback (most recent call last):
File “detect.py”, line 7, in
from scipy.spatial import distance as dist
ImportError: No module named ‘scipy’
Can you please let me know what i have to do?
Regards
Sudeep
You need to install the ScipPy library on your Raspberry Pi:
$ pip install scipy --no-cache-dirInstalling SciPy on the Raspberry Pi will take a few hours. I would suggest letting it run over night.
If you’re trying to use putty on a headless system from a Windows machine, the solution is pretty simple.
Install Xming on your windows machine, you need this as the output for the video. Once you’ve done that run Putty on Windows.
In the left had frame of putty, go to the “Connections” section and expand it. You should now see “SSH”, expand that and click on “X11”. In the right hand frame check the “Enable X11 forwarding” box.
Click “Session” in the left hand frame again and log on to your Pi as normal.
Do the Source and Workon commands, then run your python script.
You should get a Xming window open on your Windows machine which streams the video from your Pi camera.
I hope this helps fellow windows users.
Note: I can only get video ouput if I have an HDMI monitor plugged into the Pi (Even though I’m expecting video on the windows pc). If the HDMI is not plugged in, Xming displays a white window with no video. I haven’t found a workaround to this yet, if there even is one.
Dave
I should have said that the above is if you are running headless from a PC using putty and get the “Gtk-WARNING **: cannot open display” error.
Re: The no image, white window problem I was experiencing.
What I’ve found is that if you rely on the putty session to open the xming window and don’t have a display connected directly to you Pi, you may experience this issue.
The solution I have found is to run xming on your pc BEFORE starting your putty session. Now there is no need to have a display connected to the PI.
BTW Thanks for this tut Adrian, it’s been an enormous assistance to me.
Dave
Hi !
Thank you so much for you tutorial.
I have one question.
Can I choose the position of the camera’s window ?
When the window pops up, It sames pop randomly.
How can I choose the place where the camera’s window pop ?
Thank you.
Best regards
Are you referring to the location of the actual window opened by OpenCV on your desktop?
thanks for your effort, I have no video feed is displayed, i didn’t try the picture snapshot using opencv because i was only interested in the video feed, but all i have is a window titled “Frame” with nothing!
found solution in case someone face the same issue :
https://raspberrypi.stackexchange.com/questions/50439/pi-camera-opencv-my-python-script-test-video-py-has-black-screen
This blog post here on PyImageSearch covers a solution as well (the StackExchange post actually uses code from the PyImageSearch blog).
…
File “test_image.py”, line 19, in
cv2.imshow(“Image”, image)
AttributeError: ‘module’ object has no attribute ‘imshow’
Hello Guys can you help me with this error ..
How did you install OpenCV on your Raspberry Pi? Did you use one of my tutorials? It looks like, however you installed OpenCV, you did not have the GTK or QT library installed, and thus no GUI modules were built.
I am getting the following error, which I can’t understand – any ideas?
Unable to init server: Could not connect: Connection refused
(Image:1630): Gtk-WARNING **: cannot open display:
This is when I run the test_image.py script.
I am running in a virtualenv as shown in your previous tutorial.
Hey Roy — are you SSH’ing into your Raspberry Pi? If so, don’t forget to enable X11 forwarding:
$ ssh -X pi@your_ip_addressNo, I am working directly on the pi..
Have just tried through ssh with that forwarding, but still the same result..
Were any of you able to solve this issue? I have been trying to figure it out for a while but there is no luck. I always run into the error “Gtk-WARNING **: cannot open display:”
I am not using ssh either. I have tried to enable that regardless, but it still doesn’t work. Any tips would be much appreciated
How are you accessing your Raspberry Pi? Are you using a keyboard + mouse + HDMI monitor? Are you running the window display? The Raspbian GUI will need to be running in order to display the image/frame.
Thank you so much ! Its worked ! Now I can continue my personal project!
Best regards
Congrats on getting up and running with your camera module on the Raspberry Pi, Lucas! Nice job.
Thanks for the tutorials Adrian, great work!
I was probably making a silly mistake there. It didn’t work from the command line but works fine when i start X, presumably because of the image display.
Correct, you would need the X server running if you wanted to use GUI functions.
Sorry if this was already addressed above, but is there an easy way to modify this script to save the images to disk or access them via some other protocol if the Pi in question has no GUI installed (i.e., a Raspbian Lite install)? I usually just install the “lite” version for any project where the pi will be “headless,” like a camera, to save on storage space.
You can use the
cv2.imwritefunction to write individual frames to disk.Hi Adrian, I am having trouble with raw pixel value from pi camera (NoIR v2). I used the “thest_image.py” and add cv2.imwrite() to “png”. Why the maximum value is always less than 255 ( 234 in my case) in any channels, even the image looks clearly saturated?
A value of 255 vs. 234 isn’t that big of a difference. If the entire image is saturated and appears “white” then it will be a bit harder to tell the difference. I’m not sure why you are having such saturation though, it may be a problem with your filter or the environment itself.
can I do object detection and recognition from streaming video like this? e.g. i want to detect a traffic signs . thanks
You can, but it depends on the type of object detector if it will run in real-time on the Raspberry Pi. Some variants of HOG + Linear SVM and Haar cascades can run on the Pi in real-time. I would suggest taking a look at the PyImageSearch Gurus course where I demonstrate how to train your own custom traffic sign detector. Additionally, Deep Learning for Computer Vision with Python discusses how to train more robust traffic sign detectors as well.
hello Adrian can i use the USBcamera instead of picamera with ODROID-C2 using this same code ?? if i can what are the changes that i must do can u help me please !!
Sorry, I have not used the Odroid C2.
Hi Adrian, thank you for the amazing tutorial!
I’m using raspbian stretch via ssh, before i run the command “python test_image.py” or “python test_video.py” I getting the error message saying:
Unable to init server: Could not connect: Connection refused
(Image:4696): Gtk-WARNING **: cannot open display:
After installing the Xming server I’m able to display the image file and video.
Just got one problem when I click close button on the video streaming window, it will keep appearing again the next second, so I have to use keyboard interrupt to exit it.
Do you have any idea/suggestions on this problem?
This is how OpenCV handles GUI functionality. You need to click on the open window and press a key on your keyboard to terminate the script.
Hello, very nice tutorial,
I’m new to virtualenv.
How could I run a python script under correct virtualenv at startup for Raspberry PI
You first access your Python virtual environment using the “workon” command and then execute your script:
Hello Adrian,
I’m using Raspberry Pi model B+ with Pi camera v1.3 with openCV and python, when I stream video using the above method, you have mentioned above, I got a greenish image(color balance is greenish).How can I change to full-color color balance (without greenish) image?
I haven’t encountered that particular problem before. It sounds like there might be a problem with your camera sensor itself although you should check on the picamera GitHub to confirm.
Hello Adrian. How about accessing usb webcam?
You can use the “cv2.VideoCapture” function or use my custom VideoStream class which works with both the Raspberry Pi camera module and a USB webcam.
How do I go about testing the image and video with a usb webcam. It is so obvious my picamera is not working
If you are using USB webcam you can use the “cv2.VideoCapture” class or use the “VideoStream” class inside my “imutils” library. Be sure to refer to this blog post for more information.
Hi Adrian, I downgraded the picamera [array] to 1.10 and tried rpi-update.
I did get the output image but the color is wrong. how can I solve it?
Hey Tiffany — I’m not sure what you mean by “the color is wrong”. Could you elaborate on what the issue is?
Hi,
Thank u for post.
how to access datapoints?
Hey there, which data points are you referring to?
Thanks for this, i am getting this error when trying to issue the command however ssh to run the script.
$ python vid.py
Unable to init server: Could not connect: Connection refused
(Frame:825): Gtk-WARNING **: cannot open display:
any idea how to fix this?
I assume you are SSH’ing into your Pi? If so, make sure you enable X11 forwarding:
$ ssh -X pi@your_ip_addressHi,
Thanks for your tutorials.
I am too getting greenish images. However, when I capture an image with raspistill, I get a normal image with the right colors. So I guess it is not an hardware issue. Have you any idea what it might be?
It may be an issue with the picamera module itself or the firmware of the Pi. I would suggest posting on the official picamera GitHub.
Thanks worked like charm!
Glad to hear it! 🙂
hello Adrian, can you help me with my problem. im working on project with 3 push button as the menu option, one of the menu is for start streaming and recognize a face. i have success making the program back to main menu if the face recognized. but when i press the button again for recognize a face there is an error :The camera is already using port %d ‘ % splitter_port) picamera.exc.PiCameraAlreadyRecording: The camera is already using port 0 “.
is there anyway to stop the stream? i have tried break, but i close the entire program. thank you Adrian
It sounds like you are not using any code from a PyImageSearch blog post, but based on your error, you likely started recording an output video file using the picamera module but never released the pointer before trying to record a new file.
thankyou for your answer Adrian, and the code is not from PyImageSearch blog post, but it have the same way on “Accessing the video stream of your Raspberry Pi using Python and OpenCV.” so.. is there anyway to stop the stream without using break? thankyou
It’s hard to say because I don’t think you’re using the same functions as the one in this blog post. We’re not using the “PiCameraAlreadyRecording” method nor are we using splitter ports. I think you may be using a different set of functions. My primary suggestion would be to post on the picamera GitHub page where the library authors may be able to help you more.
thank you Adrian, have a great day
thankyou Adrian for the link, it just need a camera.close() at the end of the loop. have a great day
I’m getting a warning that seems to be coming from the cv2.namedWindow in in cv2.imshow.
** (Image:5465): WARNING “”: Errow retrieving accessibility bus address: org.freedesktop.DBus.Errow.ServiceUnknown: The name org.ally.Bus was not provided by any .service files
My RPi3 is hooked to a monitor. A window pops up but doesn’t display the image. The image is captured and can be saved using cv2.imwrite.
This is a warning from the GTK library and can be safely ignored (it will not affect your code).
hi Adrian! thank u so much for these amazing tutorials, all went smoothly till now. Actually i m working on a project to detect a circular cookies from pi camera streaming and then detect color and diameter of that cookie with a reference cookie, if color and diameter are same a msg displays OK but if anything does not match up display ACTION REQUIRED, cookies are running on a conveyor belt in front of pi camera. Can u write me the code for this purpose as I am a beginner at python and open cv, and running out of time to go through some tutorials to complete my project. Thnx in advance
is the installation of motion is essential or instead of this can i use raspstill for capturing photo
The
raspistillis a command that will capture an image to disk. You could technically call that command then load the image into OpenCV but normally you would want to access the Raspberry Pi camera module directly.Hi Adrian,
This a great article, but what do i do when to happen error rawCapture.truncate(0) ?
please give me a solution about this.
Q2: when i set FPS =32 that is mean 32 frame captured per second, but i don’t understand this part , in the body of a lot of lines code and this get every frame at the begin of for loop , Do i make a mistake?
The
rawCapture.truncatecall is required — is that line throwing an error for you? If so, what is the error?After executing the command of python test_image.py, I get the following error — unable to init server: Could not connect: Connection refused
(Image:864): Gtk-WARNING **: cannot open display:
What should I do??
It sounds like you are accessing your Pi over SSH. Make sure you enable X11 forwarding:
$ ssh -X pi@your_ip_addressI have this error when i test with : python test_image.py
python: can’t open file ‘test_image.py’: [Errno 2] No such file or directory
How to fix it?
Hey Eki — have you used the “Downloads” section of this blog post to download the code? If not, please make sure you do, otherwise you will not have the proper files on your system.
Thanks for this tutorial
You’re welcome Mateus, I’m glad you found it helpful!
perfect, thank you
but im having issues pi camera
What issues specifically?
good work…. please my picamera is not displaying colored , its only display gray color and I’m unable to convert from BGR2GRAY since it already gray….please help out
It might be a problem with your physical Raspberry Pi camera module. Could you try with a different one?
okay thanks,
Please can you help me with the source code to hand gestures recognition, I’m using picamera, raspberry pi3,opencv3,Python 3… I’ve seen how it was done using c++ and webcam ,please help me out.
The only source code I have for hand gesture recognition is inside the PyImageSearch Gurus course.
Hi there is there any way to process raspberry pi video that streaming on website ( by not installing python and open cv on raspberry pi)
Thanks
I have a keras model and I wanted to load the model saved as a .h5 file. I wanna use for real time object recognition with opencv. Do you have an idea how I could load the model and display predictions?
All tutorials seems to use pretrained models and none of them uses fine tuned models.
You mean something like this?
Afernoon, im having errors with the code
cnt = max(contours, key = lambda x : cv2.contourArea(x)) and errors is valueError: max() arg is an empty sequence
It looks like no contours are being detected and then you are passing an empty list into the
maxfunction. Ensure at least one contour is detected.First , thanks about that great tutorial. I have a problem with the python script that i want to use picamera with cv2.VideoCapture. I have tried to modify your code but never succeed.
Can you tell me if there is a way to use picamera with cv2.Videocapture?
Thanks from now…
If you want to use the
cv2.VideoCapturefunction with the Raspberry Pi camera module you’ll need to look into installing the VL42 drivers, otherwise the Raspberry Pi camera module will not interface with OpenCV.Can you please tell me how to fix this error? Image:3735): Gtk-WARNING **: cannot open display: :10.0
I assume you are SSH’ing into your Pi? If so, you need to enable X11 forwarding:
$ ssh -X pi@your_ip_address*** Attention Windows Users with GTK warning ***
I have a solution for the GTK warning when SSH’ing from Windows.
Tested with Windows 10 and Ubuntu Windows Subsystem for Linux (WSL).
1- On the Raspberry Pi, check that `X11Forwarding` is set to `yes` in the /etc/ssh/sshd_config file. If you had to change it make sure you restart the SSH service by running `sudo service ssh restart`.
2- On your Windows machine download and install XMing (https://sourceforge.net/projects/xming/). It’s super small and quick. After the installation verify the XMing server is running. (Should see it in your tray).
3- Open a WSL terminal and run the command `export DISPLAY=localhost:0.0`, then run `ssh -X pi@
4- Run the python script again and this time you should see a window pop up on your Windows machine. In my experience it takes a few seconds to load the image
Hope this helps!
Thanks for sharing, Faris!
Hello,
I’ve been playing around with this , trying to adapt it to my usecase, but it keeps throwing this “Incorrect buffer size” error? What is that how do I fix it?
You’re likely forgetting the call to
.truncate.ImportError: No module named ‘picamera.array’; ‘picamera’ is not a package
how to solved this one ? thank you
You didn’t install picamera properly (it’s missing the “array” subpackage):
$ pip install "picamera[array]"Hi Adrian, we have the same issue “ImportError: No module named ‘picamera.array’; ‘picamera’ is not a package” and we’ve done $ pip install “picamera[array]”. Do you have any other suggestions?
Thank you
Are you using a Python virtual environment? If so, make sure you are accessing the Python virtual environment when you install “picamera[array]” and then when you are running the script. If you need further help refer to my book, Raspberry Pi for Computer Vision.
hi
Wonderful blog ! very helpful.I was able to get the raspberry pi camera board c-82 working and capture images and video frames through it.
But the images are too warm that is they are yellow .
raspistill -o image.jpg is returning natural color images but when done with picamera they are yellow.
how can i improve it ? what setting can i use.
thank you.
That’s quite strange. What specific camera module are you using? The Raspberry Pi camera? If so, try posting the issue on the picamera GitHub Issues page.
I am also encountering the same issues. The raspistill command and picamera() function work great but when using opencv coded in the provided example I now encounter warmer colors in addition to a clear band across the image. Has anyone else encountered this and hopefully a fix as well :).
Hi Adrian
Would you be able to tell me what the Ethernet bandwidth requirements might be at different video resolutions?. Thank you
Meaning if you’ll be streaming the frames from the Pi elsewhere?
Hi, i am a student from a university and i am using your code for the streaming of picamera in our GUI in pyqt5. Is it possible to record the whole streaming and save it as a video file? How will that be possible? Thank you so much!
Yes, you can use the cv2.VideoCapture function.
hi
can we use USB web camera instead of pi camera?
Yes, refer to this tutorial where you’ll learn how to use the “VideoStream” class, compatible with both the Raspberry Pi camera module and USB webcams.
thank u
This and the installation tutorial are great. Much appreciated.
I have an issue capturing an image with the opencv module though. It basically captures it fine, however, it seems to switch RGB or BGR format in the lower 1/5th of the image. This doesn’t occur when streaming video through opencv and if I capture an image using raspistill the colour is consistent, it is just when image capturing with opencv.
Any chance you have an idea of where to point me to try and rectify this?
Many thanks
I’ve unfortunately never encountered that error before, I’m honestly not sure why that may be happening. I would post the problem on the picamera GitHub issues page.
I need to Open up a new file for “test_image.py” on desktop or can open it at anywhere?
You will need to execute the script via the command line.
ImportError: No module named ‘imutils’
I have followed all the steps
You need to install imutils on your system:
$ pip install imutilsvery good
Thanks Hui, I’m glad you liked the tutorial!
Thanks for this article!
It’s very simple to understand and apply these examples.
Congratulations!
Thanks Yhan!
Fantastic resource Adrian! Good luck with your work.
Thanks so much!
thanks for the post, it’s really good.
I have a question, how can i install the extra drivers to use the cv2.videocapture function?
Hi Adrian,
I’m using your practical python image for the rasppi. I receive teh following error:
python load_display_save.py –image /home/pi/images/trex.png
width: 350 pixels
height: 228 pixels
channels: 3
Unable to init server: Could not connect: Connection refused
(Image:1041): Gtk-WARNING **: 15:46:42.885: cannot open display:
I tried:
– I do SSH -X to my PI.
– I have enabled the desktop thrue raspi-config.
– raspistill -o output.jpg does work fine. Nice picture created in file.
– tried workon py3cv4 workenvironment and also without it.
Any ideas?
I’m trying to follow the chapter three.
The issue here is that X11 forwarding needs to be enabled to use display the “cv2.imshow” call from your RPi on your laptop/desktop screen. A few suggestions:
1. Enable X11 forwarding via ssh:
$ ssh -X pi@your_ip_address2. If the above doesn’t work make sure you have X11 installed on your host machine
3. If all else fails, use VNC to access the RPi.
Hi! Where can I find a script example that allow me to record the capture in a output file?
TKS!
See this tutorial.
Hello Adrian!
I just purchased a raspberry pi 4 model B and a pi camera module v2 and read this tutorial. It looks really great, but I’m not yet an experienced python developer.
I want to keep it really simple. Just take a picture every hour with as great resolution as possible. Save that picture locally and then probably use your k-means color clustering to finally send an email if it exceed a specific red-color threshold, otherwise probably delete it.
I was wondering if this still is “up-to-date” or is there new and better code and libraries that you would recommend? This also includes your k-means code and libraries.
Thanks alot from Sweden!
/Fred
Hey Fred — your exact project (minus the k-means clustering) is covered inside Raspberry Pi for Computer Vision — have you considered starting there?
Hey, thanks for the reply!
That would be really nice but its a bit over my budget for the moment.
Ill look around but if I don’t find anything pedagogical like this Ill definitively consider it when the salary arrives 😉
/Fred
Hi, Adrian! I had exactly the same error and tried to reinstall picamera library a couple of times. Then I figured out that the problem was in the name of my python script. I called it “picamera.py”. After I deleted it from my directory and created a new one with the different name everything worked fine. So, no “picamera.py” files in your working path. Hope it will help and save some of the learners’ time.
Yep, that would do it! Congrats on resolving the issue 🙂
Hi Adrian !
I’m currently doing a computer vision project where I have to detect behaviors such as eating in class and cheating during exam using raspberry pi 3 B+. Do you mind sharing any tip and idea on how I can start? I’m very new to this and it’ll be very helpful if I could get some guidance from you. Should I use deep learning or Haar Cascade? Any post of yours that can be helpful for my project? Thank you in advance !
Follow this tutorial on video classification as a starting point.
I managed to make this work without using the PiCamera library but instead using OpenCV’s cv2.VideoCapture(0). Had to mess around with the 4l2-ctl but got it to work. All I am wondering now is if I will encounter problems trying to process the picam feed as any other usb camera via OpenCV. Is it all the same once the frame is captured and pass on to cv2 for processing?
No, you shouldn’t run into any other issues. The V4L2 drivers simply allow you to access the RPi camera module through OpenCV’s “cv2.VideoCapture” function.
Hi Adrian
Your posts are such a piece of wonderful help. I’m about to design a face recognition on RPi 4.
I have a question here:
Does picamera have any methods like imshow in opencv?? I want to watch the output video without saving that.
I think you have some confusion on what the “picamera” module is — it’s just a Python library that can be used to read frames from your webcam. You can display them to your screen using OpenCV. I would suggest you read Practical Python and OpenCV so you can learn the basics first.
Hi Adrian
I want to send images captured by pi(client) to my system (server os:linux) as i want to do image processing on the same image.
Could you please help me as soon as posible.
Try using ImageZMQ to stream the frames.
Hi Adrian
Thanks for the reply
The article which you refered (ImageZMQ to stream the frames ) is for live streaming of video but I require to transfrer the images that is captured and saved by pi to my server.
Could you please help me.
If possible could you share your email id so that I can share my code.
If the image is already captured and saved you could still use ImageZMQ or any other message passing broker. Another option could be to use FTP/SFTP. I suggest looking into basic file passing protocols.
Thankyou