So you’re telling me that we are going to build a Pokedex in Python that can visually recognize Pokemon? Just like in the TV show? – Ash Ketchum
Yes, Ash. That’s exactly what I’m telling you. The next series of blog posts will be dedicated to building a Pokedex that can visually recognize Pokemon from the original Red, Blue, and Green (JP) games for the Game Boy.
Don’t believe me? Check out this video:
Motivation: Visually Recognizing Pokemon
I’ve been playing Pokemon (the original Red and Blue) since I was a little kid. I love these games and I know them front to back. In fact, every year my friend and I hold the Pokethon. We drink some beer and do a speed run of Pokemon Blue as fast as we can. This past year I managed to beat the entire game in 2 hours, 46 minutes. Was I pretty happy with that achievement? You bet. But there’s always room for improvement. Next year’s goal is sub 2 hours, 30 minutes.
I’m getting a little off topic here. A few years back I decided that I wanted to beat Pokemon Green. The Green version was only available in Japan, and as you might guess, all the text is in Japanese. I can’t read Japanese, but I still managed to get through the game.
And I thought to myself, this is what Ash Ketchum must have felt like.
Here I am, playing this game. I can’t read the text, I can’t follow the actual story of the game. But through random encounters and battles with rivals, I’m able to see and experience a lot of different Pokemon.
Now, I obviously know what each Pokemon is based on their sprite images. But let’s pretend that I didn’t.
How might I identify these Pokemon?
And how might I easily and conveniently lookup any stats related to the Pokemon?
Perhaps the most obvious strategy would be to go to Google and type in the name of the Pokemon in Japanese (character by character, according to what is on my screen), translate it, and then lookup the stats in english.
But we study computer vision on this blog. So instead we’re going to build a computer vision solution.
Your Phone is Your Pokedex
If you’re anything like me, you have your phone with you wherever you go. It’s probably sitting next to you when you are playing Pokemon, just in case you get a txt or a call.
And in all likelihood, your phone has a camera.
So here’s what’s going to happen:
- We are playing Pokemon on our Game Boy (either, Red, Blue, or Green version) when we encounter a Pokemon that we do not know.
- We would like to identify this Pokemon, so we take out our phone and snap a photo of our Game Boy screen.
- We then take this picture, apply our computer vision and image search engine techniques, and we are able to automatically identify the Pokemon.
Again, if you don’t believe me, just take a look at the video above.
How are we going to do this?
In order to even “find” the Game Boy screen in our query image we are first going to make use of many image processing and computer vision techniques. For example, we’ll be using techniques such as:
- Edge detection
- Thresholding
- Perspective warping
After we have our Game Boy screen, we’ll crop out the Pokemon and then describe the Pokemon. We’ll characterize the shape of the Pokemon by using shape descriptors. In this way we can abstractly represent the Pokemon using only a list of numbers (i.e. an image feature vector).
Don’t worry. I’ll go through every and every detail as we work through this series. By the time we are finished, you’ll have a firm understanding of how to use shape descriptors to describe objects in images. And in this case, the “objects” will be Pokemon.
What’s next?
Building our Pokedex will be a six step process:
- An introduction to building a Pokedex (that’s this blog post)
- Scraping the web and building our Pokemon sprite database
- Indexing our database using shape descriptors
- Finding the Game Boy screen in our image (Part 1)
- Finding the Game Boy screen (Part 2)
- Searching and identifying the Pokemon
Libraries You Will Need
I will assume that you have the following libraries installed:
- NumPy and SciPy
- OpenCV with Python bindings
- Scikit-Image
- requests
- BeautifulSoup
Don’t want spend a bunch of time installing these packages?
No problem!
I have a pre-configured Ubuntu virtual machine with all the necessary computer vision and image processing pre-installed!
Click here and check out the eBook + virtual machine package to learn more.
Gotta Identify ‘Em All
Are you ready to embark on your very own Pokemon identification journey?
I know I am.
What's next? I recommend PyImageSearch University.
30+ total classes • 39h 44m video • Last updated: 12/2021
★★★★★ 4.84 (128 Ratings) • 3,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 30+ courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 30+ Certificates of Completion
- ✓ 39h 44m on-demand video
- ✓ Brand new courses released every month, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF
Enter your email address below to join the PyImageSearch Newsletter and download my FREE 17-page Resource Guide PDF on Computer Vision, OpenCV, and Deep Learning.
