Share your repls and programming experiences

← Back to all posts
33
🕵️‍♂️ A scavenger hunt with image classification 🖼️

hey hey hey hey hey

I'm back with something I think y'all will find interesting! I built a game based on image classification - you have to get objects from around your house - whatever the app tells you to get, you fetch!

Play here 🎮

Check out this demo video I made 🎥 - I go over using the app, and the tools I used to make it!

How does this work?

4ty2 hunt has two main dependencies - ml5 and p5 - p5 lets you do cool things with graphics (like getting the video stream from the webcam, super easily) - but ml5 is the real core of this app!

ml5.js lets us run machine learning models, and train them as well, right in our browser. But the reason it exists is to make ML projects, like this mindblowingly simple 🤯

The whole of my game comes down to me writing about 150 lines of JS - that's it. ml5 allows me to pass the webcam's video stream as a parameter, and keeps telling me what it sees. I'm using the mobile-net model for this, which uses the image-net dataset - around 15 million labelled images. Besed on that huge chunk of data, this model predicts what it sees!

Challenging you

Image Net has 1000 different labels through which objects are classified - some of them include things like Komodo Dragons, Great White Sharks, and Ostriches - you know, things that can't be found in most homes 😛

So, I manually went through loads of the labels, trying to label things most of us could find.

Clearly, I was a little sleep deprived - and I probably made some mistakes. Let me know if you find some weird labels - I'm @jajoosam on the Discord 💬

Easy machine learning with JS

Machine Learning is pretty complex, but that doesn't mean making projects with it has to be too! I have gained a lot of appreciation for ml5.js - it's an amazing way to integrate ML into your projects.

I learnt all the tech behind making this project from Dan Shiffman's videos on ml5 - and I highly recommend you check them out if you found this interesting! He's a great, super energetic teacher 👨‍🏫

Before building this project, I followed the first video of the series to build this

Super excited to hear what y'all think about this!

Loading Repl
Commentshotnewtop
2

Absolutely amazing

2

Oh my! This is fantastic!!!

2

wow! This is amazing!

1

@IEATPYTHON Thanks!

Love the chess bot - but you need to give me a chess lesson sometime for me to not fail miserably 😂

1

This is a really fun idea. I'd love something like this on mobile especially.

1

@jajoosam I like that a lot! Its really easy to trick though lol