Are you paying too much to host your Machine Learning Models on AWS? Recently I had a client ping me about a new feature for their e-commerce platform. They have millions and millions of photos from customers showcasing their products. A lot of the photos with the specific product tagged in it. When a new photo was uploaded they wanted to be able to detect what product and instantly tag it. In ML terms this is called “Image Classification”.

How to host it: You could host it on AWS and spin up some GPUs and burn some cash and depending on the complexity of it you might have to. But if it is light weight enough you could use something like ml5-js to run the model in the browser. This would save you a lot of $$$.

Here are a few considerations:

How big is your model?

If it is massive it might not be able to run in the browser. Or it might take so long for the user to download the model that it ultimately creates a bad user experience. So keep this in mind when designing your system.

What Is Acceptable Latency: If a user bulk uploads 1000 images then perhaps you have time but these days people expect a super fast experience.

It's also quite possible if your model is light enough that running the model in the browser is faster then sending the query server side and waiting for the GPUs to spin up. Unless you have money to pay for an always on system (Must be nice). That leads me to my next question…

What Is Acceptable Cost: If money is tight running it in the browser could be the way to go but keep in mind then you need to serve up that model many times a day and that isn’t cheap. You should be able to save a bit by hosting it on CloudFront but that only can save you so much. Code the frontend so it caches that model locally to save a few extra $$$.

Mobile App Advantages: If you had a mobile app you could utilize some of the GPUs that now come standard on mobile devices to process. You also could pre-load the model during sign up.

How would you train it? This is the million dollar question, but this post is getting a bit long so I will save that for another time.

Question For You: Do you like hearing about AI and MLOps topics? As AI/ML becomes more prominent I am getting more requests from clients to do more MLOps(Machine Learning Operations). In case you don’t know, that is the Machine Learning equivalent of DevOps. Creating a system to train, fine tune, deploy and scale machine learning models.

If that interests you let me know in the comments or DM me.