Want to play multiple generative AI hosting providers against each other to get the lowest prices?

Check out Open Router “The Unified Interface For LLMs” which is a brilliant business model honestly. It basically proxies requests to various hosting providers based on who is offering the best prices on their services.

If one service goes down or experiences high demand and starts throttling it simply routes your requests to another one making for vastly improved uptime.

What are the downsides of Open Router? Adding anything in the middle will add latency but is it noticeable?

Running it for personal purposes is not noticeable but at scale that extra milliseconds could add up.

In this you will just have to weigh out the costs vs benefits.

If you want me to do a deeper dive let me know and perhaps I will benchmark the added latency/costs in more detail.