Thinking about adding an LLM into your product? Here is a post about creating effective agents that caught my eye:

https://www.anthropic.com/engineering/building-effective-agents

One thing that stood out to me is the idea of routing a query to 3 or more LLMs at once and then having a 4th LLM or process to see if they came to a consensus and route accordingly. If all 3 came up with different results then perhaps the certainty score is pretty low. If they all agreed then we feel pretty good about this.

I have a client that is considering a widespread use of LLMs in their e-commerce platform currently so I need to keep up on some of this stuff to ensure it can all be run efficiently on AWS.

Questions For You: How are you using AI in your products?

How are you hosting the compute resources for those models?