Want an AI agent that is NOT spying on you?

Host your own AI agent locally that is even better than the ones offered commercially with Open Web UI.

In case you missed my recent posts I will be working one handed for the next few weeks due to a surgery on my right arm. I decided to take the glass half full approach and look at this as an opportunity to review and improve the way I interact with computers to do tasks like programming, writing and everyday browsing.

This led me down a deep rabbit hole of AI/ML workflows.

One of the coolest things I have found is Open Web UI which is an open source tool you can run locally. It works a bit like Claude Desktop except it is way more extendable.

You can point it at many AI Models (With a little help from LiteLLM) so you are not locked into a single LLM provider. You can even set it up so you can pass multiple models the same prompt and compare their outputs.

It allows for multiple users and groups so if you were inclined to you could easily set it up in the cloud somewhere and give your friends, family and colleagues access to it. You can set guidelines for certain groups like prompting it to keep your kids from asking adult questions etc.

It comes with baked in dictation which since I have limited use of a keyboard will be really handy.

The most powerful feature is the library of “Tools” that you can add. These are similar to MCPs and give Open Web UI the ability to query information for context and take actions on your behalf. I added in tools that give it access to things as simple as the time or as complex as using playwright to navigate browser tabs.

In theory you could add a tool that allows the LLM to control your smart home but that might be taking it a step too far… for now.

Next up will be adding a Vector DB to it so it can have long term memory which will improve its ability to handle longer more complex tasks.

I have been setting this all up locally with Docker but I am tempted to toss this on AWS so I can access it from anywhere with my phone. If you want access to my docker compose setup or would like to see some terraform to get this spun up on AWS shoot me a message.