Jev and Astra team up to beat Minecraft

Jev and Astra team up to beat Minecraft.

Recently, Ronak Malde released a post showing how he teamed up Jev and Astra to beat the final boss in Minecraft.

Ronak admitted it had a few nuanced advantages, like “Astra built up a library of skills and information that it then used for this final playthrough,” so it wasn’t exploring a new environment, but that isn’t the interesting part to me.

Oddly enough, in between consulting projects, when I was crawling the walls with boredom, I experimented with training my own ML models from scratch to beat Minecraft so this really interests me.

I have seen a lot of people try to wire in LLMs to control Minecraft but, in my opinion, that is not the right model for it.

Too slow and clunky for one.

Second, it's a non-linear environment that the model has to understand, which isn’t great for the sequential nature of LLMs.

In my attempts, I used NEAT Neural Networks, which are much smaller, faster models that can sample the environment at much closer to 30fps making them much more adept at reacting to incoming input.

That is where Jev comes into play. Not only is it fast enough to get closer to real-time reaction speed, but due to its structured output, you force it to select from a specific set of outputs (jump, crouch, open, craft, etc.).

Why does this Minecraft have to do with anything?

The pairing of a long-term planning model with a lightning-fast model designed for strictly typed outputs might not be a major breakthrough, but it's novel.

Perhaps you have been tasked with adding an AI-powered text interface to your company’s flagship product or, as I like to say, “Slap a chatbot in it and call it AI”.

Whereas before it was clunky and slow, Jev’s own docs boast is designed for Real-time applications: “100ms speeds means you can use AI in your applications where UX is critical.”

If it's fast enough to play a realtime first person 3D game, then it's likely fast enough to help your users navigate the website.

I try not to ride the AI hype train most of the time but here and there an actual innovation pops up.

It’s not flying cars, but as a person that has dumped way too much time into training AI models from scratch to beat Minecraft, I am a bit impressed by this innovation.

What do you think?