From Punching Trees to Strategic PVP

The first thing out AI fighter in this Minecraft server has to do is punch a tree.

It needs wood before it can make tools. It needs a wooden pickaxe before stone is useful. It needs a furnace before raw iron becomes a weapon…when the glass wall between two teams disappears, it has to notice that the game has changed: There is another player somewhere on the map, and that player is trying to win too.

A language model inside Minecraft has to deal with a world that persists while it thinks discretely. It needs to observe what is around it (for some models, even without vision), remember what it has already done, choose an action, survive the consequences, and revise its plan. Add another agent on the other side of the map and the task becomes adversarial.

An Embodied Intelligence in Minecraft

Our project starts with Mindcraft, an open-source Minecraft runtime built on Mineflayer. Mindcraft provides the bridge between a language model and the game itself. It can connect an agent to a server, maintain a conversation history, parse commands, and execute Minecraft actions through a stack that includes pathfinding, block collection, combat, food management, and armor handling.

That gives the agent a body, and its brain (LLM), but it’s not yet a player that plays the entire game independently - even it has a “brain”, we need a full set of “nervous system” that interacts autonomously with the environment.

For the environment itself, which is also important for intelligence exhibition, we added an arena around them. One deployment starts a game server, sets a match director, and records the run. Each match produces logs and a summary that can be traced.

Match Director Handles the Rules, Agents Handle the Play

The separation between match director and agents is easy to miss, but it changes what the result means. A central script could tell an agent where to move, when to chase, and whom to attack. But the intelligence would then mostly sit in the script.

Our director does less than that. It controls the environment and the timetable. It puts teams at fixed spawns, creates a glass barrier during development, removes it at battle time, sets the elimination rule, records encounters, and ends the match when a team has been wiped out. It does not issue direct movement or attack commands during battle.

Instead, the agents receive compact goals. During development, a fighter is told to work through the practical sequence of Minecraft survival: gather logs, make tools, find stone, collect coal and iron, smelt the iron, and prepare combat gear. During battle, the instruction becomes much shorter: stop gathering, equip the best available gear, find the enemy, and fight.

Continuous Decision-Making

Then the self-prompting loop takes over. Each prompt includes the current goal and a phase clock. The model has to respond with a Mindcraft command. That command runs in the world. The result enters the agent’s context. The agent has to decide again.

This is where the experiment becomes interesting, because the model is not being asked to give one polished answer. It has to keep choosing the next useful thing.

The current arena is deliberately small. It is a red-versus-blue fighter duel inside a 200-block world border. The teams begin on opposite sides of a glass wall. There is a setup phase, a development phase, a battle phase, and an end state. When battle begins, the barrier disappears, the difficulty rises to hard, keep-inventory is disabled, and a death turns the agent into a spectator.

A recent verified run completed the full sequence: WAITING, SETUP, DEVELOPMENT, BATTLE, and END. Red won after red_2 eliminated blue_2.

The director sent zero direct attack/chase/sweep commands. The agents themselves generated combat actions, including !attackPlayer and equipment commands for stone and iron swords. That does not mean the system has solved autonomous combat. It means the win did not come from the director quietly puppeteering every movement. The agents had to progress through the loop themselves: gather, craft, equip, search, engage.

Making Agent Behavior Visible

A 1v1 duel sounds small, and it is. But it tests a combination that is harder than most screenshots make it look. You can see an agent spend too long mining. You can see it take the wrong route. You can see it reach an enemy with weak gear or return to a workbench when it should be fighting.

For research, that visibility matters. For teaching, it may matter even more.

Students can inspect the actual machinery of an agent instead of treating the model as a talking interface. They can read the goal, inspect the command trace, notice the phase clock, and connect an error in reasoning to something that happened in the world. The failure is concrete, thus can be quantified.

The format also works well as public media. People do not need a background in machine learning to understand “two AI agents had to prepare for battle before the wall came down.” Most benchmarks give you a number. Minecraft can give you a number and a replay.

From Duels to Team-Based Coordination

Obviously there are several limitations: Minecraft world is too noisy, model latency existsm, prompt wording still shapes behavior strongly…it is not a finished benchmark.

Still, that may be enough for now. A good benchmark reveals how AI behaves when it has to keep acting after the first answer. Minecraft gives us a world where that behavior is easy to observe, easy to explain, and hard to fake.

The next step is to make the agents more varied. A 3v3 match could introduce builders alongside fighters. Builders could gather materials and construct cover during development. Fighters could prepare weapons and armor. The question would become more about whether several agents can divide work and still react when something goes wrong.

Further ahead, the same architecture could support much richer forms of behavior if the underlying infrastructure improves. With sufficiently capable local models and hardware, we could expose more of Minecraft’s low-level state and actions to the agents. If visual observations were streamed into the loop and interpreted in real time, a builder might design its own defensive structures instead of relying on predefined templates. If agents gained access to lower-level action primitives, they could decide not only what to do, but how to do it. A fighter might choose when to crouch repeatedly as a form of taunt. At that point, the benchmark would be measuring something closer to embodied behavior, rather than the ability to select from a predefined menu of tools.

An agent that punches a tree, crafts a sword, finds an opponent, and loses has told us something. An agent that wins may have told us more. Either way, we can watch the chain of decisions unfold.