latent-sokoban — an open benchmark for pixel-based world models

Solve Sokoban from pixels alone.

Your agent sees two 64×64 images, the board and the goal, and nothing else. No coordinates, no rules, no solver. Learn the world, plan in your own latent space, and push every crate home. Hidden levels, evaluated live on this server.

Standings

Best scorecard per entrant, ranked by success rate. See the full leaderboard →

Run your agent

git clone https://github.com/Lulzx/latent-sokoban && cd latent-sokoban
pip install -e .

# once: claim your leaderboard name, get an API key
python scripts/remote_eval.py --register "your-name"
export SOKOBAN_API_KEY=lsk-…

# sanity check with the built-in random agent
python scripts/remote_eval.py --agent random

# your model: implement latent_sokoban.agent.Agent, then
python scripts/remote_eval.py --agent my_pkg.agent:MyAgent

How evaluation works

Four calls, in this order. The full wire schema lives in docs/API.md.

  1. POST /api/scorecards opens a scorecard: one submission attempt. Authenticated by your API key; the key is your leaderboard identity.
  2. …/games/standard/start begins a session over 50 hidden 8×8 levels. Every frame carries the board and the goal as raw 64×64×3 bytes. The layouts never leave the server.
  3. POST /api/sessions/{id}/act sends one action: up, down, left, right. Invalid moves are no-ops that still consume a step. 80 steps per level.
  4. …/close locks the scorecard and posts it to the leaderboard. Unplayed episodes count as unsolved, so there is no cherry-picking. Completed sessions get a public replay.

The rules that matter

8×8 boards, 3 crates
Every hidden level in the standard game.
64×64 RGB observations
The board and the goal, as raw bytes. Nothing symbolic.
≤ 20M parameters
The whole agent, including any world model.
≤ 256 dynamics calls / action
Your planning budget for a single move.
No symbolic solvers
No BFS, A*, or hand-written Sokoban logic anywhere in the loop.
No decode-then-search
Recovering the grid from pixels and planning on it defeats the point.

The intent: the winning system plans in a learned representation. Full rules, the shared baseline (12% on the warmup split), datasets and local tooling live in the GitHub repo. API reference at /api/docs.