pi-arena.
Open source command line tool·2026·Published on npm
A terminal tool that runs the same task on several AI models at once, puts the answers side by side and has a separate judge model score them without seeing which is which.
What was there
Working out which model does a job better meant typing the same prompt into each one and comparing by hand. Tiring, and biased.
Whoever compared the answers already knew which model wrote which, so the judgement was skewed from the start.
How I built it
Contestants run in parallel, each in its own process. If one stalls, the others do not wait.
The judge model sees the answers shuffled and unnamed; the winner is revealed only after scoring.
The model list is not hardcoded, it is read from what is installed, so a new model shows up without updating the tool.
Finished processes used to linger and eat the CPU; process handling was rewritten to kill the whole tree and is now covered by a test.
What I did
- Design of the race, judging and report flow.
- Terminal interface: live progress, a result browser and a model picker.
- Cost table showing tokens and spend per contestant.
- Tests for process handling and cross-platform shutdown behaviour.
- Publishing it as an npm package and preparing the gallery images.
Project details
- Client
- My own project, open to everyone
- My part
- Design, development, tests, release
- When
- Built in 2026
- Tools I used
- TypeScript · Node.js · Pi extension API · Terminal arayüzü
I can do the same for you.
If you need something close to this project, write and we will look at it together.
Let's talk →