Ali Akpoyraz
← Work

pi-arena.

Open source command line tool·2026·Published on npm

A terminal tool that runs the same task on several AI models at once, puts the answers side by side and has a separate judge model score them without seeing which is which.

Visit the site ↗·Source code ↗

What was there

Working out which model does a job better meant typing the same prompt into each one and comparing by hand. Tiring, and biased.

Whoever compared the answers already knew which model wrote which, so the judgement was skewed from the start.

How I built it

Contestants run in parallel, each in its own process. If one stalls, the others do not wait.

The judge model sees the answers shuffled and unnamed; the winner is revealed only after scoring.

The model list is not hardcoded, it is read from what is installed, so a new model shows up without updating the tool.

Finished processes used to linger and eat the CPU; process handling was rewritten to kill the whole tree and is now covered by a test.

What I did

  • Design of the race, judging and report flow.
  • Terminal interface: live progress, a result browser and a model picker.
  • Cost table showing tokens and spend per contestant.
  • Tests for process handling and cross-platform shutdown behaviour.
  • Publishing it as an npm package and preparing the gallery images.

Project details

Client
My own project, open to everyone
My part
Design, development, tests, release
When
Built in 2026
Tools I used
TypeScript · Node.js · Pi extension API · Terminal arayüzü
Next projectKalfa →

I can do the same for you.

If you need something close to this project, write and we will look at it together.

Let's talk →