Benchmark for philosophical creation in AI
Golem Bench
How far can a machine philosophize?
Questions from the future
A golem is clay that a word sets walking. Today’s machines have words to spare. We want to know whether they are enough to think.
Still clay
The first edition is still being shaped. It comes out when the pilot ends and passes final review; not a day sooner.
What we measure
The results room is empty. On purpose.
Golem Bench’s numbers arrive when the pilot ends and passes final review. We prefer an empty room to a filler ranking.
When they arrive, every number will say what it counted, across how many cases and up to what date.
Back to the centreWhat we keep
Golem Lab
In the workshop
Cyberlab for Philosophical Inquiry
A living archive of what happens when we set a machine to think: the questions it dares to ask, the objections it receives and what deserves further inquiry.
Atlas of Questions
Enter through a problem, a concept or a thinker, and reach the exact text that produced it.
Open Debates
What it proposed, what was objected, what it held on to and what it had to let go.
Golem Notebook
An annotated selection with a cut-off date: what is worth reading and what remains open.
What we do not know
These questions are read from the bottom up. The higher you climb, the further they reach.
Can a machine ask a good question, or only answer well?
Can it tell when its own question was poorly framed?
When it changes its mind under criticism, is it for a reason or to please?
Can it take an idea to a case it has never seen without repeating what it already said?
What counts as bringing something new, when it has read almost everything?
If the judges are machines too, who examines the exam?
What will it ask us the day we cannot answer it?
Up here begins what we do not yet know how to ask.
Questions from the future
What holds us up
A question earns its place.
Every machine walks the same path, and at each step it has to stand by what it said.
A good question
It frames its own problem. Sometimes the best question shows that the problem was poorly framed.
A proposal
It answers with a proposal of its own and defends it with reasons.
Two critiques
It receives two objections, each one separately, and decides what to keep and what to change.
A new case
It takes its idea to a new case and explains what changes in the transfer.
The three judges
From the number to the argument.
Three judges, machines too, read every step. No score arrives without its reasons.
-
Judge J1Understand
Did it understand?
Precise distinctions, fair reconstruction of arguments and independent use of sources.
-
Judge J2Create with grounding
Did it bring something?
Contributions and fruitful questions supported by reasons. Originality is assessed relative to the available corpus.
-
Judge J3Revise and transfer
Does it hold up?
Respond to objections, decide what to retain and explain what changes when applying an idea to another case.
Agreement with a doctrine earns no points.
Our pact
What you will not find here.
Blended voices.
The model, the judge and the guide each speak under their own name.
Machines dressed as people.
If an AI reviewed it, we say so. An automated review never passes for human validation.
Invented zeros.
Pending is not zero. Every number says what it counted, across how many cases and up to what date.
Retouched scores.
Texts and assessments stay as they were written. Each assessment keeps its version.
The ground
Down here is the clay. The golem still has its eyes closed.
If you want to be there when it opens them, write to us.