Sovereign 8s.is mirror of standardgalactic/New-AI-Benchmark
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2025-07-16 15:39:04 -04:00
AIBenchmarkFinalProblems.docx Add files via upload 2025-07-16 15:24:55 -04:00
AIBenchmarkProblems.docx Add files via upload 2025-03-24 14:31:42 -04:00
Bet.html Update Bet.txt 2025-03-24 14:46:36 -04:00
LICENSE Initial commit 2025-03-24 14:27:12 -04:00
README.md Update README.md 2025-07-16 15:39:04 -04:00

New-AI-Benchmark

AI Benchmark: To be opened 1/2/2026 This contains a collection of 100 benchmark problems to test the capacities of an AI system. The problems are currently in an encrypted file. The encryption key and an unencrypted version will be published on 1/2/2026. There is a file "Bet.html" that describes an associated bet.

IMPORTANT: Use the file AIBenchmarkProblemsFinal.docx . DO NOT use the file AIBlenchmarkOProblems.docx (without "Final"). The reason for this ex expoained below.

Any questions should be emailed to Ernest Davis, davise@cs.nyu.edu

Explanation of the two versions of the problem file. The deadline on my bet for uploading the dataset was March 24, 2025. I had completed a draft of the problems just on time, encrypted the file as AIBlenchmarkOProblems.docx, and uploaded it here. Then I got busy with other things.

When I started to prepare the answer key in June, I found to my dismay that almost 25 of the problems in the original draft had errors of sufficient severity that an answerer would be justified in answering that these were "Flawed". I therefore prepared a new draft which I trust has few or no errors, and uploaded it as AIBlenchmarkOProblemsFinal.docx. The file was prepared on my own laptop in early June and was uploaded on July 16.

I have left the original draft available so that it can be verified that the changes made to the file are only corrections, rather than substantive changes. In particular, they do not to any degree reflect any change in the state of the art of cutting-edge AI or change in my understanding of cutting-edge AI.