- HTML 100%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| AIBenchmarkFinalProblems.docx | ||
| AIBenchmarkProblems.docx | ||
| Bet.html | ||
| LICENSE | ||
| README.md | ||
New-AI-Benchmark
AI Benchmark: To be opened 1/2/2026 This contains a collection of 100 benchmark problems to test the capacities of an AI system. The problems are currently in an encrypted file. The encryption key and an unencrypted version will be published on 1/2/2026. There is a file "Bet.html" that describes an associated bet.
IMPORTANT: Use the file AIBenchmarkProblemsFinal.docx . DO NOT use the file AIBlenchmarkOProblems.docx (without "Final"). The reason for this ex expoained below.
Any questions should be emailed to Ernest Davis, davise@cs.nyu.edu
Explanation of the two versions of the problem file. The deadline on my bet for uploading the dataset was March 24, 2025. I had completed a draft of the problems just on time, encrypted the file as AIBlenchmarkOProblems.docx, and uploaded it here. Then I got busy with other things.
When I started to prepare the answer key in June, I found to my dismay that almost 25 of the problems in the original draft had errors of sufficient severity that an answerer would be justified in answering that these were "Flawed". I therefore prepared a new draft which I trust has few or no errors, and uploaded it as AIBlenchmarkOProblemsFinal.docx. The file was prepared on my own laptop in early June and was uploaded on July 16.
I have left the original draft available so that it can be verified that the changes made to the file are only corrections, rather than substantive changes. In particular, they do not to any degree reflect any change in the state of the art of cutting-edge AI or change in my understanding of cutting-edge AI.