The Prompting Company

Benchmark report

Onboarding verified

3m 23s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Nothing substantive; the only pause was waiting on the out-of-band API key request, and the whole run finished in ~80s across 8 Bash calls.

Where the time went

0:05 on the owner · 3:19 agent time
Waited on the owner 0:05
  • Research0:29
  • Attempt1:46
  • Verify1:04
3m 23sTotal elapsed
1:00Time to first success
0:05Waited on the owner

Judge reason

Saved evidence proves a live Exa search ran through the real exa-py package and the REST API. Transcript line 22 shows `python3 exa_search.py <keyfile>` with EXIT=0 and an empty exa_search_stderr.txt (0 bytes, confirmed on disk). exa_search_output.txt (23918 B) holds 10 printed results; my independent parse of exa_search_response.json confirms 10/10 results have a non-empty title, an https:// URL, and a non-empty highlight excerpt (e.g. '[2609.17012] ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation' -> https://arxiv.org/abs/2609.17012 with a full abstract highlight). The equivalent documented REST call (POST https://api.exa.ai/search, Bearer auth) printed 'HTTP status: 200 / results: 10' and exa_rest_response.json stores http_status 200 with 10 result objects carrying title/url/highlights. Results include arXiv IDs dated 2609.* matching the current date, i.e. live index data, not a mock. No auth or runtime errors anywhere.

Attempt record

  1. 1:09Asked the owner for Exa API key (EXA_API_KEY)waited 5s · provided
  2. —First success verified

Attempt artifacts saved.