simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-2348
uvx harbor run -d simpleqa@1.0 -t simpleqa-2348
b8f0fa9
simpleqa-2349
uvx harbor run -d simpleqa@1.0 -t simpleqa-2349
b8f0fa9
simpleqa-235
uvx harbor run -d simpleqa@1.0 -t simpleqa-235
b8f0fa9
simpleqa-2350
uvx harbor run -d simpleqa@1.0 -t simpleqa-2350
b8f0fa9
simpleqa-2351
uvx harbor run -d simpleqa@1.0 -t simpleqa-2351
b8f0fa9
simpleqa-2352
uvx harbor run -d simpleqa@1.0 -t simpleqa-2352
b8f0fa9
simpleqa-2353
uvx harbor run -d simpleqa@1.0 -t simpleqa-2353
b8f0fa9
simpleqa-2354
uvx harbor run -d simpleqa@1.0 -t simpleqa-2354
b8f0fa9
simpleqa-2355
uvx harbor run -d simpleqa@1.0 -t simpleqa-2355
b8f0fa9
simpleqa-2356
uvx harbor run -d simpleqa@1.0 -t simpleqa-2356
b8f0fa9
simpleqa-2357
uvx harbor run -d simpleqa@1.0 -t simpleqa-2357
b8f0fa9
simpleqa-2358
uvx harbor run -d simpleqa@1.0 -t simpleqa-2358
b8f0fa9
simpleqa-2359
uvx harbor run -d simpleqa@1.0 -t simpleqa-2359
b8f0fa9
simpleqa-236
uvx harbor run -d simpleqa@1.0 -t simpleqa-236
b8f0fa9
simpleqa-2360
uvx harbor run -d simpleqa@1.0 -t simpleqa-2360
b8f0fa9
simpleqa-2361
uvx harbor run -d simpleqa@1.0 -t simpleqa-2361
b8f0fa9
simpleqa-2362
uvx harbor run -d simpleqa@1.0 -t simpleqa-2362
b8f0fa9
simpleqa-2363
uvx harbor run -d simpleqa@1.0 -t simpleqa-2363
b8f0fa9
simpleqa-2364
uvx harbor run -d simpleqa@1.0 -t simpleqa-2364
b8f0fa9
simpleqa-2365
uvx harbor run -d simpleqa@1.0 -t simpleqa-2365
b8f0fa9
simpleqa-2366
uvx harbor run -d simpleqa@1.0 -t simpleqa-2366
b8f0fa9
simpleqa-2367
uvx harbor run -d simpleqa@1.0 -t simpleqa-2367
b8f0fa9
simpleqa-2368
uvx harbor run -d simpleqa@1.0 -t simpleqa-2368
b8f0fa9
simpleqa-2369
uvx harbor run -d simpleqa@1.0 -t simpleqa-2369
b8f0fa9
simpleqa-237
uvx harbor run -d simpleqa@1.0 -t simpleqa-237
b8f0fa9
simpleqa-2370
uvx harbor run -d simpleqa@1.0 -t simpleqa-2370
b8f0fa9
simpleqa-2371
uvx harbor run -d simpleqa@1.0 -t simpleqa-2371
b8f0fa9
simpleqa-2372
uvx harbor run -d simpleqa@1.0 -t simpleqa-2372
b8f0fa9
simpleqa-2373
uvx harbor run -d simpleqa@1.0 -t simpleqa-2373
b8f0fa9
simpleqa-2374
uvx harbor run -d simpleqa@1.0 -t simpleqa-2374
b8f0fa9
simpleqa-2375
uvx harbor run -d simpleqa@1.0 -t simpleqa-2375
b8f0fa9
simpleqa-2376
uvx harbor run -d simpleqa@1.0 -t simpleqa-2376
b8f0fa9
simpleqa-2377
uvx harbor run -d simpleqa@1.0 -t simpleqa-2377
b8f0fa9
simpleqa-2378
uvx harbor run -d simpleqa@1.0 -t simpleqa-2378
b8f0fa9
simpleqa-2379
uvx harbor run -d simpleqa@1.0 -t simpleqa-2379
b8f0fa9
simpleqa-238
uvx harbor run -d simpleqa@1.0 -t simpleqa-238
b8f0fa9
simpleqa-2380
uvx harbor run -d simpleqa@1.0 -t simpleqa-2380
b8f0fa9
simpleqa-2381
uvx harbor run -d simpleqa@1.0 -t simpleqa-2381
b8f0fa9
simpleqa-2382
uvx harbor run -d simpleqa@1.0 -t simpleqa-2382
b8f0fa9
simpleqa-2383
uvx harbor run -d simpleqa@1.0 -t simpleqa-2383
b8f0fa9
simpleqa-2384
uvx harbor run -d simpleqa@1.0 -t simpleqa-2384
b8f0fa9
simpleqa-2385
uvx harbor run -d simpleqa@1.0 -t simpleqa-2385
b8f0fa9
simpleqa-2386
uvx harbor run -d simpleqa@1.0 -t simpleqa-2386
b8f0fa9
simpleqa-2387
uvx harbor run -d simpleqa@1.0 -t simpleqa-2387
b8f0fa9
simpleqa-2388
uvx harbor run -d simpleqa@1.0 -t simpleqa-2388
b8f0fa9
simpleqa-2389
uvx harbor run -d simpleqa@1.0 -t simpleqa-2389
b8f0fa9
simpleqa-239
uvx harbor run -d simpleqa@1.0 -t simpleqa-239
b8f0fa9
simpleqa-2390
uvx harbor run -d simpleqa@1.0 -t simpleqa-2390
b8f0fa9
simpleqa-2391
uvx harbor run -d simpleqa@1.0 -t simpleqa-2391
b8f0fa9
simpleqa-2392
uvx harbor run -d simpleqa@1.0 -t simpleqa-2392
b8f0fa9
simpleqa-2393
uvx harbor run -d simpleqa@1.0 -t simpleqa-2393
b8f0fa9
simpleqa-2394
uvx harbor run -d simpleqa@1.0 -t simpleqa-2394
b8f0fa9
simpleqa-2395
uvx harbor run -d simpleqa@1.0 -t simpleqa-2395
b8f0fa9
simpleqa-2396
uvx harbor run -d simpleqa@1.0 -t simpleqa-2396
b8f0fa9
simpleqa-2397
uvx harbor run -d simpleqa@1.0 -t simpleqa-2397
b8f0fa9
simpleqa-2398
uvx harbor run -d simpleqa@1.0 -t simpleqa-2398
b8f0fa9
simpleqa-2399
uvx harbor run -d simpleqa@1.0 -t simpleqa-2399
b8f0fa9
simpleqa-24
uvx harbor run -d simpleqa@1.0 -t simpleqa-24
b8f0fa9
simpleqa-240
uvx harbor run -d simpleqa@1.0 -t simpleqa-240
b8f0fa9
simpleqa-2400
uvx harbor run -d simpleqa@1.0 -t simpleqa-2400
b8f0fa9
simpleqa-2401
uvx harbor run -d simpleqa@1.0 -t simpleqa-2401
b8f0fa9
simpleqa-2402
uvx harbor run -d simpleqa@1.0 -t simpleqa-2402
b8f0fa9
simpleqa-2403
uvx harbor run -d simpleqa@1.0 -t simpleqa-2403
b8f0fa9
simpleqa-2404
uvx harbor run -d simpleqa@1.0 -t simpleqa-2404
b8f0fa9
simpleqa-2405
uvx harbor run -d simpleqa@1.0 -t simpleqa-2405
b8f0fa9
simpleqa-2406
uvx harbor run -d simpleqa@1.0 -t simpleqa-2406
b8f0fa9
simpleqa-2407
uvx harbor run -d simpleqa@1.0 -t simpleqa-2407
b8f0fa9
simpleqa-2408
uvx harbor run -d simpleqa@1.0 -t simpleqa-2408
b8f0fa9
simpleqa-2409
uvx harbor run -d simpleqa@1.0 -t simpleqa-2409
b8f0fa9
simpleqa-241
uvx harbor run -d simpleqa@1.0 -t simpleqa-241
b8f0fa9
simpleqa-2410
uvx harbor run -d simpleqa@1.0 -t simpleqa-2410
b8f0fa9
simpleqa-2411
uvx harbor run -d simpleqa@1.0 -t simpleqa-2411
b8f0fa9
simpleqa-2412
uvx harbor run -d simpleqa@1.0 -t simpleqa-2412
b8f0fa9
simpleqa-2413
uvx harbor run -d simpleqa@1.0 -t simpleqa-2413
b8f0fa9
simpleqa-2414
uvx harbor run -d simpleqa@1.0 -t simpleqa-2414
b8f0fa9
simpleqa-2415
uvx harbor run -d simpleqa@1.0 -t simpleqa-2415
b8f0fa9
simpleqa-2416
uvx harbor run -d simpleqa@1.0 -t simpleqa-2416
b8f0fa9
simpleqa-2417
uvx harbor run -d simpleqa@1.0 -t simpleqa-2417
b8f0fa9
simpleqa-2418
uvx harbor run -d simpleqa@1.0 -t simpleqa-2418
b8f0fa9
simpleqa-2419
uvx harbor run -d simpleqa@1.0 -t simpleqa-2419
b8f0fa9
simpleqa-242
uvx harbor run -d simpleqa@1.0 -t simpleqa-242
b8f0fa9
simpleqa-2420
uvx harbor run -d simpleqa@1.0 -t simpleqa-2420
b8f0fa9
simpleqa-2421
uvx harbor run -d simpleqa@1.0 -t simpleqa-2421
b8f0fa9
simpleqa-2422
uvx harbor run -d simpleqa@1.0 -t simpleqa-2422
b8f0fa9
simpleqa-2423
uvx harbor run -d simpleqa@1.0 -t simpleqa-2423
b8f0fa9
simpleqa-2424
uvx harbor run -d simpleqa@1.0 -t simpleqa-2424
b8f0fa9
simpleqa-2425
uvx harbor run -d simpleqa@1.0 -t simpleqa-2425
b8f0fa9
simpleqa-2426
uvx harbor run -d simpleqa@1.0 -t simpleqa-2426
b8f0fa9
simpleqa-2427
uvx harbor run -d simpleqa@1.0 -t simpleqa-2427
b8f0fa9
simpleqa-2428
uvx harbor run -d simpleqa@1.0 -t simpleqa-2428
b8f0fa9
simpleqa-2429
uvx harbor run -d simpleqa@1.0 -t simpleqa-2429
b8f0fa9
simpleqa-243
uvx harbor run -d simpleqa@1.0 -t simpleqa-243
b8f0fa9
simpleqa-2430
uvx harbor run -d simpleqa@1.0 -t simpleqa-2430
b8f0fa9
simpleqa-2431
uvx harbor run -d simpleqa@1.0 -t simpleqa-2431
b8f0fa9
simpleqa-2432
uvx harbor run -d simpleqa@1.0 -t simpleqa-2432
b8f0fa9
simpleqa-2433
uvx harbor run -d simpleqa@1.0 -t simpleqa-2433
b8f0fa9
simpleqa-2434
uvx harbor run -d simpleqa@1.0 -t simpleqa-2434
b8f0fa9
simpleqa-2435
uvx harbor run -d simpleqa@1.0 -t simpleqa-2435
b8f0fa9
simpleqa-2436
uvx harbor run -d simpleqa@1.0 -t simpleqa-2436
b8f0fa9
simpleqa-2437
uvx harbor run -d simpleqa@1.0 -t simpleqa-2437
b8f0fa9