simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-2438
uvx harbor run -d simpleqa@1.0 -t simpleqa-2438
b8f0fa9
simpleqa-2439
uvx harbor run -d simpleqa@1.0 -t simpleqa-2439
b8f0fa9
simpleqa-244
uvx harbor run -d simpleqa@1.0 -t simpleqa-244
b8f0fa9
simpleqa-2440
uvx harbor run -d simpleqa@1.0 -t simpleqa-2440
b8f0fa9
simpleqa-2441
uvx harbor run -d simpleqa@1.0 -t simpleqa-2441
b8f0fa9
simpleqa-2442
uvx harbor run -d simpleqa@1.0 -t simpleqa-2442
b8f0fa9
simpleqa-2443
uvx harbor run -d simpleqa@1.0 -t simpleqa-2443
b8f0fa9
simpleqa-2444
uvx harbor run -d simpleqa@1.0 -t simpleqa-2444
b8f0fa9
simpleqa-2445
uvx harbor run -d simpleqa@1.0 -t simpleqa-2445
b8f0fa9
simpleqa-2446
uvx harbor run -d simpleqa@1.0 -t simpleqa-2446
b8f0fa9
simpleqa-2447
uvx harbor run -d simpleqa@1.0 -t simpleqa-2447
b8f0fa9
simpleqa-2448
uvx harbor run -d simpleqa@1.0 -t simpleqa-2448
b8f0fa9
simpleqa-2449
uvx harbor run -d simpleqa@1.0 -t simpleqa-2449
b8f0fa9
simpleqa-245
uvx harbor run -d simpleqa@1.0 -t simpleqa-245
b8f0fa9
simpleqa-2450
uvx harbor run -d simpleqa@1.0 -t simpleqa-2450
b8f0fa9
simpleqa-2451
uvx harbor run -d simpleqa@1.0 -t simpleqa-2451
b8f0fa9
simpleqa-2452
uvx harbor run -d simpleqa@1.0 -t simpleqa-2452
b8f0fa9
simpleqa-2453
uvx harbor run -d simpleqa@1.0 -t simpleqa-2453
b8f0fa9
simpleqa-2454
uvx harbor run -d simpleqa@1.0 -t simpleqa-2454
b8f0fa9
simpleqa-2455
uvx harbor run -d simpleqa@1.0 -t simpleqa-2455
b8f0fa9
simpleqa-2456
uvx harbor run -d simpleqa@1.0 -t simpleqa-2456
b8f0fa9
simpleqa-2457
uvx harbor run -d simpleqa@1.0 -t simpleqa-2457
b8f0fa9
simpleqa-2458
uvx harbor run -d simpleqa@1.0 -t simpleqa-2458
b8f0fa9
simpleqa-2459
uvx harbor run -d simpleqa@1.0 -t simpleqa-2459
b8f0fa9
simpleqa-246
uvx harbor run -d simpleqa@1.0 -t simpleqa-246
b8f0fa9
simpleqa-2460
uvx harbor run -d simpleqa@1.0 -t simpleqa-2460
b8f0fa9
simpleqa-2461
uvx harbor run -d simpleqa@1.0 -t simpleqa-2461
b8f0fa9
simpleqa-2462
uvx harbor run -d simpleqa@1.0 -t simpleqa-2462
b8f0fa9
simpleqa-2463
uvx harbor run -d simpleqa@1.0 -t simpleqa-2463
b8f0fa9
simpleqa-2464
uvx harbor run -d simpleqa@1.0 -t simpleqa-2464
b8f0fa9
simpleqa-2465
uvx harbor run -d simpleqa@1.0 -t simpleqa-2465
b8f0fa9
simpleqa-2466
uvx harbor run -d simpleqa@1.0 -t simpleqa-2466
b8f0fa9
simpleqa-2467
uvx harbor run -d simpleqa@1.0 -t simpleqa-2467
b8f0fa9
simpleqa-2468
uvx harbor run -d simpleqa@1.0 -t simpleqa-2468
b8f0fa9
simpleqa-2469
uvx harbor run -d simpleqa@1.0 -t simpleqa-2469
b8f0fa9
simpleqa-247
uvx harbor run -d simpleqa@1.0 -t simpleqa-247
b8f0fa9
simpleqa-2470
uvx harbor run -d simpleqa@1.0 -t simpleqa-2470
b8f0fa9
simpleqa-2471
uvx harbor run -d simpleqa@1.0 -t simpleqa-2471
b8f0fa9
simpleqa-2472
uvx harbor run -d simpleqa@1.0 -t simpleqa-2472
b8f0fa9
simpleqa-2473
uvx harbor run -d simpleqa@1.0 -t simpleqa-2473
b8f0fa9
simpleqa-2474
uvx harbor run -d simpleqa@1.0 -t simpleqa-2474
b8f0fa9
simpleqa-2475
uvx harbor run -d simpleqa@1.0 -t simpleqa-2475
b8f0fa9
simpleqa-2476
uvx harbor run -d simpleqa@1.0 -t simpleqa-2476
b8f0fa9
simpleqa-2477
uvx harbor run -d simpleqa@1.0 -t simpleqa-2477
b8f0fa9
simpleqa-2478
uvx harbor run -d simpleqa@1.0 -t simpleqa-2478
b8f0fa9
simpleqa-2479
uvx harbor run -d simpleqa@1.0 -t simpleqa-2479
b8f0fa9
simpleqa-248
uvx harbor run -d simpleqa@1.0 -t simpleqa-248
b8f0fa9
simpleqa-2480
uvx harbor run -d simpleqa@1.0 -t simpleqa-2480
b8f0fa9
simpleqa-2481
uvx harbor run -d simpleqa@1.0 -t simpleqa-2481
b8f0fa9
simpleqa-2482
uvx harbor run -d simpleqa@1.0 -t simpleqa-2482
b8f0fa9
simpleqa-2483
uvx harbor run -d simpleqa@1.0 -t simpleqa-2483
b8f0fa9
simpleqa-2484
uvx harbor run -d simpleqa@1.0 -t simpleqa-2484
b8f0fa9
simpleqa-2485
uvx harbor run -d simpleqa@1.0 -t simpleqa-2485
b8f0fa9
simpleqa-2486
uvx harbor run -d simpleqa@1.0 -t simpleqa-2486
b8f0fa9
simpleqa-2487
uvx harbor run -d simpleqa@1.0 -t simpleqa-2487
b8f0fa9
simpleqa-2488
uvx harbor run -d simpleqa@1.0 -t simpleqa-2488
b8f0fa9
simpleqa-2489
uvx harbor run -d simpleqa@1.0 -t simpleqa-2489
b8f0fa9
simpleqa-249
uvx harbor run -d simpleqa@1.0 -t simpleqa-249
b8f0fa9
simpleqa-2490
uvx harbor run -d simpleqa@1.0 -t simpleqa-2490
b8f0fa9
simpleqa-2491
uvx harbor run -d simpleqa@1.0 -t simpleqa-2491
b8f0fa9
simpleqa-2492
uvx harbor run -d simpleqa@1.0 -t simpleqa-2492
b8f0fa9
simpleqa-2493
uvx harbor run -d simpleqa@1.0 -t simpleqa-2493
b8f0fa9
simpleqa-2494
uvx harbor run -d simpleqa@1.0 -t simpleqa-2494
b8f0fa9
simpleqa-2495
uvx harbor run -d simpleqa@1.0 -t simpleqa-2495
b8f0fa9
simpleqa-2496
uvx harbor run -d simpleqa@1.0 -t simpleqa-2496
b8f0fa9
simpleqa-2497
uvx harbor run -d simpleqa@1.0 -t simpleqa-2497
b8f0fa9
simpleqa-2498
uvx harbor run -d simpleqa@1.0 -t simpleqa-2498
b8f0fa9
simpleqa-2499
uvx harbor run -d simpleqa@1.0 -t simpleqa-2499
b8f0fa9
simpleqa-25
uvx harbor run -d simpleqa@1.0 -t simpleqa-25
b8f0fa9
simpleqa-250
uvx harbor run -d simpleqa@1.0 -t simpleqa-250
b8f0fa9
simpleqa-2500
uvx harbor run -d simpleqa@1.0 -t simpleqa-2500
b8f0fa9
simpleqa-2501
uvx harbor run -d simpleqa@1.0 -t simpleqa-2501
b8f0fa9
simpleqa-2502
uvx harbor run -d simpleqa@1.0 -t simpleqa-2502
b8f0fa9
simpleqa-2503
uvx harbor run -d simpleqa@1.0 -t simpleqa-2503
b8f0fa9
simpleqa-2504
uvx harbor run -d simpleqa@1.0 -t simpleqa-2504
b8f0fa9
simpleqa-2505
uvx harbor run -d simpleqa@1.0 -t simpleqa-2505
b8f0fa9
simpleqa-2506
uvx harbor run -d simpleqa@1.0 -t simpleqa-2506
b8f0fa9
simpleqa-2507
uvx harbor run -d simpleqa@1.0 -t simpleqa-2507
b8f0fa9
simpleqa-2508
uvx harbor run -d simpleqa@1.0 -t simpleqa-2508
b8f0fa9
simpleqa-2509
uvx harbor run -d simpleqa@1.0 -t simpleqa-2509
b8f0fa9
simpleqa-251
uvx harbor run -d simpleqa@1.0 -t simpleqa-251
b8f0fa9
simpleqa-2510
uvx harbor run -d simpleqa@1.0 -t simpleqa-2510
b8f0fa9
simpleqa-2511
uvx harbor run -d simpleqa@1.0 -t simpleqa-2511
b8f0fa9
simpleqa-2512
uvx harbor run -d simpleqa@1.0 -t simpleqa-2512
b8f0fa9
simpleqa-2513
uvx harbor run -d simpleqa@1.0 -t simpleqa-2513
b8f0fa9
simpleqa-2514
uvx harbor run -d simpleqa@1.0 -t simpleqa-2514
b8f0fa9
simpleqa-2515
uvx harbor run -d simpleqa@1.0 -t simpleqa-2515
b8f0fa9
simpleqa-2516
uvx harbor run -d simpleqa@1.0 -t simpleqa-2516
b8f0fa9
simpleqa-2517
uvx harbor run -d simpleqa@1.0 -t simpleqa-2517
b8f0fa9
simpleqa-2518
uvx harbor run -d simpleqa@1.0 -t simpleqa-2518
b8f0fa9
simpleqa-2519
uvx harbor run -d simpleqa@1.0 -t simpleqa-2519
b8f0fa9
simpleqa-252
uvx harbor run -d simpleqa@1.0 -t simpleqa-252
b8f0fa9
simpleqa-2520
uvx harbor run -d simpleqa@1.0 -t simpleqa-2520
b8f0fa9
simpleqa-2521
uvx harbor run -d simpleqa@1.0 -t simpleqa-2521
b8f0fa9
simpleqa-2522
uvx harbor run -d simpleqa@1.0 -t simpleqa-2522
b8f0fa9
simpleqa-2523
uvx harbor run -d simpleqa@1.0 -t simpleqa-2523
b8f0fa9
simpleqa-2524
uvx harbor run -d simpleqa@1.0 -t simpleqa-2524
b8f0fa9
simpleqa-2525
uvx harbor run -d simpleqa@1.0 -t simpleqa-2525
b8f0fa9
simpleqa-2526
uvx harbor run -d simpleqa@1.0 -t simpleqa-2526
b8f0fa9
simpleqa-2527
uvx harbor run -d simpleqa@1.0 -t simpleqa-2527
b8f0fa9