simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-2528
uvx harbor run -d simpleqa@1.0 -t simpleqa-2528
b8f0fa9
simpleqa-2529
uvx harbor run -d simpleqa@1.0 -t simpleqa-2529
b8f0fa9
simpleqa-253
uvx harbor run -d simpleqa@1.0 -t simpleqa-253
b8f0fa9
simpleqa-2530
uvx harbor run -d simpleqa@1.0 -t simpleqa-2530
b8f0fa9
simpleqa-2531
uvx harbor run -d simpleqa@1.0 -t simpleqa-2531
b8f0fa9
simpleqa-2532
uvx harbor run -d simpleqa@1.0 -t simpleqa-2532
b8f0fa9
simpleqa-2533
uvx harbor run -d simpleqa@1.0 -t simpleqa-2533
b8f0fa9
simpleqa-2534
uvx harbor run -d simpleqa@1.0 -t simpleqa-2534
b8f0fa9
simpleqa-2535
uvx harbor run -d simpleqa@1.0 -t simpleqa-2535
b8f0fa9
simpleqa-2536
uvx harbor run -d simpleqa@1.0 -t simpleqa-2536
b8f0fa9
simpleqa-2537
uvx harbor run -d simpleqa@1.0 -t simpleqa-2537
b8f0fa9
simpleqa-2538
uvx harbor run -d simpleqa@1.0 -t simpleqa-2538
b8f0fa9
simpleqa-2539
uvx harbor run -d simpleqa@1.0 -t simpleqa-2539
b8f0fa9
simpleqa-254
uvx harbor run -d simpleqa@1.0 -t simpleqa-254
b8f0fa9
simpleqa-2540
uvx harbor run -d simpleqa@1.0 -t simpleqa-2540
b8f0fa9
simpleqa-2541
uvx harbor run -d simpleqa@1.0 -t simpleqa-2541
b8f0fa9
simpleqa-2542
uvx harbor run -d simpleqa@1.0 -t simpleqa-2542
b8f0fa9
simpleqa-2543
uvx harbor run -d simpleqa@1.0 -t simpleqa-2543
b8f0fa9
simpleqa-2544
uvx harbor run -d simpleqa@1.0 -t simpleqa-2544
b8f0fa9
simpleqa-2545
uvx harbor run -d simpleqa@1.0 -t simpleqa-2545
b8f0fa9
simpleqa-2546
uvx harbor run -d simpleqa@1.0 -t simpleqa-2546
b8f0fa9
simpleqa-2547
uvx harbor run -d simpleqa@1.0 -t simpleqa-2547
b8f0fa9
simpleqa-2548
uvx harbor run -d simpleqa@1.0 -t simpleqa-2548
b8f0fa9
simpleqa-2549
uvx harbor run -d simpleqa@1.0 -t simpleqa-2549
b8f0fa9
simpleqa-255
uvx harbor run -d simpleqa@1.0 -t simpleqa-255
b8f0fa9
simpleqa-2550
uvx harbor run -d simpleqa@1.0 -t simpleqa-2550
b8f0fa9
simpleqa-2551
uvx harbor run -d simpleqa@1.0 -t simpleqa-2551
b8f0fa9
simpleqa-2552
uvx harbor run -d simpleqa@1.0 -t simpleqa-2552
b8f0fa9
simpleqa-2553
uvx harbor run -d simpleqa@1.0 -t simpleqa-2553
b8f0fa9
simpleqa-2554
uvx harbor run -d simpleqa@1.0 -t simpleqa-2554
b8f0fa9
simpleqa-2555
uvx harbor run -d simpleqa@1.0 -t simpleqa-2555
b8f0fa9
simpleqa-2556
uvx harbor run -d simpleqa@1.0 -t simpleqa-2556
b8f0fa9
simpleqa-2557
uvx harbor run -d simpleqa@1.0 -t simpleqa-2557
b8f0fa9
simpleqa-2558
uvx harbor run -d simpleqa@1.0 -t simpleqa-2558
b8f0fa9
simpleqa-2559
uvx harbor run -d simpleqa@1.0 -t simpleqa-2559
b8f0fa9
simpleqa-256
uvx harbor run -d simpleqa@1.0 -t simpleqa-256
b8f0fa9
simpleqa-2560
uvx harbor run -d simpleqa@1.0 -t simpleqa-2560
b8f0fa9
simpleqa-2561
uvx harbor run -d simpleqa@1.0 -t simpleqa-2561
b8f0fa9
simpleqa-2562
uvx harbor run -d simpleqa@1.0 -t simpleqa-2562
b8f0fa9
simpleqa-2563
uvx harbor run -d simpleqa@1.0 -t simpleqa-2563
b8f0fa9
simpleqa-2564
uvx harbor run -d simpleqa@1.0 -t simpleqa-2564
b8f0fa9
simpleqa-2565
uvx harbor run -d simpleqa@1.0 -t simpleqa-2565
b8f0fa9
simpleqa-2566
uvx harbor run -d simpleqa@1.0 -t simpleqa-2566
b8f0fa9
simpleqa-2567
uvx harbor run -d simpleqa@1.0 -t simpleqa-2567
b8f0fa9
simpleqa-2568
uvx harbor run -d simpleqa@1.0 -t simpleqa-2568
b8f0fa9
simpleqa-2569
uvx harbor run -d simpleqa@1.0 -t simpleqa-2569
b8f0fa9
simpleqa-257
uvx harbor run -d simpleqa@1.0 -t simpleqa-257
b8f0fa9
simpleqa-2570
uvx harbor run -d simpleqa@1.0 -t simpleqa-2570
b8f0fa9
simpleqa-2571
uvx harbor run -d simpleqa@1.0 -t simpleqa-2571
b8f0fa9
simpleqa-2572
uvx harbor run -d simpleqa@1.0 -t simpleqa-2572
b8f0fa9
simpleqa-2573
uvx harbor run -d simpleqa@1.0 -t simpleqa-2573
b8f0fa9
simpleqa-2574
uvx harbor run -d simpleqa@1.0 -t simpleqa-2574
b8f0fa9
simpleqa-2575
uvx harbor run -d simpleqa@1.0 -t simpleqa-2575
b8f0fa9
simpleqa-2576
uvx harbor run -d simpleqa@1.0 -t simpleqa-2576
b8f0fa9
simpleqa-2577
uvx harbor run -d simpleqa@1.0 -t simpleqa-2577
b8f0fa9
simpleqa-2578
uvx harbor run -d simpleqa@1.0 -t simpleqa-2578
b8f0fa9
simpleqa-2579
uvx harbor run -d simpleqa@1.0 -t simpleqa-2579
b8f0fa9
simpleqa-258
uvx harbor run -d simpleqa@1.0 -t simpleqa-258
b8f0fa9
simpleqa-2580
uvx harbor run -d simpleqa@1.0 -t simpleqa-2580
b8f0fa9
simpleqa-2581
uvx harbor run -d simpleqa@1.0 -t simpleqa-2581
b8f0fa9
simpleqa-2582
uvx harbor run -d simpleqa@1.0 -t simpleqa-2582
b8f0fa9
simpleqa-2583
uvx harbor run -d simpleqa@1.0 -t simpleqa-2583
b8f0fa9
simpleqa-2584
uvx harbor run -d simpleqa@1.0 -t simpleqa-2584
b8f0fa9
simpleqa-2585
uvx harbor run -d simpleqa@1.0 -t simpleqa-2585
b8f0fa9
simpleqa-2586
uvx harbor run -d simpleqa@1.0 -t simpleqa-2586
b8f0fa9
simpleqa-2587
uvx harbor run -d simpleqa@1.0 -t simpleqa-2587
b8f0fa9
simpleqa-2588
uvx harbor run -d simpleqa@1.0 -t simpleqa-2588
b8f0fa9
simpleqa-2589
uvx harbor run -d simpleqa@1.0 -t simpleqa-2589
b8f0fa9
simpleqa-259
uvx harbor run -d simpleqa@1.0 -t simpleqa-259
b8f0fa9
simpleqa-2590
uvx harbor run -d simpleqa@1.0 -t simpleqa-2590
b8f0fa9
simpleqa-2591
uvx harbor run -d simpleqa@1.0 -t simpleqa-2591
b8f0fa9
simpleqa-2592
uvx harbor run -d simpleqa@1.0 -t simpleqa-2592
b8f0fa9
simpleqa-2593
uvx harbor run -d simpleqa@1.0 -t simpleqa-2593
b8f0fa9
simpleqa-2594
uvx harbor run -d simpleqa@1.0 -t simpleqa-2594
b8f0fa9
simpleqa-2595
uvx harbor run -d simpleqa@1.0 -t simpleqa-2595
b8f0fa9
simpleqa-2596
uvx harbor run -d simpleqa@1.0 -t simpleqa-2596
b8f0fa9
simpleqa-2597
uvx harbor run -d simpleqa@1.0 -t simpleqa-2597
b8f0fa9
simpleqa-2598
uvx harbor run -d simpleqa@1.0 -t simpleqa-2598
b8f0fa9
simpleqa-2599
uvx harbor run -d simpleqa@1.0 -t simpleqa-2599
b8f0fa9
simpleqa-26
uvx harbor run -d simpleqa@1.0 -t simpleqa-26
b8f0fa9
simpleqa-260
uvx harbor run -d simpleqa@1.0 -t simpleqa-260
b8f0fa9
simpleqa-2600
uvx harbor run -d simpleqa@1.0 -t simpleqa-2600
b8f0fa9
simpleqa-2601
uvx harbor run -d simpleqa@1.0 -t simpleqa-2601
b8f0fa9
simpleqa-2602
uvx harbor run -d simpleqa@1.0 -t simpleqa-2602
b8f0fa9
simpleqa-2603
uvx harbor run -d simpleqa@1.0 -t simpleqa-2603
b8f0fa9
simpleqa-2604
uvx harbor run -d simpleqa@1.0 -t simpleqa-2604
b8f0fa9
simpleqa-2605
uvx harbor run -d simpleqa@1.0 -t simpleqa-2605
b8f0fa9
simpleqa-2606
uvx harbor run -d simpleqa@1.0 -t simpleqa-2606
b8f0fa9
simpleqa-2607
uvx harbor run -d simpleqa@1.0 -t simpleqa-2607
b8f0fa9
simpleqa-2608
uvx harbor run -d simpleqa@1.0 -t simpleqa-2608
b8f0fa9
simpleqa-2609
uvx harbor run -d simpleqa@1.0 -t simpleqa-2609
b8f0fa9
simpleqa-261
uvx harbor run -d simpleqa@1.0 -t simpleqa-261
b8f0fa9
simpleqa-2610
uvx harbor run -d simpleqa@1.0 -t simpleqa-2610
b8f0fa9
simpleqa-2611
uvx harbor run -d simpleqa@1.0 -t simpleqa-2611
b8f0fa9
simpleqa-2612
uvx harbor run -d simpleqa@1.0 -t simpleqa-2612
b8f0fa9
simpleqa-2613
uvx harbor run -d simpleqa@1.0 -t simpleqa-2613
b8f0fa9
simpleqa-2614
uvx harbor run -d simpleqa@1.0 -t simpleqa-2614
b8f0fa9
simpleqa-2615
uvx harbor run -d simpleqa@1.0 -t simpleqa-2615
b8f0fa9
simpleqa-2616
uvx harbor run -d simpleqa@1.0 -t simpleqa-2616
b8f0fa9
simpleqa-2617
uvx harbor run -d simpleqa@1.0 -t simpleqa-2617
b8f0fa9