simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-2618
uvx harbor run -d simpleqa@1.0 -t simpleqa-2618
b8f0fa9
simpleqa-2619
uvx harbor run -d simpleqa@1.0 -t simpleqa-2619
b8f0fa9
simpleqa-262
uvx harbor run -d simpleqa@1.0 -t simpleqa-262
b8f0fa9
simpleqa-2620
uvx harbor run -d simpleqa@1.0 -t simpleqa-2620
b8f0fa9
simpleqa-2621
uvx harbor run -d simpleqa@1.0 -t simpleqa-2621
b8f0fa9
simpleqa-2622
uvx harbor run -d simpleqa@1.0 -t simpleqa-2622
b8f0fa9
simpleqa-2623
uvx harbor run -d simpleqa@1.0 -t simpleqa-2623
b8f0fa9
simpleqa-2624
uvx harbor run -d simpleqa@1.0 -t simpleqa-2624
b8f0fa9
simpleqa-2625
uvx harbor run -d simpleqa@1.0 -t simpleqa-2625
b8f0fa9
simpleqa-2626
uvx harbor run -d simpleqa@1.0 -t simpleqa-2626
b8f0fa9
simpleqa-2627
uvx harbor run -d simpleqa@1.0 -t simpleqa-2627
b8f0fa9
simpleqa-2628
uvx harbor run -d simpleqa@1.0 -t simpleqa-2628
b8f0fa9
simpleqa-2629
uvx harbor run -d simpleqa@1.0 -t simpleqa-2629
b8f0fa9
simpleqa-263
uvx harbor run -d simpleqa@1.0 -t simpleqa-263
b8f0fa9
simpleqa-2630
uvx harbor run -d simpleqa@1.0 -t simpleqa-2630
b8f0fa9
simpleqa-2631
uvx harbor run -d simpleqa@1.0 -t simpleqa-2631
b8f0fa9
simpleqa-2632
uvx harbor run -d simpleqa@1.0 -t simpleqa-2632
b8f0fa9
simpleqa-2633
uvx harbor run -d simpleqa@1.0 -t simpleqa-2633
b8f0fa9
simpleqa-2634
uvx harbor run -d simpleqa@1.0 -t simpleqa-2634
b8f0fa9
simpleqa-2635
uvx harbor run -d simpleqa@1.0 -t simpleqa-2635
b8f0fa9
simpleqa-2636
uvx harbor run -d simpleqa@1.0 -t simpleqa-2636
b8f0fa9
simpleqa-2637
uvx harbor run -d simpleqa@1.0 -t simpleqa-2637
b8f0fa9
simpleqa-2638
uvx harbor run -d simpleqa@1.0 -t simpleqa-2638
b8f0fa9
simpleqa-2639
uvx harbor run -d simpleqa@1.0 -t simpleqa-2639
b8f0fa9
simpleqa-264
uvx harbor run -d simpleqa@1.0 -t simpleqa-264
b8f0fa9
simpleqa-2640
uvx harbor run -d simpleqa@1.0 -t simpleqa-2640
b8f0fa9
simpleqa-2641
uvx harbor run -d simpleqa@1.0 -t simpleqa-2641
b8f0fa9
simpleqa-2642
uvx harbor run -d simpleqa@1.0 -t simpleqa-2642
b8f0fa9
simpleqa-2643
uvx harbor run -d simpleqa@1.0 -t simpleqa-2643
b8f0fa9
simpleqa-2644
uvx harbor run -d simpleqa@1.0 -t simpleqa-2644
b8f0fa9
simpleqa-2645
uvx harbor run -d simpleqa@1.0 -t simpleqa-2645
b8f0fa9
simpleqa-2646
uvx harbor run -d simpleqa@1.0 -t simpleqa-2646
b8f0fa9
simpleqa-2647
uvx harbor run -d simpleqa@1.0 -t simpleqa-2647
b8f0fa9
simpleqa-2648
uvx harbor run -d simpleqa@1.0 -t simpleqa-2648
b8f0fa9
simpleqa-2649
uvx harbor run -d simpleqa@1.0 -t simpleqa-2649
b8f0fa9
simpleqa-265
uvx harbor run -d simpleqa@1.0 -t simpleqa-265
b8f0fa9
simpleqa-2650
uvx harbor run -d simpleqa@1.0 -t simpleqa-2650
b8f0fa9
simpleqa-2651
uvx harbor run -d simpleqa@1.0 -t simpleqa-2651
b8f0fa9
simpleqa-2652
uvx harbor run -d simpleqa@1.0 -t simpleqa-2652
b8f0fa9
simpleqa-2653
uvx harbor run -d simpleqa@1.0 -t simpleqa-2653
b8f0fa9
simpleqa-2654
uvx harbor run -d simpleqa@1.0 -t simpleqa-2654
b8f0fa9
simpleqa-2655
uvx harbor run -d simpleqa@1.0 -t simpleqa-2655
b8f0fa9
simpleqa-2656
uvx harbor run -d simpleqa@1.0 -t simpleqa-2656
b8f0fa9
simpleqa-2657
uvx harbor run -d simpleqa@1.0 -t simpleqa-2657
b8f0fa9
simpleqa-2658
uvx harbor run -d simpleqa@1.0 -t simpleqa-2658
b8f0fa9
simpleqa-2659
uvx harbor run -d simpleqa@1.0 -t simpleqa-2659
b8f0fa9
simpleqa-266
uvx harbor run -d simpleqa@1.0 -t simpleqa-266
b8f0fa9
simpleqa-2660
uvx harbor run -d simpleqa@1.0 -t simpleqa-2660
b8f0fa9
simpleqa-2661
uvx harbor run -d simpleqa@1.0 -t simpleqa-2661
b8f0fa9
simpleqa-2662
uvx harbor run -d simpleqa@1.0 -t simpleqa-2662
b8f0fa9
simpleqa-2663
uvx harbor run -d simpleqa@1.0 -t simpleqa-2663
b8f0fa9
simpleqa-2664
uvx harbor run -d simpleqa@1.0 -t simpleqa-2664
b8f0fa9
simpleqa-2665
uvx harbor run -d simpleqa@1.0 -t simpleqa-2665
b8f0fa9
simpleqa-2666
uvx harbor run -d simpleqa@1.0 -t simpleqa-2666
b8f0fa9
simpleqa-2667
uvx harbor run -d simpleqa@1.0 -t simpleqa-2667
b8f0fa9
simpleqa-2668
uvx harbor run -d simpleqa@1.0 -t simpleqa-2668
b8f0fa9
simpleqa-2669
uvx harbor run -d simpleqa@1.0 -t simpleqa-2669
b8f0fa9
simpleqa-267
uvx harbor run -d simpleqa@1.0 -t simpleqa-267
b8f0fa9
simpleqa-2670
uvx harbor run -d simpleqa@1.0 -t simpleqa-2670
b8f0fa9
simpleqa-2671
uvx harbor run -d simpleqa@1.0 -t simpleqa-2671
b8f0fa9
simpleqa-2672
uvx harbor run -d simpleqa@1.0 -t simpleqa-2672
b8f0fa9
simpleqa-2673
uvx harbor run -d simpleqa@1.0 -t simpleqa-2673
b8f0fa9
simpleqa-2674
uvx harbor run -d simpleqa@1.0 -t simpleqa-2674
b8f0fa9
simpleqa-2675
uvx harbor run -d simpleqa@1.0 -t simpleqa-2675
b8f0fa9
simpleqa-2676
uvx harbor run -d simpleqa@1.0 -t simpleqa-2676
b8f0fa9
simpleqa-2677
uvx harbor run -d simpleqa@1.0 -t simpleqa-2677
b8f0fa9
simpleqa-2678
uvx harbor run -d simpleqa@1.0 -t simpleqa-2678
b8f0fa9
simpleqa-2679
uvx harbor run -d simpleqa@1.0 -t simpleqa-2679
b8f0fa9
simpleqa-268
uvx harbor run -d simpleqa@1.0 -t simpleqa-268
b8f0fa9
simpleqa-2680
uvx harbor run -d simpleqa@1.0 -t simpleqa-2680
b8f0fa9
simpleqa-2681
uvx harbor run -d simpleqa@1.0 -t simpleqa-2681
b8f0fa9
simpleqa-2682
uvx harbor run -d simpleqa@1.0 -t simpleqa-2682
b8f0fa9
simpleqa-2683
uvx harbor run -d simpleqa@1.0 -t simpleqa-2683
b8f0fa9
simpleqa-2684
uvx harbor run -d simpleqa@1.0 -t simpleqa-2684
b8f0fa9
simpleqa-2685
uvx harbor run -d simpleqa@1.0 -t simpleqa-2685
b8f0fa9
simpleqa-2686
uvx harbor run -d simpleqa@1.0 -t simpleqa-2686
b8f0fa9
simpleqa-2687
uvx harbor run -d simpleqa@1.0 -t simpleqa-2687
b8f0fa9
simpleqa-2688
uvx harbor run -d simpleqa@1.0 -t simpleqa-2688
b8f0fa9
simpleqa-2689
uvx harbor run -d simpleqa@1.0 -t simpleqa-2689
b8f0fa9
simpleqa-269
uvx harbor run -d simpleqa@1.0 -t simpleqa-269
b8f0fa9
simpleqa-2690
uvx harbor run -d simpleqa@1.0 -t simpleqa-2690
b8f0fa9
simpleqa-2691
uvx harbor run -d simpleqa@1.0 -t simpleqa-2691
b8f0fa9
simpleqa-2692
uvx harbor run -d simpleqa@1.0 -t simpleqa-2692
b8f0fa9
simpleqa-2693
uvx harbor run -d simpleqa@1.0 -t simpleqa-2693
b8f0fa9
simpleqa-2694
uvx harbor run -d simpleqa@1.0 -t simpleqa-2694
b8f0fa9
simpleqa-2695
uvx harbor run -d simpleqa@1.0 -t simpleqa-2695
b8f0fa9
simpleqa-2696
uvx harbor run -d simpleqa@1.0 -t simpleqa-2696
b8f0fa9
simpleqa-2697
uvx harbor run -d simpleqa@1.0 -t simpleqa-2697
b8f0fa9
simpleqa-2698
uvx harbor run -d simpleqa@1.0 -t simpleqa-2698
b8f0fa9
simpleqa-2699
uvx harbor run -d simpleqa@1.0 -t simpleqa-2699
b8f0fa9
simpleqa-27
uvx harbor run -d simpleqa@1.0 -t simpleqa-27
b8f0fa9
simpleqa-270
uvx harbor run -d simpleqa@1.0 -t simpleqa-270
b8f0fa9
simpleqa-2700
uvx harbor run -d simpleqa@1.0 -t simpleqa-2700
b8f0fa9
simpleqa-2701
uvx harbor run -d simpleqa@1.0 -t simpleqa-2701
b8f0fa9
simpleqa-2702
uvx harbor run -d simpleqa@1.0 -t simpleqa-2702
b8f0fa9
simpleqa-2703
uvx harbor run -d simpleqa@1.0 -t simpleqa-2703
b8f0fa9
simpleqa-2704
uvx harbor run -d simpleqa@1.0 -t simpleqa-2704
b8f0fa9
simpleqa-2705
uvx harbor run -d simpleqa@1.0 -t simpleqa-2705
b8f0fa9
simpleqa-2706
uvx harbor run -d simpleqa@1.0 -t simpleqa-2706
b8f0fa9
simpleqa-2707
uvx harbor run -d simpleqa@1.0 -t simpleqa-2707
b8f0fa9