simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-1718
uvx harbor run -d simpleqa@1.0 -t simpleqa-1718
b8f0fa9
simpleqa-1719
uvx harbor run -d simpleqa@1.0 -t simpleqa-1719
b8f0fa9
simpleqa-172
uvx harbor run -d simpleqa@1.0 -t simpleqa-172
b8f0fa9
simpleqa-1720
uvx harbor run -d simpleqa@1.0 -t simpleqa-1720
b8f0fa9
simpleqa-1721
uvx harbor run -d simpleqa@1.0 -t simpleqa-1721
b8f0fa9
simpleqa-1722
uvx harbor run -d simpleqa@1.0 -t simpleqa-1722
b8f0fa9
simpleqa-1723
uvx harbor run -d simpleqa@1.0 -t simpleqa-1723
b8f0fa9
simpleqa-1724
uvx harbor run -d simpleqa@1.0 -t simpleqa-1724
b8f0fa9
simpleqa-1725
uvx harbor run -d simpleqa@1.0 -t simpleqa-1725
b8f0fa9
simpleqa-1726
uvx harbor run -d simpleqa@1.0 -t simpleqa-1726
b8f0fa9
simpleqa-1727
uvx harbor run -d simpleqa@1.0 -t simpleqa-1727
b8f0fa9
simpleqa-1728
uvx harbor run -d simpleqa@1.0 -t simpleqa-1728
b8f0fa9
simpleqa-1729
uvx harbor run -d simpleqa@1.0 -t simpleqa-1729
b8f0fa9
simpleqa-173
uvx harbor run -d simpleqa@1.0 -t simpleqa-173
b8f0fa9
simpleqa-1730
uvx harbor run -d simpleqa@1.0 -t simpleqa-1730
b8f0fa9
simpleqa-1731
uvx harbor run -d simpleqa@1.0 -t simpleqa-1731
b8f0fa9
simpleqa-1732
uvx harbor run -d simpleqa@1.0 -t simpleqa-1732
b8f0fa9
simpleqa-1733
uvx harbor run -d simpleqa@1.0 -t simpleqa-1733
b8f0fa9
simpleqa-1734
uvx harbor run -d simpleqa@1.0 -t simpleqa-1734
b8f0fa9
simpleqa-1735
uvx harbor run -d simpleqa@1.0 -t simpleqa-1735
b8f0fa9
simpleqa-1736
uvx harbor run -d simpleqa@1.0 -t simpleqa-1736
b8f0fa9
simpleqa-1737
uvx harbor run -d simpleqa@1.0 -t simpleqa-1737
b8f0fa9
simpleqa-1738
uvx harbor run -d simpleqa@1.0 -t simpleqa-1738
b8f0fa9
simpleqa-1739
uvx harbor run -d simpleqa@1.0 -t simpleqa-1739
b8f0fa9
simpleqa-174
uvx harbor run -d simpleqa@1.0 -t simpleqa-174
b8f0fa9
simpleqa-1740
uvx harbor run -d simpleqa@1.0 -t simpleqa-1740
b8f0fa9
simpleqa-1741
uvx harbor run -d simpleqa@1.0 -t simpleqa-1741
b8f0fa9
simpleqa-1742
uvx harbor run -d simpleqa@1.0 -t simpleqa-1742
b8f0fa9
simpleqa-1743
uvx harbor run -d simpleqa@1.0 -t simpleqa-1743
b8f0fa9
simpleqa-1744
uvx harbor run -d simpleqa@1.0 -t simpleqa-1744
b8f0fa9
simpleqa-1745
uvx harbor run -d simpleqa@1.0 -t simpleqa-1745
b8f0fa9
simpleqa-1746
uvx harbor run -d simpleqa@1.0 -t simpleqa-1746
b8f0fa9
simpleqa-1747
uvx harbor run -d simpleqa@1.0 -t simpleqa-1747
b8f0fa9
simpleqa-1748
uvx harbor run -d simpleqa@1.0 -t simpleqa-1748
b8f0fa9
simpleqa-1749
uvx harbor run -d simpleqa@1.0 -t simpleqa-1749
b8f0fa9
simpleqa-175
uvx harbor run -d simpleqa@1.0 -t simpleqa-175
b8f0fa9
simpleqa-1750
uvx harbor run -d simpleqa@1.0 -t simpleqa-1750
b8f0fa9
simpleqa-1751
uvx harbor run -d simpleqa@1.0 -t simpleqa-1751
b8f0fa9
simpleqa-1752
uvx harbor run -d simpleqa@1.0 -t simpleqa-1752
b8f0fa9
simpleqa-1753
uvx harbor run -d simpleqa@1.0 -t simpleqa-1753
b8f0fa9
simpleqa-1754
uvx harbor run -d simpleqa@1.0 -t simpleqa-1754
b8f0fa9
simpleqa-1755
uvx harbor run -d simpleqa@1.0 -t simpleqa-1755
b8f0fa9
simpleqa-1756
uvx harbor run -d simpleqa@1.0 -t simpleqa-1756
b8f0fa9
simpleqa-1757
uvx harbor run -d simpleqa@1.0 -t simpleqa-1757
b8f0fa9
simpleqa-1758
uvx harbor run -d simpleqa@1.0 -t simpleqa-1758
b8f0fa9
simpleqa-1759
uvx harbor run -d simpleqa@1.0 -t simpleqa-1759
b8f0fa9
simpleqa-176
uvx harbor run -d simpleqa@1.0 -t simpleqa-176
b8f0fa9
simpleqa-1760
uvx harbor run -d simpleqa@1.0 -t simpleqa-1760
b8f0fa9
simpleqa-1761
uvx harbor run -d simpleqa@1.0 -t simpleqa-1761
b8f0fa9
simpleqa-1762
uvx harbor run -d simpleqa@1.0 -t simpleqa-1762
b8f0fa9
simpleqa-1763
uvx harbor run -d simpleqa@1.0 -t simpleqa-1763
b8f0fa9
simpleqa-1764
uvx harbor run -d simpleqa@1.0 -t simpleqa-1764
b8f0fa9
simpleqa-1765
uvx harbor run -d simpleqa@1.0 -t simpleqa-1765
b8f0fa9
simpleqa-1766
uvx harbor run -d simpleqa@1.0 -t simpleqa-1766
b8f0fa9
simpleqa-1767
uvx harbor run -d simpleqa@1.0 -t simpleqa-1767
b8f0fa9
simpleqa-1768
uvx harbor run -d simpleqa@1.0 -t simpleqa-1768
b8f0fa9
simpleqa-1769
uvx harbor run -d simpleqa@1.0 -t simpleqa-1769
b8f0fa9
simpleqa-177
uvx harbor run -d simpleqa@1.0 -t simpleqa-177
b8f0fa9
simpleqa-1770
uvx harbor run -d simpleqa@1.0 -t simpleqa-1770
b8f0fa9
simpleqa-1771
uvx harbor run -d simpleqa@1.0 -t simpleqa-1771
b8f0fa9
simpleqa-1772
uvx harbor run -d simpleqa@1.0 -t simpleqa-1772
b8f0fa9
simpleqa-1773
uvx harbor run -d simpleqa@1.0 -t simpleqa-1773
b8f0fa9
simpleqa-1774
uvx harbor run -d simpleqa@1.0 -t simpleqa-1774
b8f0fa9
simpleqa-1775
uvx harbor run -d simpleqa@1.0 -t simpleqa-1775
b8f0fa9
simpleqa-1776
uvx harbor run -d simpleqa@1.0 -t simpleqa-1776
b8f0fa9
simpleqa-1777
uvx harbor run -d simpleqa@1.0 -t simpleqa-1777
b8f0fa9
simpleqa-1778
uvx harbor run -d simpleqa@1.0 -t simpleqa-1778
b8f0fa9
simpleqa-1779
uvx harbor run -d simpleqa@1.0 -t simpleqa-1779
b8f0fa9
simpleqa-178
uvx harbor run -d simpleqa@1.0 -t simpleqa-178
b8f0fa9
simpleqa-1780
uvx harbor run -d simpleqa@1.0 -t simpleqa-1780
b8f0fa9
simpleqa-1781
uvx harbor run -d simpleqa@1.0 -t simpleqa-1781
b8f0fa9
simpleqa-1782
uvx harbor run -d simpleqa@1.0 -t simpleqa-1782
b8f0fa9
simpleqa-1783
uvx harbor run -d simpleqa@1.0 -t simpleqa-1783
b8f0fa9
simpleqa-1784
uvx harbor run -d simpleqa@1.0 -t simpleqa-1784
b8f0fa9
simpleqa-1785
uvx harbor run -d simpleqa@1.0 -t simpleqa-1785
b8f0fa9
simpleqa-1786
uvx harbor run -d simpleqa@1.0 -t simpleqa-1786
b8f0fa9
simpleqa-1787
uvx harbor run -d simpleqa@1.0 -t simpleqa-1787
b8f0fa9
simpleqa-1788
uvx harbor run -d simpleqa@1.0 -t simpleqa-1788
b8f0fa9
simpleqa-1789
uvx harbor run -d simpleqa@1.0 -t simpleqa-1789
b8f0fa9
simpleqa-179
uvx harbor run -d simpleqa@1.0 -t simpleqa-179
b8f0fa9
simpleqa-1790
uvx harbor run -d simpleqa@1.0 -t simpleqa-1790
b8f0fa9
simpleqa-1791
uvx harbor run -d simpleqa@1.0 -t simpleqa-1791
b8f0fa9
simpleqa-1792
uvx harbor run -d simpleqa@1.0 -t simpleqa-1792
b8f0fa9
simpleqa-1793
uvx harbor run -d simpleqa@1.0 -t simpleqa-1793
b8f0fa9
simpleqa-1794
uvx harbor run -d simpleqa@1.0 -t simpleqa-1794
b8f0fa9
simpleqa-1795
uvx harbor run -d simpleqa@1.0 -t simpleqa-1795
b8f0fa9
simpleqa-1796
uvx harbor run -d simpleqa@1.0 -t simpleqa-1796
b8f0fa9
simpleqa-1797
uvx harbor run -d simpleqa@1.0 -t simpleqa-1797
b8f0fa9
simpleqa-1798
uvx harbor run -d simpleqa@1.0 -t simpleqa-1798
b8f0fa9
simpleqa-1799
uvx harbor run -d simpleqa@1.0 -t simpleqa-1799
b8f0fa9
simpleqa-18
uvx harbor run -d simpleqa@1.0 -t simpleqa-18
b8f0fa9
simpleqa-180
uvx harbor run -d simpleqa@1.0 -t simpleqa-180
b8f0fa9
simpleqa-1800
uvx harbor run -d simpleqa@1.0 -t simpleqa-1800
b8f0fa9
simpleqa-1801
uvx harbor run -d simpleqa@1.0 -t simpleqa-1801
b8f0fa9
simpleqa-1802
uvx harbor run -d simpleqa@1.0 -t simpleqa-1802
b8f0fa9
simpleqa-1803
uvx harbor run -d simpleqa@1.0 -t simpleqa-1803
b8f0fa9
simpleqa-1804
uvx harbor run -d simpleqa@1.0 -t simpleqa-1804
b8f0fa9
simpleqa-1805
uvx harbor run -d simpleqa@1.0 -t simpleqa-1805
b8f0fa9
simpleqa-1806
uvx harbor run -d simpleqa@1.0 -t simpleqa-1806
b8f0fa9
simpleqa-1807
uvx harbor run -d simpleqa@1.0 -t simpleqa-1807
b8f0fa9