simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-1808
uvx harbor run -d simpleqa@1.0 -t simpleqa-1808
b8f0fa9
simpleqa-1809
uvx harbor run -d simpleqa@1.0 -t simpleqa-1809
b8f0fa9
simpleqa-181
uvx harbor run -d simpleqa@1.0 -t simpleqa-181
b8f0fa9
simpleqa-1810
uvx harbor run -d simpleqa@1.0 -t simpleqa-1810
b8f0fa9
simpleqa-1811
uvx harbor run -d simpleqa@1.0 -t simpleqa-1811
b8f0fa9
simpleqa-1812
uvx harbor run -d simpleqa@1.0 -t simpleqa-1812
b8f0fa9
simpleqa-1813
uvx harbor run -d simpleqa@1.0 -t simpleqa-1813
b8f0fa9
simpleqa-1814
uvx harbor run -d simpleqa@1.0 -t simpleqa-1814
b8f0fa9
simpleqa-1815
uvx harbor run -d simpleqa@1.0 -t simpleqa-1815
b8f0fa9
simpleqa-1816
uvx harbor run -d simpleqa@1.0 -t simpleqa-1816
b8f0fa9
simpleqa-1817
uvx harbor run -d simpleqa@1.0 -t simpleqa-1817
b8f0fa9
simpleqa-1818
uvx harbor run -d simpleqa@1.0 -t simpleqa-1818
b8f0fa9
simpleqa-1819
uvx harbor run -d simpleqa@1.0 -t simpleqa-1819
b8f0fa9
simpleqa-182
uvx harbor run -d simpleqa@1.0 -t simpleqa-182
b8f0fa9
simpleqa-1820
uvx harbor run -d simpleqa@1.0 -t simpleqa-1820
b8f0fa9
simpleqa-1821
uvx harbor run -d simpleqa@1.0 -t simpleqa-1821
b8f0fa9
simpleqa-1822
uvx harbor run -d simpleqa@1.0 -t simpleqa-1822
b8f0fa9
simpleqa-1823
uvx harbor run -d simpleqa@1.0 -t simpleqa-1823
b8f0fa9
simpleqa-1824
uvx harbor run -d simpleqa@1.0 -t simpleqa-1824
b8f0fa9
simpleqa-1825
uvx harbor run -d simpleqa@1.0 -t simpleqa-1825
b8f0fa9
simpleqa-1826
uvx harbor run -d simpleqa@1.0 -t simpleqa-1826
b8f0fa9
simpleqa-1827
uvx harbor run -d simpleqa@1.0 -t simpleqa-1827
b8f0fa9
simpleqa-1828
uvx harbor run -d simpleqa@1.0 -t simpleqa-1828
b8f0fa9
simpleqa-1829
uvx harbor run -d simpleqa@1.0 -t simpleqa-1829
b8f0fa9
simpleqa-183
uvx harbor run -d simpleqa@1.0 -t simpleqa-183
b8f0fa9
simpleqa-1830
uvx harbor run -d simpleqa@1.0 -t simpleqa-1830
b8f0fa9
simpleqa-1831
uvx harbor run -d simpleqa@1.0 -t simpleqa-1831
b8f0fa9
simpleqa-1832
uvx harbor run -d simpleqa@1.0 -t simpleqa-1832
b8f0fa9
simpleqa-1833
uvx harbor run -d simpleqa@1.0 -t simpleqa-1833
b8f0fa9
simpleqa-1834
uvx harbor run -d simpleqa@1.0 -t simpleqa-1834
b8f0fa9
simpleqa-1835
uvx harbor run -d simpleqa@1.0 -t simpleqa-1835
b8f0fa9
simpleqa-1836
uvx harbor run -d simpleqa@1.0 -t simpleqa-1836
b8f0fa9
simpleqa-1837
uvx harbor run -d simpleqa@1.0 -t simpleqa-1837
b8f0fa9
simpleqa-1838
uvx harbor run -d simpleqa@1.0 -t simpleqa-1838
b8f0fa9
simpleqa-1839
uvx harbor run -d simpleqa@1.0 -t simpleqa-1839
b8f0fa9
simpleqa-184
uvx harbor run -d simpleqa@1.0 -t simpleqa-184
b8f0fa9
simpleqa-1840
uvx harbor run -d simpleqa@1.0 -t simpleqa-1840
b8f0fa9
simpleqa-1841
uvx harbor run -d simpleqa@1.0 -t simpleqa-1841
b8f0fa9
simpleqa-1842
uvx harbor run -d simpleqa@1.0 -t simpleqa-1842
b8f0fa9
simpleqa-1843
uvx harbor run -d simpleqa@1.0 -t simpleqa-1843
b8f0fa9
simpleqa-1844
uvx harbor run -d simpleqa@1.0 -t simpleqa-1844
b8f0fa9
simpleqa-1845
uvx harbor run -d simpleqa@1.0 -t simpleqa-1845
b8f0fa9
simpleqa-1846
uvx harbor run -d simpleqa@1.0 -t simpleqa-1846
b8f0fa9
simpleqa-1847
uvx harbor run -d simpleqa@1.0 -t simpleqa-1847
b8f0fa9
simpleqa-1848
uvx harbor run -d simpleqa@1.0 -t simpleqa-1848
b8f0fa9
simpleqa-1849
uvx harbor run -d simpleqa@1.0 -t simpleqa-1849
b8f0fa9
simpleqa-185
uvx harbor run -d simpleqa@1.0 -t simpleqa-185
b8f0fa9
simpleqa-1850
uvx harbor run -d simpleqa@1.0 -t simpleqa-1850
b8f0fa9
simpleqa-1851
uvx harbor run -d simpleqa@1.0 -t simpleqa-1851
b8f0fa9
simpleqa-1852
uvx harbor run -d simpleqa@1.0 -t simpleqa-1852
b8f0fa9
simpleqa-1853
uvx harbor run -d simpleqa@1.0 -t simpleqa-1853
b8f0fa9
simpleqa-1854
uvx harbor run -d simpleqa@1.0 -t simpleqa-1854
b8f0fa9
simpleqa-1855
uvx harbor run -d simpleqa@1.0 -t simpleqa-1855
b8f0fa9
simpleqa-1856
uvx harbor run -d simpleqa@1.0 -t simpleqa-1856
b8f0fa9
simpleqa-1857
uvx harbor run -d simpleqa@1.0 -t simpleqa-1857
b8f0fa9
simpleqa-1858
uvx harbor run -d simpleqa@1.0 -t simpleqa-1858
b8f0fa9
simpleqa-1859
uvx harbor run -d simpleqa@1.0 -t simpleqa-1859
b8f0fa9
simpleqa-186
uvx harbor run -d simpleqa@1.0 -t simpleqa-186
b8f0fa9
simpleqa-1860
uvx harbor run -d simpleqa@1.0 -t simpleqa-1860
b8f0fa9
simpleqa-1861
uvx harbor run -d simpleqa@1.0 -t simpleqa-1861
b8f0fa9
simpleqa-1862
uvx harbor run -d simpleqa@1.0 -t simpleqa-1862
b8f0fa9
simpleqa-1863
uvx harbor run -d simpleqa@1.0 -t simpleqa-1863
b8f0fa9
simpleqa-1864
uvx harbor run -d simpleqa@1.0 -t simpleqa-1864
b8f0fa9
simpleqa-1865
uvx harbor run -d simpleqa@1.0 -t simpleqa-1865
b8f0fa9
simpleqa-1866
uvx harbor run -d simpleqa@1.0 -t simpleqa-1866
b8f0fa9
simpleqa-1867
uvx harbor run -d simpleqa@1.0 -t simpleqa-1867
b8f0fa9
simpleqa-1868
uvx harbor run -d simpleqa@1.0 -t simpleqa-1868
b8f0fa9
simpleqa-1869
uvx harbor run -d simpleqa@1.0 -t simpleqa-1869
b8f0fa9
simpleqa-187
uvx harbor run -d simpleqa@1.0 -t simpleqa-187
b8f0fa9
simpleqa-1870
uvx harbor run -d simpleqa@1.0 -t simpleqa-1870
b8f0fa9
simpleqa-1871
uvx harbor run -d simpleqa@1.0 -t simpleqa-1871
b8f0fa9
simpleqa-1872
uvx harbor run -d simpleqa@1.0 -t simpleqa-1872
b8f0fa9
simpleqa-1873
uvx harbor run -d simpleqa@1.0 -t simpleqa-1873
b8f0fa9
simpleqa-1874
uvx harbor run -d simpleqa@1.0 -t simpleqa-1874
b8f0fa9
simpleqa-1875
uvx harbor run -d simpleqa@1.0 -t simpleqa-1875
b8f0fa9
simpleqa-1876
uvx harbor run -d simpleqa@1.0 -t simpleqa-1876
b8f0fa9
simpleqa-1877
uvx harbor run -d simpleqa@1.0 -t simpleqa-1877
b8f0fa9
simpleqa-1878
uvx harbor run -d simpleqa@1.0 -t simpleqa-1878
b8f0fa9
simpleqa-1879
uvx harbor run -d simpleqa@1.0 -t simpleqa-1879
b8f0fa9
simpleqa-188
uvx harbor run -d simpleqa@1.0 -t simpleqa-188
b8f0fa9
simpleqa-1880
uvx harbor run -d simpleqa@1.0 -t simpleqa-1880
b8f0fa9
simpleqa-1881
uvx harbor run -d simpleqa@1.0 -t simpleqa-1881
b8f0fa9
simpleqa-1882
uvx harbor run -d simpleqa@1.0 -t simpleqa-1882
b8f0fa9
simpleqa-1883
uvx harbor run -d simpleqa@1.0 -t simpleqa-1883
b8f0fa9
simpleqa-1884
uvx harbor run -d simpleqa@1.0 -t simpleqa-1884
b8f0fa9
simpleqa-1885
uvx harbor run -d simpleqa@1.0 -t simpleqa-1885
b8f0fa9
simpleqa-1886
uvx harbor run -d simpleqa@1.0 -t simpleqa-1886
b8f0fa9
simpleqa-1887
uvx harbor run -d simpleqa@1.0 -t simpleqa-1887
b8f0fa9
simpleqa-1888
uvx harbor run -d simpleqa@1.0 -t simpleqa-1888
b8f0fa9
simpleqa-1889
uvx harbor run -d simpleqa@1.0 -t simpleqa-1889
b8f0fa9
simpleqa-189
uvx harbor run -d simpleqa@1.0 -t simpleqa-189
b8f0fa9
simpleqa-1890
uvx harbor run -d simpleqa@1.0 -t simpleqa-1890
b8f0fa9
simpleqa-1891
uvx harbor run -d simpleqa@1.0 -t simpleqa-1891
b8f0fa9
simpleqa-1892
uvx harbor run -d simpleqa@1.0 -t simpleqa-1892
b8f0fa9
simpleqa-1893
uvx harbor run -d simpleqa@1.0 -t simpleqa-1893
b8f0fa9
simpleqa-1894
uvx harbor run -d simpleqa@1.0 -t simpleqa-1894
b8f0fa9
simpleqa-1895
uvx harbor run -d simpleqa@1.0 -t simpleqa-1895
b8f0fa9
simpleqa-1896
uvx harbor run -d simpleqa@1.0 -t simpleqa-1896
b8f0fa9
simpleqa-1897
uvx harbor run -d simpleqa@1.0 -t simpleqa-1897
b8f0fa9
simpleqa-1898
uvx harbor run -d simpleqa@1.0 -t simpleqa-1898
b8f0fa9