simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-1628
uvx harbor run -d simpleqa@1.0 -t simpleqa-1628
b8f0fa9
simpleqa-1629
uvx harbor run -d simpleqa@1.0 -t simpleqa-1629
b8f0fa9
simpleqa-163
uvx harbor run -d simpleqa@1.0 -t simpleqa-163
b8f0fa9
simpleqa-1630
uvx harbor run -d simpleqa@1.0 -t simpleqa-1630
b8f0fa9
simpleqa-1631
uvx harbor run -d simpleqa@1.0 -t simpleqa-1631
b8f0fa9
simpleqa-1632
uvx harbor run -d simpleqa@1.0 -t simpleqa-1632
b8f0fa9
simpleqa-1633
uvx harbor run -d simpleqa@1.0 -t simpleqa-1633
b8f0fa9
simpleqa-1634
uvx harbor run -d simpleqa@1.0 -t simpleqa-1634
b8f0fa9
simpleqa-1635
uvx harbor run -d simpleqa@1.0 -t simpleqa-1635
b8f0fa9
simpleqa-1636
uvx harbor run -d simpleqa@1.0 -t simpleqa-1636
b8f0fa9
simpleqa-1637
uvx harbor run -d simpleqa@1.0 -t simpleqa-1637
b8f0fa9
simpleqa-1638
uvx harbor run -d simpleqa@1.0 -t simpleqa-1638
b8f0fa9
simpleqa-1639
uvx harbor run -d simpleqa@1.0 -t simpleqa-1639
b8f0fa9
simpleqa-164
uvx harbor run -d simpleqa@1.0 -t simpleqa-164
b8f0fa9
simpleqa-1640
uvx harbor run -d simpleqa@1.0 -t simpleqa-1640
b8f0fa9
simpleqa-1641
uvx harbor run -d simpleqa@1.0 -t simpleqa-1641
b8f0fa9
simpleqa-1642
uvx harbor run -d simpleqa@1.0 -t simpleqa-1642
b8f0fa9
simpleqa-1643
uvx harbor run -d simpleqa@1.0 -t simpleqa-1643
b8f0fa9
simpleqa-1644
uvx harbor run -d simpleqa@1.0 -t simpleqa-1644
b8f0fa9
simpleqa-1645
uvx harbor run -d simpleqa@1.0 -t simpleqa-1645
b8f0fa9
simpleqa-1646
uvx harbor run -d simpleqa@1.0 -t simpleqa-1646
b8f0fa9
simpleqa-1647
uvx harbor run -d simpleqa@1.0 -t simpleqa-1647
b8f0fa9
simpleqa-1648
uvx harbor run -d simpleqa@1.0 -t simpleqa-1648
b8f0fa9
simpleqa-1649
uvx harbor run -d simpleqa@1.0 -t simpleqa-1649
b8f0fa9
simpleqa-165
uvx harbor run -d simpleqa@1.0 -t simpleqa-165
b8f0fa9
simpleqa-1650
uvx harbor run -d simpleqa@1.0 -t simpleqa-1650
b8f0fa9
simpleqa-1651
uvx harbor run -d simpleqa@1.0 -t simpleqa-1651
b8f0fa9
simpleqa-1652
uvx harbor run -d simpleqa@1.0 -t simpleqa-1652
b8f0fa9
simpleqa-1653
uvx harbor run -d simpleqa@1.0 -t simpleqa-1653
b8f0fa9
simpleqa-1654
uvx harbor run -d simpleqa@1.0 -t simpleqa-1654
b8f0fa9
simpleqa-1655
uvx harbor run -d simpleqa@1.0 -t simpleqa-1655
b8f0fa9
simpleqa-1656
uvx harbor run -d simpleqa@1.0 -t simpleqa-1656
b8f0fa9
simpleqa-1657
uvx harbor run -d simpleqa@1.0 -t simpleqa-1657
b8f0fa9
simpleqa-1658
uvx harbor run -d simpleqa@1.0 -t simpleqa-1658
b8f0fa9
simpleqa-1659
uvx harbor run -d simpleqa@1.0 -t simpleqa-1659
b8f0fa9
simpleqa-166
uvx harbor run -d simpleqa@1.0 -t simpleqa-166
b8f0fa9
simpleqa-1660
uvx harbor run -d simpleqa@1.0 -t simpleqa-1660
b8f0fa9
simpleqa-1661
uvx harbor run -d simpleqa@1.0 -t simpleqa-1661
b8f0fa9
simpleqa-1662
uvx harbor run -d simpleqa@1.0 -t simpleqa-1662
b8f0fa9
simpleqa-1663
uvx harbor run -d simpleqa@1.0 -t simpleqa-1663
b8f0fa9
simpleqa-1664
uvx harbor run -d simpleqa@1.0 -t simpleqa-1664
b8f0fa9
simpleqa-1665
uvx harbor run -d simpleqa@1.0 -t simpleqa-1665
b8f0fa9
simpleqa-1666
uvx harbor run -d simpleqa@1.0 -t simpleqa-1666
b8f0fa9
simpleqa-1667
uvx harbor run -d simpleqa@1.0 -t simpleqa-1667
b8f0fa9
simpleqa-1668
uvx harbor run -d simpleqa@1.0 -t simpleqa-1668
b8f0fa9
simpleqa-1669
uvx harbor run -d simpleqa@1.0 -t simpleqa-1669
b8f0fa9
simpleqa-167
uvx harbor run -d simpleqa@1.0 -t simpleqa-167
b8f0fa9
simpleqa-1670
uvx harbor run -d simpleqa@1.0 -t simpleqa-1670
b8f0fa9
simpleqa-1671
uvx harbor run -d simpleqa@1.0 -t simpleqa-1671
b8f0fa9
simpleqa-1672
uvx harbor run -d simpleqa@1.0 -t simpleqa-1672
b8f0fa9
simpleqa-1673
uvx harbor run -d simpleqa@1.0 -t simpleqa-1673
b8f0fa9
simpleqa-1674
uvx harbor run -d simpleqa@1.0 -t simpleqa-1674
b8f0fa9
simpleqa-1675
uvx harbor run -d simpleqa@1.0 -t simpleqa-1675
b8f0fa9
simpleqa-1676
uvx harbor run -d simpleqa@1.0 -t simpleqa-1676
b8f0fa9
simpleqa-1677
uvx harbor run -d simpleqa@1.0 -t simpleqa-1677
b8f0fa9
simpleqa-1678
uvx harbor run -d simpleqa@1.0 -t simpleqa-1678
b8f0fa9
simpleqa-1679
uvx harbor run -d simpleqa@1.0 -t simpleqa-1679
b8f0fa9
simpleqa-168
uvx harbor run -d simpleqa@1.0 -t simpleqa-168
b8f0fa9
simpleqa-1680
uvx harbor run -d simpleqa@1.0 -t simpleqa-1680
b8f0fa9
simpleqa-1681
uvx harbor run -d simpleqa@1.0 -t simpleqa-1681
b8f0fa9
simpleqa-1682
uvx harbor run -d simpleqa@1.0 -t simpleqa-1682
b8f0fa9
simpleqa-1683
uvx harbor run -d simpleqa@1.0 -t simpleqa-1683
b8f0fa9
simpleqa-1684
uvx harbor run -d simpleqa@1.0 -t simpleqa-1684
b8f0fa9
simpleqa-1685
uvx harbor run -d simpleqa@1.0 -t simpleqa-1685
b8f0fa9
simpleqa-1686
uvx harbor run -d simpleqa@1.0 -t simpleqa-1686
b8f0fa9
simpleqa-1687
uvx harbor run -d simpleqa@1.0 -t simpleqa-1687
b8f0fa9
simpleqa-1688
uvx harbor run -d simpleqa@1.0 -t simpleqa-1688
b8f0fa9
simpleqa-1689
uvx harbor run -d simpleqa@1.0 -t simpleqa-1689
b8f0fa9
simpleqa-169
uvx harbor run -d simpleqa@1.0 -t simpleqa-169
b8f0fa9
simpleqa-1690
uvx harbor run -d simpleqa@1.0 -t simpleqa-1690
b8f0fa9
simpleqa-1691
uvx harbor run -d simpleqa@1.0 -t simpleqa-1691
b8f0fa9
simpleqa-1692
uvx harbor run -d simpleqa@1.0 -t simpleqa-1692
b8f0fa9
simpleqa-1693
uvx harbor run -d simpleqa@1.0 -t simpleqa-1693
b8f0fa9
simpleqa-1694
uvx harbor run -d simpleqa@1.0 -t simpleqa-1694
b8f0fa9
simpleqa-1695
uvx harbor run -d simpleqa@1.0 -t simpleqa-1695
b8f0fa9
simpleqa-1696
uvx harbor run -d simpleqa@1.0 -t simpleqa-1696
b8f0fa9
simpleqa-1697
uvx harbor run -d simpleqa@1.0 -t simpleqa-1697
b8f0fa9
simpleqa-1698
uvx harbor run -d simpleqa@1.0 -t simpleqa-1698
b8f0fa9
simpleqa-1699
uvx harbor run -d simpleqa@1.0 -t simpleqa-1699
b8f0fa9
simpleqa-17
uvx harbor run -d simpleqa@1.0 -t simpleqa-17
b8f0fa9
simpleqa-170
uvx harbor run -d simpleqa@1.0 -t simpleqa-170
b8f0fa9
simpleqa-1700
uvx harbor run -d simpleqa@1.0 -t simpleqa-1700
b8f0fa9
simpleqa-1701
uvx harbor run -d simpleqa@1.0 -t simpleqa-1701
b8f0fa9
simpleqa-1702
uvx harbor run -d simpleqa@1.0 -t simpleqa-1702
b8f0fa9
simpleqa-1703
uvx harbor run -d simpleqa@1.0 -t simpleqa-1703
b8f0fa9
simpleqa-1704
uvx harbor run -d simpleqa@1.0 -t simpleqa-1704
b8f0fa9
simpleqa-1705
uvx harbor run -d simpleqa@1.0 -t simpleqa-1705
b8f0fa9
simpleqa-1706
uvx harbor run -d simpleqa@1.0 -t simpleqa-1706
b8f0fa9
simpleqa-1707
uvx harbor run -d simpleqa@1.0 -t simpleqa-1707
b8f0fa9
simpleqa-1708
uvx harbor run -d simpleqa@1.0 -t simpleqa-1708
b8f0fa9
simpleqa-1709
uvx harbor run -d simpleqa@1.0 -t simpleqa-1709
b8f0fa9
simpleqa-171
uvx harbor run -d simpleqa@1.0 -t simpleqa-171
b8f0fa9
simpleqa-1710
uvx harbor run -d simpleqa@1.0 -t simpleqa-1710
b8f0fa9
simpleqa-1711
uvx harbor run -d simpleqa@1.0 -t simpleqa-1711
b8f0fa9
simpleqa-1712
uvx harbor run -d simpleqa@1.0 -t simpleqa-1712
b8f0fa9
simpleqa-1713
uvx harbor run -d simpleqa@1.0 -t simpleqa-1713
b8f0fa9
simpleqa-1714
uvx harbor run -d simpleqa@1.0 -t simpleqa-1714
b8f0fa9
simpleqa-1715
uvx harbor run -d simpleqa@1.0 -t simpleqa-1715
b8f0fa9
simpleqa-1716
uvx harbor run -d simpleqa@1.0 -t simpleqa-1716
b8f0fa9
simpleqa-1717
uvx harbor run -d simpleqa@1.0 -t simpleqa-1717
b8f0fa9