simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-1989
uvx harbor run -d simpleqa@1.0 -t simpleqa-1989
b8f0fa9
simpleqa-199
uvx harbor run -d simpleqa@1.0 -t simpleqa-199
b8f0fa9
simpleqa-1990
uvx harbor run -d simpleqa@1.0 -t simpleqa-1990
b8f0fa9
simpleqa-1991
uvx harbor run -d simpleqa@1.0 -t simpleqa-1991
b8f0fa9
simpleqa-1992
uvx harbor run -d simpleqa@1.0 -t simpleqa-1992
b8f0fa9
simpleqa-1993
uvx harbor run -d simpleqa@1.0 -t simpleqa-1993
b8f0fa9
simpleqa-1994
uvx harbor run -d simpleqa@1.0 -t simpleqa-1994
b8f0fa9
simpleqa-1995
uvx harbor run -d simpleqa@1.0 -t simpleqa-1995
b8f0fa9
simpleqa-1996
uvx harbor run -d simpleqa@1.0 -t simpleqa-1996
b8f0fa9
simpleqa-1997
uvx harbor run -d simpleqa@1.0 -t simpleqa-1997
b8f0fa9
simpleqa-1998
uvx harbor run -d simpleqa@1.0 -t simpleqa-1998
b8f0fa9
simpleqa-1999
uvx harbor run -d simpleqa@1.0 -t simpleqa-1999
b8f0fa9
simpleqa-2
uvx harbor run -d simpleqa@1.0 -t simpleqa-2
b8f0fa9
simpleqa-20
uvx harbor run -d simpleqa@1.0 -t simpleqa-20
b8f0fa9
simpleqa-200
uvx harbor run -d simpleqa@1.0 -t simpleqa-200
b8f0fa9
simpleqa-2000
uvx harbor run -d simpleqa@1.0 -t simpleqa-2000
b8f0fa9
simpleqa-2001
uvx harbor run -d simpleqa@1.0 -t simpleqa-2001
b8f0fa9
simpleqa-2002
uvx harbor run -d simpleqa@1.0 -t simpleqa-2002
b8f0fa9
simpleqa-2003
uvx harbor run -d simpleqa@1.0 -t simpleqa-2003
b8f0fa9
simpleqa-2004
uvx harbor run -d simpleqa@1.0 -t simpleqa-2004
b8f0fa9
simpleqa-2005
uvx harbor run -d simpleqa@1.0 -t simpleqa-2005
b8f0fa9
simpleqa-2006
uvx harbor run -d simpleqa@1.0 -t simpleqa-2006
b8f0fa9
simpleqa-2007
uvx harbor run -d simpleqa@1.0 -t simpleqa-2007
b8f0fa9
simpleqa-2008
uvx harbor run -d simpleqa@1.0 -t simpleqa-2008
b8f0fa9
simpleqa-2009
uvx harbor run -d simpleqa@1.0 -t simpleqa-2009
b8f0fa9
simpleqa-201
uvx harbor run -d simpleqa@1.0 -t simpleqa-201
b8f0fa9
simpleqa-2010
uvx harbor run -d simpleqa@1.0 -t simpleqa-2010
b8f0fa9
simpleqa-2011
uvx harbor run -d simpleqa@1.0 -t simpleqa-2011
b8f0fa9
simpleqa-2012
uvx harbor run -d simpleqa@1.0 -t simpleqa-2012
b8f0fa9
simpleqa-2013
uvx harbor run -d simpleqa@1.0 -t simpleqa-2013
b8f0fa9
simpleqa-2014
uvx harbor run -d simpleqa@1.0 -t simpleqa-2014
b8f0fa9
simpleqa-2015
uvx harbor run -d simpleqa@1.0 -t simpleqa-2015
b8f0fa9
simpleqa-2016
uvx harbor run -d simpleqa@1.0 -t simpleqa-2016
b8f0fa9
simpleqa-2017
uvx harbor run -d simpleqa@1.0 -t simpleqa-2017
b8f0fa9
simpleqa-2018
uvx harbor run -d simpleqa@1.0 -t simpleqa-2018
b8f0fa9
simpleqa-2019
uvx harbor run -d simpleqa@1.0 -t simpleqa-2019
b8f0fa9
simpleqa-202
uvx harbor run -d simpleqa@1.0 -t simpleqa-202
b8f0fa9
simpleqa-2020
uvx harbor run -d simpleqa@1.0 -t simpleqa-2020
b8f0fa9
simpleqa-2021
uvx harbor run -d simpleqa@1.0 -t simpleqa-2021
b8f0fa9
simpleqa-2022
uvx harbor run -d simpleqa@1.0 -t simpleqa-2022
b8f0fa9
simpleqa-2023
uvx harbor run -d simpleqa@1.0 -t simpleqa-2023
b8f0fa9
simpleqa-2024
uvx harbor run -d simpleqa@1.0 -t simpleqa-2024
b8f0fa9
simpleqa-2025
uvx harbor run -d simpleqa@1.0 -t simpleqa-2025
b8f0fa9
simpleqa-2026
uvx harbor run -d simpleqa@1.0 -t simpleqa-2026
b8f0fa9
simpleqa-2027
uvx harbor run -d simpleqa@1.0 -t simpleqa-2027
b8f0fa9
simpleqa-2028
uvx harbor run -d simpleqa@1.0 -t simpleqa-2028
b8f0fa9
simpleqa-2029
uvx harbor run -d simpleqa@1.0 -t simpleqa-2029
b8f0fa9
simpleqa-203
uvx harbor run -d simpleqa@1.0 -t simpleqa-203
b8f0fa9
simpleqa-2030
uvx harbor run -d simpleqa@1.0 -t simpleqa-2030
b8f0fa9
simpleqa-2031
uvx harbor run -d simpleqa@1.0 -t simpleqa-2031
b8f0fa9
simpleqa-2032
uvx harbor run -d simpleqa@1.0 -t simpleqa-2032
b8f0fa9
simpleqa-2033
uvx harbor run -d simpleqa@1.0 -t simpleqa-2033
b8f0fa9
simpleqa-2034
uvx harbor run -d simpleqa@1.0 -t simpleqa-2034
b8f0fa9
simpleqa-2035
uvx harbor run -d simpleqa@1.0 -t simpleqa-2035
b8f0fa9
simpleqa-2036
uvx harbor run -d simpleqa@1.0 -t simpleqa-2036
b8f0fa9
simpleqa-2037
uvx harbor run -d simpleqa@1.0 -t simpleqa-2037
b8f0fa9
simpleqa-2038
uvx harbor run -d simpleqa@1.0 -t simpleqa-2038
b8f0fa9
simpleqa-2039
uvx harbor run -d simpleqa@1.0 -t simpleqa-2039
b8f0fa9
simpleqa-204
uvx harbor run -d simpleqa@1.0 -t simpleqa-204
b8f0fa9
simpleqa-2040
uvx harbor run -d simpleqa@1.0 -t simpleqa-2040
b8f0fa9
simpleqa-2041
uvx harbor run -d simpleqa@1.0 -t simpleqa-2041
b8f0fa9
simpleqa-2042
uvx harbor run -d simpleqa@1.0 -t simpleqa-2042
b8f0fa9
simpleqa-2043
uvx harbor run -d simpleqa@1.0 -t simpleqa-2043
b8f0fa9
simpleqa-2044
uvx harbor run -d simpleqa@1.0 -t simpleqa-2044
b8f0fa9
simpleqa-2045
uvx harbor run -d simpleqa@1.0 -t simpleqa-2045
b8f0fa9
simpleqa-2046
uvx harbor run -d simpleqa@1.0 -t simpleqa-2046
b8f0fa9
simpleqa-2047
uvx harbor run -d simpleqa@1.0 -t simpleqa-2047
b8f0fa9
simpleqa-2048
uvx harbor run -d simpleqa@1.0 -t simpleqa-2048
b8f0fa9
simpleqa-2049
uvx harbor run -d simpleqa@1.0 -t simpleqa-2049
b8f0fa9
simpleqa-205
uvx harbor run -d simpleqa@1.0 -t simpleqa-205
b8f0fa9
simpleqa-2050
uvx harbor run -d simpleqa@1.0 -t simpleqa-2050
b8f0fa9
simpleqa-2051
uvx harbor run -d simpleqa@1.0 -t simpleqa-2051
b8f0fa9
simpleqa-2052
uvx harbor run -d simpleqa@1.0 -t simpleqa-2052
b8f0fa9
simpleqa-2053
uvx harbor run -d simpleqa@1.0 -t simpleqa-2053
b8f0fa9
simpleqa-2054
uvx harbor run -d simpleqa@1.0 -t simpleqa-2054
b8f0fa9
simpleqa-2055
uvx harbor run -d simpleqa@1.0 -t simpleqa-2055
b8f0fa9
simpleqa-2056
uvx harbor run -d simpleqa@1.0 -t simpleqa-2056
b8f0fa9
simpleqa-2057
uvx harbor run -d simpleqa@1.0 -t simpleqa-2057
b8f0fa9
simpleqa-2058
uvx harbor run -d simpleqa@1.0 -t simpleqa-2058
b8f0fa9
simpleqa-2059
uvx harbor run -d simpleqa@1.0 -t simpleqa-2059
b8f0fa9
simpleqa-206
uvx harbor run -d simpleqa@1.0 -t simpleqa-206
b8f0fa9
simpleqa-2060
uvx harbor run -d simpleqa@1.0 -t simpleqa-2060
b8f0fa9
simpleqa-2061
uvx harbor run -d simpleqa@1.0 -t simpleqa-2061
b8f0fa9
simpleqa-2062
uvx harbor run -d simpleqa@1.0 -t simpleqa-2062
b8f0fa9
simpleqa-2063
uvx harbor run -d simpleqa@1.0 -t simpleqa-2063
b8f0fa9
simpleqa-2064
uvx harbor run -d simpleqa@1.0 -t simpleqa-2064
b8f0fa9
simpleqa-2065
uvx harbor run -d simpleqa@1.0 -t simpleqa-2065
b8f0fa9
simpleqa-2066
uvx harbor run -d simpleqa@1.0 -t simpleqa-2066
b8f0fa9
simpleqa-2067
uvx harbor run -d simpleqa@1.0 -t simpleqa-2067
b8f0fa9
simpleqa-2068
uvx harbor run -d simpleqa@1.0 -t simpleqa-2068
b8f0fa9
simpleqa-2069
uvx harbor run -d simpleqa@1.0 -t simpleqa-2069
b8f0fa9
simpleqa-207
uvx harbor run -d simpleqa@1.0 -t simpleqa-207
b8f0fa9
simpleqa-2070
uvx harbor run -d simpleqa@1.0 -t simpleqa-2070
b8f0fa9
simpleqa-2071
uvx harbor run -d simpleqa@1.0 -t simpleqa-2071
b8f0fa9
simpleqa-2072
uvx harbor run -d simpleqa@1.0 -t simpleqa-2072
b8f0fa9
simpleqa-2073
uvx harbor run -d simpleqa@1.0 -t simpleqa-2073
b8f0fa9
simpleqa-2074
uvx harbor run -d simpleqa@1.0 -t simpleqa-2074
b8f0fa9
simpleqa-2075
uvx harbor run -d simpleqa@1.0 -t simpleqa-2075
b8f0fa9
simpleqa-2076
uvx harbor run -d simpleqa@1.0 -t simpleqa-2076
b8f0fa9
simpleqa-2077
uvx harbor run -d simpleqa@1.0 -t simpleqa-2077
b8f0fa9