simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-2078
uvx harbor run -d simpleqa@1.0 -t simpleqa-2078
b8f0fa9
simpleqa-2079
uvx harbor run -d simpleqa@1.0 -t simpleqa-2079
b8f0fa9
simpleqa-208
uvx harbor run -d simpleqa@1.0 -t simpleqa-208
b8f0fa9
simpleqa-2080
uvx harbor run -d simpleqa@1.0 -t simpleqa-2080
b8f0fa9
simpleqa-2081
uvx harbor run -d simpleqa@1.0 -t simpleqa-2081
b8f0fa9
simpleqa-2082
uvx harbor run -d simpleqa@1.0 -t simpleqa-2082
b8f0fa9
simpleqa-2083
uvx harbor run -d simpleqa@1.0 -t simpleqa-2083
b8f0fa9
simpleqa-2084
uvx harbor run -d simpleqa@1.0 -t simpleqa-2084
b8f0fa9
simpleqa-2085
uvx harbor run -d simpleqa@1.0 -t simpleqa-2085
b8f0fa9
simpleqa-2086
uvx harbor run -d simpleqa@1.0 -t simpleqa-2086
b8f0fa9
simpleqa-2087
uvx harbor run -d simpleqa@1.0 -t simpleqa-2087
b8f0fa9
simpleqa-2088
uvx harbor run -d simpleqa@1.0 -t simpleqa-2088
b8f0fa9
simpleqa-2089
uvx harbor run -d simpleqa@1.0 -t simpleqa-2089
b8f0fa9
simpleqa-209
uvx harbor run -d simpleqa@1.0 -t simpleqa-209
b8f0fa9
simpleqa-2090
uvx harbor run -d simpleqa@1.0 -t simpleqa-2090
b8f0fa9
simpleqa-2091
uvx harbor run -d simpleqa@1.0 -t simpleqa-2091
b8f0fa9
simpleqa-2092
uvx harbor run -d simpleqa@1.0 -t simpleqa-2092
b8f0fa9
simpleqa-2093
uvx harbor run -d simpleqa@1.0 -t simpleqa-2093
b8f0fa9
simpleqa-2094
uvx harbor run -d simpleqa@1.0 -t simpleqa-2094
b8f0fa9
simpleqa-2095
uvx harbor run -d simpleqa@1.0 -t simpleqa-2095
b8f0fa9
simpleqa-2096
uvx harbor run -d simpleqa@1.0 -t simpleqa-2096
b8f0fa9
simpleqa-2097
uvx harbor run -d simpleqa@1.0 -t simpleqa-2097
b8f0fa9
simpleqa-2098
uvx harbor run -d simpleqa@1.0 -t simpleqa-2098
b8f0fa9
simpleqa-2099
uvx harbor run -d simpleqa@1.0 -t simpleqa-2099
b8f0fa9
simpleqa-21
uvx harbor run -d simpleqa@1.0 -t simpleqa-21
b8f0fa9
simpleqa-210
uvx harbor run -d simpleqa@1.0 -t simpleqa-210
b8f0fa9
simpleqa-2100
uvx harbor run -d simpleqa@1.0 -t simpleqa-2100
b8f0fa9
simpleqa-2101
uvx harbor run -d simpleqa@1.0 -t simpleqa-2101
b8f0fa9
simpleqa-2102
uvx harbor run -d simpleqa@1.0 -t simpleqa-2102
b8f0fa9
simpleqa-2103
uvx harbor run -d simpleqa@1.0 -t simpleqa-2103
b8f0fa9
simpleqa-2104
uvx harbor run -d simpleqa@1.0 -t simpleqa-2104
b8f0fa9
simpleqa-2105
uvx harbor run -d simpleqa@1.0 -t simpleqa-2105
b8f0fa9
simpleqa-2106
uvx harbor run -d simpleqa@1.0 -t simpleqa-2106
b8f0fa9
simpleqa-2107
uvx harbor run -d simpleqa@1.0 -t simpleqa-2107
b8f0fa9
simpleqa-2108
uvx harbor run -d simpleqa@1.0 -t simpleqa-2108
b8f0fa9
simpleqa-2109
uvx harbor run -d simpleqa@1.0 -t simpleqa-2109
b8f0fa9
simpleqa-211
uvx harbor run -d simpleqa@1.0 -t simpleqa-211
b8f0fa9
simpleqa-2110
uvx harbor run -d simpleqa@1.0 -t simpleqa-2110
b8f0fa9
simpleqa-2111
uvx harbor run -d simpleqa@1.0 -t simpleqa-2111
b8f0fa9
simpleqa-2112
uvx harbor run -d simpleqa@1.0 -t simpleqa-2112
b8f0fa9
simpleqa-2113
uvx harbor run -d simpleqa@1.0 -t simpleqa-2113
b8f0fa9
simpleqa-2114
uvx harbor run -d simpleqa@1.0 -t simpleqa-2114
b8f0fa9
simpleqa-2115
uvx harbor run -d simpleqa@1.0 -t simpleqa-2115
b8f0fa9
simpleqa-2116
uvx harbor run -d simpleqa@1.0 -t simpleqa-2116
b8f0fa9
simpleqa-2117
uvx harbor run -d simpleqa@1.0 -t simpleqa-2117
b8f0fa9
simpleqa-2118
uvx harbor run -d simpleqa@1.0 -t simpleqa-2118
b8f0fa9
simpleqa-2119
uvx harbor run -d simpleqa@1.0 -t simpleqa-2119
b8f0fa9
simpleqa-212
uvx harbor run -d simpleqa@1.0 -t simpleqa-212
b8f0fa9
simpleqa-2120
uvx harbor run -d simpleqa@1.0 -t simpleqa-2120
b8f0fa9
simpleqa-2121
uvx harbor run -d simpleqa@1.0 -t simpleqa-2121
b8f0fa9
simpleqa-2122
uvx harbor run -d simpleqa@1.0 -t simpleqa-2122
b8f0fa9
simpleqa-2123
uvx harbor run -d simpleqa@1.0 -t simpleqa-2123
b8f0fa9
simpleqa-2124
uvx harbor run -d simpleqa@1.0 -t simpleqa-2124
b8f0fa9
simpleqa-2125
uvx harbor run -d simpleqa@1.0 -t simpleqa-2125
b8f0fa9
simpleqa-2126
uvx harbor run -d simpleqa@1.0 -t simpleqa-2126
b8f0fa9
simpleqa-2127
uvx harbor run -d simpleqa@1.0 -t simpleqa-2127
b8f0fa9
simpleqa-2128
uvx harbor run -d simpleqa@1.0 -t simpleqa-2128
b8f0fa9
simpleqa-2129
uvx harbor run -d simpleqa@1.0 -t simpleqa-2129
b8f0fa9
simpleqa-213
uvx harbor run -d simpleqa@1.0 -t simpleqa-213
b8f0fa9
simpleqa-2130
uvx harbor run -d simpleqa@1.0 -t simpleqa-2130
b8f0fa9
simpleqa-2131
uvx harbor run -d simpleqa@1.0 -t simpleqa-2131
b8f0fa9
simpleqa-2132
uvx harbor run -d simpleqa@1.0 -t simpleqa-2132
b8f0fa9
simpleqa-2133
uvx harbor run -d simpleqa@1.0 -t simpleqa-2133
b8f0fa9
simpleqa-2134
uvx harbor run -d simpleqa@1.0 -t simpleqa-2134
b8f0fa9
simpleqa-2135
uvx harbor run -d simpleqa@1.0 -t simpleqa-2135
b8f0fa9
simpleqa-2136
uvx harbor run -d simpleqa@1.0 -t simpleqa-2136
b8f0fa9
simpleqa-2137
uvx harbor run -d simpleqa@1.0 -t simpleqa-2137
b8f0fa9
simpleqa-2138
uvx harbor run -d simpleqa@1.0 -t simpleqa-2138
b8f0fa9
simpleqa-2139
uvx harbor run -d simpleqa@1.0 -t simpleqa-2139
b8f0fa9
simpleqa-214
uvx harbor run -d simpleqa@1.0 -t simpleqa-214
b8f0fa9
simpleqa-2140
uvx harbor run -d simpleqa@1.0 -t simpleqa-2140
b8f0fa9
simpleqa-2141
uvx harbor run -d simpleqa@1.0 -t simpleqa-2141
b8f0fa9
simpleqa-2142
uvx harbor run -d simpleqa@1.0 -t simpleqa-2142
b8f0fa9
simpleqa-2143
uvx harbor run -d simpleqa@1.0 -t simpleqa-2143
b8f0fa9
simpleqa-2144
uvx harbor run -d simpleqa@1.0 -t simpleqa-2144
b8f0fa9
simpleqa-2145
uvx harbor run -d simpleqa@1.0 -t simpleqa-2145
b8f0fa9
simpleqa-2146
uvx harbor run -d simpleqa@1.0 -t simpleqa-2146
b8f0fa9
simpleqa-2147
uvx harbor run -d simpleqa@1.0 -t simpleqa-2147
b8f0fa9
simpleqa-2148
uvx harbor run -d simpleqa@1.0 -t simpleqa-2148
b8f0fa9
simpleqa-2149
uvx harbor run -d simpleqa@1.0 -t simpleqa-2149
b8f0fa9
simpleqa-215
uvx harbor run -d simpleqa@1.0 -t simpleqa-215
b8f0fa9
simpleqa-2150
uvx harbor run -d simpleqa@1.0 -t simpleqa-2150
b8f0fa9
simpleqa-2151
uvx harbor run -d simpleqa@1.0 -t simpleqa-2151
b8f0fa9
simpleqa-2152
uvx harbor run -d simpleqa@1.0 -t simpleqa-2152
b8f0fa9
simpleqa-2153
uvx harbor run -d simpleqa@1.0 -t simpleqa-2153
b8f0fa9
simpleqa-2154
uvx harbor run -d simpleqa@1.0 -t simpleqa-2154
b8f0fa9
simpleqa-2155
uvx harbor run -d simpleqa@1.0 -t simpleqa-2155
b8f0fa9
simpleqa-2156
uvx harbor run -d simpleqa@1.0 -t simpleqa-2156
b8f0fa9
simpleqa-2157
uvx harbor run -d simpleqa@1.0 -t simpleqa-2157
b8f0fa9
simpleqa-2158
uvx harbor run -d simpleqa@1.0 -t simpleqa-2158
b8f0fa9
simpleqa-2159
uvx harbor run -d simpleqa@1.0 -t simpleqa-2159
b8f0fa9
simpleqa-216
uvx harbor run -d simpleqa@1.0 -t simpleqa-216
b8f0fa9
simpleqa-2160
uvx harbor run -d simpleqa@1.0 -t simpleqa-2160
b8f0fa9
simpleqa-2161
uvx harbor run -d simpleqa@1.0 -t simpleqa-2161
b8f0fa9
simpleqa-2162
uvx harbor run -d simpleqa@1.0 -t simpleqa-2162
b8f0fa9
simpleqa-2163
uvx harbor run -d simpleqa@1.0 -t simpleqa-2163
b8f0fa9
simpleqa-2164
uvx harbor run -d simpleqa@1.0 -t simpleqa-2164
b8f0fa9
simpleqa-2165
uvx harbor run -d simpleqa@1.0 -t simpleqa-2165
b8f0fa9
simpleqa-2166
uvx harbor run -d simpleqa@1.0 -t simpleqa-2166
b8f0fa9
simpleqa-2167
uvx harbor run -d simpleqa@1.0 -t simpleqa-2167
b8f0fa9