simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-2258
uvx harbor run -d simpleqa@1.0 -t simpleqa-2258
b8f0fa9
simpleqa-2259
uvx harbor run -d simpleqa@1.0 -t simpleqa-2259
b8f0fa9
simpleqa-226
uvx harbor run -d simpleqa@1.0 -t simpleqa-226
b8f0fa9
simpleqa-2260
uvx harbor run -d simpleqa@1.0 -t simpleqa-2260
b8f0fa9
simpleqa-2261
uvx harbor run -d simpleqa@1.0 -t simpleqa-2261
b8f0fa9
simpleqa-2262
uvx harbor run -d simpleqa@1.0 -t simpleqa-2262
b8f0fa9
simpleqa-2263
uvx harbor run -d simpleqa@1.0 -t simpleqa-2263
b8f0fa9
simpleqa-2264
uvx harbor run -d simpleqa@1.0 -t simpleqa-2264
b8f0fa9
simpleqa-2265
uvx harbor run -d simpleqa@1.0 -t simpleqa-2265
b8f0fa9
simpleqa-2266
uvx harbor run -d simpleqa@1.0 -t simpleqa-2266
b8f0fa9
simpleqa-2267
uvx harbor run -d simpleqa@1.0 -t simpleqa-2267
b8f0fa9
simpleqa-2268
uvx harbor run -d simpleqa@1.0 -t simpleqa-2268
b8f0fa9
simpleqa-2269
uvx harbor run -d simpleqa@1.0 -t simpleqa-2269
b8f0fa9
simpleqa-227
uvx harbor run -d simpleqa@1.0 -t simpleqa-227
b8f0fa9
simpleqa-2270
uvx harbor run -d simpleqa@1.0 -t simpleqa-2270
b8f0fa9
simpleqa-2271
uvx harbor run -d simpleqa@1.0 -t simpleqa-2271
b8f0fa9
simpleqa-2272
uvx harbor run -d simpleqa@1.0 -t simpleqa-2272
b8f0fa9
simpleqa-2273
uvx harbor run -d simpleqa@1.0 -t simpleqa-2273
b8f0fa9
simpleqa-2274
uvx harbor run -d simpleqa@1.0 -t simpleqa-2274
b8f0fa9
simpleqa-2275
uvx harbor run -d simpleqa@1.0 -t simpleqa-2275
b8f0fa9
simpleqa-2276
uvx harbor run -d simpleqa@1.0 -t simpleqa-2276
b8f0fa9
simpleqa-2277
uvx harbor run -d simpleqa@1.0 -t simpleqa-2277
b8f0fa9
simpleqa-2278
uvx harbor run -d simpleqa@1.0 -t simpleqa-2278
b8f0fa9
simpleqa-2279
uvx harbor run -d simpleqa@1.0 -t simpleqa-2279
b8f0fa9
simpleqa-228
uvx harbor run -d simpleqa@1.0 -t simpleqa-228
b8f0fa9
simpleqa-2280
uvx harbor run -d simpleqa@1.0 -t simpleqa-2280
b8f0fa9
simpleqa-2281
uvx harbor run -d simpleqa@1.0 -t simpleqa-2281
b8f0fa9
simpleqa-2282
uvx harbor run -d simpleqa@1.0 -t simpleqa-2282
b8f0fa9
simpleqa-2283
uvx harbor run -d simpleqa@1.0 -t simpleqa-2283
b8f0fa9
simpleqa-2284
uvx harbor run -d simpleqa@1.0 -t simpleqa-2284
b8f0fa9
simpleqa-2285
uvx harbor run -d simpleqa@1.0 -t simpleqa-2285
b8f0fa9
simpleqa-2286
uvx harbor run -d simpleqa@1.0 -t simpleqa-2286
b8f0fa9
simpleqa-2287
uvx harbor run -d simpleqa@1.0 -t simpleqa-2287
b8f0fa9
simpleqa-2288
uvx harbor run -d simpleqa@1.0 -t simpleqa-2288
b8f0fa9
simpleqa-2289
uvx harbor run -d simpleqa@1.0 -t simpleqa-2289
b8f0fa9
simpleqa-229
uvx harbor run -d simpleqa@1.0 -t simpleqa-229
b8f0fa9
simpleqa-2290
uvx harbor run -d simpleqa@1.0 -t simpleqa-2290
b8f0fa9
simpleqa-2291
uvx harbor run -d simpleqa@1.0 -t simpleqa-2291
b8f0fa9
simpleqa-2292
uvx harbor run -d simpleqa@1.0 -t simpleqa-2292
b8f0fa9
simpleqa-2293
uvx harbor run -d simpleqa@1.0 -t simpleqa-2293
b8f0fa9
simpleqa-2294
uvx harbor run -d simpleqa@1.0 -t simpleqa-2294
b8f0fa9
simpleqa-2295
uvx harbor run -d simpleqa@1.0 -t simpleqa-2295
b8f0fa9
simpleqa-2296
uvx harbor run -d simpleqa@1.0 -t simpleqa-2296
b8f0fa9
simpleqa-2297
uvx harbor run -d simpleqa@1.0 -t simpleqa-2297
b8f0fa9
simpleqa-2298
uvx harbor run -d simpleqa@1.0 -t simpleqa-2298
b8f0fa9
simpleqa-2299
uvx harbor run -d simpleqa@1.0 -t simpleqa-2299
b8f0fa9
simpleqa-23
uvx harbor run -d simpleqa@1.0 -t simpleqa-23
b8f0fa9
simpleqa-230
uvx harbor run -d simpleqa@1.0 -t simpleqa-230
b8f0fa9
simpleqa-2300
uvx harbor run -d simpleqa@1.0 -t simpleqa-2300
b8f0fa9
simpleqa-2301
uvx harbor run -d simpleqa@1.0 -t simpleqa-2301
b8f0fa9
simpleqa-2302
uvx harbor run -d simpleqa@1.0 -t simpleqa-2302
b8f0fa9
simpleqa-2303
uvx harbor run -d simpleqa@1.0 -t simpleqa-2303
b8f0fa9
simpleqa-2304
uvx harbor run -d simpleqa@1.0 -t simpleqa-2304
b8f0fa9
simpleqa-2305
uvx harbor run -d simpleqa@1.0 -t simpleqa-2305
b8f0fa9
simpleqa-2306
uvx harbor run -d simpleqa@1.0 -t simpleqa-2306
b8f0fa9
simpleqa-2307
uvx harbor run -d simpleqa@1.0 -t simpleqa-2307
b8f0fa9
simpleqa-2308
uvx harbor run -d simpleqa@1.0 -t simpleqa-2308
b8f0fa9
simpleqa-2309
uvx harbor run -d simpleqa@1.0 -t simpleqa-2309
b8f0fa9
simpleqa-231
uvx harbor run -d simpleqa@1.0 -t simpleqa-231
b8f0fa9
simpleqa-2310
uvx harbor run -d simpleqa@1.0 -t simpleqa-2310
b8f0fa9
simpleqa-2311
uvx harbor run -d simpleqa@1.0 -t simpleqa-2311
b8f0fa9
simpleqa-2312
uvx harbor run -d simpleqa@1.0 -t simpleqa-2312
b8f0fa9
simpleqa-2313
uvx harbor run -d simpleqa@1.0 -t simpleqa-2313
b8f0fa9
simpleqa-2314
uvx harbor run -d simpleqa@1.0 -t simpleqa-2314
b8f0fa9
simpleqa-2315
uvx harbor run -d simpleqa@1.0 -t simpleqa-2315
b8f0fa9
simpleqa-2316
uvx harbor run -d simpleqa@1.0 -t simpleqa-2316
b8f0fa9
simpleqa-2317
uvx harbor run -d simpleqa@1.0 -t simpleqa-2317
b8f0fa9
simpleqa-2318
uvx harbor run -d simpleqa@1.0 -t simpleqa-2318
b8f0fa9
simpleqa-2319
uvx harbor run -d simpleqa@1.0 -t simpleqa-2319
b8f0fa9
simpleqa-232
uvx harbor run -d simpleqa@1.0 -t simpleqa-232
b8f0fa9
simpleqa-2320
uvx harbor run -d simpleqa@1.0 -t simpleqa-2320
b8f0fa9
simpleqa-2321
uvx harbor run -d simpleqa@1.0 -t simpleqa-2321
b8f0fa9
simpleqa-2322
uvx harbor run -d simpleqa@1.0 -t simpleqa-2322
b8f0fa9
simpleqa-2323
uvx harbor run -d simpleqa@1.0 -t simpleqa-2323
b8f0fa9
simpleqa-2324
uvx harbor run -d simpleqa@1.0 -t simpleqa-2324
b8f0fa9
simpleqa-2325
uvx harbor run -d simpleqa@1.0 -t simpleqa-2325
b8f0fa9
simpleqa-2326
uvx harbor run -d simpleqa@1.0 -t simpleqa-2326
b8f0fa9
simpleqa-2327
uvx harbor run -d simpleqa@1.0 -t simpleqa-2327
b8f0fa9
simpleqa-2328
uvx harbor run -d simpleqa@1.0 -t simpleqa-2328
b8f0fa9
simpleqa-2329
uvx harbor run -d simpleqa@1.0 -t simpleqa-2329
b8f0fa9
simpleqa-233
uvx harbor run -d simpleqa@1.0 -t simpleqa-233
b8f0fa9
simpleqa-2330
uvx harbor run -d simpleqa@1.0 -t simpleqa-2330
b8f0fa9
simpleqa-2331
uvx harbor run -d simpleqa@1.0 -t simpleqa-2331
b8f0fa9
simpleqa-2332
uvx harbor run -d simpleqa@1.0 -t simpleqa-2332
b8f0fa9
simpleqa-2333
uvx harbor run -d simpleqa@1.0 -t simpleqa-2333
b8f0fa9
simpleqa-2334
uvx harbor run -d simpleqa@1.0 -t simpleqa-2334
b8f0fa9
simpleqa-2335
uvx harbor run -d simpleqa@1.0 -t simpleqa-2335
b8f0fa9
simpleqa-2336
uvx harbor run -d simpleqa@1.0 -t simpleqa-2336
b8f0fa9
simpleqa-2337
uvx harbor run -d simpleqa@1.0 -t simpleqa-2337
b8f0fa9
simpleqa-2338
uvx harbor run -d simpleqa@1.0 -t simpleqa-2338
b8f0fa9
simpleqa-2339
uvx harbor run -d simpleqa@1.0 -t simpleqa-2339
b8f0fa9
simpleqa-234
uvx harbor run -d simpleqa@1.0 -t simpleqa-234
b8f0fa9
simpleqa-2340
uvx harbor run -d simpleqa@1.0 -t simpleqa-2340
b8f0fa9
simpleqa-2341
uvx harbor run -d simpleqa@1.0 -t simpleqa-2341
b8f0fa9
simpleqa-2342
uvx harbor run -d simpleqa@1.0 -t simpleqa-2342
b8f0fa9
simpleqa-2343
uvx harbor run -d simpleqa@1.0 -t simpleqa-2343
b8f0fa9
simpleqa-2344
uvx harbor run -d simpleqa@1.0 -t simpleqa-2344
b8f0fa9
simpleqa-2345
uvx harbor run -d simpleqa@1.0 -t simpleqa-2345
b8f0fa9
simpleqa-2346
uvx harbor run -d simpleqa@1.0 -t simpleqa-2346
b8f0fa9
simpleqa-2347
uvx harbor run -d simpleqa@1.0 -t simpleqa-2347
b8f0fa9