simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-1448
uvx harbor run -d simpleqa@1.0 -t simpleqa-1448
b8f0fa9
simpleqa-1449
uvx harbor run -d simpleqa@1.0 -t simpleqa-1449
b8f0fa9
simpleqa-145
uvx harbor run -d simpleqa@1.0 -t simpleqa-145
b8f0fa9
simpleqa-1450
uvx harbor run -d simpleqa@1.0 -t simpleqa-1450
b8f0fa9
simpleqa-1451
uvx harbor run -d simpleqa@1.0 -t simpleqa-1451
b8f0fa9
simpleqa-1452
uvx harbor run -d simpleqa@1.0 -t simpleqa-1452
b8f0fa9
simpleqa-1453
uvx harbor run -d simpleqa@1.0 -t simpleqa-1453
b8f0fa9
simpleqa-1454
uvx harbor run -d simpleqa@1.0 -t simpleqa-1454
b8f0fa9
simpleqa-1455
uvx harbor run -d simpleqa@1.0 -t simpleqa-1455
b8f0fa9
simpleqa-1456
uvx harbor run -d simpleqa@1.0 -t simpleqa-1456
b8f0fa9
simpleqa-1457
uvx harbor run -d simpleqa@1.0 -t simpleqa-1457
b8f0fa9
simpleqa-1458
uvx harbor run -d simpleqa@1.0 -t simpleqa-1458
b8f0fa9
simpleqa-1459
uvx harbor run -d simpleqa@1.0 -t simpleqa-1459
b8f0fa9
simpleqa-146
uvx harbor run -d simpleqa@1.0 -t simpleqa-146
b8f0fa9
simpleqa-1460
uvx harbor run -d simpleqa@1.0 -t simpleqa-1460
b8f0fa9
simpleqa-1461
uvx harbor run -d simpleqa@1.0 -t simpleqa-1461
b8f0fa9
simpleqa-1462
uvx harbor run -d simpleqa@1.0 -t simpleqa-1462
b8f0fa9
simpleqa-1463
uvx harbor run -d simpleqa@1.0 -t simpleqa-1463
b8f0fa9
simpleqa-1464
uvx harbor run -d simpleqa@1.0 -t simpleqa-1464
b8f0fa9
simpleqa-1465
uvx harbor run -d simpleqa@1.0 -t simpleqa-1465
b8f0fa9
simpleqa-1466
uvx harbor run -d simpleqa@1.0 -t simpleqa-1466
b8f0fa9
simpleqa-1467
uvx harbor run -d simpleqa@1.0 -t simpleqa-1467
b8f0fa9
simpleqa-1468
uvx harbor run -d simpleqa@1.0 -t simpleqa-1468
b8f0fa9
simpleqa-1469
uvx harbor run -d simpleqa@1.0 -t simpleqa-1469
b8f0fa9
simpleqa-147
uvx harbor run -d simpleqa@1.0 -t simpleqa-147
b8f0fa9
simpleqa-1470
uvx harbor run -d simpleqa@1.0 -t simpleqa-1470
b8f0fa9
simpleqa-1471
uvx harbor run -d simpleqa@1.0 -t simpleqa-1471
b8f0fa9
simpleqa-1472
uvx harbor run -d simpleqa@1.0 -t simpleqa-1472
b8f0fa9
simpleqa-1473
uvx harbor run -d simpleqa@1.0 -t simpleqa-1473
b8f0fa9
simpleqa-1474
uvx harbor run -d simpleqa@1.0 -t simpleqa-1474
b8f0fa9
simpleqa-1475
uvx harbor run -d simpleqa@1.0 -t simpleqa-1475
b8f0fa9
simpleqa-1476
uvx harbor run -d simpleqa@1.0 -t simpleqa-1476
b8f0fa9
simpleqa-1477
uvx harbor run -d simpleqa@1.0 -t simpleqa-1477
b8f0fa9
simpleqa-1478
uvx harbor run -d simpleqa@1.0 -t simpleqa-1478
b8f0fa9
simpleqa-1479
uvx harbor run -d simpleqa@1.0 -t simpleqa-1479
b8f0fa9
simpleqa-148
uvx harbor run -d simpleqa@1.0 -t simpleqa-148
b8f0fa9
simpleqa-1480
uvx harbor run -d simpleqa@1.0 -t simpleqa-1480
b8f0fa9
simpleqa-1481
uvx harbor run -d simpleqa@1.0 -t simpleqa-1481
b8f0fa9
simpleqa-1482
uvx harbor run -d simpleqa@1.0 -t simpleqa-1482
b8f0fa9
simpleqa-1483
uvx harbor run -d simpleqa@1.0 -t simpleqa-1483
b8f0fa9
simpleqa-1484
uvx harbor run -d simpleqa@1.0 -t simpleqa-1484
b8f0fa9
simpleqa-1485
uvx harbor run -d simpleqa@1.0 -t simpleqa-1485
b8f0fa9
simpleqa-1486
uvx harbor run -d simpleqa@1.0 -t simpleqa-1486
b8f0fa9
simpleqa-1487
uvx harbor run -d simpleqa@1.0 -t simpleqa-1487
b8f0fa9
simpleqa-1488
uvx harbor run -d simpleqa@1.0 -t simpleqa-1488
b8f0fa9
simpleqa-1489
uvx harbor run -d simpleqa@1.0 -t simpleqa-1489
b8f0fa9
simpleqa-149
uvx harbor run -d simpleqa@1.0 -t simpleqa-149
b8f0fa9
simpleqa-1490
uvx harbor run -d simpleqa@1.0 -t simpleqa-1490
b8f0fa9
simpleqa-1491
uvx harbor run -d simpleqa@1.0 -t simpleqa-1491
b8f0fa9
simpleqa-1492
uvx harbor run -d simpleqa@1.0 -t simpleqa-1492
b8f0fa9
simpleqa-1493
uvx harbor run -d simpleqa@1.0 -t simpleqa-1493
b8f0fa9
simpleqa-1494
uvx harbor run -d simpleqa@1.0 -t simpleqa-1494
b8f0fa9
simpleqa-1495
uvx harbor run -d simpleqa@1.0 -t simpleqa-1495
b8f0fa9
simpleqa-1496
uvx harbor run -d simpleqa@1.0 -t simpleqa-1496
b8f0fa9
simpleqa-1497
uvx harbor run -d simpleqa@1.0 -t simpleqa-1497
b8f0fa9
simpleqa-1498
uvx harbor run -d simpleqa@1.0 -t simpleqa-1498
b8f0fa9
simpleqa-1499
uvx harbor run -d simpleqa@1.0 -t simpleqa-1499
b8f0fa9
simpleqa-15
uvx harbor run -d simpleqa@1.0 -t simpleqa-15
b8f0fa9
simpleqa-150
uvx harbor run -d simpleqa@1.0 -t simpleqa-150
b8f0fa9
simpleqa-1500
uvx harbor run -d simpleqa@1.0 -t simpleqa-1500
b8f0fa9
simpleqa-1501
uvx harbor run -d simpleqa@1.0 -t simpleqa-1501
b8f0fa9
simpleqa-1502
uvx harbor run -d simpleqa@1.0 -t simpleqa-1502
b8f0fa9
simpleqa-1503
uvx harbor run -d simpleqa@1.0 -t simpleqa-1503
b8f0fa9
simpleqa-1504
uvx harbor run -d simpleqa@1.0 -t simpleqa-1504
b8f0fa9
simpleqa-1505
uvx harbor run -d simpleqa@1.0 -t simpleqa-1505
b8f0fa9
simpleqa-1506
uvx harbor run -d simpleqa@1.0 -t simpleqa-1506
b8f0fa9
simpleqa-1507
uvx harbor run -d simpleqa@1.0 -t simpleqa-1507
b8f0fa9
simpleqa-1508
uvx harbor run -d simpleqa@1.0 -t simpleqa-1508
b8f0fa9
simpleqa-1509
uvx harbor run -d simpleqa@1.0 -t simpleqa-1509
b8f0fa9
simpleqa-151
uvx harbor run -d simpleqa@1.0 -t simpleqa-151
b8f0fa9
simpleqa-1510
uvx harbor run -d simpleqa@1.0 -t simpleqa-1510
b8f0fa9
simpleqa-1511
uvx harbor run -d simpleqa@1.0 -t simpleqa-1511
b8f0fa9
simpleqa-1512
uvx harbor run -d simpleqa@1.0 -t simpleqa-1512
b8f0fa9
simpleqa-1513
uvx harbor run -d simpleqa@1.0 -t simpleqa-1513
b8f0fa9
simpleqa-1514
uvx harbor run -d simpleqa@1.0 -t simpleqa-1514
b8f0fa9
simpleqa-1515
uvx harbor run -d simpleqa@1.0 -t simpleqa-1515
b8f0fa9
simpleqa-1516
uvx harbor run -d simpleqa@1.0 -t simpleqa-1516
b8f0fa9
simpleqa-1517
uvx harbor run -d simpleqa@1.0 -t simpleqa-1517
b8f0fa9
simpleqa-1518
uvx harbor run -d simpleqa@1.0 -t simpleqa-1518
b8f0fa9
simpleqa-1519
uvx harbor run -d simpleqa@1.0 -t simpleqa-1519
b8f0fa9
simpleqa-152
uvx harbor run -d simpleqa@1.0 -t simpleqa-152
b8f0fa9
simpleqa-1520
uvx harbor run -d simpleqa@1.0 -t simpleqa-1520
b8f0fa9
simpleqa-1521
uvx harbor run -d simpleqa@1.0 -t simpleqa-1521
b8f0fa9
simpleqa-1522
uvx harbor run -d simpleqa@1.0 -t simpleqa-1522
b8f0fa9
simpleqa-1523
uvx harbor run -d simpleqa@1.0 -t simpleqa-1523
b8f0fa9
simpleqa-1524
uvx harbor run -d simpleqa@1.0 -t simpleqa-1524
b8f0fa9
simpleqa-1525
uvx harbor run -d simpleqa@1.0 -t simpleqa-1525
b8f0fa9
simpleqa-1526
uvx harbor run -d simpleqa@1.0 -t simpleqa-1526
b8f0fa9
simpleqa-1527
uvx harbor run -d simpleqa@1.0 -t simpleqa-1527
b8f0fa9
simpleqa-1528
uvx harbor run -d simpleqa@1.0 -t simpleqa-1528
b8f0fa9
simpleqa-1529
uvx harbor run -d simpleqa@1.0 -t simpleqa-1529
b8f0fa9
simpleqa-153
uvx harbor run -d simpleqa@1.0 -t simpleqa-153
b8f0fa9
simpleqa-1530
uvx harbor run -d simpleqa@1.0 -t simpleqa-1530
b8f0fa9
simpleqa-1531
uvx harbor run -d simpleqa@1.0 -t simpleqa-1531
b8f0fa9
simpleqa-1532
uvx harbor run -d simpleqa@1.0 -t simpleqa-1532
b8f0fa9
simpleqa-1533
uvx harbor run -d simpleqa@1.0 -t simpleqa-1533
b8f0fa9
simpleqa-1534
uvx harbor run -d simpleqa@1.0 -t simpleqa-1534
b8f0fa9
simpleqa-1535
uvx harbor run -d simpleqa@1.0 -t simpleqa-1535
b8f0fa9
simpleqa-1536
uvx harbor run -d simpleqa@1.0 -t simpleqa-1536
b8f0fa9
simpleqa-1537
uvx harbor run -d simpleqa@1.0 -t simpleqa-1537
b8f0fa9