CLI: Leaderboards

When creating a leaderboard, optionally restrict it to specific dataset-version UUIDs in the create config:

dataset_version_ids:
  - 11111111-1111-4111-8111-111111111111
dataset_version_refs:
  - latest

IDs and refs may be combined; the API resolves refs and stores their UUIDs. Omitting both fields associates every version that exists when the leaderboard is created. Setting both to empty lists associates none. Versions published later are never added automatically.

Use harbor hub leaderboard show BOARD to display a curated leaderboard, or add --json to print the complete read API response. BOARD may be a UUID or an org/package/name slug.

Export an update-ready YAML or JSON definition, edit it, and apply it:

harbor hub leaderboard export BOARD --output board.yaml
harbor hub leaderboard update BOARD --config board.yaml

Simple definition fields can be updated directly; these flags override values from --config when both are provided:

harbor hub leaderboard update BOARD \
  --title "New title" \
  --description "New description" \
  --visibility private

Rows have the same round-trip workflow. Use --all to export every row:

harbor hub leaderboard row list BOARD
harbor hub leaderboard row create BOARD --config new-rows.yaml
harbor hub leaderboard row export ROW_ID --output row.yaml
harbor hub leaderboard row update ROW_ID --config row.yaml
harbor hub leaderboard row update ROW_ID --status hide

harbor hub leaderboard row export BOARD --all --output rows.yaml
harbor hub leaderboard update BOARD --config board.yaml --rows rows.yaml

row list shows canonical ranks, configured leaderboard columns, status, trial count, timestamps, and row IDs. It supports --limit, --page, --json, and --quiet using the same paging behavior as hub job list.

The create config contains a rows list. Each row accepts metadata, metrics, status, and optional trial_ids. You can also create a leaderboard and its initial rows atomically with harbor hub leaderboard create --rows new-rows.yaml.

Definition and batch row changes commit atomically. A schema change is rejected when any resulting row is invalid. Delete incompatible rows explicitly with harbor hub leaderboard row delete, or update them in the same command with --rows. Use --dry-run with the combined --config and --rows migration form to validate it without committing.

The API mirrors these resource boundaries: leaderboard-update changes only the definition; row create, read, update, delete, and trial changes use their dedicated endpoints. Only leaderboard-migrate accepts definition and row changes together.

Manage row provenance separately:

harbor hub leaderboard row trial list ROW_ID
harbor hub leaderboard row trial set ROW_ID --trial-id TRIAL_ID
harbor hub leaderboard row trial add ROW_ID --trial-id TRIAL_ID
harbor hub leaderboard row trial remove ROW_ID --trial-id TRIAL_ID

Leaderboard reads and mutation responses return n_trials rather than every association. row trial list pages through the associations; use --json for one page or --quiet to stream every trial ID.

set can also read the complete replacement list from YAML or JSON:

trial_ids:
  - 11111111-1111-1111-1111-111111111111
  - 22222222-2222-2222-2222-222222222222

harbor hub leaderboard row trial set ROW_ID --trial-ids-file trials.yaml

set replaces every association; use set --clear to remove them all. Trial changes do not recompute row metadata or metrics.