Labelyst

Repeatable local checks

Run a dataset quality gate before training.

Keep the decision rule with the dataset workflow. Labelyst runs locally in the browser or on your runner and records what was checked, what was limited, and what blocked release.

1. Define the acceptance contract

This example requires image, label, and split values; rejects exact cross-split duplicates; requires 25 validation images per class; and blocks analyzer errors. Tune the rule only when your team agrees on the change.

{
  "version": 1,
  "name": "Vision delivery v1",
  "requiredFields": [
    "image",
    "label",
    "split"
  ],
  "prohibitExactCrossSplitDuplicates": true,
  "minValidationImagesPerClass": 25,
  "failOnCriticalErrors": true
}
Download this contract

2. Run the gate and save evidence

npm run quality -- gate data/labels.csv --min-score 80 --contract contracts/vision-delivery-v1.contract.json --json labelyst-result.json --sarif labelyst.sarif --html labelyst-report.html

The minimum score is one gate. Contract failures and incomplete evidence can also block release. Create a portable evidence archive with npm run quality -- evidence create labelyst-result.json --out labelyst-evidence.zip.

JSON resultSARIFHTML reportEvidence archiveSample notes

3. Add the same gate to GitHub Actions

name: Dataset quality

on:
  pull_request:

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - id: labelyst
        uses: ./
        with:
          dataset: data/labels.csv
          min-score: "80"
          contract: contracts/vision-delivery-v1.contract.json
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: labelyst-quality
          path: labelyst-artifacts

The workflow uploads results even when the gate fails, so a reviewer can inspect the decision without sending dataset contents to Labelyst.

Read the outcome before acting

Exit 0: Ready

The configured checks and contract passed.

Exit 1: Ready with warnings

The gate passed, with findings to review.

Exit 2: Blocked

The score or contract blocked this dataset.

Exit 3: Analysis incomplete

Required evidence was unavailable or limited.

Exit 4: Invalid contract

The contract file could not be accepted.

The report shows score contributions, checks that did not apply, coverage limits, findings, and repairs. It is evidence from configured checks, not a certification or a prediction of model performance.

Check a dataset freeRead CLI documentation