mirror of
https://github.com/NanmiCoder/cc-haha
synced 2026-07-18 13:23:33 +08:00
9719726cd2
The repository now has a measurable PR quality path instead of a loose set of manual checks. Coverage, quarantine governance, provider smoke, desktop smoke, and workflow wiring all produce durable reports that contributors and maintainers can inspect without reconstructing terminal output. This also fixes the desktop smoke current-runtime path so browser-driven smoke runs use the desktop default active provider instead of forcing the official current model, and records that runtime decision as an artifact. Constraint: Default PR gates must remain non-live and contributor-safe while live model checks stay explicit. Constraint: Release packaging is still GitHub Actions based, so release preflight must run before the build matrix. Rejected: Make live provider or desktop smoke mandatory on every PR | secrets, quotas, and model availability are maintainer-controlled. Rejected: Let PRs lower coverage baselines in the same change | base-branch ratchet comparison must remain authoritative. Confidence: high Scope-risk: moderate Directive: Do not relax coverage or quarantine policy without a maintainer approval label and a fresh quality report. Tested: ALLOW_CLI_CORE_CHANGE=1 ALLOW_COVERAGE_BASELINE_CHANGE=1 bun run quality:gate --mode pr Tested: bun run quality:gate --mode baseline --allow-live --only provider-smoke:* --provider-model nvidia-custom:main:nvidia-custom-main --artifacts-dir /tmp/quality-gate-live-smoke Tested: bun run quality:gate --mode baseline --allow-live --only desktop-smoke:* --provider-model current:current:current-runtime --artifacts-dir /tmp/quality-gate-desktop-smoke-fixed Tested: git diff --check Not-tested: Full live release mode with multiple providers in hosted CI; provider credentials and quota remain maintainer-controlled.
193 lines
7.1 KiB
Markdown
193 lines
7.1 KiB
Markdown
# Contributing and Local Quality Gates
|
|
|
|
This guide explains how to install, develop, test, and run the local quality gates before opening a PR. The goal is to help maintainers and contributors answer one question before review: did this change break the core Coding Agent workflow?
|
|
|
|
## Setup
|
|
|
|
Install root dependencies with Bun:
|
|
|
|
```bash
|
|
bun install
|
|
```
|
|
|
|
If your change touches `desktop/`, also install desktop dependencies:
|
|
|
|
```bash
|
|
cd desktop
|
|
bun install
|
|
```
|
|
|
|
If your change touches `adapters/`, or if you run `check:adapters` / `check:native`, install adapter dependencies:
|
|
|
|
```bash
|
|
cd adapters
|
|
bun install
|
|
```
|
|
|
|
Do not commit local artifacts such as `artifacts/quality-runs/`, `node_modules/`, or `desktop/node_modules/`.
|
|
|
|
## Required PR Gate
|
|
|
|
Before opening a normal PR, run:
|
|
|
|
```bash
|
|
bun run quality:pr
|
|
```
|
|
|
|
This gate does not call real models, so every contributor can run it locally. It writes reports to:
|
|
|
|
```text
|
|
artifacts/quality-runs/<timestamp>/report.md
|
|
artifacts/quality-runs/<timestamp>/report.json
|
|
artifacts/quality-runs/<timestamp>/junit.xml
|
|
artifacts/quality-runs/<timestamp>/logs/*.log
|
|
artifacts/coverage/<timestamp>/coverage-report.md
|
|
artifacts/coverage/<timestamp>/coverage-report.json
|
|
```
|
|
|
|
Include the commands you ran and the report summary in your PR description.
|
|
|
|
`quality:pr` also runs quarantine and coverage gates. Coverage uses a ratchet policy: the current baseline lives in `scripts/quality-gate/coverage-baseline.json`, and CI compares against the base branch baseline when available. New PRs must not lower overall coverage beyond the allowed window. Changes to `coverage-baseline.json` or `coverage-thresholds.json` require the maintainer-only `allow-coverage-baseline-change` label. Quarantine entries must keep an owner, reviewAfter date, and exit criteria; once reviewAfter expires, the default server and coverage gates fail until maintainers review the entry.
|
|
|
|
## Area-Specific Checks
|
|
|
|
Run the checks that match the files you changed:
|
|
|
|
```bash
|
|
bun run check:server # Server API, WebSocket, providers, sessions, and related tests
|
|
bun run check:desktop # Desktop lint, Vitest, and production build
|
|
bun run check:adapters # IM adapter tests
|
|
bun run check:native # Desktop sidecars and Tauri native checks
|
|
bun run check:docs # Docs build, using npm ci + docs:build
|
|
bun run check:quarantine # Quarantine owners, exit criteria, and review windows
|
|
bun run check:coverage # Root, desktop, and adapter coverage reports plus ratchet enforcement
|
|
```
|
|
|
|
Focused tests are fine while developing, but run `bun run quality:pr` before sending the PR.
|
|
|
|
Production code changes must include matching tests. Changes under `desktop/src/**`, `src/server/**`, `src/tools/**`, `src/utils/**`, or `adapters/**` without a same-area test file are blocked unless a maintainer applies `allow-missing-tests`. Coverage baseline/threshold changes are also blocked unless a maintainer applies `allow-coverage-baseline-change`.
|
|
|
|
## Live Model Baseline
|
|
|
|
`quality:baseline` runs real Coding Agent tasks: it starts the local server, creates isolated fixtures, asks a model through chat to fix code, runs tests, and saves transcripts, diffs, verification logs, and a report. It also runs provider live smoke: saved or active OpenAI-compatible providers validate connectivity, proxy conversion, and streaming proxy behavior; env-only provider smoke validates upstream connectivity and the transform pipeline.
|
|
|
|
The default baseline command does not call real models:
|
|
|
|
```bash
|
|
bun run quality:baseline
|
|
```
|
|
|
|
To actually call models, pass `--allow-live` and choose a local provider.
|
|
|
|
First list your local providers and copyable selectors:
|
|
|
|
```bash
|
|
bun run quality:providers
|
|
```
|
|
|
|
Example output:
|
|
|
|
```text
|
|
Saved providers:
|
|
MiniMax
|
|
selector: minimax
|
|
main: MiniMax-M2.7-highspeed
|
|
--provider-model minimax:main:minimax-main
|
|
```
|
|
|
|
Copy one of the listed values:
|
|
|
|
```bash
|
|
bun run quality:gate --mode baseline --allow-live --provider-model minimax:main:minimax-main
|
|
```
|
|
|
|
To run only provider smoke plus desktop agent-browser smoke, use:
|
|
|
|
```bash
|
|
bun run quality:smoke --provider-model minimax:main:minimax-main
|
|
```
|
|
|
|
You can run multiple models in one pass:
|
|
|
|
```bash
|
|
bun run quality:gate --mode baseline --allow-live \
|
|
--provider-model codingplan:main:codingplan-main \
|
|
--provider-model minimax:main:minimax-main
|
|
```
|
|
|
|
Provider selectors come from the providers saved in your local Desktop Settings > Providers page. Contributors do not need the maintainer's provider UUIDs or vendor accounts. They can add their own provider locally, run `bun run quality:providers`, and choose their own model.
|
|
|
|
If you do not have a saved provider, you can run one unsaved provider smoke with environment variables:
|
|
|
|
```bash
|
|
QUALITY_GATE_PROVIDER_BASE_URL=https://example.com \
|
|
QUALITY_GATE_PROVIDER_API_KEY=... \
|
|
QUALITY_GATE_PROVIDER_MODEL=model-id \
|
|
QUALITY_GATE_PROVIDER_API_FORMAT=openai_chat \
|
|
bun run quality:gate --mode baseline --allow-live
|
|
```
|
|
|
|
## When To Run The Baseline
|
|
|
|
Run the live baseline for changes touching:
|
|
|
|
- Desktop chat, session resume, WebSocket, or the CLI bridge
|
|
- Provider, model, or runtime selection
|
|
- Permissions, tool calls, file edits, and task execution
|
|
- agent-browser smoke, Computer Use, Skills, or MCP
|
|
- Release preparation or broad cross-module refactors
|
|
|
|
If you do not have model access, still run `bun run quality:pr` and state in the PR why the live baseline was not run.
|
|
|
|
## Release Gate
|
|
|
|
Before a release, run release mode:
|
|
|
|
```bash
|
|
bun run quality:gate --mode release --allow-live --provider-model <selector>:main
|
|
```
|
|
|
|
Release mode composes PR checks, baseline catalog validation, live baseline cases, provider smoke, desktop smoke, and native checks. Reports are written to `artifacts/quality-runs/<timestamp>/`. The hosted release workflow now runs `quality:gate --mode pr` as a non-live preflight before the packaging matrix and uploads a `release-quality-gate` artifact; maintainers still need to run the live release gate explicitly with an available provider.
|
|
|
|
In release mode, live lanes are not allowed to be silently skipped. Missing providers, model quota, or external account access will fail the gate and must be recorded as a release blocker.
|
|
|
|
## PR Workflow
|
|
|
|
1. Create a product branch such as `fix/session-reconnect` or `feat/provider-quality-gate`.
|
|
2. Install dependencies and make the change.
|
|
3. Add tests for behavior changes.
|
|
4. Run focused checks for the affected area.
|
|
5. Run `bun run quality:pr`.
|
|
6. Run the live baseline for high-risk changes.
|
|
7. In the PR description, include user impact, verification commands, coverage/quality report summary, and known risks.
|
|
|
|
## FAQ
|
|
|
|
### Can I run checks without a provider?
|
|
|
|
Yes. Run the normal PR gate:
|
|
|
|
```bash
|
|
bun run quality:pr
|
|
```
|
|
|
|
Only the live baseline needs a real model. Add your provider in Desktop Settings > Providers, then run:
|
|
|
|
```bash
|
|
bun run quality:providers
|
|
```
|
|
|
|
### What if provider selectors conflict?
|
|
|
|
If two provider names produce the same selector, `quality:providers` falls back to the provider ID. Copy the `--provider-model ...` value it prints.
|
|
|
|
### What if a model ID contains a colon?
|
|
|
|
Prefer role selectors:
|
|
|
|
```bash
|
|
--provider-model custom:haiku:custom-haiku
|
|
```
|
|
|
|
The runner resolves `haiku` to the real model ID from your local provider configuration.
|