# openspec/changes/wf-switch-models-with-proof/proposal.md

## Why

A new model changes behavior in ways the tests miss, and the first sign is a production incident. Model releases come every few weeks. This makes each switch a decision with evidence instead of a gamble.

## What Changes

- Add the `switch-models-with-proof` workflow to the PromptReports workflow library, run by the workflow runner as an agent acting for a person.
- PromptCompute: Register the new version from its manifest (today: Built).
- PromptCompute: Run it in a Test first shadow test (today: Built).
- RepoOps: Compare incidents per version (today: Built).
- ContextFile: Pin the context version both read (today: Built).
- PromptCompute: A person approves the switch (today: Built).
- AI Source of Truth: Record the comparison (today: Proposed).
- Gate `engine.switch` through PromptCompute approvals; each gated step runs only after `verifyResume` passes, with `final_args`.
- Write a signed record at the end of each run and count one workflow request.

## Impact

- Products: PromptCompute, RepoOps, ContextFile, AI Source of Truth.
- Outside the family: Your model provider.
- State: Proposed. Not in the plan. Agent versions and shadow tests exist in PromptCompute.
- Plan row: none yet (Proposed).

# openspec/changes/wf-switch-models-with-proof/tasks.md

## 1. Connect and authorize

- [ ] 1.1 Confirm the org has a connection for each of: PromptCompute, RepoOps, ContextFile, AI Source of Truth. Stop and tell the user which is missing.
- [ ] 1.2 Get a PromptReports ID agent token for each product the run calls (RFC 8693 exchange). Where a product does not accept ID tokens yet, use its connect key.
- [ ] 1.3 Register policy for `engine.switch` with `require_approval` as the default.

## 2. Build the steps

- [ ] 2.1 PromptCompute: Register the new version from its manifest.
- [ ] 2.2 PromptCompute: Run it in a Test first shadow test.
- [ ] 2.3 RepoOps: Compare incidents per version.
- [ ] 2.4 ContextFile: Pin the context version both read.
- [ ] 2.5 PromptCompute: A person approves the switch.
- [ ] 2.6 AI Source of Truth: Record the comparison. Blocked: this part is Proposed; until it ships, record the step's evidence as `missing` and continue.

## 3. Verify and record

- [ ] 3.1 Run a dry run with the user's inputs: Agent = billing-fixer; New manifest = model id, prompt hash.
- [ ] 3.2 Run it once for real; approve each gate as the user.
- [ ] 3.3 Measure the outcomes: Incidents in the 14 days after a switch; Switches made with a comparison attached: target 100%.
- [ ] 3.4 Confirm the run left a signed record with a task id and one workflow request counted.

# openspec/changes/wf-switch-models-with-proof/specs/workflow-switch-models-with-proof/spec.md

## ADDED Requirements

### Requirement: No switch without a comparison

The switch step SHALL run only when the approval carries a comparison of both versions.

#### Scenario: No switch without a comparison

- **WHEN** the shadow test has fewer than 50 requests
- **THEN** the card says the sample is small and the default action is wait

### Requirement: Evidence stays honest

Each step SHALL report what it saw as measured, inferred or missing, and SHALL never report missing as measured.

#### Scenario: A product part is not built

- **WHEN** a step depends on a part that is Planned or Proposed
- **THEN** the step records `missing`, names the part, and the approval card shows the gap

### Requirement: One run is one workflow request

The runner SHALL count one workflow request when the first step starts, and SHALL NOT count retries, resumes or dry runs.

#### Scenario: A gated step resumes after two days

- **WHEN** a person approves a step and the run resumes
- **THEN** the org's usage does not change