ai-lead-ops

Dieser Auszug wurde bei der Prüfung gespeichert. Die externe Quelle enthält die vollständige und aktuelle Version.

---
name: ai-lead-ops
description: |
  Guides AI ops leadership—LLM SRE, model/prompt releases, eval/incidents, cost/capacity, vendors, and
  cross-functional cadence. Use for AI platform ops, LLM SLAs, incidents, rollout governance, unit
  economics, red-team/eval gates, and team rituals—not memory (ai-memory-developer), context code
  (ai-context-engineer), security programs (cybersecurity), token roadmaps
  (ai-token-improvement-plan-engineer), solution architecture (applied-ai-architect-commercial-enterprise),
  skills portfolio (ai-skill-manager), or vertical AI product eng management
  (engineering-manager-vertical-ai-products).
  Prompt/eval team management and golden-set release policy: engineering-manager-agent-prompts-evals.
  Safeguard inference platform: ml-infrastructure-engineer-safeguards. Safeguard model research:
  ml-research-engineer-safeguards.
---

# AI Lead Ops

## When to Use

- Standing up AI platform operations and production service reliability
- Defining SLAs/SLOs for LLM-powered features
- Running AI incident reviews and post-mortems
- Governing model, prompt, and index rollouts with tiered gates
- Tracking AI unit economics (cost per session, tokens per feature)
- Coordinating red-team and evaluation gates before releases
- Building team rituals and cadence across engineering, research, risk, and product
- Managing AI vendor relationships, contracts, and bake-offs

## When NOT to Use

- Implementing memory stores or context packing code → `ai-memory-developer` / `ai-context-engineer`
- Building RAG pipelines or agent tools → `ai-engineer`
- Designing corporate AI policy or regulatory mapping → `ai-risk-governance`
- General network penetration testing or enterprise security programs → `cybersecurity`
- Structured token/cost improvement roadmaps with backlog → `ai-token-improvement-plan-engineer`
- Commercial/enterprise AI solution architecture → `applied-ai-architect-commercial-enterprise`
- Vertical AI product engineering managers and squad roadmaps → `engineering-manager-vertical-ai-products`

## Related skills

| Need | Skill |
|---|---|
| Build RAG, agents, eval harnesses | `ai-engineer` |
| Memory and context implementation | `ai-memory-developer`, `ai-context-engineer` |
| Risk tiering and policies | `ai-risk-governance` |
| Adversarial testing execution | `ai-redteam` |
| CI/CD and platform incidents | `devops` |
| Pipeline security | `devsecops` |
| Token optimization roadmap and initiative backlog | `ai-token-improvement-plan-engineer` |
| Commercial/enterprise AI architecture | `applied-ai-architect-commercial-enterprise` |
| Skills portfolio governance | `ai-skill-manager` |
| Safeguard inference platform | `ml-infrastructure-engineer-safeguards` |
| Safety classifier research | `ml-research-engineer-safeguards` |

## Core Workflows

### 1. Operating model and cadence

| Ritual | Frequency | Outcomes |
|---|---|---|
| AI ops standup | Daily | Blockers, incidents, deploys |
| Model/prompt change review | Per release | Approvers, eval delta |
| Cost review | Weekly | Spend vs budget, top features |
| Risk & safety sync | Bi-weekly | Incidents, policy gaps |
| Quarterly capacity | Quarterly | Model roadmap, vendor contracts |

Define RACI: who owns model, prompt, index, eval suite, on-call.

**See `references/operating_model.md` for roles and escalation.**

### 2. Release governance

**Production promotion checklist:**

- [ ] Eval regression passed on golden + safety set
- [ ] Red-team sign-off for tier-2+ use cases
- [ ] Model card / change log updated
- [ ] Canary with error and cost monitors
- [ ] Rollback procedure tested (previous prompt + model version pinned)
- [ ] Comms plan for customer-visible behavior change

**See `references/release_governance.md` for tiered gates and canary metrics.**

### 3. SLOs, incidents, and observability

**Example SLIs:**

| SLI | Notes |
|---|---|
| Availability | Successful completion / total requests |
| Latency | p95 end-to-end |
| Quality proxy | T