# Claude for AI Product Managers: 36 Claude Skills to Test an AI Feature Before It Ships From "do we even need AI" to the weekly transcript read — 36 skills that spec an AI feature as tests, build its evals from real outputs and leave the verdict to a named person. - Page: https://lantern.meet-polar-bear.com/skills/ai-product-managers-pack - Download: https://lantern.meet-polar-bear.com/skills/ai-product-managers-pack.zip - Source: https://github.com/polar-bear-org/claude-skills/tree/main/packs/ai-product-managers-pack - Price: free ## Install in Claude Code ``` /plugin marketplace add polar-bear-org/claude-skills /plugin install ai-product-managers-pack@polar-bear-skills ``` ## The 36 skills you get The set follows the work, in seven stages: deciding if AI fits, speccing it, briefing the model and the agent, building the evals, shipping it safely, running and improving it, then cost, price and explaining it. ### 1 · Decide if AI fits - **AI Use Case Canvas** — use when: Someone wants “an AI feature” and nobody has said what decision it improves. Output: The canvas — prediction, judgment, action, outcome, input, feedback — a simpler-fix check, a data check, and a verdict. - **AI Failure Modes Map** — use when: Leaders expect it to be right every time and you need to show which mistakes matter. Output: A failure list per output, the cost of a false yes against a false no, severity and detection, a graceful failure, and the tolerable rate left blank for you. - **Workflow or Agent Decision** — use when: The team wants an agent and nobody asked whether a fixed workflow would do. Output: The candidate patterns, their cost, latency and failure trade-offs, the simplest pattern that passes, and the trigger for the next step up. - **AI Model Selection** — use when: You must justify the model tier and the bill before engineering commits. Output: A side-by-side test plan on real inputs, a quality, latency and cost table per tier, a pick per task, and a re-check date. - **AI Prototype Brief** — use when: Leadership saw a slick AI demo and thinks production is just more prompts. Output: What the demo tests, a pass line set in advance, a cherry-picked input check, and the gap-to-production list. ### 2 · Spec it - **AI PRD** — use when: The AI feature spec reads like a normal PRD and nobody can test it. Output: The problem, the inputs the model sees, the outputs, a behavior summary, fallbacks, data needs, the release bar, and the open questions. - **AI Success Criteria** — use when: The team says “make it good” and nobody can say what good means. Output: Measurable criteria per dimension, today's baseline, a target set by a named person, and how each one is measured. - **AI Behavior Contract** — use when: “It should be helpful and safe” is the whole spec. Output: Must, must-never and when-unsure lines written as Given-When-Then cases, a pass rate over repeated runs, and the guardrails. - **Human-in-the-Loop Design** — use when: A wrong move is costly and nobody has drawn where a person steps in. Output: The approve, edit and take-over points, their triggers, the handoff message, a response time a person can actually meet, and the log. - **AI Data Privacy Brief** — use when: Legal asks what user data reaches the model and nobody has drawn it. Output: The data flow from user to model to logs, the personal data in prompts, retention, region, and the questions for legal. ### 3 · Brief the model and the agent - **System Prompt Brief** — use when: The prompt grew by patches and nobody knows which line does what. Output: Role, audience, task, rules, examples, output format, refusals and handoffs, plus a critique of the prompt you have now. - **Context Engineering Brief** — use when: Answers are wrong because the model read stale or irrelevant documents. Output: The knowledge sources and their owners, freshness rules, what to leave out, retrieval spot checks, and a context budget. - **Agent Spec** — use when: The agent can act on its own and nobody has written where it must stop. Output: The goal, the allowed tools, permissions, stop conditions, a spend budget, the approval points, and the never-alone list. - **Tool Descriptions** — use when: The agent picks the wrong tool or misreads what a tool returns. Output: Per tool: name, purpose, inputs, outputs and error messages, plus namespacing, an overlap check, and three test tasks. ### 4 · Build the evals - **Error Analysis** — use when: Quality is judged by vibes and the dashboard tracks scores that match no real problem. Output: Notes on real outputs, open codes, failure types grouped, counts per type, and the three to fix first. - **Golden Dataset** — use when: Nobody can tell if a new prompt helped the real workflow or only the demo. Output: 20 to 50 real cases with their expected outcomes, coverage by failure type, the source and version, and add-and-retire rules. - **Synthetic Test Data** — use when: Real cases are too thin or too sensitive to cover the edge cases. Output: Generated cases for coverage gaps only, each marked synthetic, checked by a person, and kept out of the headline score. - **Eval Rubric** — use when: Scores on a 1 to 5 scale move and nobody knows what changed. Output: One pass-or-fail check per failure type, a grader per check, a pass and a fail example, and the blockers. - **LLM-as-a-Judge Prompt** — use when: You want automated grading but cannot prove the judge agrees with your experts. Output: A judge prompt for one failure type, a labelled set, agreement measured on a held-out split, and bias checks. - **AI Eval Plan** — use when: Everyone agrees evals matter and nobody owns them. Output: The capability and regression suites, when each one runs, what blocks a release, and the named owner of quality. ### 5 · Ship it safely - **AI Red Teaming Plan** — use when: An agent will read emails, tickets or web pages written by strangers. Output: Attack cases by risk class, who runs them, the pass line, and the fixes due before launch. - **AI UX Review** — use when: Users rephrase, give up or leave, and the design never said what happens when it is wrong. Output: 18 interaction guidelines checked by phase, an AI disclosure check, the correction and handoff paths, and the fixes ranked. - **AI Impact Assessment** — use when: A customer, a regulator or your legal team asks for an impact assessment. Output: Who is affected, intended use and misuse, the data, oversight, mitigations, the residual risk owner, and the questions for an adviser. - **AI Risk Register** — use when: Legal and security ask for the risks and you have a list of worries. Output: Cause, event and effect risks mapped to Map, Measure and Manage, each with an owner, a trigger and a response. - **Questions for Legal** — use when: You need legal sign-off and do not know what to ask or what to bring. Output: The questions for the legal and privacy team by topic, with the facts attached — it asks, never answers. - **AI Launch Checklist** — use when: Launch is close, the first gate has not opened, and nobody has checked the boring things. Output: A go-live list with an owner per line, from evals passed as run to the rollback rehearsed. - **AI Launch Gates** — use when: The feature is ready to go out and nobody has written what would stop it. Output: The stages, the eval and live numbers per gate, the kill criteria, the rollback path, and who signs each gate. ### 6 · Run and improve - **Weekly AI Quality Review** — use when: It worked for weeks, then broke, and nobody had been reading the transcripts. Output: A transcript sample plan, the review notes, the new failure types, the cases added to the golden dataset, and the decisions. - **AI Feedback Signals** — use when: Users rarely rate answers and you cannot see what they think. Output: A signal map — accept, edit, retry, rephrase, abandon, escalate, rating — what to log, a weekly readout, and the review triggers. - **Prompt Regression Test** — use when: A prompt edit fixed one case and quietly broke another. Output: The change note, the cases to re-run, before and after per failure type, and a ship-or-hold for a named person. - **Model Migration Plan** — use when: The model you depend on has a retirement date. Output: The deadline, the breaking changes, the eval rerun, the cost difference, the behavior diffs to read, and the rollout and fallback. - **AI Incident Response Plan** — use when: The AI told a customer something wrong and it is spreading. Output: Severity levels, containment, the customer correction, an evidence timeline, the fix carried into the golden dataset, and a blameless review. ### 7 · Cost, price and explain - **AI Unit Economics** — use when: The feature passed every eval and finance still wants to stop it. Output: Cost per call, per task and per successful outcome, the heavy-user case, the caching and batch levers, and what it costs today. - **AI Usage and Pricing Test** — use when: You need run cost and price options before you have real usage. Output: A small real-task test, a usage estimate at light, normal and heavy use, the price options to test, and what to re-measure. - **AI Feature Card** — use when: Legal, sales and support each ask what the feature does and where it fails. Output: Intended and out-of-scope use, the data, the eval results as run with dates, the known limits, the disclosures, and the owner. - **AI Exec Brief** — use when: Leaders ask “how accurate is it” and one number would mislead them. Output: One page, bottom line first: how often it is wrong by severity, what that costs, the run cost, and the decision being asked for. ## Workflows - **Decide if AI fits** (Someone asked for “an AI feature”.) AI Use Case Canvas → AI Failure Modes Map → Workflow or Agent Decision → AI Model Selection - **Write a testable spec** (The feature needs a spec you can test.) AI PRD → AI Success Criteria → AI Behavior Contract → Human-in-the-Loop Design - **Brief the model and the agent** (The system prompt grew by patches.) System Prompt Brief → Context Engineering Brief → Agent Spec → Tool Descriptions - **Build the evals** (You have real transcripts and nothing to measure them with.) Error Analysis → Golden Dataset → Eval Rubric → LLM-as-a-Judge Prompt → AI Eval Plan - **Ship it safely** (Launch is close and it needs gates that can stop it.) AI Red Teaming Plan → AI UX Review → AI Launch Gates - **Keep it working** (The feature is live.) Weekly AI Quality Review → AI Feedback Signals → Prompt Regression Test - **Cost and explain it** (Finance, legal and leadership have questions.) AI Unit Economics → AI Feature Card → AI Exec Brief ## The quality bar - One method, applied properly: each skill uses the real mechanics of its source — the canvas fields, the pattern list, the coding passes, the capability and regression split, the risk classes — not a generic “gather, analyse, recommend” - Real outputs first: failure types, golden cases and judge checks come from outputs you paste, and synthetic cases fill gaps, are marked synthetic, and never carry the headline score - No invented numbers: no made-up scores, prices, error rates or usage, and an eval result appears only when you pasted the run behind it - Errors by severity and cost per successful outcome, never a single accuracy figure on its own or a cost per call without the retries and failures behind it - Work is scored, people are not: skills score outputs, cases, risks and options, never reviewers, users or team members - Legal, privacy and regulatory points go to a qualified adviser — the skills ask the questions and never answer them - The red line: Claude drafts the tests, never the verdict — it does not set the error your users can live with, sign off a launch, or report a score it did not see run on real outputs, and a person who knows the domain reads real transcripts every week ## Questions people ask ### What is in the Claude for AI Product Managers pack? 36 Claude skills that follow AI product work in seven stages: decide if AI fits, spec it, brief the model and the agent, build the evals, ship it safely, run and improve, and cost, price and explain. Each skill is one named method or one artifact. ### How do I install these Claude skills? In Claude Code, two lines install all 36 at once. In Claude.ai or Claude Desktop, download the zip, open Settings, Skills, Add, and upload the skills you want. Either way it takes about five minutes. ### Do I need Claude Code to use them? No. The skills work in Claude.ai, Claude Desktop and Claude Code. In the app you upload them yourself; in Claude Code one command installs the whole pack. ### Do the skills work with ChatGPT or other assistants? They are built as Claude Skills, and installing them is a Claude feature. Each one is a plain Markdown file, so you can read it and reuse the method elsewhere, but it will not load and trigger by itself outside Claude. ### Will Claude sign off the launch? No. Claude drafts the tests, never the verdict. It does not set the error your users can live with, sign off a launch, or report a score it did not see run on real outputs. A person who knows the domain reads real transcripts every week. ### Is the pack free? Yes. It is free to use inside your company and in client work, and not for resale. There is no signup: the download and the install command are on this page. ## About Written by Polar Bear (https://meet-polar-bear.com/). Published on Lantern (https://lantern.meet-polar-bear.com/), which turns your notes, documents and call transcripts into social post drafts you review and publish.