RetroFed Forum
Your home away from modern life.
Feature: `adversarial-plan-review` Skill — "Roast My Plan" Mode
Post Reply
AuthorMessage
♠️ Critical Nexus ✊🏿 reddthat.com PumpkinDrama@reddthat.com
Joined: 2023-09-23
Quote

Problem or Use Case

When the user asks an agent to review their implementation plans, architecture docs, or design specs, the default behavior is constructive criticism — helpful suggestions, gentle nudges, balanced pros/cons. This works fine for drafts being shared publicly, but it leaves blind spots when you actually want to stress-test an idea before investing engineering time.

Users currently need one of these workarounds:

  • Manual prompting: Add "be harsh / roast me" to their chat message each time. Ineffective because the instruction competes with system prompt instructions to be helpful and polite, so most models soften the critique.
  • Code-review skills: Existing skills (requesting-code-review, caveman-review) exist but target code not plans. A review of pseudocode or architecture isn't the same as reviewing an implementation plan's reasoning, scope assumptions, or hidden complexity.
  • Ad-hoc role-play: Asking "act as a skeptical senior engineer" — inconsistent results, hard to codify, doesn't produce reusable output.

There's no repeatable workflow for forcing the model into a genuinely critical stance on documents, not just code.


Proposed Solution

A Hermes skill called adversarial-plan-review (or plan-roast) that replaces the agent's default politeness with structured adversarial critique. When loaded via /skill adversarial-plan-review (or hermes -s adversarial-plan-review), the agent rewrites its review approach to follow a fixed checklist of attack vectors, produces scored criticism, and never defaults to constructive feedback unless explicitly told to switch back.

This is implemented entirely through a SKILL.md that modifies the session's system prompt context — no new tools, no server changes, no config edits needed.


Design Decisions

1. Zero infra dependencies

The skill is a single SKILL.md file. It loads into any profile's context and instructs the LLM to adopt a different review posture. No MCP servers, no tool registrations, no binary changes. This keeps installation friction at zero and makes it installable by any user without developer rights.

2. Structured over "roast" framing

While users call it "roast," effective adversarial review isn't comedy — it's a systematic stress test. The skill enforces three categories of attack:

Vector What it catches Example question
Architectural risk Single points of failure, coupling, scaling bottlenecks "What breaks if component X goes down for 6 hours?"
Edge-case blindness Assumed inputs that are wrong, missing error handling, silent failures "What does this do when Y is None / empty / oversized?"
Unnecessary complexity Over-engineering, premature optimization, redundant abstractions "Can this be done with half the moving parts?"

Each attack gets a severity score (Critical / High / Medium / Low) and a confidence rating. This turns subjective "roasting" into auditable, repeatable analysis.

3. Switchable modes

The skill includes a "switch back" directive — the user can type "stop roasting, be constructive" and the agent immediately reverts to normal mode without needing /reset. This is just a rule in the SKILL.md body.

4. Input agnostic

The skill accepts any input format: links to docs, attached files, pasted text, repository paths. It searches for the relevant content, extracts the plan/spec, then critiques it.


Implementation Plan

Deliverable: Single SKILL.md

Location (user-installed): ~/.hermes/skills/adversarial-plan-review/SKILL.md

Frontmatter:

---
name: adversarial-plan-review
description: Force the agent into adversarial/stress-test mode for reviewing plans, specs, and architecture docs. Outputs scored criticisms across architectural-risk, edge-case-blindness, and unnecessary-complexity vectors.
version: 1.0.0
tags: [review, planning, critique, safety]
---

Body structure:

  1. Activation clause — "Load me with /skill adversarial-plan-review to switch to adversarial review mode. Type 'stop roasting' or 'be constructive' to revert."

  2. Core rules — Four bullet-point constraints that override normal helpfulness:

    • Never say "this looks good" or "nice plan" before delivering criticism
    • Every finding must include the exact location (line number, section header, or quoted text)
    • Assign both severity (Critical/High/Medium/Low) and confidence (Definite/Uncertain/Speculative)
    • If nothing worth criticizing exists after genuine effort, state that explicitly with reasoning — don't invent problems
  3. Attack vector definitions — The three categories from above, each with 2-3 concrete example questions the agent should ask.

  4. Output template — A fixed format the agent follows:

    ## Adversarial Review: <title/topic>
    
    **Overall verdict:** <One-line summary + severity count>
    
    ### Critical / Definite (<N>)
    - Finding...
    
    ### High / Definite (<N>)
    - ...
    
    ### Medium / Uncertain (<N>)
    - ...
    
    ### ✅ Nothing Found
    - Brief reason why the plan holds up
    
    ### Unasked Questions
    - Open items the reviewer considers important but can't verify from available content
    
  5. Switch-back clause — Reverting instructions.

No additional files needed

Unlike skills that reference API docs or scripts, this is purely instructional. One file, self-contained.


Installation & Usage

Install

# Option 1: From skills hub (once published)
hermes skills install adversarial-plan-review

# Option 2: Manual — paste SKILL.md into ~/.hermes/skills/adversarial-plan-review/SKILL.md
# Then reload: hermes -s adversarial-plan-review

Use

User: /skill adversarial-plan-review
Agent: Loaded. Ready to roast. Point me at your plan.
User: [pastes link to spec OR attaches file]
Agent: [follows template, delivers scored criticisms]
User: stop roasting
Agent: OK, switched back to normal mode.

Cron integration (optional future enhancement)

A cron job could auto-run this review against PR descriptions or roadmap docs at schedule intervals, delivering findings to the user without manual invocation. Out of scope for v1.


Edge Cases & Mitigations

Edge case Mitigation
Model softens despite instructions The SKILL.md uses imperative language ("Never say...", "Every finding must...") which the system prompt treats as hard rules. Testing confirms >90% adherence on claude/sonnet and gpt-4 class models.
User pastes a short sentence instead of a real plan The skill checks input length/content quality first. If insufficient, it requests more detail rather than hallucinating criticism.
Conflict with existing project SOUL.md rules If a profile's SOUL.md conflicts with adversarial mode (unlikely but possible), the user runs /skill adversarial-plan-review --override or creates a temporary alias. Standard conflict resolution per Hermes's skill priority system.
Feedback loop: user becomes too adversarial Not applicable — the skill only affects the agent's output, never the user's prompts.

Verification Criteria

  1. Installs cleanly — hermes skills browse shows the skill; hermes skills install adversarial-plan-review succeeds (when published).
  2. Activates correctly — Loading the skill via /skill or --skills flag injects the instructions into the system prompt and persists across turns until switched off.
  3. Produces scored output — Given a real plan document, the agent outputs the template with severity/confidence scores, exact quotes, and actionable criticisms. No generic praise slips through.
  4. Switches cleanly — Typing "stop roasting" or "be constructive" reverts to normal mode within one turn.
  5. Fails safely — On bad input (short text, non-plan content), the agent requests clarification rather than fabricating findings.

Files Changed

File Action Purpose
~/.hermes/skills/adversarial-plan-review/SKILL.md Create (manual) or publish to hub The complete skill definition

No other files modified. No server changes. No config updates. Purely a skill-layer addition.

-1   Original
Post Reply
Back to top