ai validation pipeline generation storytelling
seedling · updated 2025-10-19 13:52:00

Multi-Stage Story Generation with Validation

The generation architecture used in [[denote:20251019T134950][the drama project]]: progressively refine stories through validated stages.

Pipeline Stages

Context → Sketch → Spec → Full Text
            ↓        ↓         ↓
         Validate  Validate  Validate
            ↓        ↓         ↓
        Pass/Iterate Pass/Iterate Pass/Done

Stage 1: Sketch

Input: Premise, constraints, requirements Output: High-level story structure

Contains: - Brief premise - Character list with roles - Dramatic arcs (hamartia, peripeteia, anagnorisis per character) - Key story events - Narrative structure type

Tests: Structural, Aristotelian unity, character arcs

Size: ~100-500 tokens

Stage 2: Spec

Input: Validated sketch Output: Detailed scene breakdown

Contains: - Expanded character descriptions - Scene-by-scene breakdown - Beat-level narrative units - Dialogue sketches - Proppian functions mapped

Tests: Propp morphology, Todorov sequences, POV consistency

Size: ~1000-3000 tokens

Stage 3: Full Text

Input: Validated spec Output: Complete prose story

Contains: - Full prose narrative - Polished dialogue - Sensory details - Atmosphere and pacing

Tests: LLM judge quality, narrative consistency, completeness

Size: ~5000-15000 tokens (short story)

Why Multi-Stage?

1. Incremental Validation

Catch structural problems early (in sketch) rather than after generating 10,000 words.

2. Different Strategies Per Stage

  • Sketch: Fast, broad exploration (high temperature)
  • Spec: Systematic, constrained (medium temperature)
  • Full Text: Polished, detailed (lower temperature)

Each stage can use different: - LLMs (cheap for sketch, expensive for full text) - Prompting strategies (basic → iterative → constrained) - Validation rigor (loose → strict)

3. Caching and Reuse

Generate sketch once, try multiple spec variations. Generate spec once, try multiple prose styles for full text.

4. Human-in-the-Loop

Review/edit sketch before committing to full generation.

5. Traceability

Full lineage: sketch_id → spec_id → full_story_id Know exactly where each element came from.

Generation Strategies

Basic

Single LLM call with prompt. Simple, fast, limited control.

Iterative

Generate → Test → Fix Issues → Test Again

Loop until all tests pass or max iterations reached.

Constrained

Use grammar/format constraints to enforce structure during generation.

Multi-Agent

Different agents collaborate: - Plotter: Designs overall structure - Character Designer: Creates character arcs - Writer: Generates prose - Validator: Runs tests - Refiner: Fixes issues

Validation Across Stages

Sketch: Structural + Aristotelian Spec: + Propp + Todorov Full Text: + LLM judge + consistency checks

Tests accumulate. Full text must pass ALL tests from all stages.

Caching Strategy

Cache key = hash(generator_type + config + context + constraints)

Benefits: - Don’t regenerate identical requests - Faster experimentation - Cost reduction (LLM calls expensive)

Database Tracking

Every stage stored: - story_versions table (sketch, spec, full) - generation_attempts (all LLM calls, even failed ones) - test_results (all validation results) - lineage (parent-child relationships)

Enables analysis: - Which generators work best for which constraints? - Which tests fail most often? - What’s the distribution of iteration counts?

Example Flow

# Start with context
context = GenerationContext(
    stage="sketch",
    metadata={"premise": "A mysterious dinner party"},
    constraints=["6 characters", "single location", "3-hour timeframe"],
)

# Generate sketch
sketch_gen = SketchGenerator(config={"model": "gpt-oss-120b"})
sketch_version = await sketch_gen.generate(context)

# Validate sketch
validators = [
    SketchStructureValidator(),
    AristotelianUnityValidator(),
    PeripeteiaValidator(),
]
results = [await v.validate(sketch_version) for v in validators]

if all(r.passed for r in results):
    # Move to spec stage
    context.sketch = sketch_version.content
    context.stage = "spec"
    spec_gen = SpecGenerator()
    # ... continue
else:
    # Iterate sketch
    # Use test feedback to refine
    pass

Related Notes

Status: Seedling | Created: 2025-10-19

Backlinks