ADR-0005: Generator Architecture

Status: Accepted Date: 2025-01 Deciders: Project maintainers

Context and Problem Statement

The F5 XC API comprises 1,548 tools across 39 domains, each with complex request/response schemas, dependencies, and validation rules. Manually creating and maintaining MCP tool definitions for this API surface would be:

Prohibitively time-consuming (months of manual work)
Error-prone (inconsistencies across tools)
Unmaintainable (API updates require manual synchronization)
Incomplete (difficult to ensure full API coverage)

How can we efficiently create and maintain accurate MCP tool definitions for a large, evolving API surface?

Decision Drivers

Automation - Must generate tools from authoritative source (OpenAPI specs)
Accuracy - Generated tools must match actual API behavior
Maintainability - Updates must be easy when API changes
Completeness - All API operations must be available as tools
Performance - Generation must complete in reasonable time
Quality - Generated code must be production-ready

Considered Options

Manual Tool Creation: Write tool definitions by hand
Simple Template Generation: Basic code generation from OpenAPI
Sophisticated Generator Pipeline: Multi-stage transformation with validation

Decision Outcome

Chosen option: “Sophisticated Generator Pipeline”, because it provides accurate, maintainable tool generation with proper dependency extraction, schema resolution, and validation.

Positive Consequences

95%+ Automation: Minimal manual intervention required
API Parity: Generated tools accurately reflect OpenAPI specifications
Rapid Updates: Regenerate when API changes (< 5 minutes)
Consistent Quality: All tools follow same patterns
Dependency Tracking: Automatic resource dependency extraction
Validation Built-in: Schema validation in generated tools
Subscription Mapping: Automatic tier requirements identification

Negative Consequences

Complex Generator: Multi-stage pipeline requires maintenance
Build Time: Generation takes 2-3 minutes on initial build
Testing Complexity: Generator itself needs comprehensive tests
Debugging Difficulty: Issues may require generator fixes vs. tool fixes

Pros and Cons of the Options

Manual Tool Creation

Good: Simple, no build complexity
Good: Full control over each tool
Bad: Months of manual work for 1,548 tools
Bad: Error-prone and inconsistent
Bad: Unmaintainable when API changes
Bad: Cannot ensure completeness

Simple Template Generation

Good: Basic automation
Good: Faster than manual
Bad: Cannot handle complex dependencies
Bad: No schema resolution or validation
Bad: Limited metadata extraction
Bad: Requires significant manual post-processing

Sophisticated Generator Pipeline

Good: Full automation with high accuracy
Good: Handles complex dependencies and schemas
Good: Maintainable with API changes
Good: Comprehensive metadata extraction
Good: Built-in validation and quality checks
Bad: Complex generator requiring maintenance
Bad: Longer build times
Bad: Requires generator testing

Implementation Details

Generator Pipeline Stages

Stage 1: OpenAPI Spec Loading
|- Load 39 domain OpenAPI specs
|- Validate spec format and structure
\- Cache parsed specs

Stage 2: Schema Resolution
|- Resolve $ref references
|- Extract nested schemas
|- Handle circular references
\- Build schema dependency graph

Stage 3: Dependency Extraction
|- Parse resource references
|- Extract oneOf patterns
|- Map subscription requirements
\- Identify addon services

Stage 4: Tool Generation
|- Generate MCP tool definitions
|- Create parameter schemas
|- Build request body schemas
\- Add metadata and examples

Stage 5: Validation & Output
|- Validate generated tools
|- Check schema completeness
|- Write JSON tool definitions
\- Generate summary report

Directory Structure

src/generator/
|- index.ts                    # Main generator orchestration
|- spec-loader.ts              # OpenAPI spec loading
|- schema-loader.ts            # $ref resolution and schema handling
|- dependency-extractor.ts     # Resource dependency extraction
|- tool-generator.ts           # MCP tool definition generation
|- transformers/               # Schema transformation utilities
|   |- normalize-examples.ts   # Example normalization
|   \- ...
\- types.ts                    # Generator type definitions

Generated Output:
src/tools/generated/
|- {domain}_{operation}_{resource}.json  # 1,548 tool definitions
\- ...

Key Generator Components

1. Schema Loader (schema-loader.ts):

Resolves OpenAPI $ref pointers
Handles nested schema references
Detects circular references
Builds complete schema trees
Enforces max depth limits (20 levels)

2. Dependency Extractor (dependency-extractor.ts):

Parses resource references from schemas
Extracts oneOf field patterns (x-ves-oneof-field)
Maps resources to subscription tiers
Identifies addon service requirements
Builds resource dependency graphs

3. Tool Generator (tool-generator.ts):

Creates MCP tool definitions from operations
Generates parameter schemas
Builds request body schemas
Adds examples and descriptions
Includes metadata (domain, resource, operation)

4. Transformers (transformers/):

Normalize example payloads
Clean schema descriptions
Format metadata consistently
Remove implementation details

Generation Process

# Full regeneration
npm run generate

# Build process (includes generation)
npm run build

# Output
Generating tools from OpenAPI specs...
✓ Loaded 39 OpenAPI specifications
✓ Resolved 3,247 schema references
✓ Extracted 892 resource dependencies
✓ Generated 1,548 tool definitions
✓ Validation passed: 100% tools valid
✓ Written to src/tools/generated/
Generation complete in 2m 34s

Quality Assurance

Generator Tests (Target: 50% coverage):

Schema resolution correctness
Dependency extraction accuracy
Tool generation validation
Edge case handling
Performance benchmarks

Generated Tool Validation:

JSON schema validity
Required field presence
Reference integrity
Example payload validity
Metadata completeness

Performance Characteristics

Cold Generation: 2-3 minutes (full pipeline)
Incremental: 30-60 seconds (changed specs only)
Memory Usage: ~500MB peak during generation
Output Size: ~12MB of JSON tool definitions
Tool Count: 1,548 tools from 39 domains

Extensibility

The generator architecture supports:

New Transformers: Add custom schema transformations
Custom Validators: Additional quality checks
Metadata Enrichment: Extra tool metadata
Alternative Outputs: Other formats beyond JSON
Plugin System: Future generator plugins

Error Handling

Spec Errors:

Invalid OpenAPI format -> validation error
Missing required fields -> generation skipped
Circular references -> depth limit enforcement

Schema Errors:

Unresolvable $ref -> reference error reported
Invalid schema structure -> transformation failed
Complex nesting -> max depth protection

Generation Errors:

Tool validation failure -> tool excluded from output
Missing metadata -> default values applied
Example errors -> examples omitted

Notes

The generator architecture is critical infrastructure enabling the entire project. Without sophisticated generation, maintaining 1,548 tools manually would be impractical.

The multi-stage pipeline with validation ensures generated tools are production-ready, not just syntactically correct. Dependency extraction enables intelligent tool recommendations and workflow guidance.

Generator complexity is justified by:

95%+ automation of tool creation
Rapid API update synchronization
Consistent quality across all tools
Comprehensive dependency tracking
Built-in validation and quality checks

Future enhancements could include:

Incremental generation (only changed specs)
Parallel spec processing for faster builds
Custom tool templates per domain
Machine learning for better examples
Automated test generation for tools