Optimizing CLAUDE.md: How to Scale Your Codebase Documentation for AI Assistants

The Problem: When Your CLAUDE.md File Grows Too Large

If you're using Claude Code or similar AI coding assistants, you've likely discovered the power of the CLAUDE.md file - a project-specific instruction guide that helps the AI understand your codebase conventions, patterns, and architecture.

But as your project matures, you might notice something: your CLAUDE.md file keeps growing. And growing. Our TonyRobbins.com e-commerce platform's CLAUDE.md file recently hit 1,400+ lines, packed with form validation patterns, checkout flows, testing conventions, and architectural decisions.

At first, this feels productive - you're documenting everything! But eventually, you hit diminishing returns:

• Token bloat: Large CLAUDE.md files consume significant tokens upfront, reducing available context for actual code
• Maintenance burden: Every pattern change requires updating multiple sections
• Discovery problems: Finding relevant information becomes harder as the file grows
• Redundancy: Similar patterns get documented multiple times in different contexts

So how do you optimize your CLAUDE.md file while maintaining Claude Code's effectiveness? Let's explore strategies that work.

The Key Insight: Just-in-Time Documentation

The breakthrough realization is this: Claude Code doesn't need all information upfront. It has powerful tools like Read, Grep, and Glob that can access documentation on-demand.

Think of CLAUDE.md as an index and navigation guide, not an encyclopedia. Instead of documenting everything inline, teach Claude Code where to find information when needed.

Before: Everything in CLAUDE.md (❌ Token-heavy)

1
## Form Validation Architecture
2
 
3
### Universal Contact Form Pattern
4
 
5
For ANY form with firstName, lastName, email, phone fields:
6
 
7
1. Use the Universal Hook
8
   ```typescript import { useContactForm } from '@/hooks/useContactForm' // ... 50 more lines of code examples

1. Use Form Components

1
// ... another 40 lines

1. Validation Happens Automatically
   • ✅ Email validation (format + domain checks)
   • ✅ Phone validation (international format) // ... 30 more lines

1
### After: Reference-based approach (✅ Token-efficient)
2
 
3
```markdown
4
## Form Validation
5
 
6
**Quick reference**: All forms use react-hook-form + Zod validation.
7
 
8
| Task | Check This | Import |
9
|------|------------|--------|
10
| Contact form | `/hooks/useContactForm.ts` | `useContactForm` |
11
| Email validation | `/types/quiz-validation.ts` | `baseEmailSchema` |
12
| Examples | `/components/ui/v2/forms/LoginForm.tsx` | - |
13
 
14
**Detailed patterns**: See `/docs/claude/forms-validation.md`

The difference? The first version loads ~120 lines of examples into every conversation. The second loads ~10 lines, and Claude can Read: /docs/claude/forms-validation.md only when actually working with forms.

Strategy #1: Create a Documentation Wiki Structure

The most effective optimization is extracting detailed patterns into focused documentation files that Claude can access on-demand.

Recommended Structure

1
/docs/claude/
2
├── README.md              # Index with navigation
3
├── forms-validation.md    # Complete form patterns
4
├── checkout-patterns.md   # Payment flows (resume, discount codes)
5
├── testing-strategy.md    # Test conventions and setup
6
├── architecture.md        # Tech stack and structure
7
├── common-pitfalls.md     # Red flags and anti-patterns
8
└── quick-reference.md     # Tables and cheat sheets

What Stays in CLAUDE.md vs. Moves to /docs/claude/

Keep in CLAUDE.md (aim for ~300-500 lines):

• ✅ Project overview (2-3 paragraphs)
• ✅ Essential commands (dev, test, build)
• ✅ Critical conventions affecting all code (JSX quotes, testid patterns)
• ✅ Navigation: "See /docs/claude/X.md for..."
• ✅ Quick reference tables

Move to specialized docs:

• ❌ Long code examples with multi-step explanations
• ❌ Detailed type definitions
• ❌ Comprehensive troubleshooting guides
• ❌ Historical context (ticket numbers, migration stories)
• ❌ Technology-specific deep-dives

Example: Forms Validation Documentation

In CLAUDE.md (minimal):

1
## Form Validation
2
 
3
All forms use react-hook-form + Zod. **Never** write custom validation.
4
 
5
**Before creating any contact form:**
6
1. Check: `Grep: "useContactForm"`
7
2. Read: `/docs/claude/forms-validation.md`
8
3. Use: `useContactForm` hook from `/hooks/useContactForm.ts`
9
 
10
Red flags 🚩:
11
- Writing custom email validation → Use `baseEmailSchema`
12
- Using `useState` for forms → Use `useForm` instead

In /docs/claude/forms-validation.md (comprehensive):

1
# Form Validation Architecture
2
 
3
## Universal Contact Form Pattern (MANDATORY)
4
 
5
For ANY form with firstName, lastName, email, phone fields, follow this pattern:
6
 
7
[... detailed code examples, type definitions, 200+ lines ...]
8
 
9
## Reference Implementations
10
- `/components/ui/v2/forms/LoginForm.tsx` - Production implementation
11
- `/hooks/useContactForm.ts` - Universal hook source
12
- Storybook: http://localhost:6006/?path=/docs/07-forms-contact-form-pattern--docs
13
 
14
[... more detailed patterns ...]
15
 

Strategy #2: Use Quick Reference Tables

Replace verbose explanations with scannable tables that Claude can quickly parse:

Before: Prose format

1
When you need to create a contact form, you should use the useContactForm hook which is located in the /hooks directory at useContactForm.ts. This hook provides validation using react-hook-form and Zod schemas. For email validation specifically, check the baseEmailSchema in types/quiz-validation.ts...

After: Table format

1
| Need | Use This | Location |
2
|------|----------|----------|
3
| Contact form | `useContactForm` | `/hooks/useContactForm.ts` |
4
| Email validation | `baseEmailSchema` | `/types/quiz-validation.ts` |
5
| Phone validation | `basePhoneSchema` | `/types/quiz-validation.ts` |
6
| Form components | `Form, FormField` | `/components/ui/v2/Form` |
7
 

Tables are:

• Scannable: Claude can quickly find relevant information
• Compact: ~5 lines vs. ~20 lines for prose
• Consistent: Same structure makes patterns easier to learn

Strategy #3: Teach Discovery, Not Solutions

Instead of documenting every solution, teach Claude Code how to discover solutions in your codebase.

Discovery Checklist Pattern

1
## Before Writing New Code (MANDATORY)
2
 
3
Run this discovery process:
4
 
5
1. **Search for existing implementations**
6
   ```bash Grep: "export.*use.*Form" # Find hooks Grep: "Schema.*=.*z\.object" # Find Zod schemas

1. Check reference docs

Read: /docs/claude/[relevant-topic].md

1. Review Storybook
   • Browse: http://localhost:6006
   • Look in "04 - Organisms" for form examples
2. Document findings Create checklist:
   • ✅ Hooks available: [list]
   • ✅ Schemas available: [list]
   • ❌ Gaps requiring new code: [list]
3. Use ExitPlanMode Present plan showing reuse vs. new code

1
 
2
This pattern:
3
- **Reduces documentation**: Don't document every hook, just how to find them
4
- **Stays current**: Grep finds new patterns automatically
5
- **Encourages reuse**: Discovering existing code before writing new code
6
- **Teaches AI**: Claude learns your codebase structure
7
 
8
## Strategy #4: Red Flags > Detailed Instructions
9
 
10
Instead of documenting correct patterns in detail, highlight **anti-patterns** to avoid. This is surprisingly effective because:
11
 
12
1. **Brevity**: "Don't do X" is shorter than "Here's how to do Y correctly"
13
2. **Catches mistakes early**: Claude checks flags before deep-diving into solutions
14
3. **Prevents reinvention**: Stops duplicate implementations before they start
15
 
16
### Example: Form Validation Red Flags
17
 
18
```markdown
19
## Form Validation Red Flags 🚩
20
 
21
STOP and use standard patterns if you're about to:
22
 
23
🚩 **Write custom email validation** → Use `baseEmailSchema` instead
24
🚩 **Write custom phone validation** → Use `basePhoneSchema` instead
25
🚩 **Use `useState` for form fields** → Use `useForm` instead
26
🚩 **Track field errors manually** → react-hook-form does this
27
🚩 **Create a contact form from scratch** → Use `useContactForm` hook
28
 
29
**Before proceeding**: `Read: /docs/claude/forms-validation.md`

This takes ~10 lines instead of 100+ lines of "correct" examples, yet effectively prevents anti-patterns.

Strategy #5: Leverage File System Navigation

Claude Code can explore your file system. Use this to reduce inline documentation:

Instead of copying entire files:

❌ Don't do this:

1
## LoginForm Implementation
2
 
3
Here's the complete LoginForm.tsx file:
4
[... 200 lines of code pasted inline ...]

Point to reference implementations:

1
✅ Do this:
2
## Form Examples
3
 
4
Reference these production implementations:
5
- **Contact forms**: `/components/ui/v2/stripe-checkout/ContactInformation.tsx`
6
- **Login forms**: `/components/ui/v2/forms/LoginForm.tsx`
7
- **Quiz forms**: `/components/Quiz.tsx`
8
 
9
Common patterns:
10
- All use `useForm` + `zodResolver`
11
- All use `Form/FormField/FormControl` components
12
- All have colocated Storybook stories

When Claude needs details, it will Read the actual file - getting the current, up-to-date code instead of potentially stale documentation.

Strategy #6: Use MCP Resources for Dynamic Content

If you have data that changes frequently (API schemas, generated types, build configs), consider using Model Context Protocol (MCP) resources instead of static CLAUDE.md content.

MCP allows Claude to query resources on-demand:

• Database schemas
• API endpoint documentation
• Build tool configurations
• Generated type definitions

For example, instead of documenting Sanity types in CLAUDE.md, let Claude query them dynamically through the Sanity MCP server when needed.

Practical Example: Before and After

Let's see a real refactoring from our project:

Before: 400 lines in CLAUDE.md

1
## Resume Checkout Patterns (MT-363)
2
 
3
**Overview**: Resume checkout allows customers to return to abandoned checkouts via PaymentIntent ID in URL query params. Implemented using CheckoutOrchestrator with 'resume' context.
4
 
5
#### Architecture Components
6
 
7
1. **Entry Point**: `/app/(web)/[lang]/(lp)/complete/page.tsx` (Server Component)
8
 - Validates PaymentIntent and extracts metadata
9
 - Fetches customer and cart data from Stripe
10
 - Renders CompleteCheckoutClient with resume data
11
 
12
[... 350+ more lines of detailed implementation ...]
13
 
14
#### Testing Patterns
15
 
16
[... 50 more lines ...]
17
 
18
#### Common Pitfalls
19
 
20
[... another 40 lines ...]
21
 

After: 30 lines in CLAUDE.md + focused doc file

In CLAUDE.md:

1
## Checkout Patterns
2
 
3
Multiple checkout contexts supported:
4
- `checkout` - Standard flow
5
- `payment-link` - Token-based pre-selected products
6
- `resume` - Abandoned cart recovery
7
- `widget` - Embeddable component
8
 
9
**Implementation details**: See `/docs/claude/checkout-patterns.md`
10
 
11
**Key files**:
12
- Orchestrator: `/components/ui/v2/stripe-checkout/CheckoutOrchestrator.tsx`
13
- Resume flow: `/app/(web)/[lang]/(lp)/complete/page.tsx`
14
- Payment links: `/app/(web)/[lang]/(lp)/payment-linkv2/[token]/page.tsx`
15
 

In /docs/claude/checkout-patterns.md:

1
# Checkout Patterns
2
 
3
## Resume Checkout (MT-363)
4
 
5
### Overview
6
Resume checkout allows customers to return to abandoned checkouts via PaymentIntent ID...
7
 
8
[... full 400 lines of detailed patterns, exactly as before ...]

Result:

• CLAUDE.md: 400 lines → 30 lines (93% reduction)
• Information preserved: 100%
• Token efficiency: ~5x improvement (only loads when working on checkout)

Implementation Checklist

Ready to optimize your CLAUDE.md? Follow this checklist:

Phase 1: Audit (1-2 hours)

• [ ] Measure current CLAUDE.md size (lines and tokens)
• [ ] Identify sections over 50 lines
• [ ] Categorize content: Essential vs. Reference vs. Historical
• [ ] List repeated patterns that could be consolidated

Phase 2: Create Structure (2-3 hours)

• [ ] Create /docs/claude/ directory
• [ ] Create README.md index with navigation
• [ ] Create topic-specific files (forms, testing, architecture, etc.)
• [ ] Add frontmatter with last-updated dates

Phase 3: Extract Content (3-4 hours)

• [ ] Move detailed patterns to topic files
• [ ] Convert prose to tables where possible
• [ ] Create discovery checklists
• [ ] Add red flag sections
• [ ] Keep only essentials in CLAUDE.md

Phase 4: Update References (1-2 hours)

• [ ] Replace extracted sections with links
• [ ] Add quick reference tables
• [ ] Update file path references
• [ ] Test with Claude Code on real tasks

Phase 5: Validate (1 hour)

• [ ] Test Claude Code can find information
• [ ] Verify all links work
• [ ] Check token usage improvement
• [ ] Get team feedback on organization

Measuring Success

Track these metrics before and after optimization:

Token Efficiency:

• Before: CLAUDE.md tokens loaded per conversation
• After: Average tokens loaded (CLAUDE.md + accessed docs)
• Target: 50-70% reduction in upfront tokens

Discovery Time:

• Before: Time for Claude to find relevant patterns
• After: Time to navigate to correct doc file
• Target: Comparable or faster

Maintenance Time:

• Before: Time to update pattern across multiple sections
• After: Time to update single doc file
• Target: 60-80% reduction

Code Quality:

• Before: Pattern adherence rate
• After: Pattern adherence rate
• Target: Maintain or improve (better discovery = better adherence)

Best Practices and Tips

Do's ✅

1. Start with high-value extractions: Move the longest, most detailed sections first
2. Keep navigation obvious: Claude should find docs in ≤2 steps from CLAUDE.md
3. Use consistent structure: Same headings/organization across all doc files
4. Link bidirectionally: CLAUDE.md → docs AND docs → related docs
5. Version control everything: Track what works and what doesn't
6. Get team buy-in: Documentation is useless if the team doesn't maintain it

Don'ts ❌

1. Don't scatter randomly: Organized /docs/claude/ > random files everywhere
2. Don't duplicate: One source of truth per pattern
3. Don't over-extract: Keep critical conventions in CLAUDE.md
4. Don't forget updates: Update both CLAUDE.md and doc files when patterns change
5. Don't optimize prematurely: Wait until CLAUDE.md exceeds ~500 lines
6. Don't break working patterns: If Claude Code works well now, optimize incrementally

Advanced: Auto-Generated Documentation

For mature projects, consider automating documentation generation:

TypeScript Types

1
// Generate type documentation from TSDoc comments
2
yarn typedoc --out /docs/claude/types --includes "**/*.ts"

API Routes

1
// Extract API route metadata
2
yarn ts-node scripts/generate-api-docs.ts

Component Props

1
// Document component interfaces from Storybook
2
yarn storybook-docs --output /docs/claude/components.md

This ensures documentation stays synchronized with code changes automatically.

Real-World Results

After implementing this optimization strategy on our TonyRobbins.com project:

Before:

• CLAUDE.md: 1,400+ lines
• Token usage: ~3,500 tokens per conversation start
• Update time: 15-20 minutes to update a pattern across sections
• Pattern duplication: Same patterns documented 3-4 times

After:

• CLAUDE.md: 420 lines (70% reduction)
• Token usage: ~1,000 tokens upfront + ~800 tokens on-demand average
• Update time: 3-5 minutes to update single doc file
• Pattern duplication: Single source of truth per pattern

Impact:

• Token efficiency: 60% reduction in upfront token usage
• Maintenance speed: 4x faster to update patterns
• Code quality: Maintained 95%+ pattern adherence (no degradation)
• Discovery speed: Comparable or faster (2-3 tool calls to find info)

Conclusion: Documentation as Architecture

Optimizing your CLAUDE.md file isn't just about reducing file size - it's about architecting your documentation for AI consumption.

The key principles:

1. Just-in-time over just-in-case: Load information when needed, not upfront
2. Navigation over duplication: Point to sources of truth, don't copy them
3. Discovery over documentation: Teach how to find, not what exists
4. Red flags over instruction: Highlight what to avoid, not just what to do
5. Tables over prose: Structured data > long explanations

By treating CLAUDE.md as an index rather than an encyclopedia, you get:

• Better token efficiency
• Easier maintenance
• Improved discoverability
• More consistent patterns

Start small: extract your longest section today. Then iterate. Your AI assistant (and your future self) will thank you.

Additional Resources

• Claude Code Documentation
• Model Context Protocol (MCP)
• Example optimized CLAUDE.md: github.com/your-org/your-project
• /docs/claude/ structure template: Download

This blog post is based on real optimization work done on a production Next.js 15 e-commerce platform with 50k+ lines of TypeScript code. Your mileage may vary, but these principles apply to projects of any size.