The Git Submodule Disaster: Why We Abandoned Shared Code

Jul 25, 2025

Part 1 of the Journey: The Chaos Era - The Problem & The Pain Previous: The TypeScript Migration Journey | Next: Copy-Paste Architecture

How our brilliant plan to share code across 17 servers became a maintenance nightmare

Historical Context (November 2025): Built July 2025, during MCP ecosystem's first month. We were discovering architectural patterns through trial and error—this documents a failed abstraction attempt that taught us valuable lessons about independent server architecture.

Date: July 25, 2025 Author: Myron Koch & Claude Code Category: Architecture Lessons Learned

The Brilliant Idea

Us: "We have 17 blockchain servers. They all need logging,
     validation, error handling. Let's create a shared library!"

Also Us: "We'll use Git submodules! DRY principle! Efficiency!"

Future Us: "What were we thinking?!"

The Initial Setup

# Create shared utilities repository
mkdir blockchain-mcp-common
cd blockchain-mcp-common

# Add shared code
mkdir src
cat > src/logger.ts << 'EOF'
export const logger = createLogger();
EOF

# Publish to GitHub
git init
git add .
git commit -m "Initial shared utilities"
git push origin main

# Add to all 17 servers
cd ../ethereum-mcp-server
git submodule add https://github.com/myorg/blockchain-mcp-common.git common

cd ../solana-mcp-server
git submodule add https://github.com/myorg/blockchain-mcp-common.git common

# ... repeat 15 more times

The First Warning Signs

Problem 1: The Clone Confusion

# New developer joining project
git clone https://github.com/myorg/ethereum-mcp-server.git
cd ethereum-mcp-server
npm install

# Try to run server
npm run build

Error: Cannot find module './common/logger'

# Developer: "What's common?"
# Team: "Oh right, you need to init submodules"

git submodule init
git submodule update

# Developer: "Why isn't this automatic?"

Problem 2: The Update Hell

# We fix a bug in common logger
cd blockchain-mcp-common
vim src/logger.ts
git commit -m "Fix logger bug"
git push

# Now update all 17 servers
cd ../ethereum-mcp-server
git submodule update --remote
git add common
git commit -m "Update common submodule"
git push

cd ../solana-mcp-server
git submodule update --remote
git add common
git commit -m "Update common submodule"
git push

# ... 15 more times
# ... every single time we change common

Problem 3: The Version Drift

ethereum-mcp-server: common@v1.2.0
solana-mcp-server:   common@v1.2.0
bitcoin-mcp-server:  common@v1.1.5  # Someone forgot to update
polygon-mcp-server:  common@v1.3.0  # Someone updated too soon
avalanche-mcp-server: common@v1.2.0
...

Result: 17 servers running 5 different versions of "shared" code.

The Disaster Scenarios

Scenario 1: The Breaking Change

# We improve logger API in common
- logger.info('message', data);
+ logger.info({ message, ...data });

# Update common
cd blockchain-mcp-common
# Make changes
git commit -m "Better logger API"
git push

# Update one server
cd ../ethereum-mcp-server
git submodule update --remote

# Build breaks!
npm run build

Error: logger.info() expects object, got string

# Now we have to update logger calls in ALL servers
# Or rollback common
# Or maintain two logger APIs
# Or cry

Scenario 2: The Detached HEAD

cd ethereum-mcp-server

# Update submodule
git submodule update --remote

# Submodule is now in "detached HEAD" state
cd common
git status

HEAD detached at f7e8d92

# Developer makes changes directly in submodule
vim src/logger.ts
git commit -m "Quick fix"

# Changes lost on next submodule update!
cd ..
git submodule update --remote

# Quick fix: GONE

Scenario 3: The Circular Dependency

common/
├── logger.ts
├── validator.ts
└── errorHandler.ts

# validator.ts needs to know about blockchain types
import { EthereumAddress } from 'ethereum-mcp-server';
// 💥 Circular dependency!

# Common imports from server
# Server imports from common
# Build system explodes

Scenario 4: The CI/CD Nightmare

# .github/workflows/test.yml
name: Test

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: true  # Must remember this!

      - name: Update submodules
        run: |
          git submodule init
          git submodule update --remote

      # Now which version of common are we testing against?
      # The one in git? The one at HEAD? The one specified?

Failure rate: 37% of CI runs failed due to submodule issues.

The Maintenance Burden

Daily conversations:

Developer A: "Why is my build failing?"
Us: "Did you update submodules?"
Developer A: "What's a submodule?"

Developer B: "I updated common but now 3 servers are broken"
Us: "Did you test all 17 servers?"
Developer B: "We have 17 servers?!"

Developer C: "Git says my submodule is dirty"
Us: "Don't commit directly in submodules"
Developer C: "But I needed a quick fix"
Us: *explains entire submodule workflow again*

The Breaking Point

# The day we abandoned submodules

# Simple task: Add debug logging to all servers
git clone ethereum-mcp-server
cd ethereum-mcp-server

git submodule init
git submodule update --remote  # Wrong version
git submodule update           # Still wrong version

cd common
git checkout main              # Detached HEAD
git pull                       # Merge conflict!

# 30 minutes later, still not working

rm -rf common
cp -r ../blockchain-mcp-common common
# Just copied the directory
# Built successfully in 2 minutes

That day we knew: Submodules had to go.

The Migration Strategy

Step 1: Copy Common Code Locally

#!/bin/bash
# scripts/remove-submodules.sh

SERVERS=(
  "ethereum-mcp-server"
  "solana-mcp-server"
  "bitcoin-mcp-server"
  # ... all 17
)

for server in "${SERVERS[@]}"; do
  echo "Processing $server..."

  cd "$server"

  # Remove submodule
  git submodule deinit -f common
  git rm -f common
  rm -rf .git/modules/common

  # Copy common code directly
  mkdir -p src/utils
  cp -r ../blockchain-mcp-common/src/* src/utils/

  # Update imports
  find src -name "*.ts" -exec sed -i 's|from.*common/|from "../utils/|g' {} \;

  # Commit
  git add .
  git commit -m "Remove submodule, copy common code locally"

  cd ..
done

Step 2: Accept Strategic Duplication

// Each server now has its own:
src/utils/logger.ts         # Copied from common
src/utils/validator.ts      # Copied from common
src/utils/errorHandler.ts   # Copied from common

Yes, this is duplication. And it's BETTER.

Why Copy-Paste Won

1. Independent Evolution

// Ethereum server needs ETH-specific validation
// src/utils/validator.ts
export function validateAddress(address: string): boolean {
  return /^0x[a-fA-F0-9]{40}$/.test(address);
}

// Bitcoin server needs BTC-specific validation
// src/utils/validator.ts
export function validateAddress(address: string): boolean {
  return address.match(/^[13][a-km-zA-HJ-NP-Z1-9]{25,34}$/) ||
         address.match(/^bc1[a-z0-9]{39,59}$/);
}

Same function name, different implementations. No conflicts.

2. Faster Onboarding

# New developer
git clone ethereum-mcp-server
npm install
npm run build

# Just works! No submodule knowledge required.

3. Simpler CI/CD

# .github/workflows/test.yml
name: Test

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm install
      - run: npm test

# No submodule steps!
# No version confusion!
# Just works!

4. Clear Ownership

Each server owns its code.
Each server can change its code.
No cross-server coordination needed.

When We Still Share Code

We didn't abandon ALL sharing:

1. NPM Packages for Stable Libraries

# We published stable, rarely-changing code
npm publish @blockchain-mcp/types
npm publish @blockchain-mcp/test-utils

# Servers install as dependencies
npm install @blockchain-mcp/types

When to use NPM packages:

Stable APIs (changes rarely)
Type definitions (pure TypeScript)
Test utilities (don't affect runtime)

2. Code Generation

# Generate boilerplate from templates
npx @blockchain-mcp/generator create-server ethereum

# Generates complete server with utilities
# But the generated code is OWNED by the server
# Not linked to template anymore

3. Documentation Templates

# We share documentation templates
docs/templates/README.md
docs/templates/TOOL-DOCUMENTATION.md

# But each server copies and customizes

The Lessons Learned

1. Coupling is More Expensive Than Duplication

Maintaining submodules: 4 hours/week
Copying small utilities: 0.5 hours/week

2. Not All Code Should Be Shared

// This should be shared (pure logic)
function formatTimestamp(date: Date): string {
  return date.toISOString();
}

// This should NOT be shared (chain-specific)
function validateAddress(address: string): boolean {
  // Each chain has different rules!
}

3. Git Submodules Are Great For:

Large, stable dependencies
Monorepo alternatives
Vendor code that rarely changes

4. Git Submodules Are Terrible For:

Shared utilities that evolve
Code that needs chain-specific customization
Rapid iteration across multiple projects

5. Developer Experience Matters

Before (with submodules):
git clone
git submodule init
git submodule update
cd submodule
git checkout main
git pull
cd ..
git add submodule
git commit
git push
git submodule update --remote
# ... repeat across 17 repos

After (copy-paste):
git clone
npm install
npm run build

The Counter-Arguments We Heard

"But you're violating DRY!"

DRY is about knowledge, not code
These servers have DIFFERENT knowledge
Ethereum ≠ Solana ≠ Bitcoin

"But what if there's a bug in common code?"

Fix it in one server
Test it
Copy fix to other servers
Still faster than submodule coordination

"But you have 17 copies of the same logger!"

Each logger is <100 lines
Maintenance cost: near zero
Benefits: independence, simplicity

The Numbers

Time spent on submodule issues (per week):

Update coordination: 2 hours
Version conflicts: 1 hour
CI/CD fixes: 1 hour
Developer support: 0.5 hours
Total: 4.5 hours/week

Time spent maintaining copied code (per week):

Bug fixes that need copying: 0.5 hours
Total: 0.5 hours/week

Time saved: 4 hours/week = 208 hours/year

The Alternative We Recommend

If you must share code across multiple repos:

Option 1: NPM Packages

# Create package
mkdir blockchain-mcp-utils
npm init
npm publish

# Use in servers
npm install blockchain-mcp-utils

Pros: Version control, semantic versioning, npm ecosystem Cons: Publishing overhead for rapid changes

Option 2: Monorepo

# Everything in one repo
blockchain-mcp/
├── packages/
│   ├── ethereum-server/
│   ├── solana-server/
│   ├── bitcoin-server/
│   └── shared-utils/
└── package.json

Pros: True code sharing, atomic commits Cons: Massive repo, complex CI/CD

Option 3: Copy-Paste (Our Choice)

# Each server owns its code
ethereum-mcp-server/
├── src/utils/
│   └── logger.ts  # Copied
solana-mcp-server/
├── src/utils/
│   └── logger.ts  # Copied

Pros: Simple, fast, independent Cons: Manual synchronization

The Final Verdict

Git submodules: Good in theory, painful in practice.

For our use case:

17 independent servers
Rapid iteration
Different blockchain requirements
Small shared utilities

Copy-paste architecture wins every time.

The Migration Checklist

If you're stuck with submodules:

Identify what's actually shared vs chain-specific
Extract truly stable code to NPM packages
Copy chain-specific utilities locally
Remove submodule configuration
Update all imports
Test each server independently
Update CI/CD to remove submodule steps
Document the change
Train team on new workflow
Never look back

References

Submodule removal script: /scripts/remove-submodules.sh
Copy-paste philosophy: /blog-posts/005-copy-paste-architecture.md
NPM package creation: /docs/creating-npm-packages.md
Shared code guidelines: /docs/when-to-share-code.md

This is part of our ongoing series documenting architectural patterns and insights from building the Blockchain MCP Server Ecosystem. Sometimes the simple solution is the right solution.

Operational Semantics

Discussion about this post

Ready for more?