Warning

This is an internal project, and is not intended for public use. No support or stability guarantees are provided.

Load Isomorphic Code Source

The createLoadIsomorphicCodeSource factory builds a LoadSource function from two injected primitives — fetchSource and resolveImports — making it the platform-independent core for loading and analyzing JavaScript/TypeScript/CSS source files.

It handles everything that does not depend on the runtime: parsing imports and comments, reshaping externals, applying the chosen storeAt mode, filtering comments, and assembling extraFiles / extraDependencies for recursive loading. The only concerns left to the caller are how to read a URL and how to resolve relative imports against a real (or virtual) tree.

loadServerCodeSource is a thin wrapper that wires this factory to Node.js fs/promises. Use the isomorphic factory directly when you need to load from somewhere other than the local filesystem.

Features

Pluggable fetcher: Bring your own fetch, readFile, IndexedDB cache, etc.
Pluggable resolver: Resolve ./foo against a real filesystem, an HTTP tree, or an in-memory map
Import parsing: Extracts relative and external imports from JavaScript/TypeScript/CSS files
Path resolution: Resolves relative imports to absolute URLs the loader can fetch back
Dependency tracking: Builds extraDependencies arrays for recursive loading by loadIsomorphicCodeVariant
Import rewriting: Rewrites import statements based on configured storage mode (storeAt)
External tracking: Identifies npm package dependencies with their import specifiers
Comment processing: Strips and/or collects comments by prefix for use by sourceEnhancers

Usage

import { createLoadIsomorphicCodeSource } from '@mui/internal-docs-infra/pipeline/loadIsomorphicCodeSource';

const loadSource = createLoadIsomorphicCodeSource({
  fetchSource: async (url) => (await fetch(url)).text(),
  resolveImports: async (imports) => {
    const resolved = new Map<string, string>();
    for (const importUrl of Object.keys(imports)) {
      resolved.set(importUrl, new URL(importUrl, baseUrl).toString());
    }
    return resolved;
  },
});

const result = await loadSource('https://example.com/Demo.tsx');
// {
//   source: "import { Button } from './Button';\n...",
//   extraFiles: { './Button.tsx': 'https://example.com/Button.tsx' },
//   extraDependencies: ['https://example.com/Button.tsx'],
//   externals: { 'react': [{ name: 'default', type: 'default', isType: false }] }
// }

The returned function satisfies the LoadSource contract and can be plugged directly into CodeProvider or loadIsomorphicCodeVariant.

Injection Points

`fetchSource`

Required. Async function that resolves a URL to its raw source text. It is called once per file and is the only I/O performed by the loader itself.

fetchSource: (url: string) => Promise<string>;

In server contexts this typically wraps fs/promises#readFile; in browsers it wraps fetch.

`resolveImports`

Optional. Given the imports parsed from a JavaScript/TypeScript module, returns a Map keyed by the original import URL with values rewritten to absolute URLs that can be passed back into loadSource recursively.

resolveImports?: (imports: IsomorphicImports) => Promise<Map<string, string>>;

When omitted, JavaScript modules are processed without a resolver: import URLs are kept as-is for extraDependencies and no rewriting happens. CSS files always skip this step because parseImportsAndComments already resolves @import and url() paths.

The IsomorphicImports shape mirrors what parseImportsAndComments produces after name flattening, so resolvers can inspect each import's named bindings, type-only flag, and source positions:

type IsomorphicImports = Record<
  string,
  {
    url: string;
    names: string[];
    includeTypeDefs?: true;
    positions: Array<{ start: number; end: number }>;
  }
>;

Import Storage Modes

The storeAt option controls how imports are stored in extraFiles and how import statements are rewritten in the source.

`'flat'` (default)

Flattens all imports to the current directory and rewrites import statements accordingly. This mode uses intelligent conflict resolution to ensure unique filenames.

// Original source:
import { Button } from '../components/Button';
import styles from './styles.module.css';

// After processing with storeAt: 'flat':
import { Button } from './Button';
import styles from './styles.module.css';

// extraFiles:
{
  './Button.tsx': 'https://example.com/components/Button.tsx',
  './styles.module.css': 'https://example.com/demo/styles.module.css'
}

`'canonical'`

Preserves full relative paths including index files when they exist:

// Original source:
import { Button } from '../components/Button';

// After processing with storeAt: 'canonical':
// (source unchanged)

// extraFiles:
{
  '../components/Button/index.tsx': 'https://example.com/components/Button/index.tsx'
}

`'import'`

Uses import paths with file extensions but without index resolution:

// Original source:
import { Button } from '../components/Button';

// After processing with storeAt: 'import':
// (source unchanged)

// extraFiles:
{
  '../components/Button.tsx': 'https://example.com/components/Button/index.tsx'
}

Return Value

interface LoadSourceResult {
  source: string; // Processed source code (may have rewritten imports)
  extraFiles?: Record<string, string>; // Map of import paths to URLs
  extraDependencies?: string[]; // Array of URLs for recursive loading
  externals?: Externals; // External npm package dependencies
  comments?: Record<number, string[]>; // Comments by line number (for sourceEnhancers)
}

// Externals type
type Externals = Record<string, ExternalImportItem[]>;

interface ExternalImportItem {
  name: string; // Import name (e.g., 'Button', 'default')
  type: 'named' | 'default' | 'namespace'; // Import type
  isType?: boolean; // Whether this is a TypeScript type-only import
}

Important

The values in extraFiles are URLs, not source code. This allows loadIsomorphicCodeVariant to recursively load each dependency by calling loadSource again with the URL. Whatever URL scheme fetchSource and resolveImports produce must be one this same loader can later fetch.

Comment Processing

The function extracts comments from source files and returns them for use by sourceEnhancers. Two options control comment behavior:

`removeCommentsWithPrefix`

Strips comments starting with specified prefixes from the output source. When notableCommentsPrefix is not specified, all stripped comments are collected in the comments object:

const loadSource = createLoadIsomorphicCodeSource({
  fetchSource,
  removeCommentsWithPrefix: ['@internal', '@private'],
});

// Source file:
// // @internal This is an implementation detail
// // User-visible comment
// const x = 1;

const result = await loadSource('https://example.com/example.ts');
// result.source: "// User-visible comment\nconst x = 1;"
// result.comments: { 0: ['@internal This is an implementation detail'] }
// Note: "User-visible comment" is NOT collected because it doesn't match removeCommentsWithPrefix

This is useful for:

Removing build-time annotations from displayed code
Stripping internal documentation that shouldn't appear in demos
Hiding section markers used only for enhancer processing

`notableCommentsPrefix`

When specified, only comments matching these prefixes are collected in comments. Comments matching removeCommentsWithPrefix but not notableCommentsPrefix are still stripped but not collected:

const loadSource = createLoadIsomorphicCodeSource({
  fetchSource,
  removeCommentsWithPrefix: ['@internal', '@highlight'],
  notableCommentsPrefix: ['@highlight'],
});

// Source file:
// // @internal Implementation detail
// // @highlight Important line
// const x = 1;

const result = await loadSource('https://example.com/example.ts');
// result.source: "const x = 1;"
// result.comments: { 0: ['@highlight Important line'] }
// Note: @internal is stripped but NOT collected (not in notableCommentsPrefix)

How It Works

The function follows a straightforward pipeline:

Fetch source via the supplied fetchSource
Short-circuit for static assets (JSON, images) and when includeDependencies is false
Parse imports and comments with parseImportsAndComments — extracts relative imports (starting with ./ or ../), external dependencies (npm packages), and comments
Resolve paths by handing the parsed imports to resolveImports (JavaScript/TypeScript only)
Process imports to build extraFiles and rewrite import statements according to storeAt, via processRelativeImports
Filter comments when removeCommentsWithPrefix is set, and collect notable comments into the comments field
Build extraDependencies so that loadIsomorphicCodeVariant can recurse

Note

CSS files are handled differently—they skip step 4 because parseImportsAndComments already resolves paths for CSS @import and url() statements.

Examples

Browser loader against an HTTP source tree

const loadSource = createLoadIsomorphicCodeSource({
  fetchSource: async (url) => (await fetch(url)).text(),
  resolveImports: async (imports) => {
    const resolved = new Map<string, string>();
    for (const importUrl of Object.keys(imports)) {
      // Probe the server for the real file (extension/index resolution)
      const real = await probe(importUrl);
      resolved.set(importUrl, real);
    }
    return resolved;
  },
});

Loading a JavaScript File

// File: https://example.com/components/button/Demo.tsx
import { Button } from './Button';
import styles from './styles.module.css';

const result = await loadSource('https://example.com/components/button/Demo.tsx');
// {
//   source: "import { Button } from './Button';\nimport styles from './styles.module.css';",
//   extraFiles: {
//     './Button.tsx': 'https://example.com/components/button/Button.tsx',
//     './styles.module.css': 'https://example.com/components/button/styles.module.css'
//   },
//   extraDependencies: [
//     'https://example.com/components/button/Button.tsx',
//     'https://example.com/components/button/styles.module.css'
//   ]
// }

Loading with External Dependencies

// File: https://example.com/components/button/Button.tsx
import React from 'react';
import { Box } from '@mui/material';

const result = await loadSource('https://example.com/components/button/Button.tsx');
// {
//   source: "import React from 'react';\nimport { Box } from '@mui/material';",
//   externals: {
//     'react': [{ name: 'default', type: 'default', isType: false }],
//     '@mui/material': [{ name: 'Box', type: 'named', isType: false }]
//   }
// }

Wrapping with a cache

const cache = new Map<string, string>();

const loadSource = createLoadIsomorphicCodeSource({
  fetchSource: async (url) => {
    if (cache.has(url)) return cache.get(url)!;
    const source = await (await fetch(url)).text();
    cache.set(url, source);
    return source;
  },
  resolveImports,
});

When to Use

Use createLoadIsomorphicCodeSource when you need a LoadSource for an environment other than the local Node.js filesystem:

Browser / client code editors that fetch sources over HTTP
Remote source trees such as GitHub, GitLab, or a CMS
Virtual filesystems backed by an in-memory map or IndexedDB
Hybrid loaders that need to layer caching, auth, or rewriting on top of loadServerCodeSource

If you only need to load files from the local filesystem in a Node/server context, use loadServerCodeSource directly.

Types

Creates a LoadSource function that performs all the platform-independent work shared between server-side and client-side source loaders: fetching the file via the supplied fetchSource, parsing its imports and comments, reshaping externals, and assembling extraFiles / extraDependencies for recursive loading.

Platform-specific concerns (fs.readFile, fetch, module resolution against a real filesystem vs. a remote tree) are injected via the fetchSource and resolveImports options.

Property	Type	Description
fetchSource	`(url: string) => Promise<string>`	Async fetcher that returns the raw source text for a URL. In server contexts this typically wraps `fs/promises#readFile`, in client contexts it can wrap `fetch`. Required.
resolveImports	`\| (( imports: IsomorphicImports, ) => Promise<Map<string, string>>) \| undefined`	Resolve relative JavaScript/TypeScript imports to absolute URLs that can be passed back into a `loadSource` recursion. Returning a `Map` keyed by the import URL lets the caller rewrite identifiers (e.g. `./foo` → `https://.../foo.tsx`). When omitted, JavaScript modules are processed without a resolver: import URLs are used as-is for `extraDependencies` and no rewriting happens.
maxDepth	`number \| undefined`	Cap on the number of recursive load passes. Forwarded to consumers; this function itself only processes a single file.
maxFiles	`number \| undefined`	Cap on the total number of files surfaced via `extraFiles`.
includeDependencies	`boolean \| undefined`	When false, skip dependency parsing and return only the raw source.
storeAt	`StoreAtMode \| undefined`	Controls how imports are stored in `extraFiles`: ‘canonical’: Full resolved path (e.g., ‘../Component/index.js’) ‘import’: Import path with file extension (e.g., ‘../Component.js’) ‘flat’: Flattened to current directory with rewritten imports
removeCommentsWithPrefix	`string[] \| undefined`	Prefixes for comments that should be stripped from the source output. Comments starting with these prefixes will be removed from the returned source. They can still be collected via `notableCommentsPrefix`.
notableCommentsPrefix	`string[] \| undefined`	Prefixes for notable comments that should be collected and included in the result. Comments starting with these prefixes will be returned in the `comments` field.

Return Type

LoadSource

IsomorphicImports

Imports record passed to .

Mirrors the shape produced by parseImportsAndComments once names have been flattened, so resolvers can look up each import by its source path.

type IsomorphicImports = {
  [key: string]: {
    url: string;
    names: string[];
    includeTypeDefs?: true;
    positions: { start: number; end: number }[];
  };
}

LoadIsomorphicCodeSourceOptions

type LoadIsomorphicCodeSourceOptions = {
  /**
   * Async fetcher that returns the raw source text for a URL.
   *
   * In server contexts this typically wraps `fs/promises#readFile`, in client
   * contexts it can wrap `fetch`. Required.
   */
  fetchSource: (url: string) => Promise<string>;
  /**
   * Resolve relative JavaScript/TypeScript imports to absolute URLs that can
   * be passed back into a `loadSource` recursion. Returning a `Map` keyed by
   * the import URL lets the caller rewrite identifiers (e.g. `./foo` →
   * `https://.../foo.tsx`).
   *
   * When omitted, JavaScript modules are processed without a resolver: import
   * URLs are used as-is for `extraDependencies` and no rewriting happens.
   */
  resolveImports?: (imports: IsomorphicImports) => Promise<Map<string, string>>;
  /**
   * Cap on the number of recursive load passes. Forwarded to consumers; this
   * function itself only processes a single file.
   */
  maxDepth?: number;
  /** Cap on the total number of files surfaced via `extraFiles`. */
  maxFiles?: number;
  /** When false, skip dependency parsing and return only the raw source. */
  includeDependencies?: boolean;
  /**
   * Controls how imports are stored in `extraFiles`:
   * - 'canonical': Full resolved path (e.g., '../Component/index.js')
   * - 'import': Import path with file extension (e.g., '../Component.js')
   * - 'flat': Flattened to current directory with rewritten imports
   */
  storeAt?: 'canonical' | 'import' | 'flat';
  /**
   * Prefixes for comments that should be stripped from the source output.
   * Comments starting with these prefixes will be removed from the returned
   * source. They can still be collected via `notableCommentsPrefix`.
   */
  removeCommentsWithPrefix?: string[];
  /**
   * Prefixes for notable comments that should be collected and included in
   * the result. Comments starting with these prefixes will be returned in the
   * `comments` field.
   */
  notableCommentsPrefix?: string[];
}

loadServerCodeSource - Node.js wrapper that wires this factory to fs/promises
loadIsomorphicCodeVariant - Recursive variant loader that drives LoadSource
parseImportsAndComments - Underlying parser used to extract imports and comments
processRelativeImports - Applies the storeAt mode and rewrites import statements
resolveImportResult - Reference resolver used by loadServerCodeSource

Load Isomorphic Code Source

Features

Usage

Injection Points

fetchSource

resolveImports

Import Storage Modes

'flat' (default)

'canonical'

'import'

Return Value

Comment Processing

removeCommentsWithPrefix

notableCommentsPrefix

How It Works

Examples

Browser loader against an HTTP source tree

Loading a JavaScript File

Loading with External Dependencies

Wrapping with a cache

When to Use

Types

Related

`fetchSource`

`resolveImports`

`'flat'` (default)

`'canonical'`

`'import'`

`removeCommentsWithPrefix`

`notableCommentsPrefix`