How We Rebuilt DebuggerMe's JSON Parser to Handle 10MB Files

The DebuggerMe JSON Parser started as a simple textarea and a JSON.parse(). It worked fine for small files. Then we started getting support tickets with a consistent theme: users pasting in API responses from their production systems — and the browser tab freezing or crashing.

The largest file that reliably caused problems: 1.8MB. Not even close to what production JSON logs can look like.

This is the story of how we fixed it.

What Was Breaking and Why

The original implementation:

function parseAndRender(input: string) {
    const parsed = JSON.parse(input);      // ① Blocks the main thread
    const formatted = JSON.stringify(parsed, null, 2);  // ② Also blocks
    setOutput(formatted);                  // ③ Renders all at once
}

Three problems:

1. JSON.parse() on a 2MB string blocks the main thread for 200-800ms depending on device
2. JSON.stringify with formatting takes similar time
3. Rendering the full formatted output into a <textarea> or <pre> is a massive DOM operation — browsers try to calculate line heights for tens of thousands of lines simultaneously

The result: on mobile or mid-range laptops, the page became unresponsive for 2-4 seconds. On some mobile devices, the tab crashed.

What We Tried First (That Didn't Work)

Attempt 1: Chunked parsing

We tried splitting the input string and parsing chunks. But JSON doesn't chunk — you can't parse {"a":1,"b": as valid JSON. We'd have needed to implement a streaming JSON parser from scratch.

Attempt 2: setTimeout batching

// Process in chunks with setTimeout to yield to browser
function processChunk(chunks, index) {
    process(chunks[index]);
    if (index < chunks.length - 1) {
        setTimeout(() => processChunk(chunks, index + 1), 0);
    }
}

This helps with rendering but not with the parsing itself, which still needs the full string.

Attempt 3: requestIdleCallback

Similar problem — JSON.parse is atomic. You can't yield in the middle of it.

What Actually Worked

Solution 1: Web Worker for parsing

Move JSON.parse + JSON.stringify off the main thread entirely:

// json-worker.ts
self.onmessage = ({ data: { input } }) => {
    try {
        const parsed = JSON.parse(input);
        const formatted = JSON.stringify(parsed, null, 2);
        const lineCount = formatted.split('\n').length;
        self.postMessage({ success: true, formatted, lineCount });
    } catch (e) {
        self.postMessage({ success: false, error: (e as Error).message });
    }
};

// In the component
const workerRef = useRef<Worker | null>(null);

useEffect(() => {
    workerRef.current = new Worker(
        new URL('./json-worker.ts', import.meta.url),
        { type: 'module' }
    );
    return () => workerRef.current?.terminate();
}, []);

const parse = (input: string) => {
    setStatus('parsing');
    workerRef.current?.postMessage({ input });
};

workerRef.current.onmessage = ({ data }) => {
    if (data.success) {
        setOutput(data.formatted);
        setStatus('done');
    } else {
        setError(data.error);
    }
};

The UI stays responsive during parsing. The spinner actually spins. Users can still interact with the page.

Result: Zero UI freezing, even on 10MB files.

Solution 2: Virtual rendering for large output

Even with the worker, rendering 150,000 lines into the DOM immediately was still causing jank. We moved to a virtualised list — only rendering the visible lines:

const VISIBLE_BUFFER = 50; // Lines above/below viewport

function VirtualCodeView({ lines }: { lines: string[] }) {
    const containerRef = useRef<HTMLDivElement>(null);
    const [visibleRange, setVisibleRange] = useState({ start: 0, end: 100 });

    useEffect(() => {
        const observer = new IntersectionObserver(() => {
            const { scrollTop, clientHeight } = containerRef.current!;
            const lineHeight = 20;
            const start = Math.max(0, Math.floor(scrollTop / lineHeight) - VISIBLE_BUFFER);
            const end = Math.min(lines.length, start + Math.ceil(clientHeight / lineHeight) + VISIBLE_BUFFER);
            setVisibleRange({ start, end });
        });
        // ... simplified for clarity
    }, [lines]);

    return (
        <div ref={containerRef} style={{ height: lines.length * 20, overflow: 'auto' }}>
            <div style={{ transform: `translateY(${visibleRange.start * 20}px)` }}>
                {lines.slice(visibleRange.start, visibleRange.end).map((line, i) => (
                    <div key={visibleRange.start + i} className="line">
                        {line}
                    </div>
                ))}
            </div>
        </div>
    );
}

Result: Instant rendering regardless of file size.

Results

The 2-second parse time on 10MB is acceptable — and crucially, the UI is responsive the entire time.

Lessons

Web Workers are underused. Any computation that takes more than ~50ms should be off the main thread. The cost of setting up a worker is a few lines of code. The benefit is a responsive UI.

Measure before you optimize. We spent a week on setTimeout chunking before realising the bottleneck was JSON.parse itself, not rendering.

Virtual rendering is almost always the answer for large lists. Libraries like @tanstack/virtual make this easy — we ended up using a simplified version, but the library is excellent for complex cases.

The JSON parser now handles production-scale files that real developers throw at it every day. That's the goal.