Large JSON Parsing: High-Performance Frontend Rendering for 100MB Data
2024-06-18
Log viewers and data apps sometimes hit tens or hundreds of MB of JSON. One sync parse plus full DOM render will freeze the tab—the fix is not doing everything at once.
Where time goes
JSON.parse is sync and CPU-heavy; the parsed graph eats heap.
Rendering millions of nodes often hurts more than parse—virtualize the UI.
Parse in a Web Worker
postMessage text or buffers; parse off the main thread; return only the current page slice.
Avoid shipping the whole graph back if you can process in Worker.
Virtual lists
Array JSON: render visible rows only (react-window, virtua).
Trees: lazy-load children on expand.
Streaming and chunks
JSON Lines stream well; giant single arrays need server pagination or NDJSON exports.
fetch + ReadableStream can process incrementally.
Realistic 100MB advice
Prefer server-side filter, page, and gzip.
If client-only: Worker + virtualization + progress; monitor memory in DevTools.