JSON Lines (JSONL / NDJSON)

JSON Lines (JSONL)—also known as NDJSON (Newline Delimited JSON) or LDJSON (Line-Delimited JSON)—is a lightweight text format designed for streaming structured data. While standard JSON requires an entire dataset to be enclosed within a single top-level array ([...]), JSONL dictates that each individual line contains exactly one valid, self-contained JSON value separated by a newline character (\n). This makes JSONL the standard format for log aggregation (Elasticsearch, Loki), AI/LLM fine-tuning datasets (OpenAI, Hugging Face), and big data stream processing (Apache Spark, Kafka).

Convert JSON Lines to standard JSON arrays or CSV files using our browser-based JSONL Converter tool.

---

Technical Specifications at a Glance

Specification	Details
Community Standard	JSON Lines Specification (jsonlines.org) / NDJSON
MIME Media Type	application/x-ndjson or application/jsonlines
File Extension	.jsonl, .ndjson, .ldjson
Character Encoding	Strict UTF-8
Line Separator	\n (Unix LF) or \r\n (Windows CRLF)
Key Advantage	Streamable; process gigabyte-sized files with constant memory footprint

---

The 3 Golden Rules of JSON Lines

1. Strict UTF-8 Encoding: Every line must be encoded in valid UTF-8 without byte-order marks.
2. One Value per Line: Each line must be a valid, standalone JSON value (usually an object {...} or array [...]). Inner line breaks inside JSON strings must be escaped as \n.
3. Newline Delimiter: Records are separated by newlines. Empty trailing lines are permitted and should be ignored by compliant parsers.

{"id":1,"event":"user_signup","user":"[email protected]","ts":1725408000}
{"id":2,"event":"api_call","endpoint":"/tools/jsonl-converter","ts":1725408012}
{"id":3,"event":"payment_success","amount":29.00,"currency":"USD","ts":1725408045}

---

JSONL vs Standard JSON: Memory & Throughput Comparison

Benchmark Dimension	Standard JSON Array (.json)	JSON Lines (.jsonl)
Memory Footprint	$O(N)$ — entire file must be read into RAM	$O(1)$ — reads one line into memory at a time
10 GB File Processing	Crashes standard JSON.parse() (Heap out-of-memory)	Streams effortlessly via line-by-line generators
Append Performance	$O(N)$ — requires reading, stripping ], and rewriting file	$O(1)$ — instant atomic append to EOF (>> file.jsonl)
Corruption Resilience	Syntax error in 1 line invalidates the entire file	Bad lines can be skipped; remaining lines parse fine
Human Editability	High for small files; impossible for large files	High; easily inspected with grep, head, tail, awk

---

Code Examples: Streaming JSONL

Modern Node.js (Memory-Efficient Stream Processing)

import fs from 'fs';
import readline from 'readline';

async function processLargeJsonlFile(filePath: string) {
  const fileStream = fs.createReadStream(filePath);

  const rl = readline.createInterface({
    input: fileStream,
    crlfDelay: Infinity, // Recognizes all CR/LF line breaks
  });

  let lineCount = 0;

  for await (const line of rl) {
    if (!line.trim()) continue; // Skip empty lines

    try {
      const record = JSON.parse(line);
      lineCount++;
      // Process individual record without loading the rest into memory
    } catch (err) {
      console.warn(`Skipping malformed line ${lineCount + 1}: ${line}`);
    }
  }

  console.log(`Successfully processed ${lineCount} streaming records.`);
}

Python (Line-by-Line Reading)

import json

def stream_jsonl(file_path):
    with open(file_path, "r", encoding="utf-8") as f:
        for line_num, line in enumerate(f, start=1):
            line = line.strip()
            if not line:
                continue
            try:
                data = json.loads(line)
                yield data
            except json.JSONDecodeError as e:
                print(f"Error on line {line_num}: {e}")

# Example streaming
for record in stream_jsonl("dataset.jsonl"):
    if record.get("event") == "payment_success":
        print(f"Processed: ${record['amount']}")

---

Frequently Asked Questions

Why does OpenAI require JSONL for LLM fine-tuning?

Large language model training datasets routinely span millions of conversational turns and gigabytes of text. Using JSONL allows training scripts to stream prompt-completion pairs in batches directly to GPUs without allocating enormous memory buffers.

What is the difference between JSONL and NDJSON?

They are two names for the identical format specification. "NDJSON" stands for Newline Delimited JSON, while "JSONL" stands for JSON Lines. The file extensions .jsonl and .ndjson are interchangeable.

How do I convert standard JSON arrays into JSON Lines format?

You can paste any standard JSON array into our client-side JSONL Converter to transform array items into newline-delimited rows instantly.