Introduction
As organizations increasingly adopt large language models, the efficiency of data serialization has become an important operational cost factor. In most LLM systems, usage is billed per token, so every extra character in a payload directly increases the cost of each model call. Beyond costs, larger payloads also add computational load and slow down processing, which can affect the performance of AI-powered applications. Across millions of requests, extra whitespace, verbose field names, or redundant formatting can accumulate into significant expenses and latency. Optimizing how data is structured and serialized is therefore not just a technical detail, but a practical strategy to improve both cost efficiency and system performance.
Traditional JSON, while familiar and commonly used in APIs, contributes to this overhead due to its text-heavy format, with numerous braces, quotes, and commas. TOON (Token-Oriented Object Notation) offers an alternative by representing data as compact, semantically meaningful tokens rather than raw text. By reducing payload sizes by 30–50%, TOON can help lower inference costs and improve processing efficiency. Its schema-enforced structures also support more consistent data representation, helping reduce errors and prevent hallucinations or inconsistencies in the data.
What is TOON ?
Token-Oriented Object Notation (TOON) is a compact, human-readable JSON encoding that reduces token count without losing information. Designed for large language model input, TOON combines YAML-style indentation for nested objects with a CSV-like layout for uniform arrays. Its similarity to CSV is intentional, offering a familiar format while remaining a lossless, drop-in representation of JSON.
TOON models data like JSON using three basic building blocks:
- Primitives: strings, numbers, booleans, and null,
- Objects: mappings from string keys to values,
- Arrays: ordered sequences of values.
The following examples explore TOON’s syntax by showing JSON structures alongside their TOON equivalents, from simple cases to more complex ones, including nested structures.
Simple Object
Simple objects containing primitive values use a key: value format, with each field on its own line. Indentation takes the place of braces, and a single space follows the colon.
JSON:
{ "name": "Mario", "age": 30, "city": "Rome" }
TOON:
name: Mario
age: 30
city: Rome
Nested Object
When a key ends with : and has no value on the same line, it opens a nested object. All lines at the next indentation level belong to that object.
JSON:
{
"user": {
"id": 1,
"name": "Mario",
"details": { "age": 30, "city": "Rome" }
}
}
TOON:
user:
id: 1
name: Mario
details:
age: 30
city: Rome
These first two examples give a clear sense of TOON’s minimal syntax. By relying on indentation (similar to YAML) rather than verbose braces and brackets, TOON keeps data concise and reduces boilerplate.
Array of Values
Arrays of primitives (strings, numbers, booleans, null) are rendered inline with added metadata: the length of the array.
JSON:
{ "items": ["a", "b", "c"] }
TOON:
items[3]: a,b,c
This example highlights a key feature of TOON: the explicit array length, which helps LLMs validate the structure. When generating TOON output, these lengths make it easier to detect truncation or malformed data.
Array of Objects (Tabular)
For arrays of objects with a consistent structure, TOON avoids repeating keys by using a CSV-style tabular layout, with an explicit schema definition provided in the header.
Array headers are defined using this pattern:
key[N<delimiter?>]<{fields}>:where:
- N is the non-negative integer length,
- delimiter (optional) explicitly declares the active delimiter. Accepted values are comma (default), tab, and pipe,
- fields for tabular arrays: {field1,field2,field3}.
JSON:
{
"users": [
{
"id": 1,
"name": "Mario",
"role": "admin"
},
{
"id": 2,
"name": "Luigi",
"role": "user"
}
]
}
TOON:
users[2]{id,name,role}:
1,Mario,admin
2,Luigi,user
The leading [2] specifies the array length, while the {id, name, role} block defines the fields once up front. Each subsequent row is then just a compact, comma-separated list of values. This illustrates the core principle: declare the schema once, then stream the data as efficiently as possible, eliminating repetitive keys, excess punctuation, and the typical verbosity of JSON. The format preserves much of CSV’s compactness while retaining clear, explicit structure that models can reliably interpret.
This example illustrates one of the most efficient use cases for TOON: uniform arrays of objects. In this scenario, the token savings are substantial, a reduction of 58.3%, dropping from 60 tokens in JSON to just 25 in TOON.
Why TOON matters:
Modern LLMs such as GPT, Gemini, and Claude operate as token-based systems. Every piece of text sent to or generated by these models is broken down into tokens, and each token contributes to cost, latency, and context consumption, an unavoidable “token tax.” As shown in earlier examples, JSON’s verbosity often results in far more tokens than the underlying data actually requires. Token Oriented Object Notation (TOON) addresses this issue by declaring structure once and then streaming compact, punctuation-light rows. This approach typically provides 30–50% fewer tokens than equivalent JSON, while still preserving clear, LLM-friendly structure. These benefits are backed by benchmarks, which can be found here: https://toonformat.dev/guide/benchmarks.html. TOON’s performance has been evaluated across two key dimensions:
- Retrieval Accuracy: How well LLMs understand and extract information from different input formats.
- Token Efficiency: How many tokens each format requires to represent the same data.
Overall, TOON achieves 73.9% accuracy (compared with JSON’s 69.7%) while using 39.6% fewer tokens. The format excels with uniform arrays of objects, datasets where each item shares the same structure. Beyond token savings, TOON adds explicit structural metadata, such as array lengths and field counts, that act as guardrails for the LLM. These markers help the model validate its output, detect truncation or malformed responses, and significantly reduce parsing errors.
TOON also preserves readability and integrates naturally with common programming languages such as Python, Go, Rust, and JavaScript, making it easy to incorporate into existing workflows. Rather than replacing JSON, TOON is best used as a translation layer: applications continue to work programmatically with JSON, but encode the data as TOON when sending it to an LLM to minimize token cost.
When NOT to use TOON:
TOON is not ideal in situations where human readability is required. Because it is designed primarily for LLM interactions, its token-optimized format can feel opaque to developers who need to inspect or debug data. Traditional text-oriented formats like JSON or XML remain far more accessible in those contexts. For configuration files, YAML is usually preferred, as configurations are often deeply nested and unstructured, situations where TOON’s schema enforcement offers little benefit and readability is more important.
It may also be unsuitable when broad tooling or ecosystem support is needed. JSON, CSV, and other established formats integrate more smoothly with partner systems and production pipelines. For generic data integration, especially with flat, strictly tabular data, CSV is often a better choice. It produces smaller payloads than TOON, which introduces a modest 5–10% overhead for structural cues (such as array lengths, field headers, and delimiter scoping) intended to improve LLM reliability. If those metadata are not required, CSV remains the more efficient option.
TOON is also less effective for deeply nested or highly non-uniform structures, where the data has little or no tabular consistency. In such cases, compact JSON often uses fewer tokens and is easier to read and manage. TOON represents non-tabular arrays using list notation, with each element prefixed by a hyphen. Array elements that are objects follow the same pattern, with special indentation when the first field is a tabular array, and nested arrays are handled with separate headers for each inner array. Below is an example of an irregular and nested JSON dataset alongside its TOON equivalent. In this case, token savings are only around 20%, and also TOON’s format is noticeably harder to read and interpret.
JSON:
[
{
"timestamp": "2025-11-26T08:12:00Z",
"event": "login",
"user": {
"id": "alice123",
"role": "admin",
"email": "alice.johnson@example.com",
"metadata": {
"lastLoginIP": "203.0.113.42",
"devices": ["laptop", "tablet"],
"location": "New York, USA"
}
},
"sessions": [
{"id": "sess-001", "active": true, "startedAt": "2025-11-26T08:12:00Z"},
{"id": "sess-002", "active": false, "startedAt": "2025-11-25T17:45:00Z"}
]
},
{
"timestamp": "2025-11-26T08:25:00Z",
"event": "transaction",
"user": {"id": "dave321", "role": "customer"},
"transactions": [
{
"id": "txn-1001",
"amount": 120.50,
"currency": "USD",
"items": [
{"sku": "A100", "qty": 2},
{"sku": "B205", "qty": 1, "discount": 5.0}
],
"payment": {"method": "credit_card", "status": "approved"}
}
]
}
]
TOON:
[2]:
- timestamp: "2025-11-26T08:12:00Z"
event: login
user:
id: alice123
role: admin
email: alice.johnson@example.com
metadata:
lastLoginIP: 203.0.113.42
devices[2]: laptop,tablet
location: "New York, USA"
sessions[2]{id,active,startedAt}:
sess-001,true,"2025-11-26T08:12:00Z"
sess-002,false,"2025-11-25T17:45:00Z"
- timestamp: "2025-11-26T08:25:00Z"
event: transaction
user:
id: dave321
role: customer
transactions[1]:
- id: txn-1001
amount: 120.5
currency: USD
items[2]:
- sku: A100
qty: 2
- sku: B205
qty: 1
discount: 5
payment:
method: credit_card
status: approved
Working with TOON in Practice:
The TOON ecosystem is still emerging, but several tools make working with the format straightforward. Command-line utilities (e.g. https://toonformat.dev/cli/) and playgrounds (e.g.,https://toontools.vercel.app/ ) are available for experimentation and to analyze token savings before adopting TOON.
Official libraries in popular programming languages, including Python, Java, Go, Rust, and JavaScript, allow developers to encode and decode data efficiently, integrate TOON into existing pipelines, and perform token-aware transformations.
Using TOON as a translation layer is simple and requires minimal code changes. For example, in Python using toon-python (https://github.com/toon-format/toon-python):
from toon_format import encode, decode
# Original JSON data
data = {
"users": [
{
"id": 1,
"name": "Mario",
"role": "admin"
},
{
"id": 2,
"name": "Luigi",
"role": "user"
}
]
}
# Encode as TOON for LLM input
toon_string = encode(data)
print("TOON Encoded: ")
print(toon_string)
# Decode TOON back to Python
decoded_data = decode(toon_string)
print("\nDecoded back to Python: ")
print(decoded_data)The output will be:
TOON Encoded:
users[2]{id,name,role}:
1,Mario,admin
2,Luigi,user
Decoded back to Python:
{'users': [{'id': 1, 'name': 'Mario', 'role': 'admin'}, {'id': 2, 'name': 'Luigi', 'role': 'user'}]}
The Future of TOON
TOON and other token-oriented, deterministic formats are emerging primarily within the AI ecosystem, where minimizing token usage and ensuring predictable outputs are critical for LLM interactions. Their structured yet token-efficient approach can improve reliability, reduce parsing errors, and simplify validation in complex AI pipelines. Using TOON as a translation layer appears to be a solid strategy: applications continue to operate programmatically with familiar formats like JSON, without needing to be redesigned, while an additional layer handles encoding for LLM input. This approach preserves existing workflows and still leverages TOON’s efficiency.
While broad adoption outside AI remains uncertain, TOON’s value is clear for LLM-focused workflows, and it may gradually see more tooling, best practices, and standardization tailored specifically to AI applications.
Author:
Alessia Casagrande is an experienced Senior Data Engineer with a diverse background of working with companies ranging from startups to large enterprises. She is passionate about all things data, from building robust platforms to conducting in-depth analytics, and is always curious to explore the latest technologies in the field.