Fast TOON Encoder/Decoder for PHP

The fastest TOON (Token-Oriented Object Notation) encoder and decoder for PHP, with full support for PHP 7.0 through 8.4.

TOON is a data serialization format optimized for LLM (Large Language Model) input, providing approximately 40% token reduction compared to JSON while maintaining full semantic equivalence.

⚡ Performance Highlights

This library is specifically optimized for speed and broad compatibility:

| Feature | mfonte/fast-toon | helgesverre/toon | |---------|-------------|------------------| | PHP Version | 7.0 - 8.4 ✅ | 8.1+ only | | Encoding Speed | 3.2x faster | baseline | | Decoding Speed | 15x faster | baseline | | Memory Usage | Optimized | Standard |

Benchmark Results

Performance comparison against helgesverre/toon (the reference implementation):

| Data Size | Operation | mfonte/fast-toon | helgesverre/toon | Speedup | |-----------|-----------|-------------|------------------|---------| | Small (10 items) | Encode | 9.4 ms | 27.5 ms | 2.9x | | Small (10 items) | Decode | 9.2 ms | 33.5 ms | 3.6x | | Medium (100 items) | Encode | 85 ms | 284 ms | 3.3x | | Medium (100 items) | Decode | 27 ms | 456 ms | 16.9x | | Large (1000 items) | Encode | 88 ms | 274 ms | 3.1x | | Large (1000 items) | Decode | 24 ms | 463 ms | 19.3x |

Benchmarks run on PHP 8.3 with 1000 iterations (100 for large data). Results may vary based on hardware and PHP version.

Installation

You can install the package via composer:

composer require mfonte/fast-toon

Usage

use Mfonte\FastToon\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'active' => true],
        ['id' => 2, 'name' => 'Bob', 'active' => false],
        ['id' => 3, 'name' => 'Charlie', 'active' => true],
    ]
];

$toon = toon_encode($data);

// Output:
// users[3]{id,name,active}:
// 1|Alice|T
// 2|Bob|F
// 3|Charlie|T

$decoded = toon_decode($toon);

Fluent API

Instead of using helper functions, you may also use the fluent API:

use Mfonte\FastToon\Toon;

$toon = Toon::encode($data)
    ->withFlags(TOON_THROW_ON_ERROR | TOON_KEY_FOLDING)
    ->withDepth(100)
    ->toString();

$decoded = Toon::decode($toon)
    ->withOptions(['strict' => true])
    ->toArray();

Encoding flags

Like json_encode(), TOON supports bitmask flags for controlling encoding behavior:

// Combine flags with bitwise OR
$toon = toon_encode($data, TOON_THROW_ON_ERROR | TOON_KEY_FOLDING);

// Use tab delimiter for tabular output
$toon = toon_encode($data, TOON_TAB_DELIMITER);

| Flag | Description | |------|-------------| | TOON_THROW_ON_ERROR | Throw exception instead of returning false | | TOON_FORCE_OBJECT | Force empty/sequential arrays as objects | | TOON_PRESERVE_ZERO_FRACTION | Preserve .0 in floats like 5.0 | | TOON_TAB_DELIMITER | Use tab as tabular delimiter | | TOON_PIPE_DELIMITER | Use pipe as tabular delimiter (default) | | TOON_KEY_FOLDING | Enable key folding for nested single-key objects | | TOON_INDENT_4 | Use 4 spaces for indentation | | TOON_NUMERIC_CHECK | Encode numeric strings as numbers | | TOON_INVALID_UTF8_IGNORE | Ignore invalid UTF-8 sequences | | TOON_INVALID_UTF8_SUBSTITUTE | Substitute invalid UTF-8 with replacement char |

Decoding flags

$data = toon_decode($toon, TOON_STRICT | TOON_THROW_ON_ERROR);

| Flag | Description | |------|-------------| | TOON_THROW_ON_ERROR | Throw exception instead of returning null | | TOON_OBJECT_AS_ARRAY | Decode objects as associative arrays | | TOON_BIGINT_AS_STRING | Decode big integers as strings | | TOON_STRICT | Enable strict mode validation |

Error handling

TOON provides error handling similar to JSON:

$result = toon_encode($data);

if ($result === false) {
    echo toon_last_error_msg();
}

When using TOON_THROW_ON_ERROR, exceptions are thrown instead:

try {
    $toon = toon_encode($data, TOON_THROW_ON_ERROR);
} catch (\Mfonte\FastToon\Exceptions\EncodingException $e) {
    // Handle exception
}

TOON format overview

TOON uses a line-based syntax that eliminates redundant characters found in JSON.

Objects are represented as key-value pairs:

name:John Doe
age:30
active:T

Arrays of objects with the same structure use a highly efficient tabular format:

users[3]{id,name,email}:
1|Alice|alice@example.com
2|Bob|bob@example.com
3|Charlie|charlie@example.com

This provides approximately 50% token savings compared to the equivalent JSON for tabular data.

Testing

This package supports PHP 7.0 through 8.4 and provides PHPUnit configurations for each version range.

# Run tests on PHP 8.1+ (PHPUnit 10+)
composer test

# Run tests on PHP 7.0-7.1 (PHPUnit 6-7)
composer test:php70

# Run tests on PHP 7.2-8.0 (PHPUnit 8-9)
composer test:php72

# Run tests with coverage
composer test:coverage

The test suite is compatible with:

| PHPUnit Version | PHP Version | Config File | |-----------------|-------------|-------------| | 6.5.14 - 7.5.20 | PHP 7.0 - 7.1 | phpunit-php70.xml.dist | | 8.5.50 - 9.6.31 | PHP 7.2 - 8.0 | phpunit-php72.xml.dist | | 10.5.60+ | PHP 8.1+ | phpunit.xml.dist |

Code Style

This package uses PHP-CS-Fixer for code style enforcement. Two configurations are provided for compatibility across PHP versions.

# Check code style (PHP 7.4+, php-cs-fixer 3.x)
composer cs-check

# Fix code style (PHP 7.4+, php-cs-fixer 3.x)
composer cs-fix

# Check code style (PHP 7.0-7.3, php-cs-fixer 2.x)
composer cs-check:v2

# Fix code style (PHP 7.0-7.3, php-cs-fixer 2.x)
composer cs-fix:v2

| PHP-CS-Fixer Version | PHP Version | Config File | |----------------------|-------------|-------------| | 2.19.3 | PHP 7.0 - 7.3 | .php-cs-fixer-v2.php | | 3.x | PHP 7.4+ | .php-cs-fixer.dist.php |

PHP version compatibility

This library is designed to work across PHP 7.0 through 8.4:

• PHP 7.0-7.4: Full support with polyfills for missing functions
• PHP 8.0: Stringable interface support
• PHP 8.1+: Enum support, readonly properties

Type support

TOON supports all PHP types that json_encode() supports:

| PHP Type | TOON Representation | |----------|---------------------| | null | N | | bool | T / F | | int | Number literal | | float | Number literal | | string | Unquoted (with escapes) | | array (sequential) | Pipe-separated values | | array (associative) | Key-value pairs | | object | Key-value pairs | | JsonSerializable | Via jsonSerialize() | | DateTimeInterface | ISO 8601 string | | Enum (PHP 8.1+) | Backed value or name | | Stringable (PHP 8.0+) | String conversion |

Benchmarks

TOON vs JSON

Compared to JSON for typical API response data:

| Metric | JSON | TOON | Improvement | |--------|------|------|-------------| | Token Count | 1000 | 600 | 40% fewer | | Bytes | 5000 | 3800 | 24% smaller |

Token counts measured with OpenAI's tiktoken (cl100k_base)

Running Benchmarks

This library includes comprehensive benchmarks to measure performance:

# Compare against helgesverre/toon (requires both libraries installed)
composer benchmark:compare

# Run phpbench microbenchmarks
composer benchmark

# Run phpbench with summary report
composer benchmark:summary

# Run profiling benchmark
composer benchmark:profile

The comparative benchmark (composer benchmark:compare) will output:

• Encoding/decoding times for small, medium, and large datasets
• Operations per second
• Memory usage
• Speedup factors vs helgesverre/toon

Why is mfonte/fast-toon faster?

Key optimizations that make this library significantly faster:

1. Inline Type Checks: Direct type comparisons instead of function calls
2. Pre-computed Values: Indent strings, delimiter symbols cached at construction
3. Optimized String Building: Direct array appending instead of buffered writing
4. Fast Path Detection: Early returns for common primitive types
5. Minimal Memory Allocation: Reuse of internal buffers

Why Choose mfonte/fast-toon?

| Consideration | mfonte/fast-toon | helgesverre/toon | |---------------|-------------|------------------| | PHP 7.0-7.4 support | ✅ Yes | ❌ No | | PHP 8.0+ support | ✅ Yes | ✅ Yes | | Encoding speed | ⚡ 3x faster | baseline | | Decoding speed | ⚡ 15x faster | baseline | | Memory efficiency | ⚡ Optimized | standard | | Spec compliance | ✅ Full | ✅ Full |

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

License

The MIT License (MIT). Please see License File for more information.

Credits

• Maurizio Fonte
• TOON specification: github.com/toon-format