PHP Unicode Length: Bytes, Code Points and Graphemes

A form can show room remaining while the backend rejects a name as too long. Often the two sides count different units: bytes, Unicode code points or user-perceived character clusters.

This PHP UTF-8 example requires mbstring and intl, but no database or network. Choose the counting unit before implementing validation.

1. Three different length questions

strlen counts bytes. mb_strlen counts according to an encoding; for valid UTF-8 this counts code points. grapheme_strlen counts grapheme units and requires valid UTF-8.

Graphemes can suit interface limits, but do not measure pixel width or terminal columns. Equal grapheme counts can render at different widths.

2. Similar appearance, different representation

The letter é can be precomposed U+00E9 or e followed by combining acute U+0301. These representations have different byte and code-point counts despite commonly appearing alike. Counting alone does not normalize text.

<?php
declare(strict_types=1);

if (!extension_loaded('mbstring') || !extension_loaded('intl')) {
    throw new RuntimeException('This example requires mbstring and intl.');
}
$samples = [
    'ASCII' => ['abc', [3, 3, 3]],
    'precomposed' => ["\u{00E9}", [2, 1, 1]],
    'combining' => ["e\u{0301}", [3, 2, 1]],
];
foreach ($samples as $label => [$value, $expected]) {
    if (!mb_check_encoding($value, 'UTF-8')) {
        throw new InvalidArgumentException('Invalid UTF-8');
    }
    $actual = [strlen($value), mb_strlen($value, 'UTF-8'), grapheme_strlen($value)];
    if ($actual !== $expected) {
        throw new RuntimeException('Unexpected result: '.$label);
    }
    printf("%s: bytes=%d codepoints=%d graphemes=%d\n", $label, ...$actual);
}

Save as unicode-length.php and run php unicode-length.php with the required extensions:

ASCII: bytes=3 codepoints=3 graphemes=3
precomposed: bytes=2 codepoints=1 graphemes=1
combining: bytes=3 codepoints=2 graphemes=1

Each fixture checks its expected result. Real applications must also handle invalid UTF-8 and counting errors explicitly; a UTF-8 form does not guarantee that every integration supplies valid input.

3. Specify the validation contract

For a display name, document the unit, whitespace trimming and whether Unicode normalization occurs. Apply the same order in frontend, backend and tests. Use appropriate Unicode facilities for normalization rather than arbitrarily removing accents.

A grapheme limit does not replace request-size limits: a cluster can contain many combining code points. Retain suitable byte and resource limits. Conversely, do not describe a storage byte limit as the same number of characters when the counts differ.

4. Truncation needs the right unit too

Changing the counter while keeping byte-based truncation can produce invalid UTF-8. Code-point truncation can still separate combining marks. Choose slicing behavior for the requirement and test accents, emoji, empty input and boundaries.

Do not silently truncate passwords, tokens or identifiers. Explicit rejection is generally preferable to changing such values to fit.

5. Minimum test set

• ASCII and empty strings.
• Vietnamese accents and combining representations.
• Single and joined emoji.
• At, below and above the chosen limit.
• Invalid UTF-8 from integration sources.
• Identical frontend and backend decisions.

Control deployment Unicode/ICU versions and retest complex emoji on upgrades. Three basic fixtures are not comprehensive coverage.

No one length function fits every purpose. Choose the unit, explain it clearly and apply it consistently throughout the data path.