Skip to content

[RFC] Add grapheme_str_pad() function - #25179

Open
ChrisAbner wants to merge 10 commits into
php:masterfrom
ChrisAbner:grapheme-str-pad
Open

ChrisAbner wants to merge 10 commits into
php:masterfrom
ChrisAbner:grapheme-str-pad

Conversation

@ChrisAbner

@ChrisAbner ChrisAbner commented Nov 3, 2026 •

Copy link
Copy Markdown

RFC: https://wiki.php.net/rfc/grapheme_str_pad

Adds grapheme_str_pad() to ext/intl, the grapheme cluster counterpart of str_pad() and mb_str_pad().

function grapheme_str_pad(string $string, int $length, string $pad_string = " ", int $pad_type = STR_PAD_RIGHT): string|false {}
  • The target length and the input length are counted in grapheme clusters, like grapheme_strlen().
  • The padding string is repeated and its last copy is only cut on a grapheme cluster boundary.
  • Errors follow mb_str_pad() (ValueError for an empty padding string or an invalid $pad_type, no error when nothing needs padding). Invalid UTF-8 returns false and sets the intl error, like grapheme_substr().
  • ASCII input without "\r\n" takes a fast path and gives the same result as str_pad().

Tests: ext/intl/tests/grapheme_str_pad_basic.phpt (NFC/NFD, combining dakuten, Devanagari, emoji with modifier and ZWJ sequences, cuts of the padding string, "\r\n"), grapheme_str_pad_errors.phpt, grapheme_str_pad_oom.phpt and grapheme_str_pad_overflow.phpt.

Like mb_str_pad(), but the target length is counted in grapheme
clusters, so NFC and NFD forms of the same text, combining marks and
emoji sequences are padded to the same visible length. The padding
string is only cut on grapheme boundaries. Invalid UTF-8 makes it
return false, like grapheme_strlen().

RFC: https://wiki.php.net/rfc/grapheme_str_pad
Comment thread ext/intl/grapheme/grapheme_string.cpp Outdated
goto overflow;
}

zend_string *result = zend_string_safe_alloc(1, ZSTR_LEN(input), left_bytes + right_bytes, false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be one of the fallible zend string functions? So that it respects the allowed memory size?

  • str_pad("a", PHP_INT_MAX, "b") gives Fatal error: Allowed memory size of 134217728 bytes exhausted.
  • grapheme_str_pad("a", PHP_INT_MAX, "b") gives Hard memory limit of 268435456 bytes exhausted.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. Fixed in f397ad9: it now uses zend_string_safe_falloc() with RETURN_OOM(), like str_pad() and mb_str_pad(), so your example hits the regular memory_limit error. I added grapheme_str_pad_oom.phpt for it.

Like str_pad() and mb_str_pad(), an oversized result now hits the
regular memory_limit fatal error instead of the hard memory limit.
Fill the padding like mb_str_pad() does, with memset() for a single
byte and by doubling the copied area otherwise, instead of one memcpy()
per copy of the padding string.

Measure the padding string in one pass and stop once enough grapheme
clusters are counted, so a long padding string is not walked three
times. Stop counting the input once it is longer than the target
length, since it is then returned unchanged.
@TimWolla TimWolla added the RFC label Nov 4, 2026
@devnexen

devnexen commented Nov 4, 2026

Copy link
Copy Markdown
Member

Hi @ChrisAbner, the RFC says invalid UTF-8 in $pad_string returns false, but it is only checked when padding is needed:

var_dump(grapheme_str_pad("abc", 2, "\xff")); // string(3) "abc"

it is fine behaviour wise I think, it matches the empty $pad_string case, but it might deserve a mention in the RFC and a test wdyt ?

@ChrisAbner

Copy link
Copy Markdown
Author

Thanks, agreed. Like mb_str_pad(), neither $pad_string nor $pad_type is checked when no padding is needed. I've updated the RFC to say so and added a test for both cases in 829161e.

Comment thread ext/intl/grapheme/grapheme_string.cpp Outdated
return false;
}

ubrk_setUText(bi, ut, &ustatus);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ll have a better overall look later but I think you need to check its status here.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 333837b.
It now checks the status of ubrk_setUText(). If it fails, the function sets the intl error, closes the iterator and the UText, and returns false.
Later, 48b7896 moved the counting to UTF-16, like grapheme_strlen(), so the same check is now on ubrk_setText().

A successful call no longer leaves the error of a previous failed call, so intl_get_error_code() reflects the last call, like normalizer_normalize() and transliterator_transliterate().
Like grapheme_strlen(), convert the string with intl_convert_utf8_to_utf16(),
which also rejects invalid UTF-8, and iterate the UTF-16 text, instead of
validating with U8_NEXT() and iterating a UTF-8 UText. The byte length of a
prefix of the padding string is taken from its UTF-16 offset.

This is about a third faster on large non-ASCII strings and slightly faster
on short ones.
@ChrisAbner

Copy link
Copy Markdown
Author

Pushed three more commits besides 333837b.

ed4eee9 adds a test for the two "String size overflow" checks. They had no test before.

f164108 clears the intl error at the start of the function. normalizer_normalize() and transliterator_transliterate() already do this. No grapheme_* function does it yet.

48b7896 counts grapheme clusters through UTF-16 with intl_convert_utf8_to_utf16(), like grapheme_strlen(), instead of validating with U8_NEXT() and walking a UTF-8 UText. The status check stays on ubrk_setText(). In a release build it is about a third faster on large non-ASCII strings.

About f164108: I can drop it if you prefer consistency with the other grapheme functions. If it looks fine, I can apply the same change to the other grapheme functions in a separate PR.

@devnexen

devnexen commented Nov 5, 2026

Copy link
Copy Markdown
Member

A fragment can merge at a join, as in any concatenation, for example a padding string that starts with a combining mark or U+200D ZERO WIDTH JOINER, a lone regional indicator, Hangul conjoining jamo, a $string that starts with a combining mark, or a “\r” next to a “\n”. The result then has fewer grapheme clusters than $length, just as grapheme_strlen($a . $b) can be less than grapheme_strlen($a) + grapheme_strlen($b)

I think deserve some test coverage.

@ChrisAbner

Copy link
Copy Markdown
Author

Added in e197470: one test per merge case listed in the RFC. The results are the same with every ICU version I tried, from 57 to 76. I also added 371d81c, which checks that invalid input is rejected even when no padding is needed. Happy to add more cases if you think of others. These are edge cases, but they should be covered.


try {
grapheme_str_pad("abc", 5, "-", 3);
} catch (ValueError $e) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

...
} catch (\Throwable $e) {
     $e::class . ': ' . $e->getMessage() . "\n";
}

@devnexen

devnexen commented Nov 6, 2026

Copy link
Copy Markdown
Member

apart of the nit, I think it s good to go (or at least nearly) but will give time for other reviews before I merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants