Skip to content

Add tokenizer regressions for surrogates and buffered lookahead - #201

Open
fb55 wants to merge 2 commits into
html5lib:masterfrom
fb55:parse5/tokenizer-regressions
Open

fb55 wants to merge 2 commits into
html5lib:masterfrom
fb55:parse5/tokenizer-regressions

Conversation

@fb55

@fb55 fb55 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Adds three tokenizer regressions for lone surrogates, two-code-point character references, and lookahead after long whitespace. The surrogate case lives in unicodeCharsProblematic.test, which UTF-8-only downstream runners already exclude.

Validation: lint passes; all cases pass on parse5 #1921 with whole/chunked input. The other two cases pass html5gum’s four reader modes. Existing CI also has unrelated processing-instruction and fixture-path failures.

🤖 Generated with Codex

Copilot AI balanced review requested due to automatic review settings November 5, 2026 19:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request. Copilot's per-file or total diff limits were exceeded. Try reducing the changed content or splitting the changes into smaller pull requests, then request a review from Copilot again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants