Repository navigation
chore(docs): search and answer engines (#287) - #349
Conversation
Phase 1 of #287. Register @astrojs/sitemap explicitly with a serialize hook that reads each built page's footer <time datetime> as its <lastmod> (D1); the landing, which has no footer, gets none (D2). audit-dist fails a missing, extra, differing or non-W3C <lastmod>. Gate 1: pnpm check, typecheck (root and docs), build, docs:build, worker:bundle-check, docs:audit-dist ("28 pages and the site checks pass"), pnpm test 3826/3826, docs:test 764/764, test markers clean. sitemap-0.xml: 28 <loc>, 27 <lastmod>, none on the landing. Gate 2: reviewer subagent, FAIL (type assertions) then PASS WITH NOTES after one loopback; notes in context/notes.md.
Phase 2 of #287. docs/deploy/_headers gives /worktree/_astro/* a year's immutable Cache-Control and /worktree/*.txt X-Robots-Tag: noindex; stage:deploy copies it to the assets root (D4-D6). The audit fails any _astro/ file without a hash segment in its name, and deploy-smoke checks the exact header values plus a page carrying neither (D7). The audit fixture's 3-character hash names are lengthened to 6 because the new rule would otherwise fail them; no assertion changed. Gate 1: pnpm check, typecheck (root and docs), build, docs:build, worker:bundle-check, docs:audit-dist, pnpm test, docs:test, test markers, all exit 0. stage:deploy served by wrangler dev --local: Cache-Control immutable on _astro, noindex on llms.txt, neither on a page; deploy:smoke "every row passed (attempt 1)" (plan Q2: yes). Gate 2: reviewer subagent PASS WITH NOTES; the host then pinned the page's revalidation in the smoke row and renamed a missed fixture, re-reviewed PASS WITH NOTES. Remaining notes in context/notes.md.
…each page (#287) Phase 3 of #287. renderLlmsTxt emits the 27 documentation pages under ## Documentation and the changelog under ## Optional, the llmstxt.org section for skippable links (D9). Head.astro adds <link rel="describedby" type="text/plain"> to the absolute llms.txt URL, exported from llms-txt.ts, on every page but the 404 (D8). The audit fails a misplaced section item, a missing, doubled or wrong link, a link whose target is not built, and a link on the 404. Gate 1: pnpm check (35 warnings; the new one is Biome not seeing the Astro template's use of LLMS_TXT_URL), typecheck (root and docs), build, docs:build, worker:bundle-check, docs:audit-dist, pnpm test, docs:test 780, test markers, all exit 0. dist: 28/28 pages carry one link, 404.html none, changelog alone under ## Optional. Gate 2: reviewer subagent PASS WITH NOTES; notes in context/notes.md.
Phase 4 of #287. audit-dist fails any built index.html or 404.html over PAGE_SIZE_LIMIT (1,048,576 bytes uncompressed, D10) and names the page and its size. Source: Apple TN3156, "The main resource located at the previewed link is limited to 1 MB" (read from Apple's page JSON; it does not say 10^6 or 2^20 bytes, and D10's 2^20 is kept). Gate 1: pnpm check, typecheck (root and docs), build, docs:build, worker:bundle-check, docs:audit-dist, pnpm test, docs:test 783, test markers, all exit 0. Largest built page: commands/branch/ at 178,908 B. Gate 2: reviewer subagent PASS WITH NOTES; notes in context/notes.md.
Phase 5 of #287, the script part. docs/scripts/web-vitals.mjs measures the landing, a guide and commands/branch/: Lighthouse 13.6.0 through npx (mobile and desktop, median of 3: LCP, CLS, TBT), INP in Playwright Chromium at 4x CPU over search, the theme toggle and the mobile menu, and PageSpeed Insights read-only for a live origin (PSI_API_KEY from the environment only). Adds playwright as a docs devDependency (D11); it downloads no browser on install. Not a gate; writes no numbers. Gate 1: pnpm check, typecheck (root and docs), build, docs:build, worker:bundle-check, docs:audit-dist, pnpm test, docs:test 783, test markers, all exit 0. Gate 2: reviewer subagent PASS WITH NOTES; notes in context/notes.md. The live run is still owed (PSI quota, sandboxed Chromium), so the phase is not done.
Phase 6 of #287. A Search engines section in docs/README.md: submit sitemap-index.xml to Google Search Console and Bing Webmaster Tools, read each Pages report, what a healthy result is (28 discovered and indexed, no Duplicate or Alternate page entries), a live header spot-check, and the root robots.txt Sitemap: line as living in the northguild.dev repository. Gate 1: docs-only, so Lint (pnpm check, exit 0) plus a read of the diff. Gate 2: reviewer subagent, PASS WITH NOTES, no blocking items; one wording note applied, the rest in context/notes.md.
There was a problem hiding this comment.
code-reviewer: minor issues
This change adds <lastmod> to the sitemap from each page's built footer date, and a _headers file giving _astro/* a year's immutable caching and *.txt a noindex. It also splits llms.txt into Documentation and Optional sections, adds a describedby link on every page, adds audit rules for hashed asset names and page size, adds new smoke-test header rows, and adds a by-hand Web Vitals script. The code is careful and well tested, and I found no correctness bugs in the audit, sitemap or llms.txt logic. My one concern is that the new smoke rows make the release's docs deploy depend on edge behaviour that has not been seen live yet (#346). docs-deploy.yml is not in the diff, so I assumed its site job stages through pnpm --filter docs stage:deploy rather than its own cp. If it copies by hand, _headers is never deployed, and the new smoke rows fail every release.
Reviewed 9f0e3f8 for baldurpan · claude-opus-5-5 · 17 files · left out: pnpm-lock.yaml (lockfile or generated) · $0.66
Brings in the /dox page (#344). Seven paths conflicted: - docs/astro.config.mjs: both sides registered @astrojs/sitemap; one call now takes main's `filter` (keeps /dox/ out) and this branch's `serialize` (lastmod from the footer). - docs/package.json: main's exact pin of @astrojs/sitemap. - pnpm-lock.yaml: main's, regenerated from the merged manifests. - docs/scripts/audit-dist.mjs: both doc-comment paragraphs. - docs/scripts/audit-dist.test.mjs: this branch's lastmod-aware sitemap fixture and main's Dox fixtures; the chat-launcher fixtures and tests go, as on main; main's three-character fixture hashes lengthened for this branch's `_astro/` hash rule. - context/stack.md, context/verify.md: both sides' edits.
…287) The merge of main brought auditDox, which calls sitemapUrls; this branch replaced that function with sitemapEntries, which reads lastmod too. The bare catch around the call swallowed the ReferenceError, so the audit passed while never checking that /dox/ is kept out of the sitemap. The test 'fails a sitemap that lists /dox/' caught it.
|
Merged The call:
cc0d147, where the branch had to catch up. Main's new Verified: every line each side added since the merge base is in the result, apart from the deliberate removals above, checked per path. All of |
There was a problem hiding this comment.
code-reviewer follow-up: minor issues
Since the last review the branch merged main, which brought in #309's Dox page and the Workers AI brains. Within this PR's files that means: the sitemap integration moves after Starlight and gains a filter that drops /worktree/dox/; @astrojs/sitemap is pinned; the audit drops the launcher and shortcut rules for auditDox, header and mobile-menu Dox-link rules, and an "Ask Dox" link rule; and docs and context are updated to match. The new audit rules are consistent with their tests, and I found no new defect. Two stale descriptions sit outside the hunks: docs/README.md line 266 still lists the header's right group as theme, language, GitHub, Discord (without Dox), and the header comment of audit-dist.mjs (lines 8–9) still says the audit checks for a script that loads the chat before it is activated. My earlier comment on the deploy-smoke header rows still stands, since that file is unchanged.
Earlier comments
- Still open: Risk: New header rows can turn the first release's docs job red, and that also skips
legacy-pages
Reviewed cc0d147 (changes since 9f0e3f8) for baldurpan · claude-opus-5-5 · 7 files · left out: pnpm-lock.yaml (lockfile or generated) · $0.59
The merge of main (#309) added the "Chat with Dox" link to the header's right group and retired the chat launcher, and two descriptions in files this branch edits still said otherwise: docs/README.md listed the right group without Dox, and the audit's header comment still named the launcher-era chat-script rule. Raised by baldurs-reviewer on #349.
There was a problem hiding this comment.
code-reviewer follow-up: minor issues
Since the last review, two lines changed, both in documentation. docs/README.md now lists the "Chat with Dox" crystal first in the header's right group and counts four controls. The header comment in audit-dist.mjs now describes the Dox-code rule in place of the retired "chat before activation" rule. Both match the code they describe and raise nothing new. The earlier risk about the new smoke-check header rows in deploy-smoke.mjs is unchanged and still open.
Earlier comments
- Still open: Risk: New header rows can turn the first release's docs job red, and that also skips
legacy-pages
Reviewed f9c795f (changes since cc0d147) for baldurpan · claude-opus-5-5 · 2 files · $0.29
|
The two stale descriptions from the follow-up review (no thread to reply in) are fixed in f9c795f:
Both lines were already stale on |
Closes #287
Ports what worktree's docs still lacked from northguild/gmt#478 for search engines and AI answer engines.
Phases landed (one commit each):
<lastmod>from each page's footer date (a0bf0f6)noindexfor text files, via_headers(d4ab279)llms.txtOptional section for the changelog, and thedescribedbylink from each page (08739c5)f96ddf3)3d62eba)docs/README.md(c2707b8)Release note: none owed. The branch touches only
docs/**,context/**andpnpm-lock.yaml, whichcontext/release.mdmarks as owing no note. The changes go live with the next stable release, since a docs-only merge causes none.Owner checks left for a person (not defects): #346 (live edge applies
_headers), #347 (submit sitemap, addSitemap:line), #348 (cut a release). The live PageSpeed finding is #345.