Skip to content

feat(checks): add remediation prompts - #55

Merged
randoneering merged 7 commits into
mainfrom
feat/agent-remediation-prompts
Nov 15, 2026
Merged

randoneering merged 7 commits into
mainfrom
feat/agent-remediation-prompts

Conversation

@randoneering

@randoneering randoneering commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Pull Request Summary

Add copyable prompts for all 50 non-INFO checks, using each finding's actual objects and observed values.
Include read-only diagnosis and approval safeguards.
Use the same SQL file for installation or upgrade, preserve metadata, and generate shared helpers from canonical SQL.

Type of Change

  • New health check
  • Bug fix
  • Performance improvement
  • Documentation update
  • Refactoring/code cleanup
  • Breaking change

Related Issues

None.


Testing

PostgreSQL Version Compatibility

  • PostgreSQL 15
  • PostgreSQL 16
  • PostgreSQL 17
  • PostgreSQL 18

Testing notes:
PG17: 618 passed; PG18: 562 passed, one skipped each.
PG15/16: 94 upgrade/view tests passed per version, 10 environment-specific probes skipped.
Generation, data-sync, shell syntax, and diff checks pass.

Managed Database Platforms

  • Amazon RDS for PostgreSQL
  • Google Cloud SQL for PostgreSQL (currently unable to test)
  • Azure Database for PostgreSQL (currently unable to test)
  • Neon
  • Supabase
  • Self-managed PostgreSQL

Platform-specific notes:
Tested the function and both view variants in local containers.


Additional Notes

New and existing function installations both run the complete pgFirstAid.sql; known older signatures upgrade automatically with ownership, grants, and comments preserved.
All installers execute as one atomic statement, including through transaction pooling.
View recovery preserves permissions, comments, and security options; dependent rebuilds stop with RESTRICT.
data/agent_prompt.sql is the maintained prompt source; CI checks generated copies.


@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Add finding-specific remediation prompts to PostgreSQL health checks

✨ Enhancement 🐞 Bug fix 🧪 Tests 📝 Documentation 🕐 40+ Minutes

Grey Divider

AI Description

• Add copyable, finding-specific prompts to non-INFO results across all three SQL installers.
• Include observed evidence, read-only diagnosis, approval safeguards, and safer object
 identification.
• Document upgrade requirements and test prompt coverage, permissions, and installer compatibility.
Diagram

graph TD
  F["Function installer"] --> C["Check findings"] --> H["Prompt helper"] --> R["Result rows"] --> U["Monitoring caller"]
  V["View installers"] --> C
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Install the prompt helper from a shared SQL file
  • ➕ Keeps guidance in one maintained source instead of three copies.
  • ➖ Standalone installers would require an additional file or a build step.
2. Generate prompts outside PostgreSQL
  • ➕ Avoids changing SQL result signatures and adding a caller-visible helper grant.
  • ➖ Requires another tool and may separate prompts from the exact finding evidence.

Recommendation: Keep database-side generation so each copyable prompt travels with its finding and the installers remain standalone. The equality test helps control duplicated guidance; explicit function-recreation and helper-grant instructions are essential because the appended column changes the function signature and caller requirements.

Files changed (9) +1293 / -68

Enhancement (3) +618 / -68
pgFirstAid.sqlAppend curated prompts to function findings +206/-20

Append curated prompts to function findings

• Adds a helper with check-specific guidance for non-INFO findings and appends agent_prompt to pg_firstAid() results. Improves quoted target evidence for indexes, tables, replication slots, and foreign keys; the new return column requires explicit recreation of existing functions.

pgFirstAid.sql

view_pgFirstAid.sqlAppend prompts without dropping the self-hosted view +206/-24

Append prompts without dropping the self-hosted view

• Adds the prompt helper and an appended agent_prompt column while switching to CREATE OR REPLACE VIEW to preserve existing view identity. Improves target evidence and identifier quoting in several findings.

view_pgFirstAid.sql

view_pgFirstAid_managed.sqlAppend prompts to the managed-platform view +206/-24

Append prompts to the managed-platform view

• Adds matching prompt guidance and an appended agent_prompt column to the managed installer. Replaces the view in place and improves quoted targets and supporting evidence for affected checks.

view_pgFirstAid_managed.sql

Tests (2) +503 / -0
99_teardown.sqlRemove the prompt helper during teardown +1/-0

Remove the prompt helper during teardown

• Drops the new eight-argument helper after the pgTAP suite removes its dependent function and view.

testing/pgTAP/99_teardown.sql

test_agent_prompts.pyExercise prompt evidence and upgrade behavior +502/-0

Exercise prompt evidence and upgrade behavior

• Adds database-backed tests across all three installers for curated guidance, INFO omission, quoted targets, and column order. Covers view identity and dependencies, function recreation, restricted-role helper grants, and consistency of duplicated helper definitions.

testing/test_agent_prompts.py

Documentation (4) +172 / -0
README.mdIntroduce copyable prompts and flag function upgrades +24/-0

Introduce copyable prompts and flag function upgrades

• Adds the feature to the overview, shows how to select prompts, and clarifies that INFO prompts are NULL. Points existing function installations to the required upgrade procedure.

README.md

README.mdLink the check catalog to prompt guidance +4/-0

Link the check catalog to prompt guidance

• Explains which severities receive prompts and links to usage, safety, and upgrade documentation.

docs/health-checks/README.md

agent-prompts.mdDocument prompt safety, permissions, and upgrades +130/-0

Document prompt safety, permissions, and upgrades

• Describes copying and reviewing finding-specific prompts. Provides helper-grant guidance, view replacement expectations, and a transactional rehearsal for explicit function recreation.

docs/health-checks/agent-prompts.md

README.mdExplain how to run prompt integration tests +14/-0

Explain how to run prompt integration tests

• Adds the prompt test command, database configuration notes, and a summary of installer, evidence, upgrade, and permission coverage.

testing/integration/README.md

@qodo-code-review

qodo-code-review Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1)

Grey Divider


Informational

1. Query text reaches the agent prompt for long sessions 🟡 P2 🐞 Bug 👈 Shift left
Description
The new agent_prompt column passes p_current_value through to the generated prompt, and for the
Long Running Queries and Idle In Transaction checks that value is the raw session SQL from
pg_stat_activity. Any literal values in that SQL, including credentials or personal data, are
therefore copied into a prompt intended for an external agent, in addition to the existing output
column.
Code

pgFirstAid.sql[R2337-2340]

pgfirstaid_agent_prompt(
hr.severity, hr.category, hr.check_name, hr.object_name,
hr.issue_description, hr.current_value, hr.recommended_action,
hr.documentation_link
Relevance

●● Moderate

Security concern about SQL literals in prompts, but no matching accepted or rejected precedent
found. Subjective, so uncertain.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
pg_firstAid() now calls pgfirstaid_agent_prompt(... hr.current_value ...). For the 'Long Running
Queries' check, current_value is the raw query from pg_stat_activity, and for 'Idle In Transaction
Over 5 Minutes' it is the truncated query text, so both paths place session SQL inside the
generated prompt.

pgFirstAid.sql[1217-1243]
pgFirstAid.sql[1366-1378]
pgFirstAid.sql[2337-2340]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Session SQL text from pg_stat_activity is copied into agent_prompt through p_current_value, so literal values can reach an external agent's context.

## Fix Focus Areas
- pgFirstAid.sql[1217-1243]
- pgFirstAid.sql[1366-1378]
- pgFirstAid.sql[2337-2340]

## Recommended Fix
Do not pass raw query text into the prompt. Pass only the queryid or a short redacted form, or document that agent_prompt must be treated as sensitive. Apply the same change to the matching views.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


Grey Divider

Resolved findings

1. Shipped CI audit templates break on databases that already have pgFirstAid ✓ Resolved

Resolving commit: 8811677

Resolution reason
Installers now fail fast: psql runs with ON_ERROR_STOP=1 so failures stop the install instead of
continuing on the old function; a pgFirstAid.sql preflight DO block raises an explicit upgrade error
for legacy functions; the Python audit catches InvalidFunctionDefinition and psycopg2.Error and
exits with a logged message instead of crashing with a traceback; and the upgrade procedure is
documented.
Description
The new agent_prompt column in pg_firstAid()'s RETURNS TABLE makes `CREATE OR REPLACE
FUNCTION` fail with "cannot change return type of existing function" wherever an older copy is
already installed. The bundled workflows/pgfirstaid_audit.py pulls pgFirstAid.sql from main by
default and runs it with no preceding DROP, catching only OperationalError, so every PR audit
against an existing staging database crashes after merge. workflows/db-health-checks.yml runs `\i
pgFirstAid.sql without ON_ERROR_STOP`, so it keeps going on the old 8-column function and nobody
notices.

2. Query prompts target the wrong database ✓ Resolved

Resolving commit: 8811677

Resolution reason
Query findings now include dbid, originating database name, userid, and role in object_name, and the
prompt relabels the connection database as "Report connection database" while directing agents to
per-finding evidence for the true target. Query checks are also filtered to the connected database
before ranking.
Description
pgfirstaid_agent_prompt labels every finding with current_database(), but the query checks
retain only the query ID and omit the originating database and user. When a statistics reader sees a
query from another database, its copyable prompt identifies the connected database instead and lacks
the context needed to distinguish that query's statistics.

3. View reinstall no longer recovers from column-type drift ✓ Resolved

Resolving commit: cf9717d

Resolution reason
The view installers now call a shared _pgfirstaid_install_view helper (data/view_installer.sql,
embedded in view_pgFirstAid.sql and view_pgFirstAid_managed.sql). On an incompatible existing layout
it captures owner, grants (including grantors), comments and options, then drops and recreates
v_pgfirstaid with RESTRICT inside an atomic transaction, restores ACLs/comments, and rolls back on
failure. Compatible layouts still use CREATE OR REPLACE VIEW. This directly addresses
column-type/layout drift without the unconditional drop, though via a different mechanism than the
suggested explicit casts and CI upgrade test (the diff adds tests in
testing/test_view_installation.py covering old-layout recovery).
Description
Removing drop view if exists v_pgfirstAid means each installer now depends on `CREATE OR REPLACE
VIEW`. That only succeeds when every existing column keeps the same name, type and position. A
deployed view from an older release with a different layout, or any later check whose union branch
changes a column's resolved type, now makes the installer fail. The removed comment says the drop
existed precisely because of these "conversion issues with new health checks".

View resolved (2)
4. Agents skip transactional dry runs ✓ Resolved

Resolving commit: 8811677

Resolution reason
The shared agent_prompt template now instructs agents to rehearse transactional changes in staging
with BEGIN/ROLLBACK after explicit approval, and to handle non-transactional operations (VACUUM,
CONCURRENTLY index ops) through a separate staging rehearsal. The same wording is applied to
view_pgFirstAid.sql and view_pgFirstAid_managed.sql.
Description
The shared agent_prompt template requires staging tests but does not instruct agents to rehearse
transactional commands with BEGIN and ROLLBACK. When an agent proposes SQL for a finding, the
prompt can lead it to seek approval without the transactional dry run required by the pgFirstAid
verification workflow.

5. Prompt helper body is copied into three installers ✓ Resolved

Resolving commit: cf9717d

Resolution reason
Canonical helper moved to data/agent_prompt.sql; tools/sync_sql_helpers.py regenerates the embedded
copies in all three installers; the check mode runs in PR Safe Checks
(.github/workflows/pr-safe-checks.yml) so stale copies are rejected before merge.
Description
The entire pgfirstaid_agent_prompt body (50 curated cases plus the template) is duplicated in
pgFirstAid.sql, view_pgFirstAid.sql and view_pgFirstAid_managed.sql, and all three run `CREATE
OR REPLACE` on the same function name. A guidance edit made in one file and missed in another leaves
the copies out of sync. Whichever installer ran last then silently overwrites the shared helper for
the function and both views.

Grey Divider

Context sources
✅ Compliance rules (platform): 1 rule
✅ Web pages:
  +3 more
✅ Skills: using-pgfirstaid
Review mode: Auto: 🧠 Deep: Substantial SQL logic, duplicated installers, API changes, permissions, and many defect-prone paths.

Grey Divider

Comment thread pgFirstAid.sql Resolved
Comment thread pgFirstAid.sql Resolved
Comment thread view_pgFirstAid.sql Resolved
Comment thread view_pgFirstAid_managed.sql Resolved
Comment thread pgFirstAid.sql Resolved
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Neon Before/After Validation

Metric Value
Branch ID br-spring-river-ah59d860
Baseline CRITICAL 1
Post-change CRITICAL 1
New CRITICAL issues 0

No new critical issues -- safe to deploy.

Neon branch deleted automatically after workflow completion.

@randoneering
randoneering merged commit d8aa7f9 into main Nov 15, 2026
15 checks passed
@randoneering
randoneering deleted the feat/agent-remediation-prompts branch November 15, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant