AI Agent Security Backlog Remediation Pipeline: How to Build a Remediation Agent That Closes Its Own Issues (2026)

Section 1: What the loop is

The loop is a repeatable seven-stage cycle where a harness handles the mechanical stages, a single coding agent handles the fix, and a human owns the merge decision, returning to Stage 1 each run against a shrinking backlog.

That return trip is what separates a loop from a one-time cleanup project: the same script that pulled 400 open findings in week one pulls whatever's left in week eight, and the number keeps going down because the fixes from prior runs have already merged.

The harness runs stages 1 through 4 and stage 7. The fix agent runs stage 5. Codacy and a human jointly run stage 6, which is the one safety mechanism in the entire system that a script cannot pass on its own.

Two security metrics matter here, and only one of them should sit on a leadership dashboard:

Incident rate, meaning how often production actually breaks, is the primary outcome metric because it shows whether the reduction in unresolved findings is translating into fewer incidents

Time to remediate high-severity findings is the operational metric, useful for tuning the loop's throughput, but it's a means to the first number rather than an end in itself.

Section 2: Source the findings

Sourcing means pulling a structured and reliable list of open findings on a fixed schedule rather than whenever someone remembers to check the dashboard.

The harness calls the Codacy Cloud CLI to list open issues and security findings, filtered by repository, severity, category, and status, with JSON output enabled so the next stage can parse it without a human reading a report first.

Authentication runs through an API token so the CLI executes non-interactively, which matters because this whole system is meant to run unattended.

A GitHub Actions workflow with an on: schedule cron trigger runs the pull against the default branch on whatever timetable the team sets, and a plain cron job or Kubernetes CronJob does the identical job outside GitHub's runners.

Installing the Codacy Cloud CLI Skill puts the commands and options in front of the fix-agent without hard-coding glue code that breaks the next time an API changes. What comes out of this stage is a structured list of open findings, ready for triage rather than for a human to skim.

Section 3: Triage the findings

Triage happens in two passes: strip out what isn't real, then rank what's left. Running severity sorting on a list still full of false positives just produces a well-organized list of noise, so the order matters more than it looks.

The first pass ignores findings that aren't real issues, each with a recorded reason such as FalsePositive or NotExploitable, entered through the Cloud CLI so the decision is auditable and doesn't silently resurface on the next scan.

Codacy's Smart False Positive Triage uses broader context from the surrounding code to help assess whether findings are relevant, which matters specifically for teams processing a high volume of AI-generated commits where static analysis alone tends to overflag.

The second pass sorts what remains by severity, category, and status, and for dependency findings, the SCA data carries CVE identifiers and affected-function detail that should drive which fix lands first. What's left after both passes is an ordered and actionable list, not just a shorter one.

Section 4: Record ignores and exclude irrelevant paths

Once the finding is identified as a false positive (and therefore not actionable), it can be dismissed as such. For entire files or directories that should be excluded moving forward, such as a test folder full of intentionally insecure fixtures, those paths can be excluded with an exclude_paths entry in the repository's .codacy.yaml, committed through a pull request so the change is version-controlled and reviewed like any other code change, rather than toggled in a settings panel nobody remembers exists.

Codacy uses that configuration when analyzing the repository, so the exclusion remains in effect until someone deliberately changes the configuration. Keeping the ruleset tuned matters just as much as the exclusion itself; the Configure Codacy skill turns on the tools and patterns that actually apply to a given project and turns off the ones that don't, which is what stops irrelevant findings from reappearing every time the scanner runs.

Section 5: Route work to the fix agent

Routing converts an actionable finding into a ticket that the fix agent can actually act on, and the difference between a usable ticket and a useless one is context.

Codacy's native Jira integration creates a basic ticket, but a title and severity label without a code location or remediation guidance forces the fix agent to go rediscover information the scanner already had.

The harness closes that gap by writing detailed tickets, pulling file, line, severity, description, and remediation guidance straight from the Codacy API, grouped by team and project, so no single owner ends up buried under work meant for three different squads.

Each ticket gets assigned to the fix agent directly, turning the queue into a worklist rather than a backlog someone has to triage twice. What comes out is a queue of detailed, assigned tickets that a coding agent can pick up without a human translating scanner output into a task first.

Section 6: Fix and gate the change

The fix agent (whichever coding agent the team already prefers) takes one ticket, writes the fix, and opens a pull request scoped to that single finding, because a PR covering five unrelated fixes is a PR nobody reviews carefully.

Every pull request runs through Codacy’s AI PR Reviewer, with a merge gate configured to fail the PR if it introduces new issues or if its diff coverage falls below the configured threshold. Making things easier, it’s possible to set up coverage tracking with the setup-coverage skill rather than doing it manually.

When the gate fails, the fix agent gets the chance to revise the same PR; if it can't resolve the failure on its own, it escalates to a human rather than looping indefinitely on a fix that isn't converging. A human reviewer still approves and merges every change.

The gate checks the PR but never merges it itself, and that approval is the single point in the entire loop where a person, not a script, accepts the risk of the change going live.

Section 7: Close the ticket, and optionally verify

Closing occurs after a human has approved and merged the fix. Two approaches work here, and which one a team picks depends on how much control they want over the closing logic itself.

The Jira-native approach has the fix agent put the ticket key, something like PROJ-123, in the branch name or PR title; Jira's own GitHub integration links the PR to the ticket automatically, and a Jira Automation rule transitions the ticket to Done the moment the PR merges, with no custom code required on Codacy's side.

The harness-driven approach keeps the mapping between finding, ticket, and PR inside the harness itself, which watches for the merge event and closes the ticket through the Jira API directly, useful when a team wants the close event under its own logic rather than delegated to a Jira rule.

Verification is optional but valuable for security work specifically: the harness re-runs Codacy analysis after the merge and checks through the Cloud CLI that the finding no longer appears, confirming the fix actually removed the problem rather than just closing the paperwork around it.

Section 8: Keep humans in the loop

The loop runs unattended between checkpoints, but the checkpoints themselves stay human, and that distinction is what keeps an automated backlog cleanup from becoming an unmonitored one. A few of these checkpoints matter more than others.

• Merge approval stays the one non-negotiable gate: a person owns the final decision on every change that ships, regardless of how confidently the gate passed it.
• Suppression review happens periodically, checking that the ignores accumulated over weeks of runs were actually the right calls and not a pattern quietly hiding something real.
• Scope and prioritization decisions, meaning which repositories go first and what counts as "done" for a given vertical, stay with people who understand the business context a scanner can't see.
• Guardrails on automated ignores cap how many findings a single run can dismiss, so a misconfigured rule can't mass-ignore a category of real vulnerabilities before anyone notices.
  
  

Section 9: Roll it out in stages

Rolling out the loop works best as a staged expansion rather than an org-wide flip, because proving the architecture on a small surface catches configuration mistakes before they touch every team's backlog at once.

1. Run the loop end-to-end on a single repository, watching every stage manually the first few times through.
2. Expand one team or vertical at a time, tuning the ticket routing and gate thresholds as new codebases surface edge cases the pilot didn't.
3. Reach a steady state, where the loop processes new findings as they appear so the backlog stops rebuilding between runs.

The target worth setting is zero open critical findings within a targeted vertical, worked through at scale rather than chased finding by finding.

Closing

Once sourcing, triage, fixing, and verification run as one continuous loop instead of four separate manual steps, a backlog that used to get pushed to next sprint becomes something a small team clears on a schedule.

The result that matters to leadership is a lower incident rate, because the findings sitting in that backlog were the risk all along, and a loop that closes them continuously helps keep that risk from compounding faster than a team can review it by hand, especially as NIST reported that CVE submissions surged 263% from 2020 to 2025.