ProcessAI Cyber Defense
AI vulnerability discovery: how an engagement runs, step by step
AI vulnerability discovery uses language models to read code and configuration at a scale a small team cannot, then relies on people to decide what is real. This page walks through the sequence: authorization and rules of engagement, model-assisted review, human reproduction of each finding, fix and retest, and the report that closes the work.
On this page
- What model assistance adds to a conventional review
- The engagement from authorization to retest
- Seven steps from authorization to a closed finding
- Who does what on each finding
- Ranking real findings by evidence of exploitation
- What the rules of engagement should state
- What can go wrong, and the control against it
- A provider's own scanner shows the pattern
- Questions and answers
- Sources
What model assistance adds to a conventional review
A conventional review depends on how many hours a skilled engineer can spend reading code and configuration. Language models change the reading rate, not the judgment. They can scan a large repository, flag patterns that resemble known weakness classes, and draft a hypothesis about how a flaw might be reached.
What they cannot do is establish that a flaw matters in your environment. Reachability, compensating controls, data sensitivity and the cost of a fix are facts about your systems. That is why this process treats every model output as a lead and puts a person between the lead and the backlog.
Who does what on each finding
The split is the same for every finding, which makes the evidence trail easy to audit.
| Activity | Model's role | Person's role | Evidence kept |
|---|---|---|---|
| Spotting a candidate | Reads code and configuration and proposes a candidate with its reasoning | Skims the candidate list and sets reading priorities | Candidate record with file and reasoning |
| Confirming it is real | Suggests how to trigger it | Runs the reproduction in an isolated copy or a test window | Steps, output and the environment used |
| Judging impact | Summarizes what the code does around the flaw | Decides reachability, data exposure and business impact | Written impact statement |
| Writing the fix | Drafts a patch and a regression test | Reviews and edits the change like any other pull request | Reviewed change and test |
| Changing production | No role | A named approver authorizes the change and an engineer applies it | Approval record with time and name |
No row gives a model authority to change a production system.
Ranking real findings by evidence of exploitation
Once findings are confirmed, rank them with public exploitation signals where they exist. CISA's Known Exploited Vulnerabilities catalog lists vulnerabilities known to be exploited in the wild.1 FIRST's EPSS estimates the probability that a CVE will be exploited in the next 30 days, as a daily score between zero and one.2
Severity scores still matter, but they answer a different question. CVSS v4.0 separates a base score from threat and environmental context, so a finding on an isolated test host can score differently from the same flaw on a public service.3 CISA's SSVC decision tree ends in four outcomes (Track, Track*, Attend and Act) and uses five inputs, including exploitation status and whether exploitation can be automated.4
For new, previously unknown flaws in your own code, no exploitation data exists yet. Rank those by exposure and impact, and say so in the report.
What the rules of engagement should state
What can go wrong, and the control against it
A false positive reaches the backlog
Early signalEngineers lose time on findings nobody can reproduce.
MitigationNothing enters the backlog until a person has reproduced it. Failed candidates are logged and counted.
Sensitive code or data leaves the agreed boundary
Early signalPrompts contain secrets or customer data.
MitigationFix the model and hosting terms in the rules of engagement, strip secrets before review, and log every request.
Testing reaches a system the client does not own
Early signalA scanner follows a link to a third-party host.
MitigationEnforce an allowlist of in-scope hosts in the tooling, and treat anything else as out of scope.
A fix closes the symptom and leaves the flaw
Early signalA retest passes only on the original input.
MitigationRetest with variants of the reproduction, and add a regression test to the repository.
A provider's own scanner shows the pattern
Questions and answers
Do findings from a model count as findings?
Not until a person reproduces them. A model's output is a candidate with reasoning attached. After reproduction it becomes a finding with steps, evidence and an impact statement. Candidates that cannot be reproduced are discarded and logged, so the final report contains only issues an engineer has confirmed in your environment.
Can the engagement test third-party or cloud-provider systems?
Only with the owner's permission, and often not at all. Cloud providers and software-as-a-service vendors publish their own testing rules, and a client cannot waive them. The rules of engagement list such systems as out of scope unless written permission from the owner is attached.
What access do the reviewers need to our code?
Read access to the in-scope repositories and configuration, granted for the engagement and removed at the end. Secrets are excluded or masked before review. The rules of engagement state which models are used, where prompts are processed and how long outputs are kept.
How do you decide which confirmed findings to fix first?
Start with exploitation evidence and exposure: whether a CVE appears in the Known Exploited Vulnerabilities catalog, its EPSS score, and whether the asset is reachable from the internet. Then weigh business impact and the effort to fix. Severity scores are an input, not the whole ranking.
What happens after the report is delivered?
The backlog stays with your engineers, each item with an owner and a due date. Retests of closed items are part of the engagement. If the work should repeat, the usual next step is a managed arrangement or a second sprint scoped from the first report's measured results.
Sources
- CISA: Known Exploited Vulnerabilities Catalog — CISA · checked 19 November 2026
- FIRST: Exploit Prediction Scoring System (EPSS) — FIRST · checked 19 November 2026
- FIRST: Common Vulnerability Scoring System version 4.0 — FIRST · checked 19 November 2026
- CISA: Stakeholder-Specific Vulnerability Categorization (SSVC) — CISA · checked 19 November 2026
- Anthropic: Bringing Claude Mythos 5 to more defenders — Anthropic · checked 19 November 2026
- OWASP: Web Security Testing Guide — OWASP · checked 19 November 2026
More in AI Cyber Defense
Back to AI Cyber DefenseYour starting point
What would you like to move forward?
Start with the outcome that feels closest. We'll help give the next step a useful shape.
Let's find your next move
One conversation. A clearer direction.
Tell us what you want to improve. A few sentences are enough to start.
shayan@coldai.org