ProcessAI Cyber Defense

AI vulnerability discovery: how an engagement runs, step by step

AI vulnerability discovery uses language models to read code and configuration at a scale a small team cannot, then relies on people to decide what is real. This page walks through the sequence: authorization and rules of engagement, model-assisted review, human reproduction of each finding, fix and retest, and the report that closes the work.

Reviewed 7 min read

On this page
  1. What model assistance adds to a conventional review
  2. The engagement from authorization to retest
  3. Seven steps from authorization to a closed finding
  4. Who does what on each finding
  5. Ranking real findings by evidence of exploitation
  6. What the rules of engagement should state
  7. What can go wrong, and the control against it
  8. A provider's own scanner shows the pattern
  9. Questions and answers
  10. Sources

What model assistance adds to a conventional review

A conventional review depends on how many hours a skilled engineer can spend reading code and configuration. Language models change the reading rate, not the judgment. They can scan a large repository, flag patterns that resemble known weakness classes, and draft a hypothesis about how a flaw might be reached.

What they cannot do is establish that a flaw matters in your environment. Reachability, compensating controls, data sensitivity and the cost of a fix are facts about your systems. That is why this process treats every model output as a lead and puts a person between the lead and the backlog.

The engagement from authorization to retest

01Authorize02Map and model threats03Model-assisted review04Human reproduction05Fix and retest06Report
  1. Authorize

    The system owner signs the scope and the rules of engagement before any work starts.

  2. Map and model threats

    Engineers inventory the in-scope code, hosts and services and list the attacks that would matter most.

  3. Model-assisted review

    Models read code and configuration beside conventional scanners and produce candidate findings.

  4. Human reproduction

    A security engineer reproduces each candidate in an isolated copy or an agreed test window.

  5. Fix and retest

    Your engineers apply the fix after a named approver agrees, and the finding is retested.

  6. Report

    The report records what was found, what was fixed, the retest evidence and what remains open.

Conceptual flow of a discovery engagement. Candidates that fail reproduction are discarded and logged, not reported.

Seven steps from authorization to a closed finding

The order matters. Skipping the first step is the one mistake the rest of the process cannot repair.

  1. Authorize the scope and write the rules of engagement

    The owner of each system in scope signs a document naming the assets, the dates, the permitted techniques and the people who can pause the work. Systems you do not own or are not authorized to test are listed as out of scope.

    Output
    Signed scope and rules of engagement
    Owner
    System owner and engagement lead
  2. Inventory the assets and build a threat model

    Engineers list repositories, services, hosts and data stores, then name the attacks that would hurt most: account takeover, data exfiltration, remote code execution on an internet-facing service. The list tells the reviewers where to look first.

    Output
    Asset inventory and ranked attack scenarios
    Owner
    Security engineer with your architects
  3. Run the model-assisted code review

    Models read the in-scope code for weakness classes such as injection, broken access control and unsafe deserialization. Conventional static analysis and dependency checks run in parallel, and each candidate carries the file, the reasoning and a confidence label.

    Output
    Candidate findings with reasoning
    Owner
    Security engineer
  4. Review infrastructure and configuration

    The same method is applied to cloud configuration, network exposure, identity policies and container images. Models help read large policy files and flag combinations that grant more access than intended.

    Output
    Candidate configuration findings
    Owner
    Security engineer with your platform team
  5. Reproduce every candidate by hand

    An engineer attempts to trigger each candidate in an isolated copy or in an agreed test window. Candidates that cannot be reproduced are discarded and logged. Confirmed findings record the exact steps, the affected assets and the evidence.

    Output
    Reproduced findings with steps and evidence
    Owner
    Security engineer
  6. Fix, approve and retest

    Models can draft a patch, and your engineers review it like any other change. A named approver authorizes each production change. The original reproduction steps are then rerun, and the finding closes only if they fail.

    Output
    Approved changes and passing retests
    Owner
    Your engineers and named approver
  7. Report and hand over

    The report lists findings by priority, the fix and retest status of each, and what was out of scope. It also records detection ideas and test cases that your team can keep running after the engagement ends.

    Output
    Final report, backlog and reusable tests
    Owner
    Engagement lead

Who does what on each finding

The split is the same for every finding, which makes the evidence trail easy to audit.

ActivityModel's rolePerson's roleEvidence kept
Spotting a candidateReads code and configuration and proposes a candidate with its reasoningSkims the candidate list and sets reading prioritiesCandidate record with file and reasoning
Confirming it is realSuggests how to trigger itRuns the reproduction in an isolated copy or a test windowSteps, output and the environment used
Judging impactSummarizes what the code does around the flawDecides reachability, data exposure and business impactWritten impact statement
Writing the fixDrafts a patch and a regression testReviews and edits the change like any other pull requestReviewed change and test
Changing productionNo roleA named approver authorizes the change and an engineer applies itApproval record with time and name

No row gives a model authority to change a production system.

Ranking real findings by evidence of exploitation

Once findings are confirmed, rank them with public exploitation signals where they exist. CISA's Known Exploited Vulnerabilities catalog lists vulnerabilities known to be exploited in the wild.1 FIRST's EPSS estimates the probability that a CVE will be exploited in the next 30 days, as a daily score between zero and one.2

Severity scores still matter, but they answer a different question. CVSS v4.0 separates a base score from threat and environmental context, so a finding on an isolated test host can score differently from the same flaw on a public service.3 CISA's SSVC decision tree ends in four outcomes (Track, Track*, Attend and Act) and uses five inputs, including exploitation status and whether exploitation can be automated.4

For new, previously unknown flaws in your own code, no exploitation data exists yet. Rank those by exposure and impact, and say so in the report.

What the rules of engagement should state

0 of 7 checked

What can go wrong, and the control against it

A false positive reaches the backlog

Early signalEngineers lose time on findings nobody can reproduce.

MitigationNothing enters the backlog until a person has reproduced it. Failed candidates are logged and counted.

Sensitive code or data leaves the agreed boundary

Early signalPrompts contain secrets or customer data.

MitigationFix the model and hosting terms in the rules of engagement, strip secrets before review, and log every request.

Testing reaches a system the client does not own

Early signalA scanner follows a link to a third-party host.

MitigationEnforce an allowlist of in-scope hosts in the tooling, and treat anything else as out of scope.

A fix closes the symptom and leaves the flaw

Early signalA retest passes only on the original input.

MitigationRetest with variants of the reproduction, and add a regression test to the repository.

A provider's own scanner shows the pattern

Questions and answers

Do findings from a model count as findings?

Not until a person reproduces them. A model's output is a candidate with reasoning attached. After reproduction it becomes a finding with steps, evidence and an impact statement. Candidates that cannot be reproduced are discarded and logged, so the final report contains only issues an engineer has confirmed in your environment.

Can the engagement test third-party or cloud-provider systems?

Only with the owner's permission, and often not at all. Cloud providers and software-as-a-service vendors publish their own testing rules, and a client cannot waive them. The rules of engagement list such systems as out of scope unless written permission from the owner is attached.

What access do the reviewers need to our code?

Read access to the in-scope repositories and configuration, granted for the engagement and removed at the end. Secrets are excluded or masked before review. The rules of engagement state which models are used, where prompts are processed and how long outputs are kept.

How do you decide which confirmed findings to fix first?

Start with exploitation evidence and exposure: whether a CVE appears in the Known Exploited Vulnerabilities catalog, its EPSS score, and whether the asset is reachable from the internet. Then weigh business impact and the effort to fix. Severity scores are an input, not the whole ranking.

What happens after the report is delivered?

The backlog stays with your engineers, each item with an owner and a due date. Retests of closed items are part of the engagement. If the work should repeat, the usual next step is a managed arrangement or a second sprint scoped from the first report's measured results.

Sources

  1. CISA: Known Exploited Vulnerabilities Catalog — CISA · checked 19 November 2026
  2. FIRST: Exploit Prediction Scoring System (EPSS) — FIRST · checked 19 November 2026
  3. FIRST: Common Vulnerability Scoring System version 4.0 — FIRST · checked 19 November 2026
  4. CISA: Stakeholder-Specific Vulnerability Categorization (SSVC) — CISA · checked 19 November 2026
  5. Anthropic: Bringing Claude Mythos 5 to more defenders — Anthropic · checked 19 November 2026
  6. OWASP: Web Security Testing Guide — OWASP · checked 19 November 2026

More in AI Cyber Defense

Back to AI Cyber Defense

Next step

Send the systems you want assessed first

List the repositories, hosts and applications you own and the question the assessment must answer. We will reply with a proposed scope, the rules of engagement to sign and what each step would produce.

Scope an assessment

Your starting point

What would you like to move forward?

Start with the outcome that feels closest. We'll help give the next step a useful shape.

Let's find your next move

One conversation. A clearer direction.

Tell us what you want to improve. A few sentences are enough to start.

shayan@coldai.org