ArchitectureAI Cyber Defense
AI SOC alert triage: a reference architecture with analyst sign-off
AI alert triage puts a language model between the SIEM and the analyst to enrich alerts, summarize evidence and suggest a priority. This page sets out the layers of a reference design, where the model should sit, which actions are automated and which need a person, and how to evaluate quality and catch drift once the system is live.
On this page
- What the design has to guarantee
- Reference architecture in six layers
- One alert, from detection to analyst decision
- What is automated and what needs a person
- Where the model should sit
- Guardrails against the failure modes that matter
- Enrichment sources the model should cite
- Evaluating quality and watching for drift
- Start in shadow mode before analysts see anything
- Questions and answers
- Sources
What the design has to guarantee
A triage assistant earns its place only if analysts can trust and check what it says. The design therefore starts from four properties: every statement in a summary traces back to a log line or an enrichment lookup, the model cannot take action by itself, a person owns every closure, and quality is measured against analysts' own decisions.
NIST's Cybersecurity Framework 2.0 places this work in the Detect and Respond functions.1 NIST SP 800-61 Revision 3, published in April 2025, frames incident response as part of cybersecurity risk management and replaced Revision 2.2 Neither document is specific to AI. They set the outcomes the architecture must still meet.
Reference architecture in six layers
Each layer has one job, which keeps the model's role narrow and easy to test.
- Analyst decision
Analysts read the summary, ask follow-up questions and close, escalate or request an action.
- Guardrails and approvals
Policy checks, output validation and the approval step that gates any action on a system.
- Triage model
Summarizes evidence, suggests a priority and lists what it could not verify.
- Enrichment
Deterministic lookups for asset owner, user context, threat intelligence and vulnerability status.
- Normalization
Alerts and logs mapped to one schema, with identifiers and timestamps cleaned.
- Data sources
SIEM alerts, endpoint and identity logs, cloud audit logs, email and network telemetry.
One alert, from detection to analyst decision
- SIEM
Raises the alert from a detection rule.
- Enrichment
Adds context from deterministic sources.
- Triage model
Drafts the summary and a suggested priority.
- Analyst
Reviews, questions and decides.
- Response tooling
Executes only approved actions.
What is automated and what needs a person
Draw the line by consequence. Reading is automated. Anything that changes a system or closes a case is not.
| Action | Automation level | Who approves | Record kept |
|---|---|---|---|
| Enrich with asset, user and threat context | Automatic | No approval needed | Lookup results attached to the alert |
| Summarize evidence and suggest a priority | Automatic suggestion | Analyst accepts or changes the priority | Suggestion, final priority and any override |
| Close an alert as benign | Suggested only | The analyst who owns the queue | Reason for closure and evidence cited |
| Open or escalate an incident | Suggested only | On-call analyst or incident lead | Escalation note with linked alerts |
| Isolate a host or disable an account | Never automatic | Named approver for that system | Approver, time, action and rollback |
| Change a detection rule | Draft only | Detection engineer, through normal change control | Reviewed change and test results |
A client may decide that a narrow, low-risk action, such as adding a tag, can run without sign-off. Write that exception down and review it on a schedule.
Where the model should sit
Placement is a data decision first and a cost decision second.
- If
Alerts contain regulated data, customer identifiers or privileged credentials.
ThenRun the model inside your own cloud account or on your own hardware, and keep prompts and outputs in your logging stack.
The data never leaves your boundary, and retention follows your policy. See self-hosted LLM architecture.
- If
Alerts can be reduced to metadata and masked fields before they reach the model.
ThenA provider's hosted model under written data-handling terms may be acceptable.
Masking removes the most sensitive content, and the terms define retention and use.
- If
You cannot yet say what the model may see.
ThenRun it on historical, masked alerts only until you can.
A pilot on old data tests accuracy without exposing live telemetry.
Guardrails against the failure modes that matter
Prompt injection through alert content
Early signalAn alert field contains text that reads like an instruction, such as a user-agent string or an email subject.
MitigationTreat all log and email content as data. Separate it from instructions, validate outputs, and give the model no tool that can act. The OWASP Top 10 for LLM Applications lists prompt injection first.3
Excessive agency
Early signalThe model's connector has write permissions it does not need.
MitigationGive the triage service read-only access. Actions go through response tooling that checks for a recorded approval. OWASP lists excessive agency as its own risk.3
Analysts over-trust the summary
Early signalOverride rates fall toward zero while the model's accuracy has not been re-measured.
MitigationShow evidence beside every claim, require a reason for closure, and audit a random sample of closed alerts each week.
Quality drifts after a change
Early signalA new log source, a rule change or a model update shifts the mix of alerts.
MitigationRerun the evaluation set before and after every change, and pause suggestions if agreement drops below the agreed floor.
Enrichment sources the model should cite
Put deterministic enrichment ahead of the model so that its summary rests on facts. Useful sources include the asset inventory, identity data and vulnerability status. CISA's Known Exploited Vulnerabilities catalog can mark whether a flagged vulnerability is known to be exploited in the wild.4 EPSS adds a probability, between zero and one, that a CVE will be exploited in the next 30 days.5
Map each alert to MITRE ATT&CK techniques where a rule has a mapping. ATT&CK is a knowledge base of adversary techniques based on real-world observations, and a shared vocabulary helps analysts compare alerts and helps you measure coverage.6
Evaluating quality and watching for drift
Build an evaluation set from past alerts that analysts closed, with their final decision and reasoning. Score the model against it before go-live, and then run it in shadow mode, producing suggestions that analysts cannot see, so that you can compare its calls with theirs on live traffic.
Track four measures: how often analysts agree with the suggested priority, how many true positives the model would have deprioritized, how often analysts override it, and how long a first summary takes. The second measure matters most, because a missed true positive costs far more than a slow summary.
Drift has three common causes: detection rules change, new log sources arrive, or the model version changes. Treat each as a release. Rerun the evaluation, compare with the last result, and record the outcome next to the change ticket.
Start in shadow mode before analysts see anything
Questions and answers
Should the model be allowed to close alerts?
Not without a person. The model can suggest closure and cite the evidence, and an analyst makes the call. Some teams later allow narrow exceptions, such as closing a duplicate of an alert already under investigation. Write each exception down, limit it to a defined alert type, and review its error rate regularly.
How do we stop attackers manipulating the model through logs?
Treat every log field, email body and file name as untrusted data. Keep it apart from the model's instructions, validate the output format, and give the triage service no ability to act. Then test with adversarial alerts that contain instruction-like text before go-live and after each change.
What data should the model be allowed to see?
Start with the minimum needed to triage: alert metadata, normalized event fields and enrichment results. Mask credentials, personal data and customer identifiers unless the analyst workflow needs them and the hosting arrangement allows it. Record what each deployment may see in the same document that approves its hosting.
How do we know the system is still accurate next quarter?
Rerun the evaluation set on a schedule and after every rule, log-source or model change, and sample closed alerts for review each week. Compare results with the previous run. A drop in agreement or a rise in missed true positives pauses the suggestions until someone has found the cause.
Does this work with our existing SIEM and SOAR tools?
Usually yes. The design reads alerts through the SIEM's API, adds enrichment from existing inventories and sends approved actions to the SOAR or ticketing tool you already use. The model layer is separate, so you can replace it without rewriting detections or playbooks.
Sources
- NIST: NIST Releases Version 2.0 of Landmark Cybersecurity Framework — NIST · checked 19 November 2026
- NIST SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management — NIST · checked 19 November 2026
- OWASP: Top 10 for LLM Applications 2025 — OWASP · checked 19 November 2026
- CISA: Known Exploited Vulnerabilities Catalog — CISA · checked 19 November 2026
- FIRST: Exploit Prediction Scoring System (EPSS) — FIRST · checked 19 November 2026
- MITRE ATT&CK: Resources — MITRE · checked 19 November 2026
More in AI Cyber Defense
Back to AI Cyber DefenseYour starting point
What would you like to move forward?
Start with the outcome that feels closest. We'll help give the next step a useful shape.
Let's find your next move
One conversation. A clearer direction.
Tell us what you want to improve. A few sentences are enough to start.
shayan@coldai.org