Artificial Intelligence
Intelligence, connected to the work.
ColdAI's artificial intelligence practice builds systems in which the model is only one component. Documents, records and permissions supply the context; rules, retrieval and models share the reasoning; approved tools carry out the actions; and evaluation and human review keep the result accountable. This page explains how those parts depend on each other, how to choose between them and which specialist practice to start with.
The opportunity
Start with a decision
worth improving.
A service team needs answers from scattered documents, followed by an accurate update to its operating system. The challenge is connecting knowledge to a controlled action.
From possibility to a working system
Follow the flow.
Three connected decisions. One considered architecture.
Conceptual flow, not a live system or measured result.
Connect the context
Map the documents, records and permissions a workflow actually needs. Establish which sources are authoritative and how updates reach the model before choosing a model or interface.
Choose how to reason
Compare retrieval, structured rules and model capabilities on representative tasks. Test uncertainty and missing information, not just the ideal demonstration, so the architecture reflects the work.
Make the action accountable
Connect approved tools, capture evidence and route exceptions to people. Release against an agreed baseline for quality, cost and time, then monitor changes in real use.
The work, made concrete
What we can build
with your team.
We agree scope, dependencies and acceptance criteria before delivery. Your project can start with an assessment, a pilot or an integration into an existing system.
A workflow and data architecture
Clarify the system boundary, the responsible owners and the decisions the architecture needs to support.
An evaluated, bounded pilot
Turn the agreed design into reviewable work, evaluated against representative inputs and explicit success criteria.
A deployment and monitoring plan
Make the next step operable: document responsibilities, known limitations and the path from pilot to ongoing use.
In depth
Go deeper: methods that apply to every AI system
Cross-cutting guides that hold whichever model, platform or specialist practice you choose.
- 01GuideHow to build an evaluation set for an LLM applicationA practical method for evaluating an LLM application: choosing the unit of evaluation, sampling cases, writing rubrics, calibrating LLM judges, gating releases.
- 02Buyer's guideEstimating the total cost of an AI systemBuild a total cost of ownership model for an AI system from its parts: inference, retrieval, evaluation, human review, integration, operations and migrations.
Why an AI system is more than its model
A model on its own can draft, classify, summarize and answer. A business usually needs something narrower and more dependable: the right answer for this customer, drawn from current sources, followed by a correct change in the system where the work is recorded. Most of the engineering effort in an AI project goes into the distance between those two things.
That distance has four parts. The system has to find the context a task depends on and respect who is allowed to see it. It has to decide how to reason about each step, which is not always a job for a large model. It has to act through tools in a way that leaves evidence. And someone has to be able to tell, before and after release, whether it is doing the job well enough. The NIST AI Risk Management Framework makes that last point explicit by treating measurement as a core function alongside governing, mapping and managing risk2.
ColdAI treats each part as a design decision in its own right. That is why an engagement in this practice typically produces a workflow and data architecture, an evaluated and bounded pilot, and a deployment and monitoring plan, rather than a model recommendation on its own1. The specialist practices linked below each go deep on one part; this page gives the cross-cutting view.
The layers of a working AI system
Read from the top down. Each layer depends on the ones beneath it, so a weakness low in the stack tends to surface as a quality problem at the top.
- Workflow and owner
The recurring piece of work, the person accountable for it and the agreed definition of a good outcome.
- Oversight and approvals
The points where a person reviews, approves, corrects or stops what the system proposes.
- Tools and actions
Approved, narrowly scoped connections that read from or write to the systems of record.
- Reasoning
The mix of explicit rules, retrieval and model calls that turns context into a decision or a draft.
- Context and permissions
Authoritative documents and records, how they stay current and who is allowed to see them.
- Evaluation and monitoring
Test sets, release gates and production signals that show whether quality is holding.
Rules, retrieval or a model: choosing how each step reasons
Most workflows mix all three. The useful question is which approach should own each step, judged on representative tasks rather than on a demonstration.
| Question | Explicit rules | Retrieval over sources | Model reasoning |
|---|---|---|---|
| Best suited to | Fixed policies, thresholds, eligibility checks and anything you must explain line by line to an auditor | Questions whose answers already exist in documents or records that change over time | Ambiguous inputs, free text, drafting and judgments that weigh several signals at once |
| What it needs | Someone who can state the policy precisely and keep it current | Authoritative sources, an indexing process and access controls that follow the user | Clear instructions, worked examples and an evaluation set that covers the hard cases |
| Typical failure | Breaks silently on an input nobody anticipated | Returns a stale, partial or unauthorized passage, and the answer inherits the error | Produces a fluent but wrong answer, or behaves differently after a model update |
| How you audit it | Read the rule and the input that triggered it | Inspect which passages were retrieved and cited | Score outputs against a reviewed test set and sample production traces |
| Cost of change | Low for small edits, rising quickly as rules multiply | Moderate: re-index the sources and recheck retrieval quality | A prompt edit is cheap; a model migration needs a full re-evaluation |
When a general-purpose model still fails with good retrieval and clear rules, adapting the model becomes an option; the LLM fine-tuning practice covers that decision.
Which AI practice to start with
Each child practice of this hub solves a different problem. Start from the constraint that is hardest to change.
- If
The work needs software to take steps on its own, such as gathering missing details, calling systems and preparing an update for someone to approve
ThenStart with AI agents: one agent with a defined role, scoped tools and explicit approval points.
A single bounded agent is easier to evaluate and operate than a general assistant or a team of agents.
- If
General-purpose models misread your terminology, formats or tone even when they are given the right sources
ThenLook at LLM fine-tuning, once an evaluation set has confirmed the failure.
Adapting a model adds a training and maintenance cycle, so the evidence should come first.
- If
No existing model handles your data type, such as sensor streams, imagery or a proprietary signal
ThenConsider custom AI model development, compared against simpler baselines before committing compute.
Training from your own data is justified only when cheaper approaches have been tested and fall short.
- If
Several teams want AI at once and the data has to stay inside your own environment
ThenBegin with enterprise AI: shared identity, knowledge, model and observability layers that every workflow reuses.
Building those layers once avoids each team solving access, logging and hosting separately.
- If
Inference has to run on devices or sites with limited connectivity or tight latency limits
ThenStart with edge AI and IoT, where model size and the update strategy drive the design.
- If
You want to test decisions about physical assets or processes before making them, or act on equipment signals early
ThenExplore digital twins, and add predictive monitoring where sensor data should trigger maintenance.
A hypothetical facilities firm, rebuilt layer by layer
Where AI systems break after the demo
Sources drift out of date
Early signalAnswers cite documents that have been replaced, or miss a recent policy change.
MitigationName an owner for each source, re-index when a source changes rather than on a calendar, and add a regression case whenever a policy changes.
Retrieval ignores permissions
Early signalUsers see passages from records they could not open directly.
MitigationEnforce access at retrieval time using the requesting user's identity, and test it with accounts that ought to be refused.
The model changes underneath you
Early signalTone or accuracy shifts without any change to your prompts or code.
MitigationPin model versions where the provider allows it, and rerun the full evaluation set before adopting a new version.
Exceptions have no owner
Early signalLow-confidence cases collect in a queue that nobody checks.
MitigationRoute exceptions to a named team with a response target, and review that queue as part of normal operations.
Costs grow faster than usage
Early signalSpend rises while volumes stay flat, usually through retries, longer prompts or wider retrieval.
MitigationTrack cost per completed task rather than per call, and review it alongside quality; our AI cost guide sets out the components.
Where strategy, governance and operations are covered
This hub is about how AI systems are engineered. Deciding which use cases deserve funding, choosing an operating model for AI across the organization and meeting obligations such as the EU AI Act sit with the artificial intelligence capability. An independent review of a specific AI decision belongs with strategic consulting, and running a system after launch, including monitoring and incident handling, is part of managed services.
Frequently asked questions
What does ColdAI's AI technology practice actually build?
Systems that connect a model to the documents, records, tools and people around one specific workflow. Typical outputs are a workflow and data architecture, an evaluated pilot with a defined scope, and a plan for deploying and monitoring it. The model is chosen during that work, against your own test cases, rather than fixed at the start.
Do we need our own AI model to get value from AI?
Usually not. Many workflows are served well by an existing model combined with good retrieval, clear rules and well-designed tools. Fine-tuning or custom training becomes worth considering when an evaluation set shows a consistent failure that better context cannot fix, or when no existing model handles your type of data.
How do we choose between a chat assistant, an automated workflow and an AI agent?
Ask who decides the next step. If the steps are always the same, a deterministic workflow is cheaper and easier to audit. If people only need answers and will act themselves, an assistant with retrieval may be enough. If the system must choose its own next step and call tools to finish a task, it is an agent, and it needs scoped permissions and approval points.
How is the quality of an AI system measured before it goes live?
Against an evaluation set built from real and deliberately difficult cases, with pass criteria agreed for each category. The same set then acts as a release gate whenever prompts, models or sources change. Our guide to building an LLM evaluation set walks through the method step by step.
Which data has to be ready before an AI pilot starts?
Representative samples of the documents and records the workflow uses, the people who own those sources and can grant access, and examples of both good and failed outcomes. Perfect data is not a prerequisite. Finding out where sources are incomplete, contradictory or out of date is part of the design work itself.
Can AI systems run inside our own infrastructure?
Yes, where the workflow and the data require it. Private or on-premise inference changes the cost profile, the operating effort and the range of models available, so it is best designed as a shared foundation rather than one project at a time. Our enterprise AI practice covers that architecture.
Sources
- Artificial Intelligence technology practice: deliverables, readiness and boundaries — ColdAI
- Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 — National Institute of Standards and Technology
Connect the architecture
Go deeper into the stack.
Explore the complementary capabilities that can turn an individual technology into a complete workflow.
Your next move
Bring us the
real problem.
A workflow that takes too long. A system that cannot connect. An idea that needs a technical path. Start there, and we can define what to investigate, build and measure.
- A recurring business workflow
- Representative data and access owners
- A measurable definition of success