Edge AI & IoT
Move the decision closer to the signal.
Edge AI development puts a trained model on the device that senses the world, so it can decide without a round trip to the cloud. The hard parts are physical: limited memory and power, links that drop, enclosures that heat up and fleets that must be updated for years. ColdAI designs the sensing, local inference and fleet operation together, starting from the decision a device has to make and the evidence it should keep.
The opportunity
Start with a decision
worth improving.
A remote asset needs to recognize a condition even when connectivity is unreliable. The system must decide locally, preserve useful evidence and synchronize when the network returns.
From possibility to a working system
Follow the flow.
Three connected decisions. One considered architecture.
Conceptual flow, not a live system or measured result.
Capture what matters
Choose the signals, sampling strategy and hardware constraints for the task. Establish data quality and environmental variation before selecting an inference approach for the device.
Decide at the edge
Evaluate models against memory, power and latency limits. Define what happens when confidence is low and which decisions require a connected service or human operator.
Operate a fleet
Design secure updates, version tracking and delayed synchronization. Monitor device health and model behavior so an improvement can be rolled out without losing visibility into deployed units.
The work, made concrete
What we can build
with your team.
We agree scope, dependencies and acceptance criteria before delivery. Your project can start with an assessment, a pilot or an integration into an existing system.
Device and inference architecture
Clarify the system boundary, the responsible owners and the decisions the architecture needs to support.
A hardware-aware pilot
Turn the agreed design into reviewable work, evaluated against representative inputs and explicit success criteria.
Fleet deployment and monitoring design
Make the next step operable: document responsibilities, known limitations and the path from pilot to ongoing use.
In depth
Go deeper into edge AI engineering
Two working guides for the steps that most often stall edge projects: fitting the model onto the device and keeping a fleet of devices current.
- 01Deep diveQuantization, pruning and distillation for edge AIHow quantization, pruning and knowledge distillation shrink a model for edge hardware, the order to apply them, which runtimes fit which chips and how to test.
- 02ProcessUpdating AI models across an edge device fleetA staged process for shipping new AI models to edge devices: versioned bundles, signed updates, shadow mode, canary gates, rollback and security baselines.
Device, gateway or cloud: deciding where each inference runs
Most real systems split the work. Place each decision by its constraints, not by a single architectural preference.
- If
The decision must be made faster than a network round trip allows, such as stopping a conveyor or rejecting a part.
ThenRun inference on the device or on a controller beside it.
Network latency and outages become safety and quality problems when the decision cannot wait.
- If
Raw signals are large, such as video or high-rate vibration, and links are metered or slow.
ThenInfer at the edge and send events, summaries or short clips rather than the raw stream.
Moving raw data costs bandwidth and power continuously; sending results costs them only when something happens.
- If
Raw data is personal or commercially sensitive, such as images of people or proprietary process signals.
ThenKeep raw data on the device or site and export only what the purpose needs.
Data that never leaves the site is easier to govern than data copied to several clouds.
- If
Several nearby sensors must be combined, or the model is too large for each sensor.
ThenUse a site gateway or industrial PC as the inference point for a group of devices.
A gateway has more memory and power than a sensor while staying on the local network.
- If
The model changes weekly, needs large context or the decision can wait minutes.
ThenKeep it in the cloud and let devices filter and forward.
Frequent model changes are cheaper to ship to one service than to a fleet.
Edge hardware classes and the constraint that binds each one
Hardware choice should follow the decision and its budget. Each class below is good at something and limited by something else.
| Hardware class | Suits | Binding constraint | Weaker when |
|---|---|---|---|
| Microcontroller | Always-on sensing on batteries: vibration, sound, simple vision | On-chip memory for weights and buffers | Models need many layers or high-resolution images |
| Application SoC with NPU | Cameras, handhelds and smart devices running Linux or Android | Which operators the NPU supports | The model uses layers the accelerator cannot run |
| GPU module | Several camera streams or larger models at a site | Power draw and heat in a sealed enclosure | Power comes from a battery or a small solar supply |
| FPGA | Fixed, deterministic latency and custom signal paths | Engineering effort to change the design | The model is still changing frequently |
| Gateway or industrial PC | Combining many sensors on one site | Cabling and a single point of failure | Each sensor must decide on its own while offline |
Custom silicon sits beyond this table: it only pays off for stable workloads at volume, as the GPU, FPGA and ASIC comparison explains.
The software stack on a production edge AI device
- Fleet agent
Reports health and bundle version, receives signed updates and configuration.
- Buffer and sync
Stores decisions and evidence while offline and forwards them in order when the link returns.
- Decision and fallback
Turns model output into an action, applies confidence thresholds and safe defaults.
- Inference runtime
Runs the compiled model on CPU, NPU, GPU or FPGA.
- Signal conditioning
Filtering, windowing and feature extraction that must match training exactly.
- Sensor and board
Sensors, clocks, power management and secure storage for keys.
Energy, duty cycle and heat as first-class design inputs
On a battery or solar device, the useful question is not how accurate the model is but how many correct decisions it can make per unit of stored energy. That depends on how often the device wakes, how long the sensor and processor stay on, and how much the radio transmits. A model that is slightly less accurate but lets the processor sleep between short bursts can deliver more value over the device's life than a larger one that drains the battery early.
A common pattern is a cascade: a very cheap detector runs continuously and wakes a heavier model only when something interesting happens. Thresholds for the wake stage are tuned for few misses, while the second stage is tuned for few false alarms. Heat matters too. A model that meets its latency budget on an open bench may slow down inside a sealed enclosure in summer, so latency tests belong in realistic thermal conditions, not just at room temperature.
When the model does not fit the budget at all, compression is the next step. The guide to quantization, pruning and distillation covers how to shrink a model without losing the cases that matter.
Problems that surface only once devices are in the field
Sensors age, foul or are remounted
Early signalGradual change in input statistics at one site while others stay stable.
MitigationTrack input distributions per device and site, and schedule recalibration or cleaning as part of the operating plan.
Preprocessing on the device differs from training
Early signalGood offline accuracy, poor results on the very first devices.
MitigationShip the same conditioning code to training and device, or test both against shared reference recordings.
Device clocks drift while offline
Early signalEvents arrive out of order or appear to precede their causes.
MitigationRecord a monotonic counter with each event and correct timestamps when the device resynchronizes.
Physical access to the device
Early signalOpened enclosures, unexpected reboots or firmware that does not match the inventory.
MitigationUse secure boot, keep keys in protected storage and treat each device as untrusted until it proves its identity.
An update leaves devices unusable
Early signalUnits go silent after a release, often clustered by hardware revision.
MitigationInstall to an inactive slot with automatic revert, and stage releases as described in the fleet update process.
Frequently asked questions
What is the difference between edge AI and IoT analytics in the cloud?
IoT analytics typically sends sensor data to a central platform where models run on aggregated history. Edge AI runs the model on or near the device, so decisions happen locally and only results travel. Most production systems combine both: devices decide in the moment, and central analytics looks across the fleet to improve models and plan maintenance.
Can edge devices keep learning after deployment?
Some can, but it is rarely the first design to reach for. On-device learning raises questions about validation, memory and whether a device's behavior can still be explained. A more common approach is to collect selected examples from the field, retrain centrally, validate and redeploy. Federated learning, where devices train locally and share only model updates, suits cases where raw data must not leave the device.
How do we collect training data from devices without shipping everything?
Let devices keep a rolling buffer and upload only selected samples: low-confidence cases, disagreements with operators, random spot checks and examples around confirmed events. That gives a training set weighted toward the hard cases while keeping bandwidth and storage modest. Agree in advance who may label the samples and how long raw data is kept.
Do we need a pilot on real hardware before committing to a device?
Yes. A hardware-aware pilot is one of the deliverables of this practice because laboratory results often change on real boards, in real enclosures and with real sensors. Shortlist two or three candidate devices, run the same model and field data on each, and compare accuracy, latency, energy and the effort to update them.
What should we have ready before starting an edge AI project?
Representative sensor data from the actual environment, a clear picture of device and connectivity constraints, and the physical operating and safety requirements of the process. With those three, an architecture can be tested against reality rather than assumptions; without them, the first phase should be data collection.
Connect the architecture
The next connection.
Explore the complementary capabilities that can turn an individual technology into a complete workflow.
Your next move
Bring us the
real problem.
A workflow that takes too long. A system that cannot connect. An idea that needs a technical path. Start there, and we can define what to investigate, build and measure.
- Representative sensor data
- Known device and connectivity constraints
- Physical operating and safety requirements