Edge AI: AI at the Edge
This article explains Edge AI, where AI inference is performed on devices or nearby edge computing resources instead of relying entirely on remote cloud infrastructure.
It explores why AI is moved closer to the data source, how Edge AI works, and its applications in systems such as robotics, autonomous vehicles, cameras, and IoT devices.
1. Why Do We Need Edge AI?
In the previous articles, we discussed:
▶ On-Device vs Edge vs Cloud Computing
▶ Multi-Access Edge Computing (MEC)
▶ Content Delivery Networks (CDN)
These concepts describe where computation can be performed and where computing resources can be located.
AI is an important use case for these computing models.
AI applications often need to process large amounts of data such as images, video, audio, and sensor data. For example, a camera may continuously capture video and use an AI model to detect people, vehicles, or other objects.
The AI computation can be performed in different locations:
Camera
│
├──► On-Device AI
│
├──► Edge AI
│
└──► Cloud AI
The choice depends on the application. If a fast response is required, running the AI model on the device or at a nearby edge server can avoid sending the data to a remote cloud.
Edge AI is therefore a practical application of edge computing in which AI processing is performed close to where the data is generated.
The following sections look at how AI can be performed on the device or at the edge, using video analytics as an example.
2. On-Device AI vs. Edge AI
AI processing does not necessarily have to be performed in the cloud. Depending on the application, an AI model can run directly on the device or on a nearby edge server.
On-Device AI
In on-device AI, the AI model runs directly on the device that generates the data.
For example, a smart camera can capture an image and run an object-detection model locally.
Camera
│
▼
AI Model
(On Device)
│
▼
Detection Result
The device therefore does not need to send the complete image or video to another computer for the AI inference.
Edge AI
In Edge AI, the AI model runs on a nearby edge server instead of directly on the device.
Camera
│
│ Image / Video
▼
Edge Server
│
│ AI Model
▼
Detection Result
This is useful when the device does not have sufficient resources to run the AI model itself, but the application still requires a fast response.
The distinction can therefore be summarized as:
On-device AI runs the AI model on the device, while Edge AI runs the AI model on a nearby computing resource.
Both approaches bring AI processing closer to the source of the data than a traditional cloud-based AI system.
3. Example: AI-Based Video Analytics
Consider a security camera that continuously captures video and needs to detect people, vehicles, or other objects.
Option 1: Cloud Computing
A traditional approach is to send the video to a remote cloud server for AI processing.
Camera
│
│ Video
▼
Cloud Server
│
│ AI Inference
▼
Detection Result
This approach requires the camera to continuously transmit the video to the cloud.
Option 2: On-Device AI
The AI model can instead run directly on the camera.
Camera
│
│ Video
▼
AI Model
(On Device)
│
▼
Detection Result
The camera can process each frame locally and produce only the required result, such as:
"Person detected"
"Vehicle detected"
"No object detected"
Option 3: Edge AI
Another option is to use a nearby edge server.
Camera
│
│ Video
▼
Edge Server
│
│ AI Inference
▼
Detection Result
In this case, the camera does not need to have enough computing power to run the AI model itself. The video is sent to a nearby edge server, which performs the inference and returns the result.
This example illustrates the three possible locations for AI processing:
Cloud AI
Camera ─────────────► Remote Cloud
Edge AI
Camera ───────► Nearby Edge Server
On-Device AI
Camera ───────► AI Model on Camera
The choice depends on the capabilities of the device and the requirements of the application (as discussed in the previous writings on the edge computing series; Links are given at the beginning of this writing).
4. Takeaway
Edge AI is a practical application of edge computing where AI processing is performed close to where the data is generated.
The AI model can run directly on the device (on-device AI) or on a nearby edge server (edge AI).
This approach is particularly useful for applications that generate continuous streams of data and require fast responses, such as video analytics, robotics, autonomous systems, and industrial applications.
The main idea is simple:
Instead of sending all data to a remote cloud for AI processing, perform the AI computation on the device or on a nearby edge server.