# Edge AI or cloud AI: where should your model run? > Run a model on the device when it must work offline, answer fast or keep data local. Use the cloud when it is too big to fit. How to decide. Page: https://www.whitehatdesignstudio.com/guides/edge-ai-vs-cloud-ai/ Published by: WhiteHat Design Studio Last updated: 7 November 2026 The short answer Run the model on the device when the machine has to keep working without a network, answer quickly or keep its data to itself. Run it in the cloud when the model is too large for the device and the answer can wait for the network. Many systems do both: fast decisions on the device, heavy work on a server. By the engineers at WhiteHat Design Studio. Published 7 November 2026. ![HARP, a white and dark blue humanoid robot with a screen for a face and a second screen on its body, standing on a tiled floor.](https://www.whitehatdesignstudio.com/_astro/harp-front.DcjFrvNH_ZNkH8F.jpg) HARP holds its conversations on board, with no round trip to a server. ## The trade, side by side | What changes | On the device | In the cloud | | --- | --- | --- | | When the network drops | Keeps working | Stops | | Time to answer | No network round trip | Depends on the network | | Size of model | Limited by the device | Limited by your budget | | Your data | Stays on the device | Leaves the device | | Cost | Paid up front, in hardware | Paid with every answer | | Updates | Have to reach every device | Change it once | ## Decide by how it fails Start from the failure, not the model. Ask what happens when the network drops, and what happens when the answer arrives late. A hand that has to close on a cup cannot wait for a server. ReFlex chooses its grip on the device: a custom-trained MobileNetV2 runs through TensorFlow Lite on a Raspberry Pi 5, between the camera in the palm and the motors in the fingers. HARP runs its language model on the robot itself, so its conversation does not depend on the building’s Wi-Fi. A weekly report can wait for a server, and a server can run a model too large for any device. ## Can a Raspberry Pi 5 run a language model? Yes, if the model is small enough. A Raspberry Pi 5 has four Arm Cortex-A76 cores at 2.4 GHz and up to 16 GB of memory. llama.cpp, an open-source engine for running language models, supports integer quantisation, which shrinks a model so it uses less memory and runs faster. For more speed, the Raspberry Pi AI HAT+ 2 adds a Hailo-10H accelerator with 8 GB of its own memory, made for running language models and vision-language models locally. The AI HAT+ boards, with Hailo-8 or Hailo-8L accelerators, are for vision work such as object detection, segmentation and pose estimation. Those are capabilities, not speeds. How many words a second you get depends on the model, how far it is quantised and how well the board is cooled. Measure it on your hardware, with your prompts, before you promise anyone a response time. ## Fitting a model to a device Four things have to fit at once, and the model is only one of them: - **Memory.** The model, its working data and everything else the device runs. - **Speed.** Fast enough for the job, measured on the real device, not on a desktop. - **Power.** A robot on a battery pays for every calculation in running time. - **Heat.** A small board that overheats slows itself down, and the speed you measured on the bench is gone. Tools exist for each step. TensorFlow Lite now continues as LiteRT, Google’s runtime for running models on devices. NVIDIA’s TensorRT compiles models for low latency on NVIDIA hardware, and SKYLOCK runs its detection model through it at its ground control station. ## Where your data goes We agree that before we build. A language model can run with a provider or on your own hardware, and the choice changes what you have to tell your customers and your regulators. If the data must not leave the building, that settles the question before anything else does. If you are weighing this for a machine of your own, [our AI practice](https://www.whitehatdesignstudio.com/services/ai/) does this work, and the [HARP case study](https://www.whitehatdesignstudio.com/work/harp/) shows a model running on a robot. ## Where the answer comes from - [SKYLOCK](https://www.whitehatdesignstudio.com/work/skylock/) Finds drones in live camera feeds, predicts where each one is going and keeps a gimbal camera on it, from the operator's console or on its own. - [HARP](https://www.whitehatdesignstudio.com/work/harp/) Navigates a mapped building, tracks faces and holds a conversation with an onboard language model, built around a Raspberry Pi 5. - [ReFlex](https://www.whitehatdesignstudio.com/work/reflex/) Reads forearm muscle signals, chooses a pinch or tripod grip from what the palm camera sees, and stops each finger on encoder feedback. Practices - [Hardware](https://www.whitehatdesignstudio.com/services/hardware/) Machines that survive the real world. - [AI](https://www.whitehatdesignstudio.com/services/ai/) Intelligence that runs where the work is. A hardware, C AI ## Questions people ask next ### What is edge AI? Edge AI means running a model on the device that needs it, or close to it, rather than in a distant data centre. The answer does not have to cross a network. ### Can a Raspberry Pi 5 run a language model? Yes, if the model is small enough. Quantisation shrinks a model so it fits the memory, and an accelerator board such as the AI HAT+ 2 adds speed. Measure the speed on your own hardware with your own prompts before you rely on it. ### Is edge AI cheaper than cloud AI? It depends on how much you use it. Edge AI is paid for up front, in hardware, and costs little per answer. The cloud costs little to start and is paid for with every answer. ## Sources - [Raspberry Pi 5](https://www.raspberrypi.com/products/raspberry-pi-5/) - [Raspberry Pi AI HAT+ 2](https://www.raspberrypi.com/products/ai-hat-plus-2/) - [Raspberry Pi AI HAT+](https://www.raspberrypi.com/products/ai-hat/) - [llama.cpp, language model inference in C and C++](https://github.com/ggml-org/llama.cpp) - [Google, LiteRT](https://developers.google.com/edge/litert) - [NVIDIA, TensorRT](https://developer.nvidia.com/tensorrt) ## More guides - [How does drone detection work?](https://www.whitehatdesignstudio.com/guides/how-drone-detection-works/) - [STM32 vs ESP32: which should your product use?](https://www.whitehatdesignstudio.com/guides/stm32-vs-esp32/) - [Visual SLAM vs LiDAR SLAM for indoor robots](https://www.whitehatdesignstudio.com/guides/visual-slam-vs-lidar-slam/) [All the guides](https://www.whitehatdesignstudio.com/guides/) ## Tell us what has to work. One sentence is enough. An engineer will reply. [Talk to an engineer](https://www.whitehatdesignstudio.com/contact/) [WhatsApp an engineer](https://wa.me/923112721007?text=Hello%20WhiteHat%20Design%20Studio%2C%20I%20would%20like%20to%20talk%20to%20an%20engineer%20about%20my%20project.)