Axon driver
The following guide outlines the requirements that you need to fulfill before you start working with the Axon NPU using Axon API directly, without the abstraction layer of nRF Edge AI Library API. It also explains how to use the Axon NPU driver API to run TensorFlow Lite models directly on the Axon NPU. This solution is ideal if you require low‑level control over inference execution, memory usage, and system integration.
To follow this guide, you should be familiar with embedded systems development and C‑based APIs. Compared to higher‑level frameworks such as nRF Edge AI Library, using the driver API requires more manual setup but enables finer control over performance and resource utilization.
After completing this guide, you will have compiled a TensorFlow Lite model for the Axon NPU and deployed a custom application that performs inference using the Axon driver API.
Hardware requirements
Axon NPU library is included as part of the Axon NPU: Hardware Accelerated Machine Learning and provided as compiled binaries for Cortex-M33F architectures. Axon NPU is currently available on the nRF54LM20B device.
Software requirements
To start working with the Axon NPU, complete the setup based on your use case:
If you want to deploy models on the device, you just need to complete Setting up the SDK to install nRF Connect SDK and toolchain.
If you want to prepare models for deployment, you only need to set up a Python environment to run the Axon NPU TFLITE compiler. Follow instructions in Setting up Axon TFlite Compiler to set up the environment.
Setting up Axon TFlite Compiler
Before you can run the Axon NPU TFLITE compiler, you need to set up a Python environment with the required dependencies.
The executor of the compiler is compatible with Python 3.11.
You can set up the Python environment using one of the methods below.
Using a virtual environment is strongly recommended to isolate dependencies. You can use any virtual environment tool you prefer. This section shows one example using Miniforge (Conda):
Install Miniforge
Ensure the Conda
scriptsdirectory is added to your systemPATH, for example,C:/Users/<user>/AppData/Local/miniforge3/Scripts.Create a new environment with the supported Python version:
conda create -n <env_name> python=3.11Activate the environment. All installation and execution commands must be run from the activated environment:
conda activate <env_name>Install the required Python packages using the
requirements.txtfile:cd tools/axon/compiler/scripts pip install -r requirements.txt
Note
On macOS, you may encounter the ERROR: No matching distribution found for tensorflow==2.15.1 error.
To fix it, install TensorFlow using Conda instead:
conda install -c conda-forge tensorflow=2.15.1
Once you complete the setup, you can try running the compiler by following instructions in Axon NPU TFLITE compiler.
Docker provides a fully isolated way to run the compiler without installing dependencies locally.
Before using Docker with the compiler, you must install it on the system. You should also verify that Docker is working correctly by building and running a simple Docker container.
The following links provide introductory material and best-practice guidance for Docker:
Once you have installed and verified Docker, you can follow Using Docker (Optional) to build and run a Docker container for the Python compiler executor.
Podman is a daemonless alternative to Docker. Follow the steps below to set up Podman and run the compiler in a Podman container:
Install Podman by following the Podman installation guide.
Set up and run a simple container with Podman
Once you have installed and verified Podman, you can follow Alternative to Docker: Podman to build and run a Podman container for the Python compiler executor.
Model compilation
With the compiler environment ready, you can transform your TensorFlow Lite model into Axon-optimized code and verify that inference produces correct results.
Compile your model
The Axon compiler analyzes your model’s operations, maps them to hardware accelerators, and generates efficient code for the NPU. Whether you’re using a pre-trained model or one you have trained yourself, you will need to run it through this compilation process.
Follow the Setting up the executor instructions to transform your TFLite model into an Axon-optimized model.
Verify compilation
Test your compiled model to ensure it works correctly before integrating it into your application.
Run Test: NN Inference to confirm your compiled model produces correct results. This validation step checks for compilation issues early in the development process.
Application development
Once you have a compiled model, you must integrate it into your embedded application. Use the Axon driver API to load your model, manage memory, and execute inference directly on the NPU hardware.
Get compatible hardware
The Axon driver requires direct access to NPU hardware. Obtain a development board with Axon NPU. Keep in mind the NPU is only available on select Nordic devices, so verify compatibility before starting.
Set up the Axon driver
Install the Axon runtime library and driver components on your development system. Follow the Axon driver setup instructions to prepare your environment for building and deploying Axon applications.
Verify your setup
Before developing your own application, verify that everything is working correctly. Run the Hello Axon sample application to confirm the driver can communicate with the NPU hardware.
Note
Successfully running this sample means your development environment is ready. Any issues at this stage are typically related to hardware setup or driver installation.
Develop your application
With your environment set up and model compiled, you can start building your Axon application. Use the Axon driver API to load your compiled model, manage memory buffers, and execute inference.
Tip
Start by modifying the Hello Axon sample to understand the basic API flow before building your custom application from scratch.
Follow the Axon integration guide for detailed instructions on:
Initializing the Axon driver
Initializing your compiled model for synchronous or asynchronous inference
Executing inference and handling results
Integrating the model into your application
Deploy and optimize
Build your application and flash it to your Nordic device.
To build an application, follow the instructions in Building an application for your preferred building environment. See also Programming for programming steps and Testing and optimization for general information about testing and debugging in the nRF Connect SDK.
Monitor performance metrics like inference time and current consumption to ensure your application meets requirements. The direct driver access gives you the control needed to fine-tune performance for demanding embedded AI applications.
Next steps
See further documentation:
Add runtime monitoring of model outputs with nRF Edge AI Observability Library.