Neural Networks on Arm Cortex-M with CMSIS-NN

Introduction

CMSIS-NN (Cortex Microcontroller Software Interface Standard — Neural Network Library) is an open-source library designed for efficient neural network computation on microcontrollers. It provides a set of optimized and compact functions for performing common neural network layers, such as convolution, pooling, and fully-connected operations. This library is specifically designed for low-power, resource-constrained embedded systems, making it ideal for applications such as sensor data processing, edge computing, and Internet of Things (IoT) devices.

The goal of CMSIS-NN is to provide a standardized interface for deep learning on microcontrollers, promoting compatibility and collaboration across different hardware and software platforms. It adheres to the Cortex Microcontroller Software Interface Standard (CMSIS), which is a software framework for Cortex-M microcontroller devices. This makes the library easily portable and compatible with a wide range of microcontrollers from different manufacturers.

Understanding CMSIS-NN

The Cortex Microcontroller Software Interface Standard (CMSIS) is a software library developed by Arm that provides a standardized programming interface for Cortex-M processors. CMSIS-NN (Neural Network) is an extension of this library that includes optimized functions for implementing neural network layers on Cortex-M processors.

The role of CMSIS-NN is to accelerate the execution of neural network models on Cortex-M processors by providing efficient implementations of commonly used operations such as convolution, matrix multiplication, and activation functions. This allows developers to easily port their neural network models to Cortex-M processors and achieve improved performance without extensive optimization efforts.

Some key features of CMSIS-NN include:

• Optimized functions for neural network layers: CMSIS-NN provides highly optimized functions for common operations used in neural networks, such as convolution, matrix multiplication, and activation functions. These functions are specifically designed for Cortex-M processors, taking into account their unique architecture and instruction set.

• Small code size: The functions in CMSIS-NN are designed to be efficient in terms of code size, making it suitable for memory-constrained embedded systems.

• Hardware acceleration support: CMSIS-NN takes advantage of hardware features such as DSP and SIMD (Single Instruction Multiple Data) instructions on Cortex-M processors to improve performance and reduce power consumption.

• Easy integration with other CMSIS components: As a part of the larger CMSIS library, CMSIS-NN can be easily integrated with other CMSIS components, such as CMSIS-DSP for signal processing tasks.

Building Custom Neural Networks with CMSIS-NN

Process for building custom neural networks:

• The first step in building a custom neural network using CMSIS-NN is to define the network architecture. This includes choosing the number and type of layers, the size of each layer, and the connections between layers.

• The next step is to prepare the data that will be used to train the network. This includes collecting and formatting the data in a way that is suitable for the network.

• Once the data is prepared, it is used to train the neural network. This involves adjusting the weights and biases of the network based on the input data and desired outputs.

• After the network has been trained, the weights and biases are converted into a format that can be used by the CMSIS-NN library. This step is necessary because CMSIS-NN uses a specific format for weights and biases that is optimized for efficient inference on MCUs.

• Once the weights and biases have been converted, an inference code can be created using the CMSIS-NN library. This code will take in an input and use the optimized implementations of neural network layers to perform inference and produce an output.

• The final step is to test the network and optimize it for performance on the specific MCU. This might involve tweaking the network architecture, adjusting the input data, or fine-tuning the CMSIS-NN library parameters.

CMSIS-NN provides optimized implementations for various neural network layers such as convolution, fully connected, pooling, activation, and normalization layers. These implementations are optimized for efficient execution on an MCU and take into consideration the limited resources and processing capabilities of these devices.

The CMSIS-NN library also supports various data types, allowing for efficient usage of limited memory resources on MCUs. For example, CMSIS-NN supports fixed-point arithmetic, which is more suitable for MCUs compared to floating-point arithmetic. This allows for more compact networks and improves performance on MCUs.

Integration and Deployment

• Download and install the CMSIS-NN library from Arm’s website. This library is available as a standalone package or as part of the CMSIS-5 repository.

• Make sure that your project is using a CMSIS-compliant toolchain, such as Arm Keil MDK or GNU ARM Embedded Toolchain.

• Add the CMSIS-NN library to your project by including the necessary header files and source files. The specific files you will need to include will depend on the network model you want to use.

• Configure the CMSIS-NN library by setting the necessary parameters in the configuration file (usually named “arm_nn_config.h”). These parameters include things like the fixed-point format used in your project, the maximum buffer size, and the compiler flags to use.

• Initialize the CMSIS-NN library by calling the “arm_nn_init” function. This will set up internal variables and memory buffers needed for CMSIS-NN to work properly.

• Use the available CMSIS-NN functions to perform the desired operations on your network model. These functions include initialization, inference, and optimization routines.

• When your project is finished using CMSIS-NN, be sure to clean up any resources it has allocated by calling the “arm_nn_reset” function.

• Test your project on a microcontroller that uses an Arm Cortex-M CPU. You can use a debugger or a serial terminal to view the output of your network model and make sure it is performing as expected.

• Optimize your project by tweaking the configuration file and using different CMSIS-NN functions to find the best performance for your application.

• Continue updating and maintaining your project with CMSIS-NN as needed. You can also refer to the CMSIS-NN user guide and reference manual for more information on the available functions and how to use them.