Arm is supporting this new initiative with engineering resources and by opening up Arm's Neural Network (NN) inference engine to external contributions. Arm, Linaro, and the other members of the Machine Intelligence Initiative will collaborate to reduce redundant engineering and fragmentation in the deep learning and neural network acceleration ecosystem and accelerate development of new technology solutions. "The development of a common software interface supporting industry-leading frameworks and tools is one of the biggest requirements in accelerating adoption of machine learning by developers," said Robert Elliott, director of applied machine learning, Arm. "Arm is addressing this with our donation of the Arm NN inference engine to the Machine Intelligence Initiative, which will quickly enable the Linaro community and Arm ecosystem to deploy machine learning across the widest number of applications."
Neural network acceleration in Arm®-based platforms provides an unprecedented opportunity for new intelligent devices. Today however, every IP vendor forks existing models and frameworks to integrate their hardware blocks and then tunes for performance. This leads to duplication of effort, an increasing perpetual cost of re-integration for every new rebasing, and an overall increased total cost of ownership. In addition, the growing amount of data captured by sensors and connected devices, coupled with real-time constraints and the cost to move large data sets from the edge to the cloud, intensifies the need to manage and execute big data analytics and Machine Learning (ML) inference engines at the edge, wherever possible.
"In order to accelerate innovation in machine intelligence on Arm, players in the Arm ecosystem need to collaborate," said Andrea Gallo, VP of Segments and Strategic Initiatives at Linaro. "Through the Machine Intelligence Initiative, Linaro and members of the initiative aim to adopt a unified model description format and framework runtime API, an optimized inference engine for Arm application processors and a flexible plug-in architecture to integrate each NN solution and use members' internal resources to focus on product competitive advantage."
Linaro's Machine Intelligence Initiative will initially focus on inference for Arm Cortex®-A SoCs and Cortex-M MCUs running Linux, Android, and Zephyr, both for edge compute and smart devices. As part of the remit, the team will collaborate on defining an API and modular framework for an Arm runtime inference engine architecture based on plug-ins supporting dynamic modules and optimized shared Arm compute libraries. The work will rapidly develop to support a full range of processors, including CPUs, NPUs, GPUs, and DSPs and it is expected that the Arm NN will be a crucial part of this.
Linaro expects to quickly expand the scope of this new initiative to include Cortex-M microcontrollers.
"The TensorFlow team is excited to work with Arm and Linaro to expand support for edge devices, and we're looking forward to integrating with the Arm NN library," said Pete Warden, Technical lead of the TensorFlow mobile and embedded team at Google. "We think this kind of standard, open source interface for neural computing will improve the experience for product developers across the Arm ecosystem."
