Tengine AIoT应用开发平台

发布日期:2019-08-26 浏览次数:1101

Tengine AIoT应用开发平台


Tengine基本信息

名称:Tengine

编程语言:C++

开发者:OPEN AI LAB

软件大小:<3MB

软件授权:Apache 2.0

软件版本:Tengine初始版本 2017年12月31日  

更新时间:稳定版 1.7.1  2019年8月2日

特点:轻量AI推理部署、适配主流AIoT芯片,支持异构计算、 模型压缩加密功能


Tengine概述

Tengine是一个专门针对嵌入式设备开发的神经网络推理引擎,目标是成为AIoT应用级操作系统。支持Arm CPU, GPU, AIPU以及第三方神经网络加速器异构计算,专注提升推理性能,解耦神经网络应用于底层硬件,提升应用部署效率。


Tengine由OPEN AI LAB开发和维护,于2017年12月31日在OAID(OPEN AID)社区开源。2018年11月9日Tengine作为Arm中国AI解决方案周易平台软件前端在“2018年乌镇世界互联网大会”发布。



Tengine主题信息

产生背景:

2018年边缘AI推理应用爆发,现有嵌入式软硬件不能满足AI应用开发需求,云端AI软件工具迁移至端侧遇到技术困难。产业急需开放高效的端侧AI开发平台。


为了解决边缘AI应用开发难题,赋能AIoT产业,Arm联合产业合作伙伴发起成立OPEN AI LAB。由OPEN AI LAB开发的Tengine(Tensor Engine),解决了边缘侧AI应用开发的难题。


发展历程:

OPEN AI LAB在2017年先后推出Caffe-HRT,TensorFlow-HRT,MXNet-HRT,将当时云端主流的AI框架移植到Arm平台上,满足了AI应用落地边缘侧的部分需求。然而在移植过程中发现这些框架并不能很好适用于边缘侧AI应用。


通过收集整理嵌入式开发者需求,重新定义嵌入式端AI框架的功能,包括只做推理,异构计算,兼容各训练框架模型格式等,最终推出了转为AIoT应用打造的AI推理框架Tengine。


主要功能:

支持Arm CPU, GPU, AIPU以及第三方神经网络加速器异构计算(nnie,rk3399pro)

支持直接读取Caffe/TF/TF-Lite/MXNet/ONNX模型

支持Linux/Android/RTOS/Baremetal执行环境

支持调用HCL高性能计算库,与ACL,OpenBLAS等开源计算库

支持FP32/FP16/INT8等多精度计算

开放支持第三方DSP,NPU,FPGA

模型二进制加密



版本更新:


version 1.7.1 - 2019/08/02

Optimization

Add Winograd implementation

LSTM operator neon implementation

GRU operator neon implementation

New features

GPU backend support with ACL 19.02

Static tool for convert other models to tengine

Tengine graph use origin layout to execute

Add CV implementation


version 1.3.2 - 2019/04/19

tengine model 2.0

New apis

get_graph_node_number() get_graph_node_by_idx()

New features

Separate CPU operator as a independent so: hclcpu.so

Add Reference Operator

Update Testcase & Update permute for mxnet

Update lstm grun mxnet serializer

Support MXNET serializer in CMakelist.txt

Support TFLITE serializer in CMakelist.txt

Support eltwise in TFLITE serializer

More operator support

RNN operator definition and blas implementation

LSTM operator definition and blas implementation

GRU operator definition and blas implementation



version 1.0.0 - 2018/12/31

tengine API 2.0 -- New API set for NN inference

Support perf stat and tensor dump

Support to build Android NN Driver

more tensorflow models inceptionv3/v4, resnet_v2_101, mobilenet v1/v2 models


version 0.8.0 - 2018/11/15

Support GPU/CPU Heterogeneous Computing

Support ACL in Android


version 0.7.2 - 2018/10/15

update ONNX module with new onnx proto version


version 0.7.0 - 2018/9/15

support saving model as c files

support ACL GPU FP16

support mobilenet v2 in examples

support cross-building arm32 library

support building on raspberry pi 3b


version 0.6.0 - 2018/7/02

Support Tengine model file. protobuf is optional now.


version 0.5.0 - 2018/6/15

Support GPU: using ACL (Arm computing library) as a backend graph device

Support blas operator implementation: Tengine can run on x86 without caffe now

Support new NN: Inception-v3/vgg16/faster-rcnn/ssd/yolo-v2

Support Android build: includes 32bit and 64bit

Support cross-compile on x86 (experimental): debian example contributed by mcharleb and Mani-Sadhasivam @ Linaro

Support Tensorflow serializer: load inception-v3 and mobilenet TF model directly

Support Tensorflow wrapper: label_image.cpp from tensorflow repo

Tengine will automatically probe the CPU arch/part settings, and there is just one CPU driver now.


version 0.3.0 - 2018/2/6

Introduce the driver/device model to support MT(Multi-Thread)

Support new NN: Inception-v4

Caffe Wrapper examples: squeezenet/mobilenet/mtcnn

MXNet model load examples: squeezenet/mobilenet


version 0.2.0 - 2018/1/24

Support new operator: Eltwise, PReLU, Slice

Support new NN: mtcnn, resnet and lighten_cnn

Experimental caffe API wrapper: caffe based application just needs to recompile to use Tengine


version 0.1.2 - 2017/12/30

Update documents, as well a few fixes.


version 0.1.0 - 2017/12/29

Initial release of single A72 support