HailoチップでEdge AIを切り開く

HailoチップでEdge AIを 切り開く Nori Kojima 14 December 2023

Hailo in Numbers AI Patented & field proven ML architecture Chipmaker 1 st Generation Founded in 2017 in mass production 5x >300 10s YoY Revenue Growth 2021-2022 Customer Programs of products in production $224M Including 2 © Hailo 2023 funding & >240 70% of R&D are software engineers

Empowering Edge Devices with AI Powerful Easy to Use ▪ High performance AI processing ▪ Comprehensive & mature AI software suite ▪ Real-time, low latency AI inferencing ▪ ▪ High accuracy for low false alarms Common machine learning development frameworks ▪ Wide range of interfaces & runtime frameworks ▪ Rich neural network model zoo ▪ Quarterly software updates Versatile ▪ Complex models and AI pipelines → Multi-source, multi-task & dynamic model reconfiguration → Native transformers support ▪ Extended temperature range for industrial & automotive* applications 3 © Hailo 2023 * Hailo-8™ is AEC-Q100, Grade B Efficient ▪ Highest cost-efficiency (FPS/$) ▪ Best-in-class power efficiency (FPS/W)

Hailo’s Unique Dataflow Architecture World’s Most Efficiency AI Architecture ▪ Dataflow-led approach ▪ Physically distributed computation ▪ Software abstraction allows quickly running a variety of NN models ▪ Smaller elements lead to lower power consumption ▪ >30 patents granted 4 © Hailo 2023 Neural Network Graph Resource processing breakdown Resource Graph Physical resource mapping Hailo NN Core Dynamic configuration & execution

Empowering Edge Applications with AI Hailo’s powerful & scalable AI technology enables new capabilities in various markets Security Automotive ITS, Perimeter Protection, Access Control Industrial Automation AMR/AGV, AOI, Building Management, Robotics, Machinery 5 © Hailo 2023 ADAS, Autonomous Driving, In-Cabin Monitoring Retail Point of Sale, Supply Chain Management * ITS – Intelligent Traffic System; AMR – Autonomous Mobile Robots; AGV – Autonomous Guided Vehicles; AOI – Automatic Optical Inspection

A Portfolio of AI Processors & A Single Software Suite Serving all Edge Deployments AI Inference Accelerators AI Vision Processors Hailo-8™ NN Core Hailo-15™ NN Core Processor DSP Vision Subsystem Host Processor Processor DSP 6 © Hailo 2023

AI Accelerators Hailo-8™ Portfolio

Hailo-8™ Highlights Starting automotive production in Q3 2024 Powerful & Efficient Versatile High Performance Comprehensive Software Suite ▪ Efficient AI architecture ▪ No external DRAM ▪ Low power consumption ▪ Low latency Mature dataflow compiler ▪ Efficient RT library ▪ Rich Model Zoo Industrial & Automotive Grades ▪ Industrial: -40°C to 85°C ▪ Automotive: -40°C to 105°C Automotive Grade ▪ AEC-Q100 Grade 2 qualified ▪ PPAP is available 8 © Hailo 2023 Scalable & Flexible ▪ Multi-streams ▪ Multi-model ▪ Multi-chip

Hailo-8™ Product Portfolio Hailo-8L Entry-Level AI Accelerator Hailo-8 AI Accelerator CoB – 13 TOPS M.2 Entry-Level AI Acceleration Modules 13 TOPS Hailo-8 Century High Performance PCIe Cards CoB – 26 TOPS Small Form 52-104 Factor 0802, 0803, TOPS 0804 M.2 AI Acceleration Module Hailo-8R™ mPCIe AI Acceleration Module 26 TOPS 13 TOPS Wide Host Compatibility 104-208 TOPS 0804S, 0805S, 0806S, 0807S, 0808S Key B+M 2 lanes Key A+E 2 lanes 9 © Hailo 2023 Key M 4 lanes Key B+M 2 lanes Key A+E 2 lanes

Hailo-8 Measured Benchmarks1,2,3 NN Model Input Resolution FPS Power (W) FPS/W ResNet-50 v1 224x224 1,312 3.4 386 MobileNet_v2_1.0 224x224 2,444 2.0 1,222 EfficientNet_M 240x240 890 3.5 254.2 ViT_base* 224x224 136 2.66 51.12 SSD_MobileNet_v1 300x300 1,016 2.2 461.8 YOLOv5m 640x640 218 4.6 47.3 1024x1920 54 2.8 19.2 Classification Object Detection Segmentation stdc1 * Hailo has been adding transformer based models to the model zoo for a year now, and was the first edge AI accelerator able to run ViT-B in real-time. ViT/DETR/CLIP/YoloV5-c3TR are currently supported. 10 © Hailo 2023 Notes: 1. Batch 8 2. Measurements were taken at room temperature through PCIe interface on Hailo-8 evaluation board 3. System host: Intel® Core™ i5-9400 CPU @ 2.90GHz; Models compiled with Hailo Dataflow Compiler version 3.25.0 (SW version 2023-10)

Hailo8 採用例 11 © Hailo 2023

AI Vision Processors

Hailo-15™ AI Vision Processors A series of System-on-a-Chip (SoC) processors that set new standards in computer vision and deep learning model processing Hailo-15H 20 TOPS 13 Hailo-15M 11 TOPS © Hailo 2023 Hailo-15L 7 TOPS Smarter Camera, Unmatched Results.

Hailo-15™ Highlights Complete Solution ▪ Single integrated SoC with full vision capabilities ▪ A series of vision processors supporting a wide range of camera requirements ▪ Industrial grade Unparalleled AI Video Analytics ▪ Full range of 7, 11, and 20 TOPS for every camera budget ▪ High FPS DL model processing for fast and accurate detection ▪ Best AI performance at a standard camera power consumption & cost Superior Image Quality Comprehensive Software Suite ▪ AI-powered vision enhancement ▪ 4k60 or 2x4k30 resolution ▪ 12MP ISP video pipeline ▪ Comprehensive vision library 14 © Hailo 2023 ▪ Yocto-based Linux distribution ▪ Industry-standard ML frameworks

Hailo-15™ Product Offering (1/2) NN Core Hailo-15M Hailo-15L 20 TOPS 11 TOPS 7 TOPS Quad-core ARM™ A53 @ 1.3 GHz 12 kDMIPs Quad-core ARM™ A53 @ 1.1 GHz 10.1 kDMIPs Vector DSP, 256 MACs@ 700 MHz supports up to 350 GOPs Vector DSP, 256 MACs@ 750 MHz supports up to 380 GOPs AP DSP Hailo-15H Max resolution 12MP, 600 Mpixel/s ISP 4k @ 60 FPS - DeWarp 2 Support Cameras Memory Interface Encoding PCIe Physical 15 © Hailo 2023 LPDDR4/4X 32 bit @ 4266 MT/s LPDDR4 32 bit @ 4266 MT/s 4k30 + 720p30, 10/8 bit 4k30 + 1080p30 + 720p30, 8 bit, MJPEG, OSD, Output downscale Gen3.0 x 4 lanes (Endpoint or RC) Gen3.0 x 2 lanes (Endpoint or RC) 15 x 15 mm 13 x 13 mm

Next Edge AI World

The Power of AI Vision & AI Analytics Combined ▪ Semantic awareness to selectively apply denoising techniques based on the semantic significance of elements and processing differently by Region of Interest (RoI) ▪ Noise reduction while preserving important image details and textures, for higher SNR & SSIM ▪ Parallel processing for faster input and higher FPS, mainly in high-resolution video Input Stream Denoising Exposure Correction ROI Detection Deblu r ABC-123 17 © Hailo 2023

AI-Powered Video Enhancement Noise reductio n Dynamic range & 3A Auto Focus, Auto White-Balance, Auto Exposure 18 © Hailo 2023 Distortion correctio n Motion compensatio n & image stabilization Digital zoom

AI-Powered Denoising Results ▪ Removing noise while preserving important image details and textures ▪ Resulting in higher SNR and SSIM ▪ Hailo-15 provides best image quality at a low light condition <5 Lux by up to 3 TOPS @4K/30FPS ▪ Leaving enough TOPS for AI video analytics at a very low power consumption of ~1W Sub 1 Lux AI-powered denoising by Hailo-15 Before (top) and after (bottom) 19 © Hailo 2023

AI Low Light Denoising DEMO 20 © Hailo 2023

ティアフォー様との取り組み C1 CameraとHailo8を使ってmulti task AI DEMOを展 示会へ出展 Object Detection, Semantic Segmentation, Lane Detection and Depth Estimation 21 © Hailo 2023

今後の展望 ▪ High-end perception pipeline on chip → ▪ ▪ User-controlled sparsity support → Supports both structured/unstructured pruning → User controlled sparsity/performance/power tradeoffs Enhanced compute data-path elements to efficiently accelerate common non-neural operators → ▪ Seamlessly supports multi-sensor, multi-stream and data-driven multi-network switching without loading the host Natively supporting Vision Transformers and LLMs/Generative AI Targeting best-in-class performance/price & performance/power ratios 22 © Hailo 2023

hailo.ai [email protected]

• http://www.hailo.ai/