Sensing & Perception 技術概要と性能課題

14.7K Views

July 24, 23

#tieriv #自動運転 #computing #tierivmeetup #Autoware #TIER IV #Sensing Technology #Perception Technology #Autonomous Vehicles

スライド概要

2023/7/13「自動運転におけるAIコンピューティング」
発表者：大里 章人

profile-image
スライド一覧

TIER IV（ティアフォー）は、「自動運転の民主化」をビジョンとし、Autowareを活用したソフトウェアプラットフォームと統合開発環境を提供しています。 #Autoware #opensource #AutonomousDriving #deeptech

シェア

またはPlayer版

埋め込む »CMSなどでJSが使えない場合

（ダウンロード不可）

関連スライド

slide-thumbnail

Edge AIのためのDNN推論処理の最適化
tieriv 自動運転 computing tierivmeetup
user-img TIER IV 101.3K
slide-thumbnail

Hailo-8上の低消費電力Edge AI
tieriv 自動運転 computing hailo edgeai autoware dnn
user-img TIER IV 43K
slide-thumbnail

Openな資産とFPGAを活用して、領域特化型のProcessorを作ろう！
自動運転 tieriv fpga rtl autoware riscv llvm
user-img TIER IV 42.2K
slide-thumbnail

小さく始める Blue/Green Deployment
tieriv sre cicd
user-img TIER IV 33.7K
slide-thumbnail

Autoware紹介と自動運転ハードウェアアーキテクチャ
自動運転 autoware tieriv fpga rtl
user-img TIER IV 31.7K
slide-thumbnail

Webエンジニアが自動運転企業でやっていること
web 自動運転
user-img TIER IV 26.2K
各ページのテキスト
1.

Sensing & Perception 技術概要と性能課題 株式会社ティアフォー 大里 章人

2.

TIER IV Speaker 経歴 ~ 2016 .3 東京理科大学大学院 理工学研究科機械工学専攻 修士課程修了 ~ 2017.7 ソニー株式会社 深層強化学習ベースの自律移動ロボットの研究開発に従事 2017.7~ Akihito Ohsato Architect 株式会社ティアフォー 4人目の正社員として入社 Autoware 開発および複数の自動車両開発プロジェクトに携わった後 Sensing/Perception チームリーダーを経て 現在は自動運転システム開発におけるアーキテクトを担当

3.

TIER IV TIER IV 1. System Overview 2. Sensing Technology 3. Perception Technology 4. Issues

4.

System Overview

5.

TIER IV What is Autoware? Autowareは、LinuxとROSをベースとした、世界初のオープン ソース⾏動運転ソフトウェアで、⾏動運転を設計する上で必要な 全ての機能を有している。 ティアフォーはAutoware開発をその設立当初からリードし、安心 ・安全な自動運転の実装のために数々の実地⾏⾏を世界中のさ まざまな環境下で⾏っている。 AutowareはThe Autoware Foundation (AWF)の登録商標 30+ 20+ 500+ VEHICLES COUNTRIES COMPANIES

6.

Perception Planning Scenario Dynamic Object Detection Tracking Scenario Selector Prediction Control Mission Lane Driving Traffic Light Detection Parking Etc. Classifier Localization Sensing Sensors Map Data Vehicle Interface Vehicle

7.
8.

Sensing Technology

9.

Perception Planning Scenario Dynamic Object Detection Tracking Scenario Selector Prediction Control Mission Lane Driving Traffic Light Detection Parking Etc. Classifier Localization Sensing Sensors Map Data Vehicle Interface Vehicle

10.

Perception Planning Scenario Dynamic Object Detection Tracking Scenario Selector Prediction Control Mission Lane Driving Traffic Light Detection Parking Etc. Classifier Localization Sensing Sensors Map Data Vehicle Interface Vehicle

11.

TIER IV Robo-taxi Sensors JPNTaxi model Velo d yne VLS -128 u-blox F9P TamagawaSeiki TAG300N (GNSS / Localization) (IMU/ Localization) AP Velodyne VLS-128 Alpha Prime (LiDAR / 360° Detection) Velo d 6 LP-1 ne V dy Velo yne V LP-1 6 TIER IV C1 85deg + TIER IV C2 62 deg TIER IV C1 85 deg x 6 LiDAR Configuration (FRONT) Velodyne VLP-16 Velo d (LiDAR / Side/Rear Detection) yne VLS -128 AP Camera Configuration (TOP) Velo d yne Continental ARS408-21 Continental ARS408 VLP -16 TIER IV C1 + TIER IV C2 w/ Jetson AGX (RADAR / Long-range Detection) LiDAR Configuration (SIDE) (Camera / Traffic Light Recognition) TIER IV C1 w/ JetsonAGX (Camera / Object Recognition)

12.

TIER IV Shuttle-bus Sensors J6 model Septentrio mosaic x5 (GNSS / Localization) HES AI P anda r 40P TamagawaSeiki TAG300N (IMU/ Localization) x4 HESAI Pandar QT x4 HESAI Pandar40P (LiDAR / Long-range Detection) TIER IV C1 w/ JetsonAGX (Camera / Object Recognition) HESAI PandarQT (LiDAR / Short-range Detection) LiDAR Configuration (SIDE view) TIER IV C1 + TIER IV C2 w/ Jetson AGX (Camera / Traffic Light Recognition) Continental ARS408-21 (RADAR / Long-range Detection) TIER IV C1 85deg + TIER IV C2 62 deg TIER IV C1 85 deg x6 Continental ARS408 x6 RADAR Configuration (TOP view) Camera Configuration (TOP view)

13.

TIER IV Sensor/ECU system (example) Camera Perception Pipeline Object Recognition Camera #1 Object Recognition Camera #2 Object Recognition Camera #3 Object Recognition Camera #4 Object Recognition Camera #5 Object Recognition Camera #6 Traffic Light Recognition Camera #1 Traffic Light Recognition Camera #2 LiDARs RADARs GMSL2 Perception ECU #1 (Jetson Xavier) Ethernet Perception ECU #2 (Jetson Xavier) Network Switch Ethernet Perception ECU #3 (Jetson Xavier) Perception ECU #4 (Jetson Xavier) LiDAR Perception & Fusion Pipeline Ethernet CAN Network Switch CAN Hub Main ECU (x86 + Discrete GPUs) CAN VCU

14.

TIER IV Sensor Calibration & Synchronization Sensor Calibration (tier4/CalibrationTools) Perception Components Sensor Synchronization (tier4/sensor_trigger)

15.

TIER IV

16.

Perception Technology

17.

Perception Planning Scenario Dynamic Object Detection Tracking Scenario Selector Prediction Control Mission Lane Driving Traffic Light Detection Parking Etc. Classifier Localization Sensing Sensors Map Data Vehicle Interface Vehicle

18.

TIER IV ML-based Object Detection 2D Object Detection Sensor Fusion Camera based → YOLO, SSD… 3D Object Detection Camera based → CenterNet, SMOKE… LiDAR based → PointPillars, CenterPoint… Camera-LiDAR based → Frustum PointNet… Detected Object ● Class label ● 3D Pose ● 3D Shape ● Velocity ● …

19.

TIER IV Object Recognition Pipeline (example) Perception / Detection Layer Sensing layer Object Recognition Cameras Pre-processing YOLO X LiDAR-Camera BoundingBox Fusion LiDAR Clustering LiDARs Pre-processing RADARs Pre-processing Perception / Tracking Layer Shape Estimation Detection by Tracker Centerpoint Merger RADAR Fusion Multi Object Tracker

20.

TIER IV Camera-LiDAR Fusion 1. BoundingBox level Fusion LiDAR-Camera BoundingBox Fusion Easy to Deploy YOLO X LiDAR Clustering Github - autowarefoundation/autoware.universe image_projection_based_fusion 2. Pointpainting YOLO X Pointpainting LiDAR Pointcloud Centerpoint Vora, Sourabh, et al. "Pointpainting: Sequential fusion for 3d object detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. Performance 3. Sensor level Fusion Transformer based Fusion Camera Image 2D Backbone LiDAR Pointcloud 3D Backbone Bai, Xuyang, et al. "Transfusion: Robust lidar-camera fusion for 3d object detection with transformers." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

21.

Perception Planning Scenario Dynamic Object Detection Tracking Scenario Selector Prediction Control Mission Lane Driving Traffic Light Detection Parking Etc. Classifier Localization Sensing Sensors Map Data Vehicle Interface Vehicle

22.

TIER IV Traffic Light Recognition Pipeline (example) Perception / Traffic Light Recognition Layer Sensing layer Traffic Light Recognition Camera (Wide) Pre-processing Traffic Light Recognition Camera (Narrow) Pre-processing Map-based Detector YOLO X (Fine Detector) MobileNet V2 (Classifier) Map / TF Merger Map-based Detector YOLO X (Fine Detector) MobileNet V2 (Classifier)

23.

Sensing & Perception Issues

24.

TIER IV Computation Cost ↕ Performance

25.

TIER IV 1. Huge Computation Cost :( YOLO X x8 process 10 Hz @ Jetson Xavier x 4 Centerpoint x1 process 10 Hz @ NVIDIA Quadro RTX 5000

26.

TIER IV 1. Huge Computation Cost CPU resource analysis @ MainECU (Xeon E-2278GE 8core/16threads) LiDAR pre-processing + Perception components > 50 % Power consumption analysis (w/o Sub/Logging System) x3 Main ECU ECU (- GPU) ~ 95 [W] GPU ~ 50 [W] x1 Perception ECU Object Recog x3 ~ 90 [W] Traffic Light Recog x1 ~ 20 [W] Total ≒ 255 [W]

27.

TIER IV 2. More Performance :( Long-range Detection Case Usecase: Obstacle Stop YOLOX-tiny FP16 (608x960 input) Usecase : Intersection e.g. 幼児ターゲット H = 70 cm Φ = 16 cm ↑FP, FN YOLOX-sPlusOpt Int8 (960x960 input) 自車速度 [km/h] 60 緩減速 [m] 160.56 準急減速 [m] 急減速 [m] 自車が緩加速 75.81 54.4 自車が準急加速 他車速度 [km/h] 60 他車が減速しないケース [m] 182.96 他車が途中から緩減速 [m] 165.05 他車が減速しないケース [m] 114.81 他車が途中から緩減速 [m] 111.92

28.

TIER IV 2. More Performance :( TLR camera upgrade C1 (2.5 MP) → C2 (5.4 MP) Low-latency Detection Case Usecase: Traffic Light Recognition 自車速度 [km/h] 60 緩減速 [m] 160.56 急減速1 [m] 75.81 急減速2 [m] 54.4 Sensing ~ Recognition Delay < 150 [ms]

29.

物体検出 DNN の高効率化で 課題解決をめざす 29

30.

TIER IV CONTACT US https://tier4.jp/ Thanks Again !