ROS Japan UG #57 mesh2gazebo: Building a simulator environment using mesh data of 3D Scan Application

ROS Japan UG #57 mesh2gazebo: Building a simulator environment using mesh data of 3D Scan Application CyberAgent, Inc. AI Lab Activity Understanding Team Yasuhiro Yoshimura

About me Yasuhiro Yoshimura CyberAgent, Inc. AI Lab Activity Understanding Team Research Engineer I research robotics and the sensing to understand environment. OpenCV Contributor 2

Motivation ● I hope to quickly build a simulator environment that is the same scale as the real environment for developing software for an autonomous robot. ● So, I want to build a simulator environment by simply taking photos of the environment with my smartphone. 3

Problem of previous method(pointcloud2gazebo) ● Previous method pointcloud2gazebo has the following problem. ○ ○ ○ The quality of mesh data is NOT good. There is no color data in mesh data. This tool supports only Gazebo Classic. Real world 3D Point Cloud Gazebo World 4

• https://www.docswell.com/s/dandelion1124/ZENDP8-2024-02-13-094148

mesh2gazebo ● mesh2gazebo solves these problems! ○ ○ ○ The quality of mesh data is good. Mesh data has color data. This tool supports Gazebo Ignition. Real World 3D Mesh data Gazebo World 5

mesh2gazebo I easily built a simulator environment using mesh data of 3D Scan Application. 3D Scan Application mesh2gazebo Mesh data (obj data, texture) Gazebo Gazebo World 6

mesh2gazebo demo : Launch Gazebo world https://www.youtube.com/watch?v=z63dIfPx9iE 7

• https://www.youtube.com/watch?v=z63dIfPx9iE

mesh2gazebo demo : Navigation https://www.youtube.com/watch?v=x92R9y_1isM 8

• https://www.youtube.com/watch?v=x92R9y_1isM

Reference : Camera image of Real world https://www.youtube.com/watch?v=4dIjLJ6BXzM 9

• https://www.youtube.com/watch?v=4dIjLJ6BXzM

Reference : Visualization mesh data in RViz2 I visualized mesh data in RViz2. 10

Test environment This time, I used the following environment. ● ROS 2 Jazzy ● Gazebo Harmonic v8.7.0 ● Scaniverse ※iOS application 11

• https://scaniverse.com/

Technical briefing ● 3D Scanning ○ I used Scaniverse for the 3D Scanning. ■ ○ ○ Reference : https://note.com/iwamah1/n/nc8a5427157ef Take a picture of the floor as well for post processing. I export OBJ format and convert to DAE format by MeshLab. Exported mesh data Texture data 12

• https://scaniverse.com/
• https://note.com/iwamah1/n/nc8a5427157ef
• https://www.meshlab.net/

Technical briefing ● Post Processing(1/2) ○ ○ If there is the difference between Gazebo and mesh, I need to rotate. I used MeshLab. ■ I used Transform: Rotate to rotate mesh data. Y Z X 13

• https://www.meshlab.net/

Technical briefing ● Post Processing(2/2) ○ ○ ○ It is difficult to run robot if the floor surface is NOT flat. I adjust the height of the mesh data so that the ground is flat. I used MeshLab. ■ I used Transform：Translate to adjust the height of the mesh. before after Z Z 14

• https://www.meshlab.net/

Technical briefing ● Making of Gazebo World（1/2） ○ I specified the file path of mesh data in <collision> tag in SDF file. <model name="mesh2gazebo"> <static>true</static> <link name="link"> <collision name="collision"> <geometry> <mesh> <scale>1 1 1</scale> <uri>file:///home/jazzy/Scaniverse_2024_11_18_162729.dae</uri> </mesh> </geometry> </collision> 《中略》 </link> </model> 15

Technical briefing ● Making of Gazebo World（2/2） ○ I specified the file path of mesh data in <visual> tag in SDF file. <model name="mesh2gazebo"> <static>true</static> <link name="link"> 《中略》 <visual name="visual"> <geometry> <mesh> <scale>1 1 1</scale> <uri>file:///home/jazzy/Scaniverse_2024_11_18_162729.dae</uri> </mesh> </geometry> </visual> </link> </model> 16

Future plans ● I’ll publish the repository of mesh2gazebo. ● This repository has the template of Gazebo World. 17

Acknowledgment The in-store image was collected at EZOHUB TOKYO with the cooperation of SATUDORA HOLDINGS CO.,LTD. 18

• https://ezohub.jp/office/tokyo/

Advertisement ● I published an article to introduce ffmpeg_image_transport package. ● If you are interested in the capture of camera, please read this article. 19

• https://qiita.com/dandelion1124/items/deed014872624fd9a50c
• https://github.com/ros-misc-utilities/ffmpeg_image_transport

Appendix：Tips ● Decrease of CPU usage ○ ○ I disabled cast shadows to decrease CPU usage. So, I set cast_shadows to 0. <light type="directional" name="sun"> <cast_shadows>0</cast_shadows> <pose>0 0 10 0 0 0</pose> <diffuse>0.8 0.8 0.8 1</diffuse> <specular>0.2 0.2 0.2 1</specular> <attenuation> <range>1000</range> <constant>0.9</constant> <linear>0.01</linear> <quadratic>0.001</quadratic> </attenuation> <direction>-0.5 0.1 -0.9</direction> </light> 20

• http://sdformat.org/spec?ver=1.11&amp;elem=light#light_cast_shadows