[2023 ver.] Getting Started with WebXR using Babylon.js

Getting Started with WebXR using Babylon.js for HoloLens 2 / Meta Quest

Download sample files https://github.com/TakashiYoshinaga/AR-Fukuoka/ raw/main/Babylonjs_20230211/Sample.zip

• https://github.com/TakashiYoshinaga/AR-Fukuoka/raw/main/Babylonjs_20230211/Sample.zip

Todayʼs Goal https://youtu.be/iok646xGtd0 Introduction to the basic usage of Babylon.js and the use of MRTK for interaction

• https://youtu.be/iok646xGtd0

Duplicate Template Source Code https://glitch.com/~babylon-template GET STARTED

• https://glitch.com/~babylon-template

Duplicate Template Source Code Remix Your Own

Duplicate Template Source Code Click on index.html and confirm that the code is displayed index.html

Duplicate Template Source Code Click on index.html and confirm that the code is displayed Editor Preview

Tip: If Preview is Not Displayed PREVIEW

Tip: If Preview is Not Displayed Open preview pane

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

Check the Template

Check the Template Loading Library Description of drawing and interaction (today's main focus) Layout of drawing area

Check the Template Layout of drawing area

Explanation of HTML Description
<!-- Area to be drawn -->
<canvas id="renderCanvas" style="width: 100%; height: 100%;"></canvas>

ID is used when accessing from javascript

Paste the canvas in the window at the full size.

Check the Template Loading Library

テンプレートの確認 Description of drawing and interaction (today's main focus)

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

Declaration of Variables Add variable here //Initialization of variables after page loading is completed window.onload = function() { } //Initialization of 3D scene function createScene() { } //Initialization of WebXR settings async function initializeXR(){ } //Initialization of 3D objects to be drawn function addObjects(){ }

Declaration of Variables let canvas; //canvas element (drawing area) ← "where" to draw let engine; //Drawing functionality with Babylon.js ← "What" to draw with let scene; //Virtual 3D space ← "What" to draw //Initialization of variables after page loading is completed window.onload = function() { } //Initialization of 3D scene function createScene() { } //Initialization of WebXR settings async function initializeXR(){ } Lesson02

Initialization of Drawing Functions of Babylon.js
let canvas; //canvas element (drawing area) ← "where" to draw
let engine; //Drawing functionality with Babylon.js ← "What" to draw with
let scene; //Virtual 3D space ← "What" to draw
//Initialization of variables after page loading is completed
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//カメラやライトの設定を⾏ったscene(3D空間)を取得
scene= createScene();
}
//Initialization of 3D scene
function createScene() {
}

Lesson03

Initialization of Scene (3D Space)
let canvas; //canvas element (drawing area) ← "where" to draw
let engine; //Drawing functionality with Babylon.js ← "What" to draw with
let scene; //Virtual 3D space ← "What" to draw
//Initialization of variables after page loading is completed
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//Create a scene (3D space) with camera and light settings
createScene();
}
// Initialization of 3D scene
function createScene() {
}

Description of the initialization process of a scene (next page)

Lesson04

Initialization of Scene (3D Space)
function createScene() {
//Create 3D space
scene = new BABYLON.Scene(engine);
//Set background color
scene.clearColor = new BABYLON.Color3.Black();
//Set Light (arguments: name, direction of reflection, destination)
let light = new BABYLON.HemisphericLight("light",
new BABYLON.Vector3(1, 1, -0.5) ,scene);
//Create a camera[Details]
let camera = new BABYLON.ArcRotateCamera(
"camera",
//name of camea
-Math.PI/2, Math.PI/2, 3, //alpha,beta,distance form origin
new BABYLON.Vector3(0, 0, 0)); //look at
// Enables camera operation with mouse or keyboard
camera.attachControl(canvas, true);
}

Lesson05

• https://doc.babylonjs.com/divingDeeper/cameras/camera_introduction

Start Drawing
//Initialization of variables after page loading is completed
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//Create a scene (3D space) with camera and light settings
createScene();
//Update drawing every frame
engine.runRenderLoop(function () {
if (scene && scene.activeCamera) {
scene.render(); //Drawing!
}
});
}

Lesson06

Operation Check Preview background becomes the specified color (=black)

Adding a Display Object
// Initialization of variables after page loading is completedう
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//Create a scene (3D space) with camera and light settings
createScene();
//Creating Drawing Objects
addObjects();
//Update drawing every frame
engine.runRenderLoop(function () {
if (scene && scene.activeCamera) {
scene.render();
}
});
Lesson07
}

Adding a Display Object Edit here next

Adding a Display Object
function addObjects(){

}

// Create ground (for visual checking the location of the origin.)
let ground = BABYLON.MeshBuilder.CreateGround(
"ground", {width: 1, height: 1});
//箱状のオブジェクト作成 (30cm)
let box = BABYLON.MeshBuilder.CreateBox(
"box", {width: 0.3, height: 0.3, depth: 0.3});
box.position.x = 0.0;
box.position.y = 1; //地⾯(ground)から1mの⾼さ
//マテリアルを作成
let boxMaterial = new BABYLON.StandardMaterial("material");
//箱の⾊を設定 (今回はランダムに変化)、そのあとboxに適⽤
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//8⾯体を作成
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
Lesson08

Operation Check The board that corresponds to the ground can be observed by changing the viewpoint with the mouse operation. Initial Viewpoint Left-click and drag to rotate Scroll back and forward Right-click and drag to move vertical/horizontal

Adding a Display Object
function addObjects(){

}

// Create ground (for visual checking the location of the origin.)
let ground = BABYLON.MeshBuilder.CreateGround(
"ground", {width: 1, height: 1});
// Creation of Box object (30cm)
let box = BABYLON.MeshBuilder.CreateBox(
"box", {width: 0.3, height: 0.3, depth: 0.3});
box.position.x = 0.0;
box.position.y = 1; // 1m above the ground
// Create Material
let boxMaterial = new BABYLON.StandardMaterial("material");
// Set the color of the box (in this case randomly changing) and apply it to the box
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//8⾯体を作成
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
Lesson09

Operation Check Box is shown

Adding a Display Object
function addObjects(){

}

//地⾯を作成 (原点の位置確認⽤。後で削除してOK)
let ground = BABYLON.MeshBuilder.CreateGround(
"ground", {width: 1, height: 1});
//箱状のオブジェクト作成 (30cm)
let box = BABYLON.MeshBuilder.CreateBox(
"box", {width: 0.3, height: 0.3, depth: 0.3});
box.position.x = 0.0;
box.position.y = 1; //地⾯(ground)から1mの⾼さ
//マテリアルを作成
let boxMaterial = new BABYLON.StandardMaterial("material");
//箱の⾊を設定 (今回はランダムに変化)、そのあとboxに適⽤
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//8⾯体を作成 [詳細]
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
Lesson10

• https://doc.babylonjs.com/features/featuresDeepDive/mesh/creation/polyhedra/polyhedra_by_numbers

Operation Check Box and octahedron is shown

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

Experience the Content on Quest or HoloLens2 window.onload = function() { /* Omitted */ //Creating Drawing Objects addObjects(); //Initialization of WebXR settings initializeXR(); } //Update drawing every frame engine.runRenderLoop(function () { if (scene && scene.activeCamera) { scene.render(); } }); //Initialization of WebXR settings async function initializeXR(){ let xr = await scene.createDefaultXRExperienceAsync({ /*options*/ }); } Lesson11

Change the URL Before Testing on a Device Click Settings

Change the URL Before Testing on a Device Edit project details Change the name to something easy to remember Save

Check URL of Your XR Content Click Copy Link

Check URL of Your XR Content Check the URL by pasting it into a text editor, etc. https://your-content.glitch.me/

Operation Check HoloLens2 Input the URL Goggle Button Quest Input the URL Goggle Button

Operation Check

How to Exit HoloLens2 Home Button Quest Close Button Exit content via Home window shown by wrist tap (HoloLens2) or Oculus button (Quest)

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

Object Manipulation
function addObjects(){
/*Omitted*/
// Set box color randomly
let boxMaterial = new BABYLON.StandardMaterial("material");
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//Create an octahedron
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
//Add manipulatable attribute to the box object
let dragBehavior = new BABYLON.SixDofDragBehavior();
dragBehavior.allowMultiPointer = false;
dragBehavior.rotateWithMotionController = false;
box.addBehavior(dragBehavior);
}

Lesson12

Operaton Check on PC Only objects to which DragBehavior is applied can be manipulated

Operation Check on HMD Reload Reload Please click the Reload button everytime before testing

Operation Check on HMD Quest users currently have hand tracking displaying hands incorrectly (to be fixed later)

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

Turn On Hand Visualization (For Quest)
//Initialization of WebXR settings
async function initializeXR() {
let xr = await scene.createDefaultXRExperienceAsync({});
// Check if XR mode available or not
if (!xr.baseExperience) { }
else {
//Turn on hand tracking if XR mode available
xr.baseExperience.featuresManager.enableFeature(
BABYLON.WebXRFeatureName.HAND_TRACKING,
"latest",
{xrInput: xr.input}
);
}
}

Lesson13

Operation Check on XR Device

That's it for the basic steps!

Extra Tutorial

Using 3D Models (GLB, OBJ, etc.)

Loading a 3D Model File Loading Libraries Enables the object loading functionality provided by Babylon.js

Loading a 3D Model File
<!– Loading babylon.js-->
<script src="https://cdn.babylonjs.com/babylon.js"></script>
<script src="https://cdn.babylonjs.com/loaders/babylonjs.loaders.min.js"></script>
<!-- Actual process implementation -->
<script type="text/javascript">
let canvas; //canvas element (drawing area) ← "where" to draw
let engine; //Drawing functionality with Babylon.js ← "What" to draw with
let scene; //Virtual 3D space ← "What" to draw
// Initialization of variables after page loading is completed
window.onload = function () {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
/*Omitted below */

Lesson14

Upload 3D Model to Glitch Assets

Upload 3D Model to Glitch Drag and drop in the browser figure.glb

Upload 3D Model to Glitch Back to index.html

Loading a 3D Model File
function addObjects(){
/* Omitted due to space limitation. */
let boxMaterial = new BABYLON.StandardMaterial("material");
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
// Create octahedron
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
// Added a hand-grabable attribute to the box.
let dragBehavior = new BABYLON.SixDofDragBehavior();
dragBehavior.allowMultiPointer=false;
dragBehavior.rotateWithMotionController=false;
box.addBehavior(dragBehavior);
}

Add code to load object (next page)

Loading a 3D Model File function addObjects(){ /* Omitted due to space limitation */ dragBehavior.rotateWithMotionController=false; box.addBehavior(dragBehavior); // Load 3D models using SceneLoader BABYLON.SceneLoader.LoadAssetContainer( "URL of 3D Model ", // Folder where the 3D model is placed OR URL of the 3D model "", //File name of the 3D model. Empty is OK if you specified the URL of the model above. scene, // Scene to which the object is added function (container) { //For Babylon.js, the 0th mesh is just the root; the 1st is the model entity.[Detail] let glbMesh = container.meshes[1]; //Add object to scene (3D space) scene.addMesh(glbMesh); } ); } Lesson15

• https://doc.babylonjs.com/features/featuresDeepDive/importers/glTF

Getting the URL of 3D Model function addObjects(){ /* Omitted due to space limitation */ dragBehavior.rotateWithMotionController=false; box.addBehavior(dragBehavior); // Load 3D models using SceneLoader BABYLON.SceneLoader.LoadAssetContainer( "URL of 3D Model ", // Folder where the 3D model is placed OR URL of the 3D model "", //File name of the 3D model. Empty is OK if you specified the URL of the model above. scene, // Scene to which the object is added function (container) { //For Babylon.js, the 0th mesh is just the root; the 1st is the model entity.[Detail] let glbMesh = container.meshes[1]; //Add object to scene (3D space) scene.addMesh(glbMesh); } ); }

• https://doc.babylonjs.com/features/featuresDeepDive/importers/glTF

Getting the URL of 3D Model Assets

Getting the URL of 3D Model Click figure.glb

Getting the URL of 3D Model Copy URL

Getting the URL of 3D Model Close

Getting the URL of 3D Model Back to index.html

Loading a 3D Model File function addObjects(){ /* Omitted due to space limitation */ dragBehavior.rotateWithMotionController=false; box.addBehavior(dragBehavior); // Load 3D models using SceneLoader BABYLON.SceneLoader.LoadAssetContainer( "URL of 3D Model ", // Folder where the 3D model is placed OR URL of the 3D model "", //File name of the 3D model. Empty is OK if you specified the URL of the model above. Paste URL between " and ". scene, // Scene to which the object is added function (container) { //For Babylon.js, the 0th mesh is just the root; the 1st is the model entity.[Detail] let glbMesh = container.meshes[1]; //Add object to scene (3D space) scene.addMesh(glbMesh); } ); }

• https://doc.babylonjs.com/features/featuresDeepDive/importers/glTF

Operation Checl Too Small

Modification //Load 3D models using SceneLoader BABYLON.SceneLoader.LoadAssetContainer( "URL of 3D Model", // Folder where the 3D model is placed OR URL of the 3D model "", //File name of the 3D model. Empty is OK if you specified the URL of the model above. scene, //Scene to which the object is added function (container) { // For Babylon.js, the 0th mesh is just the root; the 1st is the model entity let glbMesh = container.meshes[1]; //Scale x10 glbMesh.scaling=new BABYLON.Vector3(10,10,10); //Add object to scene (3D space) scene.addMesh(glbMesh); } ); Lesson16

Modification
//Load 3D models using SceneLoader
BABYLON.SceneLoader.LoadAssetContainer(
"URL of 3D Model ", // Folder where the 3D model is placed OR URL of the 3D model
"", //File name of the 3D model. Empty is OK if you specified the URL of the model above.
scene, //Scene to which the object is added
function (container) {
// For Babylon.js, the 0th mesh is just the root; the 1st is the model entity
let glbMesh = container.meshes[1];
//Scale x10
glbMesh.scaling=new BABYLON.Vector3(10,10,10);
// Rotate 180 degrees (=π radians) around Y-axis (=vertical axis)
glbMesh.rotation =new BABYLON.Vector3(0, Math.PI, 0);
//Add object to scene (3D space)
scene.addMesh(glbMesh);
}
);

Lesson17

Make 3D Model Manipulatable
// Load 3D models using SceneLoader
BABYLON.SceneLoader.LoadAssetContainer( "URL of 3D Model", "", scene,
function (container) {
//For Babylon.js, the 0th mesh is just the root; the 1st is the model entity
let glbMesh = container.meshes[1];
// Scale x10
glbMesh.scaling=new BABYLON.Vector3(10,10,10);
//Rotate 180 degrees (=π radians) around Y-axis (=vertical axis)
glbMesh.rotation =new BABYLON.Vector3(0, Math.PI, 0);
// Create manipulation behavior for loaded objects
let dragBehavior2 = new BABYLON.SixDofDragBehavior();
dragBehavior2.allowMultiPointer=false;
dragBehavior2.rotateWithMotionController=false;
glbMesh.addBehavior(dragBehavior2);
//Add object to scene (3D space)
scene.addMesh(glbMesh);
}
);

Lesson18

Operation Check

参考 Camera https://doc.babylonjs.com/divingDeeper/cameras/camera_introduction WebXR Experience Helper https://doc.babylonjs.com/divingDeeper/webXR/webXRExperienceHelpers#webxr-defaultexperience-helper MRTK https://doc.babylonjs.com/divingDeeper/gui/mrtk SixDofDragBehaviour https://doc.babylonjs.com/typedoc/classes/BABYLON.SixDofDragBehavior Hand Tracking https://doc.babylonjs.com/divingDeeper/webXR/WebXRSelectedFeatures#hand-tracking SceneLoader https://doc.babylonjs.com/typedoc/classes/BABYLON.SceneLoader AR mode (Quest pass-through supported) https://doc.babylonjs.com/features/featuresDeepDive/webXR/webXRARFeatures#features

• https://doc.babylonjs.com/divingDeeper/cameras/camera_introduction
• https://doc.babylonjs.com/divingDeeper/webXR/webXRExperienceHelpers
• https://doc.babylonjs.com/divingDeeper/gui/mrtk
• https://doc.babylonjs.com/typedoc/classes/BABYLON.SixDofDragBehavior
• https://doc.babylonjs.com/divingDeeper/webXR/WebXRSelectedFeatures
• https://doc.babylonjs.com/typedoc/classes/BABYLON.SceneLoader
• https://doc.babylonjs.com/features/featuresDeepDive/webXR/webXRARFeatures

Source code is available on GitHub https://github.com/TakashiYoshinaga/babylonjs-webxr

• https://github.com/TakashiYoshinaga/babylonjs-webxr