Getting Started with ARCore Geospatial API & Unity

Getting Started with ARCore Geospatial API

Goal of This Tutorial Low Accuracy! Display Tracking Results Tracking accuracy check High Accuracy! CG display at specified latitude and longitude

Download Sample Data https://github.com/TakashiYoshinaga/ARFukuoka/raw/main/GeospatialAPI_20220604/Smples.zip

• https://github.com/TakashiYoshinaga/AR-Fukuoka/raw/main/GeospatialAPI_20220604/Smples.zip

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

Launch Unity Hub Unity Hub

Project Creation (1/6) ①Projects ②New Project

Project Creation (2/6) Open Editor Version

Project Creation (3/6) 2020.3.xxf1 This document assumes 2020.3.x for explanation.

Project Creation (4/6) ①3D ②Project name ③ Storage location

Project Creation (5/6) Create project

Project Creation (6/6) When the Unity Editor appears, you are ready to go!

Build Settings for Android/iOS (1/4) File

Build Settings for Android/iOS (2/4) Build Settings

Build Settings for Android/iOS (3/4) ①Select Android or iOS ②Switch Platform

Build Settings for Android/iOS (4/4) Player Settings

Build Settings for Android For Android

Build Settings for Android (1/3) ①Other Settings ②Click Vulkan ③ Delete

Build Settings for Android (2/3) ①Identification ②Minimum API Level ③ Select Android 7.0 ʻNougatʼ API Level 24

Build Settings for Android (3/3) ①Configuration ②Scripting Backend ③Select IL2CPP ④Select ARM64

Build Settings for iOS For iOS

Build Settings for iOS ①Configuraton ② Set [Required for AR] for each Usage Description

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

Prepare to download packages (1/3) Window

Prepare to download packages (2/3) Package Manager

Prepare to download packages (3/3) Select Unity Registry from Packages

Installation of AR Foundation (1/2) Search AR

Installation of AR Foundation (2/2) ①ARFoundation ② Install

Install ARCore or ARKit ①Android: ARCore XR Plugin iOS: ARKit XR Plugin ② Install

Installing ARCore Extensions (1/3) Click + [Note] ARFoundation and ARCore/ARKit are only tools to realize AR. The ARCore Extensions is necessary to use Geospatial API, which is the main topic of this article.

Installing ARCore Extensions (2/3) Add package from git URL

Installing ARCore Extensions (3/3) Search below https://github.com/google-ar/arcore-unity-extensions.git

• https://github.com/google-ar/arcore-unity-extensions.git

Note If the PC you are using is not configured for GitHub connection, an error will occur.

Installation of ARCore Extensions (supplement) Access the ARCore Extensions repository https://github.com/google-ar/arcore-unity-extensions.git Click Releases

• https://github.com/google-ar/arcore-unity-extensions.git

Installation of ARCore Extensions (supplement) v.1.35.0 Scroll down a little

Installation of ARCore Extensions (supplement) Assets Download arcore-unity-extensions-1.35.0.tgz

Installation of ARCore Extensions (supplement) Move arcore-unity-extensions-1.35.0.tgz to the Packages folder in the folder of the project you have just created for this tutorial.

Installation of ARCore Extensions (supplement) Add package from git tarball

Installation of ARCore Extensions (supplement) Select arcore-unity-extentions

End of supplementary information

Confirmation of installation results Select In Project from Packages

Confirmation of installation results You can find ARCore Extensions

Get a sample of GeospatialAPI ①ARCore Extensions ②Samples ③ Import Geospatial Sample

Note Might have errors in environments that do not have the iOS build option installed.

Resolving iOS-related errors (Supplemental) You may get a red text error.

Resolving iOS-related errors (Supplemental) Samples -> ARCore Extensions ->1.35.0 ->GeospatialSample ->Scripts

Resolving iOS-related errors (Supplemental) Delete Editor

Resolving iOS-related errors (Supplemental) Console If the error disappears after clicking Clear, OK.

Resolving iOS-related errors (Supplemental) Display Project tab

End of supplementary information

Installation of ARCore Extensions is complete! Close Package Manager

Next, configure the tools you have installed so far for use in this project.

ARCore or ARKit Usage Settings (1/4) ① Edit ② Project Settings

ARCore or ARKit Usage Settings (2/4) XR Plug-in Management

ARCore or ARKit Usage Settings (3/4) For Android Turn on ARCore

ARCore or ARKit Usage Settings (4/4) For iOS Turn on ARKit

ARCore Extensions Usage Settings (1/6) ARCore Extensions

ARCore Extensions Usage Settings (2/6) [For iOS] Check the iOS Support Enabled checkbox

ARCore Extensions Usage Settings (3/6) Notice the item drop-down for Android or iOS

ARCore Extensions Usage Settings (4/6) Select APIKey from the drop-down menu

ARCore Extensions Usage Settings (5/6) API Key input pre-created in Google Cloud ※ See [Set up a Google Cloud Project] of the official website for how to create an API Key.[Link]

• https://developers.google.com/ar/develop/c/geospatial/developer-guide

ARCore Extensions Usage Settings (6/6) Turn on Geospatial

ARCore Extensions usage setup completed! Close Project Settings

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

Creating a scene (1/4) File

Creating a scene (2/4) Save As...

Creating a scene (3/4) Sample Save

Creating a scene (4/4) Assets Sample should be added

First, set up ARFoundation to realize AR experience in this scene (*GeospatialAPI will appear a little later)

Setting up AR Foundation (1/7) Delete MainCamera

Setting up AR Foundation (2/7) Right-click on a blank space in Hierarchy with nothing selected.

Setting up AR Foundation (3/7) ①XR ②AR Session Origin

Setting up AR Foundation (4/7) AR Session Origin should appear

Setting up AR Foundation (5/7) Right-click on a blank space in Hierarchy with nothing selected.

Setting up AR Foundation (6/7) ①XR ②AR Session

Setting up AR Foundation (7/7) AR Session should appear

Prepare to use Anchor, which ties an object to be displayed in AR to a specific location. The ability to align when looking there again even if the tracking is lost. Even if the CG placed at the specified latitude and longitude is lost, it will come back!

Setting up AR Anchor Manager (1/3) ①AR SessionOrigin ②Add Component

Setting up AR Anchor Manager (2/3) ①Search AR ②AR Anchor Manager

Setting up AR Anchor Manager (3/3) ARAnchorManager should be added

Enable the results of position and direction estimation by Geospatial API to be used in this scene

Setting up AR Earth Manager (1/3) ①AR SessionOrigin ②Add Component

Setting up AR Earth Manager (2/3) ①Search AR Earth ②AR Earth Manager

Setting up AR Earth Manager (3/3) AREarthManager should be added

Using the Geospatial API also requires permission to access location information, etc. But… You don't want to write the Permission-related details, do you?

Take advantage of official scripts for easy setup!

Adding a object to set up permission (1/2) Right-click on a blank space in Hierarchy with nothing selected.

Adding a object to set up permission (2/2) ②AR Extensions ①XR

Permission-related script settings (1/9) ARCore Extensions In this area, you can pass the following script that is running in this scene and it will do the Permission-related settings. ・ARSession ・ARSessionOrigin ・ARCameraManager ・ARCoreExtensionConfiguration

Permission-related script settings (2/9) of Sessoin

Permission-related script settings (3/9) Scene AR Session [Explanation] The ARSession script is attached to the AR Session object in this scene (in the Hierarchy)

Permission-related script settings (4/9) of SessoinOrigin

Permission-related script settings (5/9) Scene AR Session Origin

Permission-related script settings (6/9) of CameraManager

Permission-related script settings (7/9) Scene AR Camera

Permission-related script settings (8/9) of ARCoreExtensionsConfig

Permission-related script settings (9/9) Scene GeospatialConfig

Permission setup completed! Allocation completed (CameraConfigFilter is not required)

現状確認 AR Earth Manager It is possible to obtain the terminal support status, the success or failure of Geospatial API initialization, and the results of position and orientation estimation. Permission setting is done by Extensions. However, it is available only when tracking by GeospatialAPI has started. Start tracking by script!

Aside: Official Sample There is a scene in the official sample to learn the steps from initialization to Geospatial API utilization.

Aside: Official Sample Samples ->ARCore Extensions ->1.35.0 ->Geospatial Sample

Aside: Official Sample Scripts GeospatialController

Aside: Official Sample The GeospatialController script contains code for initialization, position acquisition, and saving and reproducing object placement. However, both initialization and data utilization are described together, making it difficult to reuse for other purposes

I have prepared a script that isolates only the initialization code to start tracking. Aside from the details, let’s use it now!

Importing sample assets (1/3) ①Samples Folder ②Double click AR_Fukuoka.unitypackage

Importing sample assets (2/3) Import

Importing sample assets (3/3) AR_Fukuoka should be added

Creating an object to start tracking (1/3) Right-click on a blank space in Hierarchy with nothing selected.

Creating an object to start tracking (2/3) CreateEmpty

Creating an object to start tracking (3/3) ①GameObject ②Change name to VPS_Demo

Add tracking start script (1/7) ①Assets -> AR_Fukuoka ->Scripts ②See VpsInitializer (Do not click it)

Add tracking start script (2/7) VPS Demo Drag & drop VpsInitializer into Inspector of VPS_Demo

Add tracking start script (3/7) VPS Visualizer should be added

Add tracking start script (4/7) of EarthManager

Add tracking start script (5/7) Scene AR Session Origin

Add tracking start script (6/7) of ARCoreExtensions

Add tracking start script (7/7) Scene AR Core Extensions

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

Added tracking results display script (1/4) ①Assets -> AR_Fukuoka ->Scripts ②See SampleScript (Do not click it)

Added tracking results display script (2/4) VPS_Demo

Added tracking results display script (3/4) Drag & Drop SampleScript into Inspector of VPS_Demo

Added tracking results display script (4/4) SampleScript should be added Double click to open script file

Script to display tracking results (1/4) Add a description in advance to the empty script generated by Unity to use the following functions ・ ARCoreExtensions ・ ARFoundation

Script to display tracking results (2/4) public class SampleScript : MonoBehaviour { //Tracking information using GeospatialAPI public AREarthManager EarthManager; //GeospatialAPI and ARCore initialization and results public VpsInitializer Initializer; //UI for displaying results public Text OutputText; void Start() { } void Update() { } } 01.txt

Script to display tracking results (3/4)
void Update()
{
string status = "";
//If initialization fails or you do not want to track, do nothing and return
if ( ! Initializer.IsReady ||
EarthManager.EarthTrackingState != TrackingState.Tracking)
{
return;
}
//Get tracking results
GeospatialPose pose = EarthManager.CameraGeospatialPose;
//Show results
ShowTrackingInfo(status, pose);
}
void ShowTrackingInfo(string status, GeospatialPose pose)
{
//Displays lat, lng, alt and their precision
}

02.txt

Script to display tracking results (4/4)
void ShowTrackingInfo(string status, GeospatialPose pose)
{
OutputText.text = string.Format(
"Latitude/Longitude: {0}°, {1}°¥n" +
"Horizontal Accuracy: {2}m¥n" +
"Altitude: {3}m¥n" +
"Vertical Accuracy: {4}m¥n" +
"Heading: {5}°¥n" +
"Heading Accuracy: {6} °¥n" +
"{7} ¥n"
,
pose.Latitude.ToString("F6"), //{0}
pose.Longitude.ToString("F6"), //{1}
pose.HorizontalAccuracy.ToString("F6"), //{2}
pose.Altitude.ToString("F2"), //{3}
pose.VerticalAccuracy.ToString("F2"), //{4}
pose.EunRotation.ToString("F1"), //{5}
pose.OrientationYawAccuracy.ToString("F1"), //{6}
status //{7}
);
}

03.txt

Ctrl/Command + S to save script

Mapping scripts to variables (1/4) VPS Demo of ARCoreExtensions

Mapping scripts to variables (2/4) Scene AR Session Origin

Mapping scripts to variables (3/4) Initializer

Mapping scripts to variables (4/4) Scene VPS_Demo

Add UI for display (1/3) ①Assets -> AR_Fukuoka ②See Canvas

Add UI for display (2/3) Drag & Drop Canvas into Hierarchy

Add UI for display (3/3) Open Canvas Confirm that TrackingMonitorText

Mapping scripts to variables (1/2) Click VPS Demo of OutputText

Mapping scripts to variables (2/2) Scene TrackingMonitorText

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

For Android

実機インストール for Android Connect ARCore device and PC with USB cable

Install for Android File

Install for Android Build And Run

Install for Android Name(e.g. test) Save

For iOS

Install for iOS File

Install for iOS Build Settings...

Install for iOS Build

Install for iOS Create new folder

実機インストール for iOS Set folder name (e.g. build) Create

Install for iOS Choose

Install for iOS Open build folder

Install for iOS Unity-iPhone.xcworkspaceを開く

Install for iOS Unity-iPhone Setting & Capabilities

Install for iOS Automatically manage signing Select your team

Install for iOS Connect iOS device and PC with USB cable

Install for iOS Select a device

Install for iOS Click

Android and iOS

Result

Tutorial Procedure p Creating a Unity Project p Installing AR Foundation and ARCore Extensions p Setting up a scene using the Geospatial API and ARCore/ARKit p Getting and displaying tracking information via Geospatial API p Application installation on smartphone p Display objects at specified latitude and longitude

Development Policies pSpecify latitude, longitude, and height in advance and display objects at those locations • Check position and orientation accuracy every frame • Make objects appear when position and orientation are more accurate. Lat・Lng Matching AR

Development Policies pSpecify latitude, longitude, and height in advance and display objects at those locations • Check position and orientation accuracy every frame • Make objects appear when position and orientation are more accurate. Lat・Lng Matching AR

Discrimination of tracking accuracy (1/3) public class SampleScript : MonoBehaviour { //Tracking information using GeospatialAPI public AREarthManager EarthManager; //GeospatialAPI and ARCore initialization and results public VpsInitializer Initializer; //UI for displaying results public Text OutputText; //Allowable accuracy of azimuth public double HeadingThreshold = 25; //Allowable accuracy of horizontal position public double HorizontalThreshold = 20; } void Start(){ } void Update(){ /*略*/ } 04.txt

Discrimination of tracking accuracy (2/3)
void Update()
{
string status = "";
// If initialization fails or tracking is not done, do nothing and return
if ( ! Initializer.IsReady ||
EarthManager.EarthTrackingState != TrackingState.Tracking)
{
return;
}
// Get tracking results
GeospatialPose pose = EarthManager.CameraGeospatialPose;
// Describe here the behavior according to tracking accuracy

}

//Show result
ShowTrackingInfo(status, pose);

Discrimination of tracking accuracy (3/3)
// Get tracking results
GeospatialPose pose = EarthManager.CameraGeospatialPose;
// Tracking accuracy is worse than the threshold (larger value)
if (pose.OrientationYawAccuracy > HeadingThreshold ||
pose.HorizontalAccuracy > HorizontalThreshold)
{
status = "Low Tracking accuracy ";
}
else //許容誤差以内の場合
{
status = "High Tracking Accuracy ";
}
//結果を表⽰
ShowTrackingInfo(status, pose);

05.txt

Result Accuracy is displayed as good or bad.

Development Policies pSpecify latitude, longitude, and height in advance and display objects at those locations • Check position and orientation accuracy every frame • Make objects appear when position and orientation are more accurate. Lat・Lng Matching AR

指定した位置にオブジェクトを出現 (1/3) public class SampleScript : MonoBehaviour { public AREarthManager EarthManager; //GeospatialAPIを⽤いたトラッキング情報 public VpsInitializer Initializer; //GeospatialAPIとARCoreの初期化と結果 public Text OutputText; //結果表⽰⽤のUI public double HeadingThreshold = 25; //位置・⽅位の許容精度 public double HorizontalThreshold = 20; public double Latitude; // Latitude to place the object public double Longitude; // Longitude to place the object public double Altitude; // Height to place the object public double Heading; // Object orientation (North=0°) public GameObject ContentPrefab; // Original data of display object GameObject displayObject; // Actual object to be displayed public ARAnchorManager AnchorManager; // Used to create anchors } void Start(){ } void Update(){/*略*/ } 06.txt

指定した位置にオブジェクトを出現 (2/3)
// Get tracking results
GeospatialPose pose = EarthManager.CameraGeospatialPose;
if (pose.HeadingAccuracy > HeadingThreshold ||
pose.HorizontalAccuracy > HorizontalThreshold)
{
status = "Low Tracking accuracy ";
}
else
{
status = "High Tracking accuracy ";
if (displayObject == null)
{
}

}

//結果を表⽰
ShowTrackingInfo(status, pose);

07.txt

指定した位置にオブジェクトを出現 (3/3)
if (displayObject == null)
{
// Height of the phone - 1.5m to be approximately the height of the ground
Altitude = pose.Altitude - 1.5f;
// Angle correction (because the Anchor generation function assumes South=0)
Quaternion quaternion = Quaternion.AngleAxis(180f - (float)Heading, Vector3.up);
// Create anchors at specified position and orientation.
ARGeospatialAnchor anchor
= AnchorManager.AddAnchor(Latitude, Longitude, Altitude, quaternion);

}

//If you want to use the height of the terrain (Terrain) provided
//by the API instead of Altitude, use the following
/*ARGeospatialAnchor anchor
= AnchorManager.ResolveAnchorOnTerrain(Latitude, Longitude, 0, quaternion);*/
// Materialize the object if the anchor is correctly created
if (anchor != null)
{
08.txt
displayObject = Instantiate(ContentPrefab, anchor.transform);
}

Development Policies pSpecify latitude, longitude, and height in advance and display objects at those locations • Check position and orientation accuracy every frame • Make objects appear when position and orientation are more accurate. Lat・Lng Matching AR

Specify latitude and longitude VPS Demo Specify latitude and longitude

Open Google Map

Specify latitude and longitude (1/5) Right-click on the location where you want to place the object.

Specify latitude and longitude (2/5) Click on latitude and longitude to copy to clipboard

Specify latitude and longitude (3/5) Pasete Latitude and Longitude

Specify latitude and longitude (4/5) Altitude is not necessary this time because it is written in a script.

Specify latitude and longitude (5/5) Heading is set to 0 (=northward)

Registering Display Objects (1/2) ①Assets -> AR_Fukuoka ->Content ② Find ARPrefab (Do not click it)

Registering Display Objects (2/2) VPS Demo AR PrefabをContentPrefab にドラッグ&ドロップ

Registering AnchorManager (1/2) of AnchorManager

Registering AnchorManager (2/2) Scene AR Session Origin

Result Object appears at specified position

Completed!

ARCore Geospatial API official page For more information on the API, check out the official page!https://developers.google.com/ar/develop/geospatial

• https://developers.google.com/ar/develop/geospatial

Sample Project for This Tutorial https://github.com/TakashiYoshinaga/ GeospatialAPI-Unity-StarterKit

• https://github.com/TakashiYoshinaga/GeospatialAPI-Unity-StarterKit

Follow Me • Twitter: @Tks_Yoshinaga • LinkedIn: https://www.linkedin.com/in/tks-yoshinaga/

• https://twitter.com/Tks_Yoshinaga
• https://www.linkedin.com/in/tks-yoshinaga/