Workflow for Making Roads with a Curve Editor

Workflow for Making Roads with a Curve Editor Let's get started with Workflow for Making Roads with a Curve Editor. This presentation will focus on the creation of roads in open world games; I‘ll introduce the curve editing tool developed in RE ENGINE and the production flow using the tool. ©CAPCOM 1

How to express the road First, lets go over the features and requirements of the roads we want to create. 2 ©CAPCOM 2

Road Features and Functions ・Entire area is connected Wheel tracks Lots of undulation Branching and converging ・Followable by NPCs ・Displayable on maps Drawing Method Road Shape Mesh Non-uniform width ＋ Terrain Height Map This presentation will deal primarily with unpaved natural terrain roads. Compared to urban roads, natural terrain roads are more undulating and less uniform in width, making them difficult to3represent in simple shapes. Also, wheel ruts are often seen on roads used by wheeled vehicles. This also looks unnatural if it does not follow the curves of the road. Based on these considerations, the road should be drawn on top of the base terrain height map, covered with a mesh that has the shape of the road. In terms of the data structure considerations, the roads have many branching and merging paths. And because almost all roads in the world are connected, it is difficult to divide the data by area. Other features such as guiding NPCs to walk along the paths, or the ability to display the paths on the map screen, may also become necessary. ©CAPCOM 3

Data Structure of Road Assets Graph structure with connected cubic Bezier curves Vertex (point) ・Road width ・Direction ・Falloff Edge (segment) ・Strength of curve ・Mesh type ・Tag information To handle these requirements, we defined the data of the road asset in the engine in this way. The entire road asset is managed in a single graph structure. Each point has information such as path width and direction, and each segment represents a cubic Bezier curve. Additional data used, such as the type of road, is carried in the segments as tag information. ©CAPCOM 4 4

Data Structure of Road Assets
Asset differences between collaborators are merged using Id
"PointsData":[
{
"Id":"2f89cb70",
"Width":10.0,
...
},
{
"Id":"f6db770c",
"Width":10.0,
...
}],

"SegmentsData":[
{
"Id":"38ade50e",
"StartPointId":"2f89cb70",
"EndPointId":"f6db770c",
...
}]

The contents of an asset are structured like this.
Every point and segment has a unique Id, and a segment refers to a point by its Id.
When multiple people edit and commit a road asset at the same time, the assets are merged via Id;
The system is designed to prevent conflicts unless they edit the same location.

©CAPCOM

5

5

Road Editing Tools Next I'll describe the road editing tools. 6 ©CAPCOM 6

SplineEditor Placement and Editing of Road Data in the Runtime View SplineEditor is an engine function for editing road data in the runtime. Here are some basic functions of the editor. 7 ©CAPCOM 7

Basic SplineEditor Operation Road Addition Movement and Rotation Manipulation of Road Width Bezier Editing First, to add a path, we place a point on the plane and draw a segment. When connecting, automatic correction of orientation is used to ensure smooth connections. 8 ©CAPCOM 8

Basic SplineEditor Operation Road Addition Movement and Rotation Manipulation of Road Width Bezier Editing Placed points can be moved and rotated with manipulators. 9 ©CAPCOM 9

Basic SplineEditor Operation Road Addition Movement and Rotation Manipulation of Road Width Bezier Editing The road width is edited by moving the handles on the points. Here you can see that by selecting two points, you can select all the points between them, and adjust the path width on all of them together. 10 ©CAPCOM 10

Basic SplineEditor Operation Road Addition Movement and Rotation Manipulation of road Width Bezier Editing To edit the gradualness of the curve, manipulate the Bezier handles to adjust their distance to the control point. We use a shortcut key here to switch to symmetric mode. 11 ©CAPCOM 11

SplineMesh Component Deform mesh to fit on curve Next, I'll describe the SplineMesh component. This is the ability to draw a mesh assigned to a segment by placing it on a curve. Here are some descriptions of the placement that can be made with SplineMesh. ©CAPCOM 12 12

SplineMesh Placement Properties Appearance ratios for each mesh type Ordinary fence Ratio: 100 Broken fence Ratio: 15 When multiple meshes are assigned to one segment, they are placed in a random order. At this time, by setting different appearance ratios for each mesh, you can create an arrangement in which broken fences appear once in a while. 13 ©CAPCOM 13

SplineMesh Deformation Types Deformation Bend and place vertices along curves Visual artifacts: None / Processing cost: High Use cases: roads, rivers, caves, etc. No deformation Affine transformation on a curve Visual artifacts: Present / Processing cost: Low Use cases: fences, poles, paving stones, etc. SplineMesh also allows you to turn deformation on and off. When deformation is enabled, the mesh is bent vertex by vertex along the curve. When disabled, the mesh vertices are placed on the curve using only affine transformations. When making roads, the vertices of the instance must overlap each other exactly or the appearance will be broken. Therefore, deformation is almost always enabled for them. 14 On the other hand, road side objects such as fences, telephone poles, paving stones, etc. do not necessarily need to overlap exactly. Deformation can be disabled for them. ©CAPCOM 14

Road Data Conversion So far, we have introduced how to edit roads and the functions of the tool. From here, we will discuss the baking functionality provided by the engine to convert the edited road assets into the most 15 suitable format for drawing. ©CAPCOM 15

Drawing Roads Road width Falloff Mesh placed along curves ＋ Rut depth Terrain Height Map At the beginning, I explained that the drawing of a road is represented by a combination of heightmap and mesh. The boundary between the road and the ground should not be noticeable. The curve information of the road must be burned into the heightmap. ©CAPCOM 16 16

Bake to Height Map Blend according to distance from curve Road width Falloff Road width Rut depth Rut depth Rut depth Exponential blend Falloff Cosine blend There are two processes here: One, dig into the ground a bit so the ground doesn't poke through the road. Two, blend the ground height as it moves away from the path. 17 The inner part is blended with an exponential function so that the width and smoothness can be adjusted for each type of mesh. The outer part is blended with a cosine function to create a smooth transition to the ground level. ©CAPCOM 17

Bake to Height Map Blend according to distance from curve Distance calculation from curve Road width d𝒄 𝑡 d𝑡 Rut depth 𝒑−𝒄 𝑡 𝒄 𝑡 Calculate each pixel's nearest neighbor on the curve using Newton's method Rut depth Exponential blend Falloff Cosine blend 𝒑−𝒄 𝑡 ∙ d𝒄 𝑡 =𝟎 d𝑡 The blending process is calculated using the distance from the center of the road. The distance is calculated for each pixel of the height map. The equation for its nearest neighbor on the curve is solved using Newton's method. ©CAPCOM 18 18

Can SplineMesh be used in-game? Impact of dynamic deformation on performance Processing cost Memory Render mesh: deformation processing VRAM consumption: original mesh only Physics mesh: deformation processing Physics mesh: BVH construction processing Road information: read control Next, we will discuss SplineMesh conversion, but before that, we will explain what happens when SplineMesh is brought into a game as it is. 19 First of all, the cost of bending the mesh along the curves will be directly applied to the game. On the other hand, by dynamically deforming the mesh at rendering time, VRAM consumption is limited to the original mesh. This is an advantage. However, for physics meshes, this advantage disappears. The additional processing cost of the BVH construction, which is usually done in advance, is also added. Also, it is not easy to handle loading the road data while accounting for the drawing range of the path in the game. ©CAPCOM 19

Can SplineMesh be used in-game? Impact of dynamic deformation on performance Processing cost Memory Render mesh: deformation processing VRAM consumption: original mesh only Physics mesh: deformation processing Physics mesh: SplineMesh is BVH only used during development construction processing Its output needs to be pre-baked Road information: read control Because there are many challenges in bringing SplineMesh directly into the game, in this case, we decided to take the results of its placement and pre-bake it. 20 ©CAPCOM 20

Can SplineMesh be used in-game? Impact of dynamic deformation on performance Processing cost Memory Render mesh: deformation processing VRAM consumption: original mesh only Physics mesh: deformation processing Physics mesh: SplineMesh is BVH only used during development construction processing Its output needs to be pre-baked Bake Method A) Mesh output B) Prefab placement C) Joint conversion Road information: read control The engine offers three ways to bake. I'll explain each of them. 21 ©CAPCOM 21

Conversion Method A: Mesh output Mesh output specified by the size of the area to be loaded SplineMesh can be accurately represented VRAM consumption proportional to number of instances The first is to output the deformation results to an FBX mesh. Specify a block that matches the size of the mesh to be loaded in the game, The SplineMesh instances within that area22 are solidified into a mesh and output. Since the deformed path is output as is, there is no visual degradation. However, since all vertices are allocated separately in memory, VRAM consumption increases in proportion to the number of instances. ©CAPCOM 22

Conversion Method A: Mesh output Separate output for each mesh type Combine into one mesh and output The block may also contain more than one type of mesh. In this case, there are two ways to output each type of mesh as a separate asset, one way is to combine them all into a single mesh and output them as one. 23 ©CAPCOM 23

Conversion Method A: Mesh output Separate output for each mesh type Combine into one mesh and output Maintain original material connections Bundle material connections Render LOD is disjointed Rendered with unified LOD Complex conversion process LOD criteria between meshes must be aligned When output separately, the relationship between the original mesh and material can be maintained. This has the advantage of being easy to use. 24 However, when different LODs are selected between meshes at the time of drawing, the meshes may end up looking clumsy. This problem does not occur when using merged output, as it is always displayed with the same LOD. However, this requires processing to bind the mesh and material connections. This can lead to accidents during the change detection and conversion process, increasing labor cost. Also, the criteria for the LOD to be created for each mesh must be aligned in advance. ©CAPCOM 24

Conversion Method B: Prefab Placement Replace with transformed instances of original mesh (no deformation possible) Lightweight due to instanced drawing SplineMesh Placed Prefabs Next, a conversion method that works by arranging prefabs. This is a limited option where deformation is disabled, i.e., placement without bending, which just retrieves transformation 25 information for each instance. By using this to replace SplineMesh with prefab placement, instanced drawing can be used. ©CAPCOM 25

Conversion Method C: Joint Conversion Approximate SplineMesh shape with joint deformation of mesh Use of original mesh geometry saves VRAM Trade-off between drawing cost and approximation accuracy Finally, we will discuss joint conversions. This is to handle the case where VRAM usage is bottleneck, preventing the use of the baked FBX mesh method, but deformation is 26 required, proscribing the prefab method. The workaround is to bake the SplineMesh deformation information into joints, then rendering it via skinning. This method does increase the cost of drawing at runtime by using skinning, but, it has the advantage of not consuming VRAM for each instance. However, the joint-based deformation is only an approximate calculation, and may cause visual artifacts depending on the way the joints are divided and the degree of curvature. ©CAPCOM 26

Data Conversion Flow Summary Height map Bake Background Artist deformation used SplineMesh edits road Yes No VRAM limited Yes Placed prefabs No Package Mesh output Joint conversion WayPoint Bake The above is a description of the road data conversion process currently supported by the engine, summarized as an example of a data conversion flow. 27 When an artist edits a road, the height map is first baked. SplineMesh is then replaced with placed prefabs if it has no deformation used. If it does use deformation, it outputs to a mesh if VRAM budget is not a priority, or converts to joints if it is. WayPoint assets for guiding NPCs are also baked using macros. ©CAPCOM 27

Asset Management Challenges Large scale of road assets handled Number of points: approx. 18,000 Number of segments: approx. 19,000 Processing the entire road is computationally expensive • Loading and saving assets • Frustum culling • Baking processing, etc. This is probably the absolute limit for the current single-asset management method Split and manage as multiple assets Finally, I will describe future issues. In an actual game we developed, road assets weighed in at approximately 18,000 points and 19,000 segments, which is 28larger than we had anticipated. As a result, the process of loading assets, frustum culling during rendering, etc., was intensive, making it difficult to use the tool comfortably. Because there is a limit to managing a larger scale than this with a single asset, we are considering moving to a split management system in the future. ©CAPCOM 28

Workflow Challenges Change the shape of the road To reflect the change in the exported game package all roads in the game world need to be re-baked. High-cost conversion flow hinders game operation checks. Also, there are still issues to be addressed in terms of workflow. For example, assume a case where changes are made to the original mesh that is being deformed in SplineMesh. In this case, the change impacts all the road shapes that deforming this mesh. To reflect this in the package, we would need to re-bake the entire world's road mesh. This can be an obstacle when checking game behavior in some cases. We consider this a candidate for improvement in our workflow. ©CAPCOM 29 29

Workflow Solutions Single generic technique Location-specific techniques Tool In-editor representation In-game representation SplineEditor SplineMesh Mesh SplineMesh Mesh SplineEditor Decal Modular layout In this case, we were operating using only the versatile SplineMesh, we could not avoid this problem. In the future, we'll include techniques that don't need baking, such as decals and modular placement. The goal is a workflow that lets us match the method to the location. ©CAPCOM 30 30

This concludes the presentation. Thank you very much. 31 ©CAPCOM 31