Google Summer of Code Final Work Product

• Name: Shivam Anand

• Organisation: Python Software Foundation

• Sub-Organisation: FURY

• Project: FURY - glTF Integration

Proposed Objectives

• Ability to load glTF models

  • Should be able to load static glTF models

  • Should be able to add model to the scene.

• Exporting scene data as glTF file

• Materials & Textures

  • Textures

  • PBRs

• Animations

  • Simple Actor animations

  • Skinning

  • Morphing

• Stretch Goals

  • Ability to load .glb files

Objectives Completed

Loading Static glTF models

A glTF file is a JSON like file format containing required data for 3D scenes. VTK has two built-in glTF loaders. However, they lack ability to animate and apply materials. Added methods to load binary data and create actors from them. These actors can be added directly to the scene. The glTF class reads texture data from either base64 encoded string or from the image file, and maps the texture to the actor using the given UV data. It is capable of doing the following:

• Loading both gltf and glb files. Get actors from the model.

• Applying textures and colors from materials.

• Setting cameras if the model contains multiple cameras.

• Apply normals (for a smoother surface).

Pull Requests:

• Importing glTF files: (Merged) fury-gl/fury#600

• Loading glTF Demo (with textures) (Merged): fury-gl/fury#600

Exporting Scene as a glTF

The fury scene can contain multiple objects such as actors, cameras, textures, etc. We need to get the primitive information (such as Vertices, Triangles, UVs, Normals, etc.) from these objects and store them into a .bin file. Added methods that export these information to a .gltf or .glb file format.

Pull Requests:

• Exporting scene as glTF: (Merged) fury-gl/fury#630

• Exporting scene as glTF Tutorial: (Merged) fury-gl/fury#630

Simple Actor Animations

Added simple actor animations (translation, rotation & scale of actors) support. The animation data (transformation and timestamp) is stored in buffers. It converts the binary data to ndarrays and creates a timleline for each animation in glTF animations. This timeline contains actors an can be added to the scene. We can animate the scene by updating timeline inside a timer callback.

Pull Requests:

• Simple Animations in glTF: (Merged) fury-gl/fury#643

• Simple Animations in glTF Tutorial: (Merged) fury-gl/fury#643

Morphing in glTF

glTF allows us to animate meshes using morph targets. A morph target stores displacements or differences for certain mesh attributes. At runtime, these differences may be added to the original mesh, with different weights, to animate parts of the mesh. Added methods to extract this information, update the timeline and apply morphing to each actor in the scene.

Pull Requests:

• Morphing support in glTF: (Under Review) fury-gl/fury#700

• Morphing in glTF demo: (Under Review) fury-gl/fury#700

Skeletal Animations (Skining)

Another way of animating a glTF is by skinning. It allows the geometry (vertices) of a mesh to be deformed based on the pose of a skeleton. This is essential in order to give animated geometry. It combines every parameter of a glTF file. While working with skinning, we need to keep track of the parent-child hierarchy of transformations. Vertex Skinning takes full advantage of newly implemented Timeline & Animation modules to track hierarchical transformation order. Though the current version of the skinning implementation works with most of the glTF sample modes, It struggles with models that have multiple actors (e.g. BrainStem). It can be fixed by using the vertex shader to update the vertices. The current implementation of skinning supports the following:

• Multiple animation support

• Multiple node and multiple actor animation with textures

• Show or hide bones/skeleton of the model.

Pull Requests:

• Skinning support in glTF: (Under Review) fury-gl/fury#685

• Skinning in glTF demo: (Under Review) fury-gl/fury#685

Objectives in Progress

PBR and emission materials in glTF

The glTF format supports Physically based rendering also. PBR allow renderers to display objects with a realistic appearance under different lighting conditions, the shading model has to take the physical properties of the object surface into account. There are different representations of these physical material properties. One that is frequently used is the metallic-roughness-model. We have various material properties already in FURY, we need to apply it to glTF models as well.

Skinning for models with no indices

The glTF format supports non-indexed geometry (e.g., the Fox model). We currently do not know how to render the model without indices. I tried estimating it in this branch. However, It fails to render in skinning.

Branch URL:

• Rendering glTF with no indices: (in-progress) xtanion/fury

Other Objectives

Fetcher for importing glTF files from Khronos-glTF-Samples

The KhronosGroup/gltf-samples contain multiple glTF sample models to test a glTF viewer for free. Implemented new methods in fetcher that can load all of these models by (using type) asynchronously. The glTF fetcher is capable of the following:

• Downloading multiple models asynchronously.

• Get the path to the downloaded model using it - Download any model using the URL of the model.

Pull Requests:

• Fetching glTF sample models from github: (Merged) fury-gl/fury#602

• Fixing github API limit: (Merged) fury-gl/fury#616

Other Pull Requests

• Sphere actor uses repeat_primitive by default: fury-gl/fury/#533

• Cone actor uses repeat primitive by default: fury-gl/fury/#547

• Updated code of viz_network_animated to use fury.utils: fury-gl/fury/#556

• Added simulation for Tesseract: fury-gl/fury/#559

• GLTF actor colors from material fury-gl/fury/#689

GSoC weekly blogs

• My blog posts can be found on the FURY website and the Python GSoC blog.

Timeline

Date	Description	Blog Post Link
Week 0 (24-05-2022)	My journey to GSoC 2022	FURY - Python
Week 1 (20-06-2022)	A basic glTF Importer	FURY - Python
Week 2 (29-06-2022)	Improving Fetcher and Exporting glTF	FURY - Python
Week 3 (04-07-2022)	Fixing fetcher adding tests and docs	FURY - Python
Week 4 (12-07-2022)	Finalizing glTF loader	FURY - Python
Week 5 (19-07-2022)	Creating PR for glTF exporter and fixing the loader	FURY - Python
Week 6 (25-07-2022)	Extracting the animation data	FURY - Python
Week 7 (01-08-2022)	Fixing bugs in animations	FURY - Python
Week 8 (09-08-2022)	Fixing animation bugs	FURY - Python
Week 9 (17-08-2022)	First working skeletal animation prototype	FURY - Python
Week 10 (25-08-2022)	Multi-node skinning support	FURY - Python
Week 11 (31-08-2022)	Multiple transformations support and adding tests	FURY - Python
Week 12 (08-09-2022)	Adding skeleton as actors and fix global transformation	FURY - Python
Week 13 (15-09-2022)	Multi bone skeletal animations	FURY - Python
Week 14 (28-09-2022)	Morphing is here !	FURY - Python

Google Summer of Code Final Work Product   Contribute to FURY via Google Summer of Code 2023