gsoc
fury

Google Summer of Code Final Work Product#

Proposed Objectives#

  • Ability to load glTF models

    • Should be able to load static glTF models

    • Should be able to add model to the scene.

  • Exporting scene data as glTF file

  • Materials & Textures

    • Textures

    • PBRs

  • Animations

    • Simple Actor animations

    • Skinning

    • Morphing

  • Stretch Goals

    • Ability to load .glb files

Objectives Completed#

Loading Static glTF models#

A glTF file is a JSON like file format containing required data for 3D scenes. VTK has two built-in glTF loaders. However, they lack ability to animate and apply materials. Added methods to load binary data and create actors from them. These actors can be added directly to the scene. The glTF class reads texture data from either base64 encoded string or from the image file, and maps the texture to the actor using the given UV data. It is capable of doing the following:

  • Loading both gltf and glb files. Get actors from the model.

  • Applying textures and colors from materials.

  • Setting cameras if the model contains multiple cameras.

  • Apply normals (for a smoother surface).

image

Pull Requests:

Exporting Scene as a glTF#

The fury scene can contain multiple objects such as actors, cameras, textures, etc. We need to get the primitive information (such as Vertices, Triangles, UVs, Normals, etc.) from these objects and store them into a .bin file. Added methods that export these information to a .gltf or .glb file format.

Pull Requests:

Simple Actor Animations#

Added simple actor animations (translation, rotation & scale of actors) support. The animation data (transformation and timestamp) is stored in buffers. It converts the binary data to ndarrays and creates a timleline for each animation in glTF animations. This timeline contains actors an can be added to the scene. We can animate the scene by updating timeline inside a timer callback.

https://user-images.githubusercontent.com/74976752/217645594-6054ea83-12e5-4868-b6a1-eee5a154bd26.gif

Pull Requests:

Morphing in glTF#

glTF allows us to animate meshes using morph targets. A morph target stores displacements or differences for certain mesh attributes. At runtime, these differences may be added to the original mesh, with different weights, to animate parts of the mesh. Added methods to extract this information, update the timeline and apply morphing to each actor in the scene.

https://user-images.githubusercontent.com/74976752/217645485-153ec403-6c87-4282-8907-30d921106b34.gif

Pull Requests:

Skeletal Animations (Skining)#

Another way of animating a glTF is by skinning. It allows the geometry (vertices) of a mesh to be deformed based on the pose of a skeleton. This is essential in order to give animated geometry. It combines every parameter of a glTF file. While working with skinning, we need to keep track of the parent-child hierarchy of transformations. Vertex Skinning takes full advantage of newly implemented Timeline & Animation modules to track hierarchical transformation order. Though the current version of the skinning implementation works with most of the glTF sample modes, It struggles with models that have multiple actors (e.g. BrainStem). It can be fixed by using the vertex shader to update the vertices. The current implementation of skinning supports the following:

  • Multiple animation support

  • Multiple node and multiple actor animation with textures

  • Show or hide bones/skeleton of the model.

https://user-images.githubusercontent.com/74976752/217645367-f901c6ed-ca20-40d6-92dd-f1cd8899ac7a.gif

Pull Requests:

Objectives in Progress#

PBR and emission materials in glTF#

The glTF format supports Physically based rendering also. PBR allow renderers to display objects with a realistic appearance under different lighting conditions, the shading model has to take the physical properties of the object surface into account. There are different representations of these physical material properties. One that is frequently used is the metallic-roughness-model. We have various material properties already in FURY, we need to apply it to glTF models as well.

Skinning for models with no indices#

The glTF format supports non-indexed geometry (e.g., the Fox model). We currently do not know how to render the model without indices. I tried estimating it in this branch. However, It fails to render in skinning.

Branch URL:

  • Rendering glTF with no indices: (in-progress) xtanion/fury

Other Objectives#

Fetcher for importing glTF files from Khronos-glTF-Samples#

The KhronosGroup/gltf-samples contain multiple glTF sample models to test a glTF viewer for free. Implemented new methods in fetcher that can load all of these models by (usind type) asynchronously. The glTF fetcher is capable of the following:

  • Downloading multiple models asynchronously.

  • Get the path to the downloaded model using it - Download any model using the URL of the model.

Pull Requests:

Other Pull Requests#

GSoC weekly blogs#

Timeline#

Date

Description

Blog Post Link

Week 0
(24-05-2022)

My journey to GSoC 2022

FURY - Python

Week 1
(20-06-2022)

A basic glTF Importer

FURY - Python

Week 2
(29-06-2022)

Improving Fetcher and Exporting glTF

FURY - Python

Week 3
(04-07-2022)

Fixing fetcher adding tests and docs

FURY - Python

Week 4
(12-07-2022)

Finalizing glTF loader

FURY - Python

Week 5
(19-07-2022)

Creating PR for glTF exporter and fixing the loader

FURY - Python

Week 6
(25-07-2022)

Extracting the animation data

FURY - Python

Week 7
(01-08-2022)

Fixing bugs in animations

FURY - Python

Week 8
(09-08-2022)

Fixing animation bugs

FURY - Python

Week 9
(17-08-2022)

First working skeletal animation prototype

FURY - Python

Week 10
(25-08-2022)

Multi-node skinning support

FURY - Python

Week 11
(31-08-2022)

Multiple transformations support and adding tests

FURY - Python

Week 12
(08-09-2022)

Adding skeleton as actors and fix global transformation

FURY - Python

Week 13
(15-09-2022)

Multi bone skeletal animations

FURY - Python

Week 14
(28-09-2022)

Morphing is here !

FURY - Python