Visualize the results of the Generate Signed Distance Fields for arbitrary meshes using both traditional methods and the one described in the DeepSDF paper

Used Rerun types

Tensor, Asset3D, Points3D, AnnotationContext, TextLog

Background

This example illustrates the visualization of the results obtained from generating Signed Distance Fields (SDFs) for arbitrary meshes using both traditional methods and the approach described in the DeepSDF paper. DeepSDF introduces a learned continuous representation of shapes using SDFs, enabling high-quality shape representation, interpolation, and completion from partial and noisy 3D input data. This novel approach offers improved performance and reduced model size compared to previous methods. The generated SDFs help with accurate 3D reconstruction and visualization.

Logging and visualizing with Rerun

The visualizations in this example were created with the following Rerun code:

3D asset

# Internally, `mesh_to_sdf` will normalize everything to a unit sphere centered around the center of mass.
bs1 = mesh.bounding_sphere
bs2 = mesh_to_sdf.scale_to_unit_sphere(mesh).bounding_sphere
scale = bs2.scale / bs1.scale
center = bs2.center - bs1.center * scale

# Logging the 3D asset with the unit sphere
mesh3d = rr.Asset3D(path=path)
mesh3d.transform = rr.OutOfTreeTransform3DBatch(rr.TranslationRotationScale3D(translation=center, scale=scale))
rr.log("world/mesh", mesh3d)

Sample SDF

The sampled points and their corresponding signed distances are visualized using the Points3D archetype within the world/sdf/points entity.

# Points inside the object are highlighted in red, while those outside are marked in green.
rr.log("world/sdf", rr.AnnotationContext([(0, "inside", (255, 0, 0)), (1, "outside", (0, 255, 0))]), timeless=False)

rr.log("world/sdf/points", rr.Points3D(points, class_ids=np.array(sdf > 0, dtype=np.uint8))) # Visualizing Sample SDF

Volumetric SDF

The computed distances for each voxel are visualized using the Tensor archetype to the tensor entity, which represents a 3D grid with dimensions for width, height, and depth.

rr.log("tensor", rr.Tensor(voxvol, dim_names=["width", "height", "depth"])) # Visualizing Volumetric SDF

Run the code

Known issue: On macOS, this example may present artefacts in the SDF and/or fail.

To run this example, make sure you have the Rerun repository checked out and the latest SDK installed:

# Setup
pip install --upgrade rerun-sdk  # install the latest Rerun SDK
git clone git@github.com:rerun-io/rerun.git  # Clone the repository
cd rerun
git checkout latest  # Check out the commit matching the latest SDK release

Install the necessary libraries specified in the requirements file:

pip install -r examples/python/signed_distance_fields/requirements.txt

To experiment with the provided example, simply execute the main Python script:

python examples/python/signed_distance_fields/main.py # run the example

You can specify the mesh:

python examples/python/signed_distance_fields/main.py --mesh {lantern,avocado,buggy,brain_stem}

If you wish to customize it, explore additional features, or save it use the CLI with the --help option for guidance:

python examples/python/signed_distance_fields/main.py --help