Logging Data in Rust

In this section we'll log and visualize our first non-trivial dataset, putting many of Rerun's core concepts and features to use.

In a few lines of code, we'll go from a blank sheet to something you don't see everyday: an animated, interactive, DNA-shaped abacus:

This guide aims to go wide instead of deep. There are links to other doc pages where you can learn more about specific topics.

At any time, you can checkout the complete code listing for this tutorial here to better keep track of the overall picture.

Prerequisites

We assume you have a working Rust environment and have started a new project with the rerun dependency. If not, check out the setup page.

For this example in particular, we're going to need all of these:

[dependencies]
rerun = "0.2.0"
itertools = "0.10"
rand = "0.8"

While we're at it, let's get imports out of the way:

use std::f32::consts::TAU;

use itertools::Itertools as _;

use rerun::{
    components::{ColorRGBA, LineStrip3D, Point3D, Quaternion,
                 Radius, Rigid3, Transform, Vec3D},
    demo_util::{bounce_lerp, color_spiral},
    external::glam,
    time::{Time, TimeType, Timeline},
    MsgSender, MsgSenderError, Session,
};

Already you can see the two most important types we'll interact with:

• Session, our entrypoint into the logging SDK.
• MsgSender, a builder-like type that we'll use to pack our data in order to prep it for logging.

Initializing the SDK

The first thing we want to do is creating a Session and name the dataset we're working on by setting its ApplicationId:

fn main() {
    let session = Session::init("DNA Abacus", true);
}

Among other things, a stable ApplicationId will make it so the Rerun Viewer retains its UI state across runs for this specific dataset, which will make our lives much easier as we iterate.

Check out the reference to learn more about how Rerun deals with applications and sessions.

Starting the Viewer

Next up, we want to spawn the Rerun Viewer itself.

fn run(mut session: Session) -> Result<(), MsgSenderError> {
    Ok(())
}

fn main() {
    let session = Session::init("DNA Abacus", true);

    session.spawn(run).unwrap();
}

Now you can run your application just as you would any other Rust program:

$ cargo run

And with that, we're ready to start sending out data:

When using a spawn callback like we just did, the SDK will start a viewer on the main thread and execute your callback on another, automatically piping the data through. There are other means of sending data to a viewer as we'll see at the end of this section, but for now this will work great as we experiment.

Logging our first points

The core structure of our DNA looking shape can easily be described using two point clouds shaped like spirals. Add the following to your run callback:

const NUM_POINTS: usize = 100;

let (points1, colors1) = color_spiral(NUM_POINTS, 2.0, 0.02, 0.0, 0.1);
let (points2, colors2) = color_spiral(NUM_POINTS, 2.0, 0.02, TAU * 0.5, 0.1);

MsgSender::new("dna/structure/left")
    .with_component(&points1.iter().copied().map(Point3D::from).collect_vec())?
    .with_component(&colors1.iter().copied().map(ColorRGBA::from).collect_vec())?
    .with_splat(Radius(0.08))?
    .send(&mut session)?;

MsgSender::new("dna/structure/right")
    .with_component(&points2.iter().copied().map(Point3D::from).collect_vec())?
    .with_component(&colors2.iter().copied().map(ColorRGBA::from).collect_vec())?
    .with_splat(Radius(0.08))?
    .send(&mut session)?;

Run your program once again and you should now see this scene in the viewer.

This is a good time to make yourself familiar with the viewer: try interacting with the scene and exploring the different menus. Checkout the Viewer Walkthrough and viewer reference for a complete tour of the viewer's capabilities.

Under the hood

Although there's not that much code yet, there's already quite a lot that's happening under the hood.

Entities & hierarchies

Note the two strings we're passing in when creating our MsgSenders: "dna/structure/left" & "dna/structure/right".

These are Entity Paths, which uniquely identify each Entity in our scene. Every Entity is made up of a path and one or more Components. Entity paths typically form a hierarchy which plays an important role in how data is visualized and transformed (as we shall soon see).

Components

The Rerun Rust SDK works at a lower-level of abstraction than the Python one. In particular, when using the Rust SDK, you work directly with components instead of higher-level primitives.

By logging multiple components to an Entity, one can build up Primitives that can later be visualized in the viewer. For more information on how the rerun data model works, refer to our section on entities and components.

Logging components is a only a matter of calling MsgSender::with_component using any type that implements the Component trait. We provide a few of those).

Batches

One final observation: notice how we're logging a whole batch of points and colors all at once here. Batches of data are first-class citizens in Rerun and come with all sorts of performance benefits and dedicated features. You're looking at one of these dedicated features right now in fact: notice how we're only logging a single radius for all these points, yet somehow it applies to all of them.

A lot is happening in these two simple function calls. Good news is: once you've digested all of the above, logging any other Component will simply be more of the same. In fact, let's log everything else in the scene right now.

Adding the missing pieces

We can represent the scaffolding using a batch of 3D line segments:

let all_points = points1.iter().interleave(points2.iter()).copied();
let scaffolding = all_points
    .map(Vec3D::from)
    .chunks(2)
    .into_iter()
    .map(|positions| LineStrip3D(positions.collect_vec()))
    .collect_vec();
MsgSender::new("dna/structure/scaffolding")
    .with_component(&scaffolding)?
    .with_splat(ColorRGBA::from([128, 128, 128, 255]))?
    .send(&mut session)?;

Which only leaves the beads:

use rand::Rng as _;
let mut rng = rand::thread_rng();
let offsets = (0..NUM_POINTS).map(|_| rng.gen::<f32>()).collect_vec();

let (beads, colors): (Vec<_>, Vec<_>) = points1
    .iter()
    .interleave(points2.iter())
    .copied()
    .chunks(2)
    .into_iter()
    .enumerate()
    .map(|(n, mut points)| {
        let (p1, p2) = (points.next().unwrap(), points.next().unwrap());
        let c = bounce_lerp(80.0, 230.0, offsets[n] * 2.0) as u8;
        (
            Point3D::from(bounce_lerp(p1, p2, offsets[n])),
            ColorRGBA::from_rgb(c, c, c),
        )
    })
    .unzip();
MsgSender::new("dna/structure/scaffolding/beads")
    .with_component(&beads)?
    .with_component(&colors)?
    .with_splat(Radius(0.06))?
    .send(&mut session)?;

Once again, although we are getting fancier and fancier with our iterator mappings, there is nothing new here: it's all about building out vectors of Components and feeding them to the Rerun API.

Animating the beads

Introducing Time

Up until this point, we've completely set aside one of the core concepts of Rerun: Time and Timelines.

Even so, if you look at your Timeline View right now, you'll notice that Rerun has kept track of time on your behalf anyways by memorizing when each log call occurred.

Unfortunately, the logging time isn't particularly helpful to us in this case: we can't have our beads animate depending on the logging time, else they would move at different speeds depending on the performance of the logging process! For that, we need to introduce our own custom timeline that uses a deterministic clock which we control.

Rerun has rich support for time: whether you want concurrent or disjoint timelines, out-of-order insertions or even data that lives outside of the timeline(s)... you'll find a lot of flexibility in there.

Let's add our custom timeline:

let stable_time = Timeline::new("stable_time", TimeType::Time);

for i in 0..400 {
    let time = i as f32 * 0.01;

    let times = offsets.iter().map(|offset| time + offset).collect_vec();
    let (beads, colors): (Vec<_>, Vec<_>) = points1
        .iter()
        .interleave(points2.iter())
        .copied()
        .chunks(2)
        .into_iter()
        .enumerate()
        .map(|(n, mut points)| {
            let (p1, p2) = (points.next().unwrap(), points.next().unwrap());
            let c = bounce_lerp(80.0, 230.0, times[n] * 2.0) as u8;
            (
                Point3D::from(bounce_lerp(p1, p2, times[n])),
                ColorRGBA::from_rgb(c, c, c),
            )
        })
        .unzip();

    MsgSender::new("dna/structure/scaffolding/beads")
        .with_time(stable_time, Time::from_seconds_since_epoch(time as _))
        .with_component(&beads)?
        .with_component(&colors)?
        .with_splat(Radius(0.06))?
        .send(&mut session)?;
}

First we declare a name and type for our Timeline, then we pass it to MsgSender::with_time along with a timestamp for our data. You can add as many timelines and timestamps as you want when logging data.

⚠️ If you run this code as is, the result will be.. surprising: the beads are animating as expected, but everything we've logged until that point is gone! ⚠️

Enter...

Latest At semantics

That's because the Rerun Viewer has switched to displaying your custom timeline by default, but the original data was only logged to the default timeline (called log_time). To fix this, go back through the previous logging calls we made and add this:

.with_time(stable_time, 0)

This fix actually introduces yet another very important concept in Rerun: "latest at" semantics. Notice how, with our latest fix, entities "dna/structure/left" & "dna/structure/right" have only ever been logged at time zero, and yet they are still visible when querying times far beyond that point.

Rerun always reasons in terms of "latest" data: for a given entity, it retrieves all of its most recent components at a given time.

Transforming space

There's only one thing left: our original scene had the abacus rotate along its principal axis.

As was the case with time, (hierarchical) space transformations are first class-citizens in Rerun. Now it's just a matter of combining the two: we need to log the transform of the scaffolding at each timestamp.

Expand the previous loop to also include:

for i in 0..400 {
    // ...everything else...
    MsgSender::new("dna/structure")
        .with_time(stable_time, Time::from_seconds_since_epoch(time as _))
        .with_component(&[Transform::Rigid3(Rigid3 {
            rotation: Quaternion::from(glam::Quat::from_axis_angle(
                glam::Vec3::Z,
                time / 4.0 * TAU,
            )),
            ..Default::default()
        })])?
        .send(&mut session)?;
}

Voila!

Other ways of logging & visualizing data

Session::spawn is great when you're experimenting on a single machine like we did in this tutorial, but what if the process that's doing the logging doesn't have a graphical interface to begin with?

Rerun offers several solutions for these use cases.

Logging data over the network

At any time, you can start a Rerun Viewer by running rerun (you need to install it first: cargo install rerun). This viewer is in fact a server that's ready to accept data over TCP (it's listening on 0.0.0.0:9876 by default).

On the logger side, simply use Session::connect instead of Session::spawn to start sending the data over to any TCP address.

Checkout rerun --help for more options.

Saving & loading to/from RRD files

Sometimes, sending the data over the network is not an option. Maybe you'd like to share the data, attach it to a bug report, etc.

Rerun has you covered:

• Use Session::save to save all the data logged so far to disk.
• Visualize it via rerun path/to/recording.rrd

You can also save a recording (or a portion of it) as you're visualizing it, directly from the viewer.

⚠️ RRD files don't yet handle versioning! ⚠️

Closing

This closes our whirlwind tour of Rerun. We've barely scratched the surface of what's possible, but this should have hopefully given you plenty pointers to start experimenting.

To go further, have a look at some of our other examples.