Logging Data in Python
In this section we'll log and visualize our first non-trivial dataset, putting many of Rerun's core concepts and features to use.
In a few lines of code, we'll go from a blank sheet to something you don't see everyday: an animated, interactive, DNA-shaped abacus:
This guide aims to go wide instead of deep. There are links to other doc pages where you can learn more about specific topics.
At any time, you can checkout the complete code listing for this tutorial here to better keep track of the overall picture.
We assume you have working Python and
rerun-sdk installations. If not, check out the setup page.
For this tutorial you will also need to
pip install numpy scipy.
Initializing the SDK
Start by opening your editor of choice and creating a new file called
The first thing we want to do is to import
rerun and name the dataset we're working on by setting its
import rerun as rr rr.init("DNA Abacus")
Among other things, a stable
ApplicationId will make it so the Rerun Viewer retains its UI state across runs for this specific dataset, which will make our lives much easier as we iterate.
Check out the reference to learn more about how Rerun deals with applications and sessions.
Starting the Viewer
Next up, we want to spawn the Rerun Viewer itself.
To do this, you can add the line:
Now you can run your application just as you would any other python script:
(venv) $ python dna_example.py
And with that, we're ready to start sending out data:
By default, the SDK will start a viewer in another process and automatically pipe the data through. There are other means of sending data to a viewer as we'll see at the end of this section, but for now this default will work great as we experiment.
The following sections will require importing a few different things to your script. We will do so incrementally, but if you just want to update your imports once and call it a day, add the following to the top of your script right now:
from math import tau import numpy as np from rerun_demo.data import build_color_spiral from rerun_demo.util import bounce_lerp, interleave from scipy.spatial.transform import Rotation
Logging our first points
The core structure of our DNA looking shape can easily be described using two point clouds shaped like spirals. Add the following to your file:
# new imports from rerun_demo.data import build_color_spiral from math import tau NUM_POINTS = 100 # points and colors are both np.array((NUM_POINTS, 3)) points1, colors1 = build_color_spiral(NUM_POINTS) points2, colors2 = build_color_spiral(NUM_POINTS, angular_offset=tau*0.5) rr.log_points("dna/structure/left", points1, colors=colors1, radii=0.08) rr.log_points("dna/structure/right", points2, colors=colors2, radii=0.08)
Run your script once again and you should now see this scene in the viewer. Note that if the viewer was still running, Rerun will simply connect to this existing session and replace the data with this new recording.
This is a good time to make yourself familiar with the viewer: try interacting with the scene and exploring the different menus. Checkout the Viewer Walkthrough and viewer reference for a complete tour of the viewer's capabilities.
Under the hood
This tiny snippet of code actually holds much more than meets the eye...
Although the Rerun Python SDK exposes concepts related to logging primitives such as points, and lines, under the hood these primitives are made up of individual components like positions, colors, and radii. For more information on how the rerun data model works, refer to our section on entities and components.
Our Python SDK integrates with the rest of the Python ecosystem: the points and colors returned by
build_color_spiral in this example are vanilla
Rerun takes care of mapping those arrays to actual Rerun components depending on the context (e.g. we're calling
log_points in this case).
Entities & hierarchies
Note the two strings we're passing in:
These are Entity Paths, which uniquely identify each Entity in our scene. Every Entity is made up of a path and one or more Components. Entity paths typically form a hierarchy which plays an important role in how data is visualized and transformed (as we shall soon see).
One final observation: notice how we're logging a whole batch of points and colors all at once here. Batches of data are first-class citizens in Rerun and come with all sorts of performance benefits and dedicated features. You're looking at one of these dedicated features right now in fact: notice how we're only logging a single radius for all these points, yet somehow it applies to all of them.
A lot is happening in these two simple function calls. Good news is: once you've digested all of the above, logging any other Component will simply be more of the same. In fact, let's log everything else in the scene right now.
Adding the missing pieces
We can represent the scaffolding using a batch of 3D line segments:
# new imports from rerun_demo.util import interleave points = interleave(points1, points2) rr.log_line_segments("dna/structure/scaffolding", points, color=[128, 128, 128])
Which only leaves the beads:
# new imports import numpy as np from rerun_demo.util import bounce_lerp offsets = np.random.rand(NUM_POINTS) beads = [bounce_lerp(points1[n], points2[n], offsets[n]) for n in range(NUM_POINTS)] colors = [[int(bounce_lerp(80, 230, offsets[n] * 2))] for n in range(NUM_POINTS)] rr.log_points("dna/structure/scaffolding/beads", beads, radii=0.06, colors=np.repeat(colors, 3, axis=-1))
Once again, although we are getting fancier and fancier with our
there is nothing new here: it's all about building out
numpy arrays and feeding them to the Rerun API.
Animating the beads
Up until this point, we've completely set aside one of the core concepts of Rerun: Time and Timelines.
Even so, if you look at your Timeline View right now, you'll notice that Rerun has kept track of time on your behalf anyways by memorizing when each log call occurred.
Unfortunately, the logging time isn't particularly helpful to us in this case: we can't have our beads animate depending on the logging time, else they would move at different speeds depending on the performance of the logging process! For that, we need to introduce our own custom timeline that uses a deterministic clock which we control.
Rerun has rich support for time: whether you want concurrent or disjoint timelines, out-of-order insertions or even data that lives outside of the timeline(s)... you'll find a lot of flexibility in there.
Let's add our custom timeline:
# new imports from rerun_demo.util import bounce_lerp time_offsets = np.random.rand(NUM_POINTS) for i in range(400): time = i * 0.01 rr.set_time_seconds("stable_time", time) times = np.repeat(time, NUM_POINTS) + time_offsets beads = [bounce_lerp(points1[n], points2[n], times[n]) for n in range(NUM_POINTS)] colors = [[int(bounce_lerp(80, 230, times[n] * 2))] for n in range(NUM_POINTS)] rr.log_points("dna/structure/scaffolding/beads", beads, radii=0.06, colors=np.repeat(colors, 3, axis=-1))
A call to
set_time_seconds will create our new
Timeline and make sure that any logging calls that follow gets assigned that time.
⚠️ If you run this code as is, the result will be.. surprising: the beads are animating as expected, but everything we've logged until that point is gone! ⚠️
Latest At semantics
That's because the Rerun Viewer has switched to displaying your custom timeline by default, but the original data was only logged to the default timeline (called
To fix this, go back to the top of the file and add:
rr.spawn() rr.set_time_seconds("stable_time", 0)
This fix actually introduces yet another very important concept in Rerun: "latest at" semantics.
Notice how entities
"dna/structure/right" have only ever been logged at time zero, and yet they are still visible when querying times far beyond that point.
Rerun always reasons in terms of "latest" data: for a given entity, it retrieves all of its most recent components at a given time.
There's only one thing left: our original scene had the abacus rotate along its principal axis.
As was the case with time, (hierarchical) space transformations are first class-citizens in Rerun. Now it's just a matter of combining the two: we need to log the transform of the scaffolding at each timestamp.
Expand the previous loop to also include:
# new imports from scipy.spatial.transform import Rotation for i in range(400): # [...] rr.log_rigid3( "dna/structure", parent_from_child=( [0, 0, 0], Rotation.from_euler("z", time / 4.0 * tau).as_quat(), ), )
Other ways of logging & visualizing data
rr.spawn is great when you're experimenting on a single machine like we did in this tutorial, but what if the process that's doing the logging doesn't have a graphical interface to begin with?
Rerun offers several solutions for these use cases.
Logging data over the network
At any time, you can start a Rerun Viewer by running
python -m rerun. This viewer is in fact a server that's ready to accept data over TCP (it's listening on
0.0.0.0:9876 by default).
On the logger side, simply use
rr.connect instead of
rr.spawn to start sending the data over to any TCP address.
python -m rerun --help for more options.
Saving & loading to/from RRD files
Sometimes, sending the data over the network is not an option. Maybe you'd like to share the data, attach it to a bug report, etc.
Rerun has you covered:
rr.saveto save all the data logged so far to disk.
- Visualize it via
python -m rerun path/to/recording.rrd
You can also save a recording (or a portion of it) as you're visualizing it, directly from the viewer.
⚠️ RRD files don't yet handle versioning! ⚠️
This closes our whirlwind tour of Rerun. We've barely scratched the surface of what's possible, but this should have hopefully given you plenty pointers to start experimenting.
To go further, have a look at some of our other examples.