SDK Micro Batching
The Rerun SDK automatically handles micro-batching in a background thread in order to find a sweet spot between latency and throughput, reducing metadata overhead and thus improving both bandwidth and CPU usage.
The flushing is triggered by both time and space thresholds, whichever happens to trigger first.
You can configure these thresholds using the following environment variables:
Sets the duration of the periodic tick that triggers the time threshold, in seconds.
Sets the size limit that triggers the space threshold, in bytes.
Sets the number of rows that drives the space threshold.