Append data and set data difference

[sreekarreddy21]

Hi, I am a little confused with cb_set_data and cb_append_data_point. Source code says, it "replaces current data" for cb_set_data, and "appends new data point" for cb_append_data_point. My understanding is that, when using cb_append_data_point, we can retain old data. For instance, if the plot widget is set to max_points of 600, the plot will start scrolling when it reaches max_points, but we will be able to pan and look at the past data if cb_append_data_point is used. Whereas, if we use cb_set_data, points in the view box will be replaced by a new set of data points when max_point is reached. Please let me know if this is correct. Thanks.

[domarm-comat]

Yes, you got it right. DataConnector has two methods to update plot data, cb_set_data and cb_append_data_point. By default cb_append_data_point is used whenever you have continues stream of data. It adds one point to the queue at the time. If you have 600 max_points points, it removes first point from the queue and adds new one in the end when point 601 is added. On the other hand cb_set_data can be used for non continuous data, such as one frame in the movie. In the current state of implementation is cb_set_data not used, but in the future I'm planning to implement displaying stream from camera or other sources. cb_set_data will be used for that purpose.
You can use cb_set_data manually if you want to replace all data in the plot (or reset it empty queue).

[sreekarreddy21]

Great, Is it possible to retain data when using cb_append_datapoint? I would like to pan and look at the past data when streaming is stopped. Do I need to modify the append_datapoint implementation to be able store past data? Or do we have any other way to hold old data? Thanks.

[domarm-comat]

No, in the current implementation you display everything what is in the queue. You can remove max_point parameter so no data will be thrown away, but that would slow down plotting performance drastically (if you want to plot in high frequencies). I'm currently working on a new version employing range_controller, which would enable you to plot only part of the collected data, pretty much doing what you want. But it still need some development and I'm planning to release it soon.

[domarm-comat]

You can try version 0.4.2, which supports scrolling view for X or Y axis. You don't have to reset or remove any data in DataConnector now (it's still preferable for performance to set some value for max_point of DataConnector). There might be possible issues as it's quite complex addition, but you can give it a try.

[sreekarreddy21]

Thank you, I just looked at the update. I will give it a try. Thanks again.

[sreekarreddy21]

Hi,
I tried plotting this, it is great. I just want to add a parameter to make the plotting look continuous (like the crosshair example). Currently, the view is shifting based on the roll_on_tick. I want to have a plot that holds like 30min of data (max_points), updating each second (could be roll_on_tick each second) and be able to see 15min of data in the view (tried to use offset, but that is interfering with roll_on_tick). Please let me know if there are any quick tweaks in the current code that could achieve this. If not, I will go through the code and work on implementing this. Thanks for the update.

[domarm-comat]

So the whole concept of new update is to calculate offset based on the space between ticks. If you base your data generator on time, you can then calculate how many ticks to set in max_points and how to set offsets.
In new update your view width is basically roll_on_tick size * space between two ticks. Then offset is calculated as a fraction of this view width (or height for Y axis).
So if you set offset_left to 1 it always pan left by view width. You can of course set it to higher number or negative number.

Let's say we have data generator set to 100Hz (100 ticks / second). We want to hold 2 minutes in our max_points and then show full one minute worth of data with roll_on_tick every second.

You would create:
LivePlotWidget(title="Line Plot @ 100Hz", x_range_controller=LivePlotWidget(title="Line Plot @ 100Hz", x_range_controller=LiveAxisRange(roll_on_tick=100, offset_left=60))

And data connector like this (assuming 100Hz data generator):
data_connector = DataConnector(plot, max_points=120000)

This would do the job. But there is one problem, that before we accumulate 60000 points we would see big empty space on the left side. That's normal, because that what our offset_left = 60 produces. x[0] - 60 * (x[max] - x[min]).
To avoid this, you can introduce small check to make sure, offset left is always cropped to at least x[0].

Try following example if it does what you need. You can then scale ticks sizes to your situation. If you really need to stick to external timers, then of course you need to change other things.

import pglive.examples_pyqt6 as examples
import signal
from threading import Thread
from copy import copy
from pglive.sources.data_connector import DataConnector
from pglive.sources.live_axis_range import LiveAxisRange
from pglive.sources.live_plot import LiveLinePlot
from pglive.sources.live_plot_widget import LivePlotWidget

"""
In this example Line plot is displayed.
"""

class LiveAxisRangeCropLeftOffsetToData(LiveAxisRange):

    def get_x_range(self, data_connector, tick):
        axis_range = data_connector.plot.data_bounds(ax=0, offset=self.roll_on_tick if self.roll_on_tick > 1 else 0)
        final_range = self._get_range(axis_range, tick, (self.offset_left, self.offset_right))
        if final_range is None:
            return self.final_x_range
        else:
            """
            This makes our range to always crop left view to the first point of x until 
            buffer is filled to offset_left * roll_on_tick size.
            """
            if final_range[0] < data_connector.plot.getData()[0][0]:
                final_range[0] = data_connector.plot.getData()[0][0]
            offset_x = data_connector.plot.pos().x()
            final_range[0] += offset_x
            final_range[1] += offset_x
        self.x_range[data_connector.__hash__()] = copy(final_range)
        for x_range in self.x_range.values():
            if final_range[0] > x_range[0]:
                final_range[0] = x_range[0]
            if final_range[1] < x_range[1]:
                final_range[1] = x_range[1]
        if self.final_x_range != final_range:
            self.final_x_range = final_range
        return self.final_x_range

"""
Roll view every second (100 ticks)
If Offset_left = 1 is always equal roll_on_tick * offset_size so in this case offset_left = 1 would be view size of 100 ticks.
Set offset left to 60 * 100 ticks = 60 seconds.
"""
win = LivePlotWidget(title="Line Plot @ 100Hz", x_range_controller=LiveAxisRangeCropLeftOffsetToData(roll_on_tick=100, offset_left=60))
plot = LiveLinePlot()
win.addItem(plot)

# Hold 120 seconds of data (120000 ticks)
data_connector = DataConnector(plot, max_points=120000)

win.show()

Thread(target=examples.sin_wave_generator, args=(data_connector,)).start()
signal.signal(signal.SIGINT, lambda sig, frame: examples.stop())
examples.app.exec()
examples.stop()

[domarm-comat]

I've added option to plot data with range controller and with version 0.4.4 it's also possible to plot multiple plots inside one view with different plot rates (one_plot_multiple_plot_rates.py example). I will close this issue as resolved. Thank you for your interest and ideas, which helped to improve pglive and make it more useful for others.