Visualisation of fixed-point signals as Analog

[umarcor]

First off, thanks a lot for keeping this awesome tool alive for +20 years!

I'm using v3.3.104 installed through MSYS2 (Windows 10). I'm trying to visualise the signals of a closed-loop control system. These are modelled in VHDL as std_logic_vector, simulated with GHDL and saved to a GHW file. Inside some components, ieee.fixed_generic_pkg is used to perform arithmetic operations. Since these type of signals are typically seen as analog signals (see http://ctms.engin.umich.edu/CTMS/index.php?example=MotorPosition&section=ControlPID) I'm using the Analog -> Step Data Format of GtkWave. I'm finding it cumbersome to work with these, and I feel I might be missing something. Hence, these are just some questions regarding the user experience:

• I found that the vertical resizing can be adjusted depending on the data in the screen or to all data. However, it seems not to be possible to adjust it to the actual range of the signal. For instance, for an std_logic_vector of 8 bits, I'd expect the range to be [0,255] (unsigned) or [-128,127] (signed), regardless of the actual values that were logged.

• The fixed-point format can be seen in the range of the signals. For example, signal d : sfixed(4 downto -4); is shown as d[4:-4] in GtkWave. However, the Fixed Point Shift needs to be specified manually. I'd expect to be able to get the number of shifts from the right index of the range. Even if it needs to be manually set, it'd be helpful to have the value preloaded (as a default suggestion).

• It seems not possible to see the binary value of a signal, if the Data Format is set to any other option. I believe it would be handy to see the actual binary representation of the numbers that are been otherwise shown as floats.

• When comparing Analog outputs of multiple blocks, it would be useful to visualise them on the same axis. This is common in control applications where a target needs to follow a given setpoint or trajectory.

• I think it makes sense for the height of analog signals to be a multiple of the height of a regular digital row. However, it doesn't feel intuitive to move an analog signal along with the corresponding additional spacers. One needs to select the first hone, then hold Shift and move the last one. I found that moving any other, or selecting them in a different order won't work. I think it would be more intuitive if the height was a parameter of the signal (just as the Data Format), so that moving the first row would take others with it.

• It would be handy to perform arithmetic operations on two or more signals, similar to oscilloscopes.

• I found a reference to X11 colors in the docs, which points to https://github.com/gtkwave/gtkwave/blob/master/gtkwave3-gtk3/src/rgb.c. However, it seems not to be possible to set one of those colors as the signal color through the GUI. Is it limited to TCL scripting and/or to set the background?

Once again, thanks for maintaining this great tool!

[umarcor]

For the record, I evaluated PulseView as a complement to GtkWave for visualization of "analog channels" (i.e. real/float values or fixed-point types). See Reading waveforms from HDL simulators with PulseView. The TL;DR is that sigrok/PulseView do not support multi-bit signals, hierarchy or 9-value logic, and they have serious performance issues with small time scales (fs, ps or even ns). However, protocol decoders written in Python are a really nice feature.

[gtkwave]

Thanks for pointing me to your article. Memory management in wave viewers seems to be neglected until the software hits a memory brick wall and a redesign is necessary. I was lucky because 2GB limits causing an out of memory crash was a very real problem I hit quickly back when code was predominantly 32-bit. I hit the memory/performance problem and everyone else does as well. At some point I need to put together a "Ted Talk" of sorts regarding dumpfile formats, memory management, and viewer performance. None of it is rocket science, but the problems encountered haven't really been documented, mainly because there are probably half a dozen to a dozen viewers out there tops which means the number of experienced developers isn't all that large, either, and it's not in the interest of commercial tools to arm open source competition with knowledge. Regarding commercial tools, I will say that the Novas guys were probably about 10 years ahead of open source until about 2009. I think the key insight they stumbled onto is building the waveform viewer around (the interface to) a very efficient database/dumpfile format, not the other way around where one would just interface the viewer to an existing reader or whatever format needed to be read.

…

-Tony On Thursday, May 7, 2020, 03:25:22 AM EDT, umarcor <notifications@github.com> wrote: For the record, I evaluated PulseView as a complement to GtkWave for visualization of "analog channels" (i.e. real/float values or fixed-point types). See Reading waveforms from HDL simulators with PulseView. The TL;DR is that sigrok/PulseView do not support multi-bit signals, hierarchy or 9-value logic, and they have serious performance issues with small time scales (fs, ps or even ns). However, protocol decoders written in Python are a really nice feature. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[umarcor]

Tony, thanks a lot for taking time to provide your very insightful comments. I believe there is lots of knowledge in this project's codebase, and I find painful to see it getting superficially old because of contraints imposed by the toolkit, at the same time that other projects seem to step on the same stones that you did many years ago.

Regarding your idea of a "Ted Talk" (maybe a FOSDEM talk?), I would love to see any related content. Maybe it exists already, but I didn't find it :(. Shall I expect to find "guidelines for developers" anywhere in the docs or the codebase?

Overall, it feels to me that GtkWave is both a C toolkit for dumpfile formats and a viewer. Does GtkWave implement some mechanism as what you described about Novas? I'd like to know whether it is possible to extract a kind of API for dumpfile parsing, seeking, rewinding, keeping it in memory or in file, etc. How do you feel about this 30s (talk referenced in #12)?

The idea of starting some "waveform post-processing tool" by reusing GtkWave's codebase has been bouncing in my head for long. Either for chopping VCD or GHW files, or for converting them to other formats. For instance, metrics of web services are kind of similar to dumpfiles and in the last years nice web frontends (https://play.grafana.org/d/000000016/graph-styles) and specilized databases (https://github.com/influxdata/influxdb) have been developed. Grafana feels quite reponsive, even used remotely. Since screens are no wider than 1-4k pixels, I wonder if performance of the database would suffice. To write an exporter, I'd like to know whether GtkWave has some "unique" internal format that unifies the different support input formats, or conversely, if each format is parsed to some data model specific for Gtk.

Once again, thanks for your time and congratulations for this great tool.

[gtkwave]

The easiest thing to do is to interface directly with the FST reader API code. Mainly, you call an function that calls your callback function and you process the read data accordingly. Looking at fst.c will show how to call the fstReader functions. It's fairly easy. FSDB's reader is similar and you can see how I call it in extload.c. The main difference is they don't do callbacks. The gtkwave viewer needs to massage some of the data for its own internal structs so having to do extra work isn't really advisable where gtkwave would function as a C toolkit to get to waveform data. There actually is a Tcl feature that does that already as I added it for a guy around 5-10 years ago, but I haven't really used it. He was working on his own Verdi-like environment with time vs signal annotation in sourcecode. Having gtkwave as a C toolkit would be on my to-do list if I developed EDA tools as my day job. I write/debug RTL so the tool gets shaped into what I need to get my job done. Mainly, what a viewer would want to do is cache signals in their entirety when imported into the viewer and keep them around for some amount of time after the last instance of them has been deleted from the wave window. For rendering speed, that Grafana example is acceptable, but for my day job I would get frustrated quickly with something that runs that slow*. For those writing viewers for large amounts of data, the main gist is: 1) Query a database to get the signal *in its entirety*. This is mainly because you'll need it all at full zoom anyway. The trickiest part of the database is that you don't want to have to traverse all the data to get to a signal. Appendix F in the user manual in doc/ explains how to do this in detail. FSDB employs some of the same methods I use for FST. 2) Store the data in some time vs data pair structure that you have random access into such as an array. This can chew up a lot of memory. You need to do this because you do not want to be traversing a multi-GB dumpfile in real-time as a user scrolls around. 3) Make sure you map time vs pixels and pixels vs time values to speed up overdraws. So if you are zoomed way out, you can skip to the time value corresponding to the next pixel on screen. I don't know if any other viewers do this. There are a lot of ways to speed up drawing. 4) The biggest enemies here are TLB misses so you really might want to employ your own malloc/free (or use something like slabs) in order to make sure small allocations exist in very large malloc() pools that don't need a 4k TLB entry for every page. If you do the above, you will have something quite usable. I use gtkwave for my day job and have not come across any limitations holding me back for quite a long time. I'll put together a presentation/discussion sometime. I was asked late last year to do something like that this spring for an OSS conference, but I had to get clearance from the legal department at work first, and I would've missed the yes/no deadline so I had to decline for this year.

…

-Tony (*) When I say "slow," this is a relative term. If you run the GTK3 version of gtkwave on a touchscreen-equipped laptop, you can use your finger to fling the wave and it will scroll through time/decelerate like people are accustomed to seeing scrolling on a tablet. I can achieve full frame rate if we're zoomed in at a level where the text for multi-bit vectors is more-or-less visible. I doubt Grafana can achieve that, but its design points are completely different than mine. On Thursday, May 7, 2020, 05:40:33 PM EDT, umarcor <notifications@github.com> wrote: Tony, thanks a lot for taking time to provide your very insightful comments. I believe there is lots of knowledge in this project's codebase, and I find painful to see it getting superficially old because of contraints imposed by the toolkit, at the same time that other projects seem to step on the same stones that you did many years ago. Regarding your idea of a "Ted Talk" (maybe a FOSDEM talk?), I would love to see any related content. Maybe it exists already, but I didn't find it :(. Shall I expect to find "guidelines for developers" anywhere in the docs or the codebase? Overall, it feels to me that GtkWave is both a C toolkit for dumpfile formats and a viewer. Does GtkWave implement some mechanism as what you described about Novas? I'd like to know whether it is possible to extract a kind of API for dumpfile parsing, seeking, rewinding, keeping it in memory or in file, etc. How do you feel about this 30s (talk referenced in #12)? The idea of starting some "waveform post-processing tool" by reusing GtkWave's codebase has been bouncing in my head for long. Either for chopping VCD or GHW files, or for converting them to other formats. For instance, metrics of web services are kind of similar to dumpfiles and in the last years nice web frontends (https://play.grafana.org/d/000000016/graph-styles) and specilized databases (https://github.com/influxdata/influxdb) have been developed. Grafana feels quite reponsive, even used remotely. Since screens are no wider than 1-4k pixels, I wonder if performance of the database would suffice. To write an exporter, I'd like to know whether GtkWave has some "unique" internal format that unifies the different support input formats, or conversely, if each format is parsed to some data model specific for Gtk. Once again, thanks for your time and congratulations for this great tool. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[umarcor]

For rendering speed, that Grafana example is acceptable, but for my day job I would get frustrated quickly with something that runs that slow*.

Agree. It's mostly a demo strategy to fight a battle at a time: dump formats first, viwer toolkits then.

Appendix F in the user manual in doc/ explains how to do this in detail.

I had completely overlooked the content of Appendix F!

Thanks again for your explanations.