Progress bar while plotting.

[patakk]

Hello!

Recently I've started utilizing the new command-line API and I really love it. In combination with Raspberry Pi it really makes plotting more enjoyable since I only have to connect the Raspberry Pi to Axidraw and the rest can be done over ssh, and I can use my laptop for other tasks even when plotting (because with ssh and RPi there's no need to have my laptop next to Axidraw).

Anyway, one simple feature that I'd really love to have is a simple progress bar showing, well, the progress of the plotting process. I've implemented my own solution using tqdm python library and it does its job pretty well, and I've been thinking of making a pull request if anyone else would be interested in using this feature.
But before I do that, I think it would be good to get some input from @oskay on whether my approach makes sense. The explanation of my process follows bellow.

In order for any kind of progress bar to be possible, the program needs to know the total amount of steps (or the total amount of time) that's needed for the plotting process. The easiest way of obtaining such information is to first plot a drawing in preview mode, in order to calculate the needed information, and then export that information to that same file (I found that to be the most convenient). So, in order to enable progress bar while plotting, one should run axicli /path/to/file.svg --preview -o /path/to/file.svg prior to the actual plotting part. The exporting part of the steps/time data happens in the UpdateSVGWCBData method with the rest of the info that's being exported.
As for the data itself, I chose to make a simple counter that counts the total amount of steps needed to make the plot, and I increase it each time the doXYMove() is called. I figured putting it here gets me the most detailed info about how much steps the plot needs, but I'd be happy to hear better suggestions.

After exporting the steps data, in the actual plotting part the counter would again be counting the same thing at the same place, but in this non-preview mode the program would know the total amount of plotting steps (calculated in the previous step) and would know to update the progress bar accordingly. I put all this in one small function that, depending on the plotting mode either increases the counter (preview mode), or increases the counter + updates the progress bar (not-preview mode)

The biggest dilemma I had regarding all this is what should actually be the data we export and use for the progress bar. In my first attempt I exported the total amount of time needed for plotting (in seconds), and that's fine, but in the plotting part the program would need to periodically calculate the elapsed time and use that to adjust the bar, but that seemed less clean to me, so I went with the raw steps approach where I count the actual axidraw moves. The total duration of the plotting can be simply printed above the progress bar in order for the user to know how much it will take, and the bar itself would show the number of completed steps vs the total amount of steps needed (the default tqdm info).

As for the necessity of the initial preview run in which the data is calculated, we could maybe avoid this step by first checking if the necessary data exists in the svg file, and if not, the program runs the axicli /path/to/file.svg --preview -o /path/to/file.svg line before the plotting part, but I'm not exactly sure where would be the proper place to call this or if this is the right approach, so all suggestions are welcome.

Let me know how this all sounds to you!

[oskay]

I did see your issue this morning, and have not had time to finish a response yet.

This is a feature that has been on our roadmap for some time, but implemented somewhat differently.

[patakk]

I understand, and in that case feel free to decline my pull request. I decided to share it because it works well enough for me.

[oskay]

I appreciate you making the request.

We've had this on our roadmap for about a year, since a user request. We've made a lot of changes to the CLI since then, and it's not a terrible time to revisit it.

The "weight" of an additional dependency in terms of support requirements should not be underestimated. tqdm looks very good as far as these things go, but I am hesitant to add any library when one isn't absolutely positively required. We have had several software projects over the years paralyzed by issues with dependencies-- too many dependencies, too much reliance on a library that turns out to have issues, and/or poorly maintained libraries.

I like the idea of printing out progress. I am not convinced that a real progress bar is necessary-- a printout of time elapsed, time remaining, and percentage complete would (to me) cover all necessary data.

The time estimator built into the AxiDraw software is very good. The method that you're using to estimate progress is not necessarily well correlated to how long different things take. Not all plotted segments take the same amount of time to draw, and not all things that take time are plotted segments. Using the time may not be as "clean", but it is likely to be more accurate in most cases.

One thing that I need to give some additional thought to is whether there needs to be a CLI argument for showing the progress. It may be good enough to just have it on by default, if we provide an option to disable it in the conf file. Do you see any clear reason that people would need to turn it on and off?

Finally, the method that you're suggesting -- running the file twice -- is easy to implement, but is a hassle to the user. I think that users should expect to be able to run it once and have this feature work. We'll keep to our original plan of run the preview (within the CLI program) to find the time, and then run it again to plot showing progress.

[patakk]

Thank you for the reply.

Yeah, when I reconsider it, showing "time left" or similar outputs is a more accurate indicator of progress, and I agree on the rest of your points about not using tqdm.

About turning the indicator on/off, maybe someone wouldn't want any output whatsoever so that's why it may be useful to be able to turn it off, but I think it's a good idea to have it on by default.

[oskay]

Great. I'm going to close the pull request, but leave the issue open. I encourage you to continue to use this solution if it's working well for you, and I hope to add this as a supported feature in the near future.

[patakk]

Hey, I couldn't resist so I updated my progress bar to use time.time() instead of the "steps" as a measure of progress, and also replaced tqdm's progress "renderer" with my own that also prints accurate "time left".

What I was wondering is whether you could help me with one small issue (which actually manifested in my previous version too) regarding the frequency at which I'm updating the bar/message. As mentioned before, I only do it after the doXYMove() call, but I was wondering if you could suggest a better point in code, one which gets called more frequently, in order to get more frequent and consistent updates?

[oskay]

One possibility would be to spawn it in a separate thread.

If you need more frequent updates, I'd suggest searching for the phrase "pause before issuing next command" -- and add updates at sub-durations within those sleep periods.

[patakk]

I've just tried the separate thread approach that calls the update function at some frequency and it works wonderfully.

[StarsoftAnalysis]

Is progress information in axicli still being worked on?

[oskay]

It is still something that we are thinking about; we have discussed it regularly, including this past week. (We also had a related feature request recently, for adding webhooks to report progress, which we are considering as well.)

We were hoping to implement this in the GUI and CLI at the same time, but getting GUI things working (1) reliably (2) on all platforms has been a slog for multiple unrelated reasons.

We do have a proof of concept for a CLI-only progress bar, and I'll bump it to see if it can go in our next release.