Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternatives to Bezier #547

Closed
DJGoossens opened this issue Aug 5, 2021 · 18 comments
Closed

Alternatives to Bezier #547

DJGoossens opened this issue Aug 5, 2021 · 18 comments

Comments

@DJGoossens
Copy link

Hi. I really like Veusz. I am curious about how the smoothing seems to deliver some odd results at times. The attached image shows some lines drawn in Excel (top) and Veusz (bottom). I prefer Veusz over Excel almost always, but I think you can see that the smoothing here is Veusz is delivering some strange results. In particular, the big peak -- the line seems to go backwards, with the x value decreasing near (2015,0.125), which I think would be an arror in most circumstances. I cannot pretend to be able to offer a solution. If there is a sort of 'tension' parameter, that might help? Any advice welcome.

2smooths

@korintje
Copy link
Contributor

korintje commented Aug 9, 2021

I searched a bit about this issue.

Veusz's interpolation is internally treated by C++ code derived from Inkscape (veusz/veusz/helpers/src/qtloops/beziers.cpp).
Excel's interpolation algorithm is not officially open, but have been figured out by Brian Murphy.

Both algorithms are similar in many points:

  • Use Bézier Fitting
  • Use four Bézier control points (bz0, bz1, bz2, bz3) determined by four data points (i0, i1, i2, i3).
  • The start and end points of the Bézier control points (bz0 and bz3) are fixed at the positions of initial and final data points (i0 and i3).

As far as I glanced through the both codes, the difference between the two algorithms comes from the way of determining the positions of the remaining two control points. Inkscape's algorithm seems to optimize the positions of bz(1) and bz(2) by moving them on "tangent vectors" (Core functions: generate_bezier and estimate_lengths). On the other hand, Excel's algorithm seems to use the following rules (copied from Brian's Bezier example file ).

  • bz(1) = on a line through i1 parallel to i2-i0, at a distance from i1 = 1/6th the length |i2-i0|
  • bz(2) = on a line through i2 parallel to i1-i3, at a distance from i2 = 1/6th the length |i1-i3|
  • for bz(1), limit [1/6th the length of i2-i0] to never be more than 1/2 the length of |i2-i1|
  • for bz(2), limit [1/6th the length of i1-i3] to never be more than 1/2 the length of |i2-i1|
  • in cases where just bz(1) is being limited to |i2-i1|/2, for bz(2) reduce the length of [i1-i3]/6 by replacing it with (i2-i1)/2 * |i3-i1|/|i2-i0|
  • in cases where just bz(2) is being limited to |i2-i1|/2, for bz(1) reduce the length of [i2-i0]/6 by replacing it with (i2-i1)/2 * |i2-i0|/|i3-i1|
  • Also, endpoint intervals (designed by i0=i1 or i2=i3) are handled a little differently. In this case use 1/3 instead of 1/6.

Therefore, I think the issue may be solved by translating Brian's VBA code to beziers.cpp of Veusz. Unfortunately, I have no experience to write C++ code. So, it will be a tough work for me.

@DJGoossens
Copy link
Author

Wow, thanks for that very careful response. It seems to me that the Inkscape formula -- being from a drawing program -- will not necessarily include the constraint that the graph cannot go 'backwards', because a drawing does not have such a concept, whereas a graph does. That would seem to make the Excel-type approach more appropriate for a graphing program. I have not installed Veusz from source, nor coded in C++, so I can have a look but I would struggle to know where to start. I only know Fortran ... but I think this is a great step, knowing what to do. Thank you for a very clear explanation.

@korintje
Copy link
Contributor

korintje commented Aug 13, 2021

I think I have successfully implemented what we want, though it is still just under testing (and the code is still dirty).
Please see my test branch (https://github.com/korintje/veusz/tree/bezier) and check that whether it works fine or not.

Bezier fitting result by current ver. of Veusz

Bezier interpolation result by newly tested Veusz

Bezier interpolation by MS Excel

1

@korintje
Copy link
Contributor

korintje commented Aug 13, 2021

This seems to work well by a simple code described below. Such fitting-less method might be also beneficial in the aspect of performance.
But I am a complete novice in interpolation techniques, so helps or comments from other users and developers would be needed.

QPolygonF bezier_interpolate_Excel(const QPolygonF& data) {
  // Excel-like low-cost Bezier interpolation, translated from
  // https://blog.splitwise.com/2012/01/31/mystery-solved-
  // the-secret-of-excel-curved-line-interpolation/
  double tension=6.0
  const int len = data.size();
  if (len <= 1) {
    return QPolygonF();
  }else if (len == 2) {
    QPolygonF bezier_ctrls(4);
    bezier_ctrls << data[0] << data[1] << data[1] << data[1];
    return bezier_ctrls;
  }else{
    QPolygonF bezier_ctrls(4 * (len - 1));
    QPolygonF ctrls(4);
    QPointF pt0; 
    QPointF pt1;
    QPointF pt2;
    QPointF pt3;
    double f1;
    double f2;
    double d02;
    double d12;
    double d13;
    bool b1;
    bool b2;
    for (int i = 1; i < len; i++) {
      pt1 = data[i-1];
      pt2 = data[i];
      if (i == 1) {
        pt0 = data[i-1]; 
        pt3 = data[i+1];
        f1 = 1.0 / tension * 2;
        f2 = 1.0 / tension; 
      }else if  (i == len - 1) {
        pt0 = data[i-2]; 
        pt3 = data[i];
        f1 = 1.0 / tension;
        f2 = 1.0 / tension; 
      }else{
        pt0 = data[i-2]; 
        pt3 = data[i+1];
        f1 = 1.0 / tension;
        f2 = 1.0 / tension * 2; 
      }
      ctrls[0] = pt1;
      ctrls[3] = pt2;
      d02 = QLineF(pt0, pt2).length();
      d12 = QLineF(pt1, pt2).length();
      d13 = QLineF(pt1, pt3).length();
      b1 = d02 / 6.0 < d12 / 2.0;
      b2 = d13 / 6.0 < d12 / 2.0;
      if (b1 && b2) {
        ctrls[1] = pt1 + (pt2 - pt0) * f1;
        ctrls[2] = pt2 + (pt1 - pt3) * f2;
      }else if (!b1 && !b2) {
        ctrls[1] = pt1 + (pt2 - pt0) * d12 / 2.0 / d02;
        ctrls[2] = pt2 + (pt1 - pt3) * d12 / 2.0 / d13;
      }else if (!b1) {
        ctrls[1] = pt1 + (pt2 - pt0) * d12 / 2.0 / d02;
        ctrls[2] = pt2 + (pt1 - pt3) * d12 / 2.0 / d13 * d13 / d02;
      }else{
        ctrls[1] = pt1 + (pt2 - pt0) * d12 / 2.0 / d02 * d02 / d13;
        ctrls[2] = pt2 + (pt1 - pt3) * d12 / 2.0 / d13;
      }
      bezier_ctrls += ctrls;
    }
    return bezier_ctrls;
  }
}

I also added and examined a "Tension parameter" (I'm not sure this naming is appropriate) to control the curvature.
As can be seen in the following figures, Tension = 6.0-12.0 seems to work well.


@DJGoossens
Copy link
Author

That is great. I am working on being able to build Veusz but Debian stable which I use provides low versions of Qt and the build commands throw lots of errors to do with pyqt. I don't know how to fix it because the qt version is not something you can back port -- that would break existing things apparently I have to see if I can build it. But your changes really look like they have done a great job.

@DJGoossens
Copy link
Author

Thanks again. I am still trying to get a working development environment. I am not a software developer, and I am finding it very difficult. I am following https://github-wiki-see.page/m/veusz/veusz/wiki/DevStart but without success. If I ever get it to work, I'll report back!

@korintje
Copy link
Contributor

Can you tell me what error messages you are facing and what your OS?
Current "Dev start" recipie is fragile, so I should collect as many people's cases as possible.

@DJGoossens
Copy link
Author

DJGoossens commented Aug 20, 2021

Hi

I should preface this by saying I am very ignorant of development environments. I find the jungle of dependencies with tools like Python and Qt to be incomprehensible. It is entirely possible that the instructions are fine and that I am just too ignorant. I did get a working development environment, and have compiled the vanilla Veusz successfully. Please note that the computer in question had no development tools to speak of on it before I started.

Next I need to see if I can compile your version and text the new algorithm.

Veusz development environment on Windows

OK, we start with https://github-wiki-see.page/m/veusz/veusz/wiki/DevStart

Was unclear to me which bits of Qt I needed. Did I only need the MSVC 2017? Or did I need lots of stuff as well as MSVC 2017? In the end I installed the whole 12 GB of stuff. (The downloads needed to get this project running are very large. I guess their fine if you are a developer and work on several projects using Qt and so on, but for me the overhead was a shock. I spent many hours waiting for downloads to happen -- Qt and the MS C++ tools are very big downloads.)

In the end, I installed all 12 GB of the Qt stuff; after the 2 GB download I thought I might as well.

Added the QMAKE_EXE environment variable for my user. The path was not quite the same as in the docs, but it was easy enough to find
the correct one:

Windows button > type 'env' > Edit environment variables for your account > created QMAKE_EXE and made it point to directory and file as
noted. For the session I was already in, I ran:

C:\Users\username>set QMAKE_EXE=C:\Qt\Qt5.9.9\5.9.9\msvc2017_64\bin\qmake.exe

When I typed:

python3 -m venv sandbox

I got redirected to the MS store to install it. OK, something did not work. Maybe I need to add something to my path? No, eventually I worked out that the Python installer above just installs it as python.exe:

python --version
Python 3.9.6

So had to just use 'python' for 'python3'.

python -m venv sandbox
sandbox\Scripts\activate.bat

OK, that seemed to be working.

(sandbox)> pip install numpy sip==5.5.0 PyQt5 pyqt5-tools

Got a message to update pip:

(sandbox)> python.exe -m pip install --upgrade pip

Then continued the steps as given:

(sandbox)> pip install h5py astropy iminuit Ghostscript Sphinx

Watched lots of dependencies install.

(sandbox)> cd sandbox
(sandbox)> git clone https://github.com/jeremysanders/pyemf
(sandbox)> cd pyemf
(sandbox)> python setup.py install

Seemed OK. Now, it said we go to:

(sandbox) C:\Users\user\veusz>

but that does not exist. What step have I missed?

Ah, Windows instructions omit this:

(sandbox)> cd c:\Users\username
(sandbox)> git clone https://github.com/jeremysanders/veusz

I guess we do not issue it ftom within the sandbox folder? I guess not.

(sandbox)> cd veusz
(sandbox)> python setup.py build

Threw an error -- but of course this is an error only a complete nondeveloper would make. I did not have C++ installed from Microsoft. I guess you would reasonably assume this. Not me!

building 'veusz.helpers.threed' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with
"Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

OK, we go to that website. How big is this download?

OK, ran the installer from MS and installed the C++ tools. Not sure what was needed, so installed using defaults from this category.

Rebooted.

cd c:\Users\username
python -m venv sandbox
sandbox\Scripts\activate.bat
(sandbox)> cd veusz
(sandbox)> python setup.py build
(sandbox)> python setup.py install
(sandbox)> veusz

And it worked.

Summary: The supplied document gave enough information to get it to work, it just omitted a few steps that were probably seen as too obvious to need listing. (I am not a programmer.) The name of the Python binary in Windows as a small trap, and given the relatively compact size of Veusz when it is all done, the overhead of Python, Qt and MSVC is kind of large. I guess that is inevitable.

Getting it to work on Debian stable was more of a problem. In the end, I think it just cannot be done. The show stopper was the version of Qt5, which is too low on Debian stable.

Some output from Debian:

username@lauequad:~$ cat /etc/debian_version 10.10
username@lauequad:~$ uname -a
Linux lauequad 4.19.0-17-amd64 #1 SMP Debian 4.19.194-1 (2021-06-10) x86_64 GNU/Linux

$ python setup.py build
Traceback (most recent call last):
File "setup.py", line 60, in
import pyqtdistutils
File "/home/goossens/installs/veusz/veusz-3.3.1/pyqtdistutils.py", line 16, in
import PyQt5.QtCore
ImportError: /usr/lib/x86_64-linux-gnu/libQt5Core.so.5: version `Qt_5.15' not found (required by /usr/local/lib/python3.7/dist-packages/PyQt5-5.15.4-py3.7-linux-x86_64.egg/PyQt5/QtCore.abi3.so)

$ ls -l /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
lrwxrwxrwx 1 root root 20 Sep 14 2020 /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 -> libQt5Core.so.5.11.3

And of we look at the Debian package database, stable contains:

https://packages.debian.org/buster/libqt5core5a
Package: libqt5core5a (5.11.3+dfsg1-1+deb10u4)
If we look in unstable:
Package: libqt5core5a (5.15.2+dfsg-9)

And I am not prepared to change my whole distribution just to be able to compile Veusz. I expect Debian will move to the new libraries soon enough anyway.

@korintje
Copy link
Contributor

korintje commented Aug 20, 2021

Thank you for the detailed report, which must be very useful to improve the wiki.
"Dev Start" should literally be a start point, but the current version (which I wrote before) is not very friendly.
I would like to add some explanations, especially the part of developing in Windows which need a major revision.

I usually develop on Windows machine using WSL2 (Ubuntu). Setting up the development environment with WSL2 is easier, so I may add some descriptions for WSL2 case.

@DJGoossens
Copy link
Author

DJGoossens commented Aug 20, 2021 via email

@korintje
Copy link
Contributor

I revised https://github-wiki-see.page/m/veusz/veusz/wiki/DevStart
, reflecting your informative report.

I checked that I can make a devevopment environment on a fresh win 10 by this recipie.

@DJGoossens
Copy link
Author

That looks fantastic. Thanks again. I think Veusz is really great.

@DJGoossens
Copy link
Author

DJGoossens commented Aug 27, 2021 via email

@DJGoossens
Copy link
Author

Hi. I have managed to try your branch out (took me a while, I had a lot to learn, even just about using git). It looks great. If we refer back to the original image with which I began this thread, here is the same graph, with Bezier-tight:

Look fantastic.

AIR_10_01_5Aug21

Thanks again.

@korintje
Copy link
Contributor

Thanks for the report.
I updated the Wiki DevStart, and have sent a pull request for this issue.

@korintje
Copy link
Contributor

korintje commented Oct 2, 2021

@jeremysanders
Thanks for the review and merge!
I assume that we can close this issue now?

@jeremysanders
Copy link
Collaborator

Yes - I forgot

@DJGoossens
Copy link
Author

Great experience with the Veusz community. Many thanks. It is appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants