Improve split lines with lines #2864

bstroebl · 2016-03-03T11:22:15Z

No description provided.

bstroebl · 2016-03-03T13:04:47Z

why is the test failing again?

sbrunner · 2016-03-03T13:27:37Z

Why spaming everybody by closing and opening new pullrequest? And losing the discussion history by the way ...

bstroebl · 2016-03-03T15:03:27Z

Sorry for that, Matthias already pointed it out to me.

sbrunner · 2016-03-03T15:19:38Z

OK, sorry to blame you again :-)

m-kuhn · 2016-03-03T19:02:03Z

why is the test failing again?

Seems the CRS in your reference files and the one produced by running the test on travis don't match.

https://travis-ci.org/qgis/QGIS/jobs/113374867#L1294

assertLayersEqual
    _TestCase.assertEqual(self, layer_expected.dataProvider().crs().authid(), layer_result.dataProvider().crs().authid())
AssertionError: u'EPSG:4326' != u''
- EPSG:4326
+

Plus there are some indentation problems. Best to setup a pre-commit hook to avoid that in the future:
https://github.com/m-kuhn/QGIS-Website/blob/qtcreator.rst/source/site/getinvolved/development/git.rst#setup-pre-commit-hook

bstroebl · 2016-03-04T11:00:23Z

Hi Matthias,
thank you for looking into this, again. The gml files in expected now are the ones produced when running the algorithm from the toolbox and directly saving them (not outputting to temporary layers).
Where are indentation problems? In the Python code or in the yaml file?

m-kuhn · 2016-03-04T11:05:04Z

In the python file, have a look at the Details link as soon as the test finished for details.

But I would really recommend setting up the pre-commit git hook as explained in the link above, it will save you a lot of time.

bstroebl · 2016-03-04T11:29:54Z

But I would really recommend setting up the pre-commit git hook as explained in the link above, it will save you a lot of time.

When I (manually) runprepare-commit.sh it outputs No working copy, I enabled the hook nonetheless.

m-kuhn · 2016-03-04T11:36:40Z

Can you do now:

echo '' >> python/plugins/processing/algs/qgis/SplitLinesWithLines.py
git add python/plugins/processing/algs/qgis/SplitLinesWithLines.py
git commit

It should bring print a warning that there are indentation problems and correct them for you. If you re-add the file and commit, it should be all fixed and good.

jef-n · 2016-03-04T11:40:04Z

Better follow the tip on the dash page. The symbolic link is wrong in that forked branch.

bstroebl · 2016-03-04T11:45:34Z

Better follow the tip on the dash page. The symbolic link is wrong in that forked branch.

I already tried with autopep (that what you mean, Jürgen?) as proposed there, which resulted in the whole Python file being marked as changed when doing git diff

m-kuhn · 2016-03-04T11:50:50Z

Thanks for pointing out @jef-n
Since I hope to merge this forked branch to our developers guide soon I updated it there as well

bstroebl · 2016-03-04T12:00:49Z

Maybe two empty lines are not allowed?

jef-n · 2016-03-04T13:35:59Z

@bstroebl no, the symlink was wrongly created (see also d2c1668). And you need autopep8 1.2.1 (at least 0.9.1 is not enough - which would be what debian unstable currently ships with).

bstroebl · 2016-03-04T20:52:26Z

ok, seems like the indentation error is fixed now (two consecutive empty lines)
Any help with the test errors appreciated

m-kuhn · 2016-03-05T07:12:20Z

There is still a mismatch in the CRS

https://travis-ci.org/qgis/QGIS/jobs/113640270#L1298

bstroebl · 2016-03-07T06:45:22Z

There is still a mismatch in the CRS

Yes I have seen that in the test result but why? The CRS is declared in the gfs file and all features carry the CRS information, too. So the question is, why does layer_result.dataProvider().crs().authid() retun an empty string instead of EPSG:4326?

m-kuhn · 2016-03-07T10:55:23Z

This seems to be related to the .gfs file which is present in your expected data but none of the others. Was this .gfs file produced by processing or a leftover from your save-as approach?

It looks as if processing would not create this file and this leads to results end up without a properly specified CRS.

I'm still investigating if this is a processing, QGIS or test infrastructure issue.

bstroebl · 2016-03-07T11:22:12Z

Yeah, you are right, looks like a left over (by file date). I will remove and recreate all files in expected and try them as a new commit, ok?

m-kuhn · 2016-03-07T12:02:05Z

Yep, that would be good.

Can you also limit the coordinate comparison precision (the results I get here are slightly less precise)

    compare:
      geometry:
        precision: 7

bstroebl · 2016-03-07T14:39:29Z

done both, see what the test result will be

bstroebl · 2016-03-08T06:58:36Z

seems that the test crashes because of the proposed coordinate precision limitation(?)

m-kuhn · 2016-03-08T11:28:47Z

seems that the test crashes because of the proposed coordinate precision limitation(?)

why that? to me it looks like it's still the same CRS issue...

Interesting enough a very similar commit fixed things
m-kuhn@1859de8

bstroebl · 2016-03-08T12:10:27Z

why that? to me it looks like it's still the same CRS issue...

because the red cross marks this particular commit but checking the test result shows you are right

Interesting enough a very similar commit fixed things

How can I improve my PR?

m-kuhn · 2016-03-09T08:07:33Z

Not sure, I'll check

m-kuhn · 2016-03-10T11:09:41Z

So it seems it's failing for a good reason this time.

I have improved debug output for the tests, I'll push this soon. Meanwhile you can already work on fixing the algorithm.

Problem description is here: https://travis-ci.org/qgis/QGIS/jobs/115020339#L1316-L1320

bstroebl · 2016-03-10T11:30:06Z

Problem description is here: https://travis-ci.org/qgis/QGIS/jobs/115020339#L1316-L1320

Hihi, that's the code you proposed the other day :-)
Still, why is there no runtime error when running the algorithm in QGIS desktop with the same input? I could check additionally
if len(aLine.asPolyline()) > 2 or (len(aLine.asPolyline()) == 2 and aLine.asPolyline()[0] != aLine.asPolyline()[1]):

m-kuhn · 2016-03-10T11:33:58Z

I get an exception when I run it on QGIS desktop and when I run the tests locally

In the build folder :

ctest -R QgisAlg -V

Just push your changes to this branch, the next results should include the backtrace

bstroebl · 2016-03-10T11:50:28Z

ah, maybe it's because I use QGIS 2.14.0

m-kuhn · 2016-03-14T09:57:45Z

Manually merged

Thanks a lot for your patience

bstroebl · 2016-03-14T10:13:31Z

any hint why the tests threw errors?

m-kuhn · 2016-03-14T10:15:21Z

indentation
there were still the manually added expected result files instead of the ones produced from the test

bstroebl · 2016-03-14T10:35:41Z

indentation

hmm, I deleted the blank line and thought this fixed it, you mean the backslash that you deleted in d6fca7e ? Why is it not allowed?

there were still the manually added expected result files instead of the ones produced from the test

how can I create them with the test? Any link that explains what one has to do?

m-kuhn · 2016-03-14T10:45:18Z

hmm, I deleted the blank line and thought this fixed it, you mean the backslash that you deleted in d6fca7e ? Why is it not allowed?

No idea, autopep8 deleted it here and is happy now

how can I create them with the test? Any link that explains what one has to do?

https://github.com/qgis/QGIS/blob/master/python/plugins/processing/tests/README.md

bstroebl · 2016-03-14T10:54:53Z

the README says

Run the algorithm you want to test in QGIS from the processing toolbox.

That's what I did and it is is different from

result files ... produced from the test

I do not want to blame you (or anybody else) but as it is now I am reluctant to write more tests (and so are others as the processing tests thread in the dev ML shows). Or would you neverthelesss encourage me to write tests and produce pull requests (with probaly fainling tests) so you can debug?

m-kuhn · 2016-03-14T11:04:08Z

The readme also says

Redirect output to python/plugins/processing/tests/testdata/expected

didn't you create the output somewhere else and then re-save it or copy it to this location?

Or would you nevertheless encourage me to write tests and produce pull requests (with probaly fainling tests) so you can debug?

As I also said on the thread in the ML: absolutely!

I am glad you opened this pull request. As a result I spent quite a bit of time in improving the test infrastructure and now there's better debug output if a test fails.

bstroebl · 2016-03-14T11:10:49Z

didn't you create the output somewhere else and then re-save it or copy it to this location?

No (if I recall correctly) at least in the end I clicked the three-dots button and directly saved the result to testdata/expected as gml

As I also said on the thread in the ML: absolutely!

ok, so I will probably add some more tests (at least for the other algorithm I provided)

I am glad you opened this pull request. As a result I spent quite a bit of time in improving the test infrastructure and now there's better debug output if a test fails.

So how do I proceed with this PR? Shall I delete the branch or leave it as it is?

m-kuhn · 2016-03-14T11:13:52Z

No (if I recall correctly) at least in the end I clicked the three-dots button and directly saved the result to testdata/expected as gml

Strange, because here it does not create a .gfs file in this case but it does create a .xsd file - not sure if that's actually better but at least it's what succeeds on travis.

ok, so I will probably add some more tests (at least for the other algorithm I provided)

👍

So how do I proceed with this PR? Shall I delete the branch or leave it as it is?

The PR is merged and closed, so feel free to delete your branch.

bstroebl · 2016-03-14T11:18:05Z

Strange, because here it does not create a .gfs file in this case but it does create a .xsd file - not sure if that's actually better but at least it's what succeeds on travis.

OK, will see what the next test does

bstroebl · 2016-03-14T12:35:50Z

ok, next PR is ont its way

Bernhard Ströbl added 5 commits March 2, 2016 14:54

[BUG] Avoid trying to split a line with itself

4eb0d63

[BUG] Do not output lines shorter than 0.00001

f2bc101

[BUG] access geometry the right way

bec07bf

[BUG] Do not output lines with two identical points

0237b50

[processing] test cases for splitlineswithlines

ffaadb4

[processing] update test output files

2eac37e

Try to fix indentation error

f55c6ee

Bernhard Ströbl added 2 commits March 7, 2016 15:43

[processing] test: remove gfs output files

630bd80

[processing] test: limit coordinate comparison precision

4980ab7

bstroebl force-pushed the improveSplitLinesWithLines branch from 0a5ae6c to 4980ab7 Compare March 10, 2016 12:24

[BUG] make sure that result line consists of two nodes

bda0e19

m-kuhn closed this Mar 14, 2016

bstroebl deleted the improveSplitLinesWithLines branch March 14, 2016 12:35

Improve split lines with lines #2864

Improve split lines with lines #2864

Conversation

bstroebl commented Mar 3, 2016

bstroebl commented Mar 3, 2016

sbrunner commented Mar 3, 2016

bstroebl commented Mar 3, 2016

sbrunner commented Mar 3, 2016

m-kuhn commented Mar 3, 2016

bstroebl commented Mar 4, 2016

m-kuhn commented Mar 4, 2016

bstroebl commented Mar 4, 2016

m-kuhn commented Mar 4, 2016

jef-n commented Mar 4, 2016

bstroebl commented Mar 4, 2016

m-kuhn commented Mar 4, 2016

bstroebl commented Mar 4, 2016

jef-n commented Mar 4, 2016

bstroebl commented Mar 4, 2016

m-kuhn commented Mar 5, 2016

bstroebl commented Mar 7, 2016

m-kuhn commented Mar 7, 2016

bstroebl commented Mar 7, 2016

m-kuhn commented Mar 7, 2016

bstroebl commented Mar 7, 2016

bstroebl commented Mar 8, 2016

m-kuhn commented Mar 8, 2016

bstroebl commented Mar 8, 2016

m-kuhn commented Mar 9, 2016

m-kuhn commented Mar 10, 2016

bstroebl commented Mar 10, 2016

m-kuhn commented Mar 10, 2016

bstroebl commented Mar 10, 2016

m-kuhn commented Mar 14, 2016

bstroebl commented Mar 14, 2016

m-kuhn commented Mar 14, 2016

bstroebl commented Mar 14, 2016

m-kuhn commented Mar 14, 2016

bstroebl commented Mar 14, 2016

m-kuhn commented Mar 14, 2016

bstroebl commented Mar 14, 2016

m-kuhn commented Mar 14, 2016

bstroebl commented Mar 14, 2016

bstroebl commented Mar 14, 2016