homebrew compatibility (qt 5.10)

[thomaslima]

I would like to help with building klayout with qt 5.10. I use a mac, and would use the source from brew: https://github.com/Homebrew/homebrew-core/blob/master/Formula/qt.rb

The first step is to regenerate those gsiqt5 headers in klayout. I have tried the script in /scripts/mkqtdecl.sh but so far I have issues in the parsing step. Essentially qt5 uses c++11 and the grammar in /scripts/mkqtdecl_common/c++.treetop fails to parse the headers. For example, here's my output:

$ ./scripts/mkqtdecl.sh -qt5 -dry
Running gcc preprocessor ..
Stripping hash lines ..
Parsing and producing db ..
Failure in line: 5222
Expected one of [ \t\n\r], "const", "__const", "volatile", "__volatile", "throw", "__attribute__", "__asm", "__extension__", "[", "(", "=", ",", ":", "{", ";", <semantic predicate> at line 5222, column 105 (byte 158859) after __attribute__ ((__visibility__("default"))) void* operator new(std::size_t __sz, const std::nothrow_t&)

and here's the offending line in allofqt.e:

__attribute__ ((__visibility__("default"))) void* operator new(std::size_t __sz, const std::nothrow_t&) noexcept __attribute__((__malloc__));

the grammar in c++.treetop expects a throw in a function declaration:

  rule func_spec
    "(" s fa:( a:func_args )? s ")" cvspec:( s cv:cv )? ( s "throw" s "(" s ( type_wo_comma s )? ")" )* a <PFuncSpec>
  end

, but c++11 deprecated throw in favor of noexcept. http://en.cppreference.com/w/cpp/language/noexcept_spec

That was not the first occurrence of noexcept in allofqt.e. For example,

__attribute__ ((__visibility__("default"))) new_handler get_new_handler() noexcept;

passed without a problem, though I don't know why. So my understanding is that the grammar is inconsistent. Somebody compiled a full c++11 grammar in html: https://www.codeproject.com/Tips/325689/Cplusplus-and-Cplusplus-Syn

Maybe rewriting the c++.treetop grammar is a good way to start. It's a huge endeavor, so I would ask for advice and ideas. What do you think, @klayoutmatthias ? Did you write the current grammar on your own?

[thomaslima]

Here's a big advantage if we get this to work. python3, ruby and qt 5 are all available in homebrew and they are relatively well maintained. If we managed to make the build steps compile properly, we could deploy klayout as a brew formula! Installation would be a lot more straightforward for mac users. Especially for those who depend on particular python packages and versions, like me.

PS: this could solve some of the issue in #67 .

[thomaslima]

On maintaining a klayout tap: https://github.com/Homebrew/brew/blob/master/docs/How-to-Create-and-Maintain-a-Tap.md

[klayoutmatthias]

Hi Thomas,

don't try this. The Qt binding headers usually don't need to be rebuilt.

The headers are not built dynamically and they need to compile on Windows and a variety of Linuxes. The API basis is Qt 5.5.1 which is where I derived the headers from. My goal is not to provide a full-flavoured Qt binding binding. So I am providing some API version which compiles on the other Qt's. This treetop parser is just saving me the effort of having to create the bindings manually. It's a minimum parser that allows extracting the Qt method signatures, not more.

Is there a specific reason you need Qt 5.10 bindings?

Matthias

[thomaslima]

Not really. I will try compiling with 5.10 and can report the errors here. I thought re-binding would be crucial to this step. To be honest I don't particularly understand the purpose of the "gsi" source code files. And again I am no expert in qt or even gui development...

[thomaslima]

This commit 8bec896 also fixed rendering issue in Qt5.10. FYI @Kazzz-S @lukasc-ubc .

[klayoutmatthias]

What exactly are the errors that happen for Qt 5.10 with the sources from GitHub? Anything I can fix within the normal sources?

[thomaslima]

Hi Matthias. Previously when I compiled with Qt5.10, not only the screenshots didn't display the colors, but also the layout canvas was blank. With that fix, that was gone. So now qt5.10 is working fine with me. My installation process is very simplified now: just brew install python3 qt, then python3 ./build4mac.py -p B36 -q Qt5Custom.

It even works in Travis CI using one of their osx images, but the compilation lasts longer than 55min, their build limit for opensource projects. If we fix that, we can have automatic klayout build not only for Mac but also for windows and linux. This is why I opened #92

Here, I just want to make a stable build script for homebrew so we can deploy as a formula, so people can just install with brew install klayout, and brew would figure out the dependencies.

[thomaslima]

Another update:

• I have created a pull request in (macOS) Adding support for qt and python3 installed from brew #97 with a stable build script from homebrew dependencies.
• I submitted a job to travis with commit 8795f82 . It built and deployed fine! But some unit tests failed. I'd like some helping parsing them.

klayout_unittest_log_8795f82.txt

[klayoutmatthias]

Hi Thomas,

Some unit tests are skipped: long runners (you can enable them with 'ut_runner -s') and a few for which I'm not supposed to publish the test data (yet) :-(

Others may be skipped if Ruby or Python or Qt support isn't compiled in.

The other fails I have seen on the MacOS VirtualBox of mine but waived as minor issues:

The four OASIS tests fail because the object's order is different. That's not a true issue as OASIS does not imply a certain order of objects. I assume that's an effect of the STL implementation: Linux + Windows use gcc's libstdc++ even with clang, while MacOS uses libc++ from clang. But it spoils the unit tests here.

The klayout_main test fails because it tries to lookup the "klayout" main executable in the current directory. It should pass if you cd to the build directory first.

The pya:dbTransTest fails because of a rounding issue (or rather non-rounding issue) I have not seen so far. Probably not a real issue, but maybe also an effect of different runtime libraries.

And the two ThreadedWorker tests are notorious for failing on different systems or even sporadically. Threading is subject to deviations and unpredicability, so that's to be expected. I'm not able to fully bring them down on all systems myself.

Overall the yield is pretty good already. No serious fails. And when the long runners pass as well, I'd say this is well acceptable result.

Best regards,

Matthias

[thomaslima]

Hi Matthias, here's the log while running ut_runner -s. 18 tests failed... I've also symlinked klayout to the same folder. Thank you for the analysis!

klayout_unittest_log_49d672b.txt

[klayoutmatthias]

Is this problem still active? I'm trying to clean up my open issue list :-)

[thomaslima]

I was waiting for you to check the above log. if these unit tests failures are acceptable, I will continue with the travis CI work. I've been in contact with them to allow for extra build time, and they granted it. So I should try that out more later.

Please take a look at the log above and close the pull request.

[thomaslima]

I'm closing this because I've been able to consistently compile klayout with qt5.11. Later on we can do a major test-fixing hackaton.