Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance CI building for Mingw-w64 (Part 1) #4071

Closed

Conversation

SlySven
Copy link
Member

@SlySven SlySven commented Sep 16, 2020

This adds threefour shell scripts that try to replace some PowerShell ones to produce both 32 and 64-bit Windows packages. At present they should each produce a zip archive file containing all the files needed to execute the Mudlet application on a Windows PC that does NOT have MSYS2 installed in it. Further work is needed to get executable installers built from that set of files and to upload them to the normal location - at present they will be available as build artefacts on the AppVeyor CI platform...

Also: add some more existing files to QMake project file, so that they show up in the Qt Ctreator IDE!

Signed-off-by: Stephen Lyons slysven@virginmedia.com

So that they show up in Qt Ctreator IDE!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
This adds three shell scripts that try to replace some powershell ones to
produce both 32 and 64-bit Windows packages. At present they should each
produce a zip archive file containing all the files needed to execute the
Mudlet application on a Windows PC that does NOT have MSYS2 installed in
it. Further work is needed to get excecutable installers built from that
set of files and to upload them to the *normal* location - at present they
will be available as build artifacts on the AppVeyor CI platform...

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@SlySven SlySven requested a review from a team September 16, 2020 23:09
@SlySven SlySven self-assigned this Sep 16, 2020
@add-deployment-links
Copy link

add-deployment-links bot commented Sep 16, 2020

Hey there! Thanks for helping Mudlet improve. 🌟

Test versions

You can directly test the changes here:

No need to install anything - just unzip and run.
Let us know if it works well, and if it doesn't, please give details.

…able

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
in the past I had used a trick in a script to get past an issue caused by
an overlarge font resources file that caused compilation failures. As that
file is no longer present the explict mention of it broke things!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Copy link
Member

@vadi2 vadi2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this start! Could you rename the files so they keep the same naming structure as the macOS and Linux ones? So setup, build, after_success and such.

@SlySven
Copy link
Member Author

SlySven commented Sep 17, 2020

Could you rename the files so they keep the same naming structure as the macOS and Linux ones? So setup, build, after_success and such.

I thought I had (apart, obviously from the file extension as they are shell scripts {NOT bash ones BTW}) - except for a new "after_build" one - which is not done in the same place in the CI process as the "after_success" stage ...?

@SlySven SlySven changed the title Enhance ci building for mingw w64 part1 Enhance CI building for Mingw-w64 (Part 1) Sep 17, 2020
@vadi2
Copy link
Member

vadi2 commented Sep 17, 2020

after_build does what after_success does now, right?

…uilds

I had forgotton it!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@SlySven
Copy link
Member Author

SlySven commented Sep 17, 2020

Not completely - I need it currently as that is the point at which the build artefacts (a .zip file) can be captured and made available in the AppVeyor build web-pages. It may be that after_success could be used but that wasn't clear to me when I first came up with this set of scripts...

@vadi2
Copy link
Member

vadi2 commented Sep 17, 2020

Okay - well when it's ready, I'd like it to be consistent with other build system setup so the entire thing is easier to work on.

So that the envirmental variables needed are available in all parts of the
CI build.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Using the `--release` option breaks things!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@SlySven
Copy link
Member Author

SlySven commented Sep 17, 2020

Oh blast, windeployqt.exe is borked since Qt 5.14 - the --release option doesn't work (it seems to be assumed unless --debug is given and it breaks if it is given): https://bugreports.qt.io/browse/QTBUG-84567

Because of the wrong order the script was using a `tr` from a non-MSYS2
element that is also present on the CI platform (Cygwin) and there was
a dll mis-match.

Also included a missing project related environment variable in the build
environment that was needed.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
I had copy/pasted a script from the Travis CI arrangement and forgotten to
tweak the CI variable names to suit!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@SlySven
Copy link
Member Author

SlySven commented Sep 17, 2020

Okay - main problem with those Windows builds is that they do not contain the Lua translation files that would be found in
the ./translations/lua/translated/ folder (relative to the executable) of a Window's installation.

This might actually also be an issue for Linux packagers - as those files are NOT generated by the build process and won't be present in the source-code Edit: they are, but are not listed in the qmake file so do not show up in the Qt Creator IDE. For comparison the main mudlet_xx_YY.qm are generated from the corresponding mudlet_xx_YY.ts files during the build process (qmake or cmake based) but the latter are only there because we have a semi-automated process to get them from CrowdIn every so often - it looks as though that process needs to be extended to include those Lua ones Edit: it seems like they are!

Just added another commit to address this issue!

Also add them to qmake project file so that they show up in Qt Creator!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@vadi2
Copy link
Member

vadi2 commented Sep 18, 2020

Also upload to make.mudlet.org/snapshots instead of appveyor, that's what we use :)

I don't seem to be having as much sucess with rsync as I had in the past,
maybe because I was trying to be a bit more selective in my file selection.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
…ravis]

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Python 2.x has finally left the MSYS2 / Mingw-w64 project - so we just need
to specify python to get python 3.x as asking for python2 no longer works.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
`ntldd` was missing from package list but since I am now using it to
identify the shared libraries that Mudlet needs besides the Qt and some
other ones loaded dynamically.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
CI/appveyor.after_build.sh Outdated Show resolved Hide resolved
CI/appveyor.after_build.sh Outdated Show resolved Hide resolved
We are preferring it over the other one as it does not depend on zziplib.

The other one missed getting into the previous commit for some reason.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
CI/appveyor.install.sh Outdated Show resolved Hide resolved
…vis]

The snapshot build code should now be complete. The PT builds should also
work now. However the release build is still not complete. For testing
purposes the NEXT commit should try and use SCP to upload the snapshot
code so that it can be verified to be usable for the release builds - this
needs some extra encryption stuff and a key (and salt) will probably need
to be created to use the MSYS2 `scp` rather than the WINSCP NET module
that the original PowerShell implimentation used.

Includes a debug line to locate the secure-file utility that might be
needed.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
…[skip gha] [skip travis]

Conflicts resolved in:
* CI/travis.linux.after_success.sh
* CI/travis.osx.after_success.sh

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
I had used an `end` instead of `fi` to close an `if` and
https://www.shellcheck.net/ was helpful to find that AND strengthen all
the AppVeyor scripts I have produced.

IDEA: use it on the Linux and MacOs ones as well...! 8-)

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
When using native (to MSYS2) paths (with '/' directory separators) it is
probably not necessary to use the $APPVAYOR_BUILD_DIR - we can hard code
the path '/c/projects/mudlet/` which makes the quoting and processing of
paths simpler - if harder to revise later one if that value ever changed.

Also needed to fix some uses of `rsync` with a wild-card, especially when
the path containing them is double-quoted. If rsync is processing the
wildcard (instead of the shell?) it will stop at directories if a single
`*` is used - apparently one needs to use s `**` to cause it to decend.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
…vis]

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
…ha] [skip travis]

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
…vis]

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
…travis]

Also swap 32 / 64 bit build job order to get results from latter first.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Also do not attempt to sign the Windows installer when we do not have the
password - which we won't have in a PR testing build like the one that
inserts this commit!

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Parts of this can only be tested AFTER it has been merged in the
development branch because it uses some secure environment varables that
are NOT present for PR/testing builds.

HOWEVER ONCE IT HAS BEEN MERGED THINGS MUST BE REVERTED OR SUBQUENTLY
REVISED BEFORE THE development BRANCH IS USED FOR A RELEASE BUILD.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
`pushd`/`popd` are bash shell built-ins and not present in a generic `sh`.

The string equality test in a generic shell `test` is `=` nor `==`.

One check for PT Builds used an old variable/logic which I had forgotten to
update.

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@SlySven
Copy link
Member Author

SlySven commented Nov 8, 2020

development branch merged in and the code forked to a new branch: 64Bit_Windows.

@SlySven
Copy link
Member Author

SlySven commented Nov 26, 2020

Not sure now if this is the way to go - Vadim seems to have cobbled together produced a working GH Actions method to build a 64-Bit Windows Mudlet so that is probably a more desirable (on a speed basis if nothing else) basis to proceed. Given that I do prefer to work in a pure MSYS2/Mingw-w64 Windows environment and have no desires to touch the MSVC toolchain i will probably keep picking away at this at odd times...

Needed to pull in recent main line changes.

Conflicts resolved in:
* .appveyor.yml
* src/mudlet.pro

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
Forgot that I had left this in place... 8-(

Signed-off-by: Stephen Lyons <slysven@virginmedia.com>
@vadi2
Copy link
Member

vadi2 commented Feb 20, 2021

Alright, sounds good. We don't need a PR for this then, it can stay in a branch.

@vadi2 vadi2 closed this Feb 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants