Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: add Windows CE / CeGCC support, with CI jobs #15975

Closed
wants to merge 121 commits into from

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented Jan 12, 2025

Make it possible to build curl for Windows CE using the CeGCC toolchain.
With both CMake and autotools, including tests and examples, also in CI.
The build configuration is the default one with Schannel enabled. No
3rd-party dependencies have been tested.

Also revive old code to make Schannel build with Windows CE, including
certificate verification.

Builds have been throughougly tested. But, I've made no functional tests
for this PR. Some parts (esp. file operations, like truncate and seek)
are stubbed out and likely broken as a result. Test servers build, but
they do not work on Windows CE. This patch substitutes fstat() calls
with stat(), which operate on filenames, not file handles. This may or
may not work and/or may not be secure.

About CeGCC: I used the latest available macOS binary build v0.59.1
r1397 from 2009, in native mingw32ce build mode. CeGCC is in effect
MinGW + GCC 4.4.0 + old/classic-mingw Windows headers. It targets
Windows CE v3.0 according to its _WIN32_WCE value. It means this PR
restores portions of old/classic-mingw support. It makes the Windows CE
codepath compatible with GCC 4.4.0. It also adds workaround for CMake,
which cannot identify and configure this toolchain out of the box.

Notes:

  • CMake doesn't recognize CeGCC/mingw32ce, necessitating tricks as seen
    with Amiga and MS-DOS.
  • CMake doesn't set MINGW for mingw32ce. Set it and MINGW32CE
    manually as a helper variable, in addition to WINCE which CMake sets
    based on CMAKE_SYSTEM_NAME.
  • CMake fails to create an implib for libcurl.dll, due to not
    recognizing the platform as a Windowsy one. This patch adds the
    necessary workaround to make it work.
  • headers shipping with CeGCC miss some things curl needs for Schannel
    support. Fixed by restoring and renovating code previously deleted
    old-mingw code.
  • it's sometime non-trivial to figure out if a fallout is WinCE,
    mingw32ce, old-mingw, or GCC version-specific.
  • WinCE is always Unicode. With exceptions: no wmain,
    GetProcAddress().
  • _fileno() is said to convert from FILE * to void * which is
    a Win32 file HANDLE. (This patch doesn't use this, but with further
    effort it probably could be.)
    https://stackoverflow.com/questions/3989545/how-do-i-get-the-file-handle-from-the-fopen-file-structure
  • WinCE has no signals, current directory, stdio/CRT file handles, no
    _get_osfhandle(), no errno, no errno.h. Some of this stuff is
    standard C89, yet missing from this platform. Microsoft expects
    Windows CE apps to use Win32 file API and FILE * exclusively.
  • revived CeGCC here (not tested for this PR):
    https://building.enlyze.com/posts/a-new-windows-ce-x86-compiler-in-2024/

On UNDER_CE vs. _WIN32_WCE: (This patch settled on UNDER_CE)

  • A custom VS2008 WinCE toolchain does not set any of these.
    The compiler binaries don't contain these strings, and has no compiler
    option for targeting WinCE, hinting that a vanilla toolchain isn't
    setting any of them either.
  • UNDER_CE is automatically defined by the CeGCC compiler.
    https://cegcc.sourceforge.net/docs/details.html
  • UNDER_CE is similar to _WIN32, except it's not set automatically
    by all compilers. It's not supposed to have any value, like a version.
    (Though e.g. OpenSSL sets it to a version)
  • _WIN32_WCE is the CE counterpart of the non-CE _WIN32_WINNT macro.
    That does return the targeted Windows CE version.
  • _WIN32_WCE is not defined by compilers, and relies on a header
    setting it to a default, or the build to set it to the desired target
    version. This is also how _WIN32_WINNT works.
  • _WIN32_WCE default is set by windef.h in CeGCC.
  • _WIN32_WCE isn't set to a default by MSVC Windows CE headers (the
    ones I checked at least).
  • CMake sets _WIN32_WCE=<ver>, UNDER_CE, WINCE for MSVC WinCE.
  • _WIN32_WCE seems more popular in other projects, including CeGCC
    itself. zlib is a notable exception amongst curl dependencies,
    which uses UNDER_CE.
  • Since _WIN32_WCE needs "certain" headers to have it defined, it's
    undefined depending on headers included beforehand.
  • curl/curl.h re-uses _WIN32_WCE's as a self-guard, relying on
    its not-(necessarily)-defined-by-default property:
    #if defined(_WIN32) && !defined(_WIN32_WCE) && !defined(__CYGWIN__)

Toolchain downloads:


w/o whitespace: https://github.com/curl/curl/pull/15975/files?w=1

@vszakats vszakats added build Windows Windows-specific labels Jan 12, 2025
@vszakats vszakats marked this pull request as draft January 12, 2025 05:07
@testclutch

This comment was marked as outdated.

@vszakats vszakats changed the title [WIP] build: try adding Windows CE support with CeGCC toolchain [WIP] build: try adding Windows CE support with CeGCC/mingw32ce toolchain Jan 12, 2025
@vszakats vszakats changed the title [WIP] build: try adding Windows CE support with CeGCC/mingw32ce toolchain [WIP] build: try adding Windows CE support via CeGCC/mingw32ce Jan 12, 2025
@github-actions github-actions bot added CI Continuous Integration tests labels Jan 12, 2025
@vszakats vszakats changed the title [WIP] build: try adding Windows CE support via CeGCC/mingw32ce build: add Windows CE / CeGCC support, with CI jobs Jan 13, 2025
@vszakats vszakats force-pushed the cegcc branch 5 times, most recently from 27fe00d to e618b6b Compare January 13, 2025 21:04
@vszakats vszakats marked this pull request as ready for review January 13, 2025 21:23
@vszakats vszakats force-pushed the cegcc branch 4 times, most recently from 4163f40 to cf2f23b Compare January 14, 2025 12:16
vszakats added a commit that referenced this pull request Jan 14, 2025
- fix `find` commands to not miss items.

- call `file` on the built files in `curl -V` steps.
  To give more feedback on what was built.

- add `curl info` step for cross-jobs that can't do a `curl -V`.
  It lists the files built and calls `file` on them.

- appveyor: make a VS2010 32-bit to match the VS2008 job it replaced.
  Follow-up to d34aeec #15934

- GHA/windows: drop the word "old" from standalone mingw-w64 jobs to not
  conflate it with "old mingw" we no longer support (while also keeping
  it short).

Cherry-picked from #15975
Closes #16001
@vszakats vszakats force-pushed the cegcc branch 4 times, most recently from c5f56be to 56e28ae Compare January 15, 2025 15:22
It'd a build case when using VS2015 for WinCE. I haven't tried
this and also not tested in CI.

Excluding it because ALPN is very unlikely to have support under
CE, but if it is, it's possible to adjust the detection code,
and even the wind detection can be made WinCE compatible by adding
a couple of lines.
@talregev
Copy link
Contributor

talregev commented Feb 21, 2025

Are you thinking to add vcpkg to compile with windows CE?

@vszakats
Copy link
Member Author

Are you thinking to add vcpkg to compile with windowxe?

Does vcpkg support Windows CE via mingw32ce?

@talregev
Copy link
Contributor

vcpkg support mingw in community. I can try to see if it works with that mingw.

@vszakats
Copy link
Member Author

vcpkg support mingw in community. I can try to see if it works with that mingw.

Sounds good to me! This is a rather niche mingw (based on the old-mingw
aka mingw 1.0).

Another interesting challenge would be making a WinCE build with either
VS2008 or ideally with VS2015. I have no idea how to bring those toolchains
up and running, locally or CI.

There is also the "revived" mingw32ce, but it needs to be built from source
and doesn't seem to improve much on the old-mingw-style SDK headers, which
is one pain point with mingw32ce. But I haven't made a try with it.

Also, actually running any of these builds under an emulator would be interesting,
but to me, this amount of retro-computing was I think more than enough for
a while! :)

@talregev
Copy link
Contributor

@vszakats why you run wince under macos? can you try on linux?
on macos there isn't vcpkg maintain (same problem for ios vcpkg).

@vszakats
Copy link
Member Author

@vszakats why you run wince under macos? can you try on linux? on macos there isn't vcpkg maintain (same problem for ios vcpkg).

Pre-built CeGCC binaries are available for macOS and Windows.

Of these, the to unixy macOS runner is faster and easier to work
with, and also what I use locally for tests.

Linux would be nicer but this isn't officially packaged, external
binaries aren't available for download (sadly, this still isn't a thing
on Linux). Building from source is in theory an option, but
a separate project on itself, if someone is interested in this (I am
personally not). Maybe there is a docker image somewhere, but
I haven't really looked (and it's also an extra layer, that I prefer to
avoid if there is a way).

@talregev
Copy link
Contributor

will you try from here?
https://sourceforge.net/projects/cegcc/files/cegcc/0.59.1/

@vszakats
Copy link
Member Author

will you try from here? https://sourceforge.net/projects/cegcc/files/cegcc/0.59.1/

These are the binaries I was working with, and what's used in CI.
The direct links are in the PR message, to the specific archives in this directory.

vszakats added a commit to vszakats/curl that referenced this pull request Feb 24, 2025
vszakats added a commit that referenced this pull request Feb 24, 2025
@vszakats
Copy link
Member Author

Forgot to mention it publicly:

With ./configure, I added the ac_cv_prog_cc_c99=no hack
to avoid these unexplained compiler warnings:

../../lib/ftp.c: In function 'Curl_GetFTPResponse':
../../lib/ftp.c:726: warning: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
../../lib/ws.c: In function 'ws_dec_pass_payload':
../../lib/ws.c:304: warning: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
../../lib/ws.c: In function 'ws_enc_write_head':
../../lib/ws.c:581: warning: format '%zd' expects type 'signed size_t', but argument 3 has type 'long int'

Maybe there is a better solution; I didn't explore this rabbit hole in depth.
Off hand it looks like a compiler issue and/or combined with mis(?)-detection
of C99 mode?

icing pushed a commit to icing/curl that referenced this pull request Feb 25, 2025
@jay
Copy link
Member

jay commented Feb 26, 2025

warning: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'

As long ssize_t has a size of signed size_t type (int I'd assume in this case) I don't see a problem

@vszakats
Copy link
Member Author

warning: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'

As long ssize_t has a size of signed size_t type (int I'd assume in this case) I don't see a problem

Yes, it should be the case.

Which made me think this is possibly a GCC bug and not worth addressing.
But it affects default autotools builds due to it detecting C99 and hitting this.

Maybe there is a #if combo to detect this case to set CURL_NO_FMT_CHECKS?

@vszakats
Copy link
Member Author

test: #16492 (comment)

@vszakats
Copy link
Member Author

vszakats commented Feb 26, 2025

This patch seems to work to fix (=silence) it: e6359ca

The next question is why does autotools want to detect C11 and C99 in these builds, and not in others:

checking whether /Users/runner/opt/mingw32ce/bin/arm-mingw32ce-gcc accepts -g... yes
checking for /Users/runner/opt/mingw32ce/bin/arm-mingw32ce-gcc option to enable C11 features... unsupported
checking for /Users/runner/opt/mingw32ce/bin/arm-mingw32ce-gcc option to enable C99 features... -std=gnu99
checking whether /Users/runner/opt/mingw32ce/bin/arm-mingw32ce-gcc -std=gnu99 understands -c and -o together... yes

https://github.com/curl/curl/actions/runs/13534146058/job/37822577315?pr=16492#step:6:74

vs. a normal mingw builds:

checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes

https://github.com/curl/curl/actions/runs/13534146058/job/37822579122?pr=16492#step:7:73

vszakats added a commit that referenced this pull request Feb 26, 2025
`./configure` mingw32ce builds enable C99 mode automatically, that
triggers compiler warnings in gcc 4.4.0. We initially worked it around
in CI by suppressing the detection of C99 with `ac_cv_prog_cc_c99=no`.

Replace it with automatically silencing the bogus warnings in C99 mode,
for all build systems:
```
lib/ftp.c: In function 'Curl_GetFTPResponse':
lib/ftp.c:726: error: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
lib/ws.c: In function 'ws_dec_pass_payload':
lib/ws.c:304: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/ws.c: In function 'ws_enc_write_head':
lib/ws.c:581: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'long int'
lib/vtls/schannel.c: In function 'schannel_connect_step1':
lib/vtls/schannel.c:1122: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_connect_step2':
lib/vtls/schannel.c:1311: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_send':
lib/vtls/schannel.c:1793: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c:1810: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_shutdown':
lib/vtls/schannel.c:2286: error: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
lib/vtls/vtls.c: In function 'ssl_cf_recv':
lib/vtls/vtls.c:1422: error: format '%zd' expects type 'signed size_t', but argument 5 has type 'ssize_t'
```
Ref: https://github.com/curl/curl/actions/runs/13533841306/job/37821720902?pr=16492#step:9:20

Also: simplify Windows CE job configuration in GHA/windows.

Follow-up to 2a292c3 #15975
Closes #16492
vszakats added a commit that referenced this pull request Feb 26, 2025
…allouts

- GHA/windows/WinCE:
  - set `-O3 -DNDEBUG` C flags manually for the CMake mingw32ce build.
    CMake doesn't recognize the platform and fails to add them. To match
    autotools (using `-O2`), and hit similar compiler warnings.
  - enable parallel builds for cmake.
  - tune parallelism for cmake using unity batches.
  - tune parallelism for autotools.

  Follow-up to 2a292c3 #15975

- tests: fix potentially uninitialized value in `readline()` in
  `getpart.c`. Detected by gcc 4.4.0 `-O2` (Windows CE) jobs:
  ```
  tests/server/getpart.c: In function 'getpart':
  tests/server/getpart.c:298: error: 'datalen' may be used uninitialized in this function
  ```
  Ref: https://github.com/curl/curl/actions/runs/13522595237/job/37785147505?pr=16476#step:11:25
  Follow-up to 592880a

- vtls_scache: rework returning pointer to avoid compiler warning seen
  with `-O3` gcc 4.4.0 builds (Windows CE/schannel):
  ```
  lib/vtls/schannel.c: In function 'schannel_connect_step1':
  lib/vtls/vtls_scache.c:975: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
  lib/vtls/vtls_scache.c:985: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
  lib/vtls/schannel.c:959: note: initialized from here
  ```
  Ref: https://github.com/curl/curl/actions/runs/13523868335/job/37789610845#step:9:25
  Follow-up to fa0ccd9 #15774

Closes #16476
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build CI Continuous Integration cmdline tool feature-window A merge of this requires an open feature window libcurl API tests Windows Windows-specific
Development

Successfully merging this pull request may close these issues.

4 participants