Make Chromium build #3
Comments
Should go away once bitwiseworks/libc#64 is done. Needed for #3.
|
Then next rather big step is to port the |
|
The |
Based on a (much) older version of the port from Mozilla. Needed for #3.
|
I'm getting build issues in |
|
With LIBC and GCC fixed, the string functions build fine now. Next stops besides pthread TLS are these (all expected):
|
|
LaunchProcess requires some tricks to be done. In POSIX, they sometimes want the child to inherit a specific FD of the parent. Sometimes this FD is required to be inherited under a specific file number in the child (i.e. not the original one). All this is achievable (with dup2) but not out of the box. The main problem is thread safety. I fear Chromium starts processes from many threads. So doing it w/o locking might mix things up. Also, inheritance is a big question. On OS/2 all file handles are inherited by default and this is surely not what Chromium wants. In |
|
With the new LIBCx spawn2 update, doing LaunchProcess was quite easy (and much easier than Unix or even Windows code!). I've also stumbled upon psmedley/gcc#34 when building histogram code: It can be worked around the way I did it in QtDeclarative but I guess it's time to fix GCC instead as it should be easy and will free us from working around these bits in other software. |
|
bitwiseworks/gcc-os2#15 weil known id say |
|
With bitwiseworks/gcc-os2#15 closed, histogram code builds fine. I will rebuild QtDeclarative too. |
These are in POSIX since long. Taken from FreeBSD whose numeric values match OS/2 TCP/IP 4.21 PR. Needed for bitwiseworks/qtwebengine-chromium-os2#3.
|
Just as a side note. It's kind of hard to work on things because there is basically a single Chromium "makefile" which builds everything in parallel and in random order. There is no way to tell "build me only threading sources" etc. So I have to grab individual gcc calls and run them manually if I want to work specifically on them. This is very inconvenient. Takes too much time to sort these things out. There is some level of control via |
System versions always contain latest OS/2 fixes maintained in their own repositories and it doesn't make sense to do monkey work by applying them to the Chromium tree. Needed for #3.
|
Moved a bit further. With latest pthread fixes TLS code also builds fine out of the box (and should be thread-safe). I'm seeing lots of small build breaks here and there. Will commit them next once done. |
This is necessary to avoid -Wsign-compare warnings (enabled with -Wall) in C++ code that does comparisons like `siginfo.si_code == BUS_ADRALN` because si_code member of siginfo is defined as int (per POSIX) while BUS_ADRALN and many other constants are defined as 0x8xxxxxxx which produces an unsigned int literal (since it won't fit in signed int). Needed for bitwiseworks/qtwebengine-chromium-os2#3.
|
Moved to a separate issue. Theseus is great but not very useful under heavy load. |
|
When using tools like Theseus on a heavily loaded systems, it is
recommended that they be run at time-critical priority.
|
|
I wonder then how to run Theseus at time-critical priority... Some nice-like command line tool? I worked around the OOM condition in EMXOMF but this didn't solve all the problems. Now I'm hitting OOM in CC1PLUS.EXE while compiling (i.e. generating that huge assembly): I will look at that in Theseus. But I already noticed earlier that CC1PLUS.EXE was taking more than 1G of "shared" memory on heavy source files. Quite expectable I would say. BTW, as source file locations show, all the OOM problems started to happen when I came to building the browser component of Chromium. I guess it is its major component so no surprise here. Its sources drag LOTS of stuff from all other support libraries. |
|
The above was solved by changing VAL from 1536 to 3072. Going further. |
|
To run at time critical, I tend to use setpri.exe. I really should spend
some time with nice6. It's probably much more functional. It was written
by an IBM employee and Dave Yeo got it released under the Apache license,
which is a good thing.
The cc1plus OOM message looks a bit odd. Maybe it was not well tested.
Given the size of the source files, the memory utilization is not
unexpected. The question is how to reduce it. Back when I was a kid and
64-bit addressing was a dream, we would never write a production compiler
that could not spill data to a work file when needed. It would not
surprise me if it is not possible to build chromium on a 32-bit Linux.
We might need to look a refactoring some of the larger files. The problem
is how to merge future updates into the the refactored files. This issue
came up for me recently in a slightly different context. My brother is
involved in a project where a piece of his code is getting refactored for
buzzword compilance (aka micro-services). The problem is how to backport
cherry-picked fixes from the trunk to the maintenance branches that are
based on the code as it existed before refactoring. Git provides some
support for these kinds of merges, but we have not you come up with a
fully automatable methodology.
|
|
With about 2900 compilation units left to go after some source changes, I'm starting getting kernel panic (and reboots) again. Sigh. Here is the postmortem (or almost postmortem) screenshot:
My guess is that it's OOM again from CC1PLUS.EXE (there are 6 processes again as this is what NINJA does by default here). I will try to lower it to 3 processes... |
|
@StevenLevine yes, I second your thoughts about 64/32 bits and compiler optimization. It's clear that current GCC builds target 64-bit systems and Chromium compilation failing on a 32-bit Linux wouldn't make me surprised either. Refactoring Chromium sources? Yes, that's an option but given how numerous and complex these these sources are and also further merging problems you mentioned this might become a real head ache. Frankly, I would invest this (potentially big) amount of time & effort into optimizing GCC instead. By using temporary on-disk "caches" or whatever. This will of course slow down compilation further but will make it at least possible. Another option (which I'm currently exploiting) is reducing parallelism (to make sure there is no more than 1-2 heavy processes at a time). But this is going to slow down compilation time even more... |
|
With a few bits fixed (and a few changes still pending — collecting more patches) the number of compilation units left to go is around 1200. As part of this process I need to add PM support to some of the remaining parts. |
This also includes usual DLL name shortening with TARGET_SHORT. Needed for #1 and for bitwiseworks/qtwebengine-chromium-os2#3.
This is required with ICU >= 65. Needed for #3.
Zygote process is a parent for all worker processes which is intended to optimize some resource usage in case of forking. Since OS/2 will use spawn rather than fork, it's not really needed. See docs/linux_zygote.md for more info about zygote. Needed for #3.
|
With a bunch of the above commits I got all 15k compilation units built! I'm pretty sure there will be a bunch of unresolved externals (platform specific parts of C++ classes intended to reside in separate source files which we miss on OS/2 yet). I expect it several dozen of missing exports. A link stage will show the exact list but I'm stuck there due to too long command line. Will continue in bitwiseworks/qtwebengine-os2#1 with that as it belongs to the Qt side of things (this command is generated by qmake). |
APIENTRY is _System on OS/2 but this definition only appears if <os2.h>
("base/os2/os2_toolkit.h") is included before OpenGL headers. Otherwise
many things will have a different sgnature and both compiling and
execution will break. Always define APIENTRY in OpenGL headers to avoid
this (just like Windows does).
Needed for #3.
On OS/2, we use fontconfig and freetype just as Linux does. Needed for #3.
C calling convention used for these exports requires this. Needed for #3.
On OS/2, we use modern CUPS for printing in cross platform software and ChromeOS uses it as well. Needed for #3.
Stubs are mostly based on stubs made for Fuchsia OS. Needed for #3.
This is due to out of memory problems of the EMX toolchain that seems to be unable to handle extremely large amount of debug info produced for C++ extensive Chromium code. See bitwiseworks/libc#82 and bitwiseworks/libc#84. Needed for #3.
|
The above commits is a result of ~two weeks of work (including all distractions like bitwiseworks/libc#82, bitwiseworks/libc#83 and bitwiseworks/libc#84). With those commits the, DLL containing Chromium successfully links now. It's named |
|
Note that there is a lot of stubs for the OS/2 platform instead of real code (done so to speed up getting first results ASAP). These stubs are mostly grabbed from the code for Fuchsia OS and basically do nothing ATM. It's about to be seen which ones of them are actually needed for Qt (and therefore will have to be implemented). This will be done in separate tickets. Closing this one as its main ticket is reached. |
This enhances cf7a5da. On OS/2, both _SC_NPROCESSORS_CONF and _SC_NPROCESSORS_ONLN return the same value (the total number of CPU cores available to the system at startup) and it doesn't change over time because OS/2 doesn't suupport hot plugging. While it's theoretically possible to return only the number of cores being actually online via _SC_NPROCESSORS_ONLN, there is no practical point in doing so because dynamic core activation and deactivation is a measure of power management (i.e. all cores should go online if the application wants as many threads and such) and the application should not make its logic dependent on it. Needed for sys_info_posix.cc in Chromium (bitwiseworks/qtwebengine-chromium-os2#3).
This is a continuation of bitwiseworks/qtwebengine-os2#1 dedicated to building the original Chromium source code.
Note that the build process itself is intended to be started using
qmakemakefiles from within theqtwebenginerepository. Butqmakedoes no more than simply runsgnto generateninjabuild files and then runsninja. I.e. prepares and starts a native Chromium build process. While this process can be then started manually from$(BLDROOT)/qtwebengine/src/core/debug(orrelease) by callingninjalike this:it's still recommended to use qmake makefiles to avoid any de-synchronization with Qt sources.
The text was updated successfully, but these errors were encountered: