Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tier 1 FreeBSD Support for x86_64 #1759

Open
andrewrk opened this Issue Nov 19, 2018 · 20 comments

Comments

Projects
None yet
5 participants
@andrewrk
Copy link
Member

commented Nov 19, 2018

Here's the issue to track FreeBSD support.

History:

  • @tiehuis started an initial freebsd branch.
  • @myfreeweb picked up the figurative baton in #1661 and that work is merged into freebsd2 branch.
    • See this PR description and comments for some useful information.
    • I believe the current state of this branch is that the tests pass in debug mode but not release mode. I haven't personally tried it yet.
  • freebsd2 is now merged into master, and FreeBSD has Tier 2 Support status.

Checklist:

  • switch to using libc as the stable kernel ABI for freebsd. citation
  • #1217 Bus error on simple code
  • get all the tests passing for freebsd
  • (@andrewrk) audit the freebsd2 branch
  • CI for FreeBSD. We actually have a few options:
    • get my raspberry pi (aarch64) hooked up to the CI somehow
    • use https://builds.sr.ht/ which has freebsd x86_64 support
    • on azure dev ops, use qemu and a custom freebsd image as demonstrated by @andrewchambers in #941 (although this was for openbsd not freebsd). This should be possible because we get a full 6 hours to run the tests with azure dev ops.
  • make sure stack traces are working
  • Solve cross compilation when targeting FreeBSD

Once this is complete, we can mark FreeBSD x86_64 as having Tier 1 support.

@bnoordhuis

This comment has been minimized.

Copy link
Member

commented Nov 20, 2018

get my raspberry pi (aarch64) hooked up to the CI somehow

That's a raspberry pi 3? We have a small armada of those in the Node.js CI. They're a lot faster than the pi 1 and 2 but still so much slower than regular server hardware that we cross-compile and shard the test suite across 10 or 12 machines.

That's a long-winded way of saying they're not really an option unless you're really patient. :-)

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Nov 20, 2018

That's a raspberry pi 3?

Yes it is. Thanks for the heads up!

@myfreeweb

This comment has been minimized.

Copy link
Contributor

commented Nov 20, 2018

re: CI, I'm "starting" a buildbot cluster at https://porting.club — currently it's just a Scaleway 4-core ThunderX VPS (the master as well as the first worker, which is a jail on the same box).
The "plan" is to have a cluster with many different OSes and CPU architectures specifically for stuff that has portability needs — compilers/interpreters/VMs, libraries that touch OS specific stuff, etc.
The idea is to not use throwaway containers or VMs, but instead set up persistent environments — so that they can be used not just for CI tests or nightly builds, but also for development (i.e. project developers should be able to ssh into the workers).

(btw ThunderX cores are comparable to Cortex-A72s / significantly faster than the A53s the RPi has)

(@bnoordhuis I guess Node.js builds its own copy of V8, while Zig wouldn't need to build any monster dependencies, and the compiler itself is kinda lightweight. Oh wait, LLVM can be a memory hog though, and the Pi only has 1GB.)

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Nov 27, 2018

@myfreeweb that's exciting. Keep me updated.

In other news I updated the support table in the README to use a "tier" system: https://github.com/ziglang/zig#support-table

I think what I'll do is review the freebsd2 branch, make any changes that I see fit, and then merge it into master. Then freebsd becomes Tier 2 for x86_64 and i386. And then this issue can remain open to track Tier 1 support.

@andrewrk andrewrk changed the title Official FreeBSD Support Tier 1 FreeBSD Support for x86_64 Nov 28, 2018

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2018

I just merged the freebsd2 branch into master, and so now FreeBSD has Tier 2 Support. This issue is tracking the effort to achieve Tier 1 Support.

@ararslan

This comment has been minimized.

Copy link

commented Dec 9, 2018

FreeBSD is now available on Cirrus CI. I set it up on one of my repositories to test it out. As an example, you can see the config here and a build log here.

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Dec 13, 2018

I rented a freebsd server for an hour and poked at it. Here's what I accomplished:

  • Fixed bug in os_self_exe_path which was making the build cache not work
  • Fixed a bunch of incorrect declarations in std/os/freebsd/index.zig. For example NSIG was copied from linux and it was incorrect. Be careful copying this stuff from other systems, we have to hand-verify that the declarations are correct for every single one that we add. Also the Sigaction struct was knowingly incorrect. I didn't catch this in the pull request. Similarly some syscall functions returned success even though they did nothing. I removed these. I think it's better that they are compile errors until we can have the actual implementations for them.
  • Implemented raise syscall and hence made abort work. I might have found a bug in freebsd libc: https://lists.freebsd.org/pipermail/freebsd-hackers/2018-December/053712.html Depending on the answer to that mailing list question, we need to add signal handling protection in raise or abort.
  • Updated syscall code and fixed some bugs

Now Hello World works and the behavior tests pass in debug mode. ReleaseFast mode doesn't pass, and the standard library tests require more porting work in order to compile. But they're in a state where I think it's easy to contribute to.

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Dec 13, 2018

One of the developers from FreeBSD stopped by and clarified that the stable kernel ABI is in fact through libc, and so we will now switch to doing it that way, like we do for MacOS.

mgxm added a commit to mgxm/zig that referenced this issue Dec 19, 2018

freebsd: link against libc
Since the stable kernel ABI is through libc ziglang#1759
@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Dec 20, 2018

Thanks @ararslan and @mgxm for the CI pull requests. We now have sr.ht building successfully on x86_64 FreeBSD on every pull request and master branch commit. However, all tests are disabled except for the behavioral tests in debug mode (which all pass).

So - we've now ensured that FreeBSD won't regress accidentally. And we can progress on Tier 1 Support by getting more tests to pass, and then making sure those tests are run by CI. Once we get all the tests passing, I'll work on static FreeBSD builds being available on ziglang.org/download.

Progress!

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 8, 2019

See this comment for a proof of concept of cross compiling glibc: #514 (comment)

With regards to FreeBSD here are the next steps:

  • Follow the example set by glibc, using the wiki article and the new tool libc/process_headers.zig, creating or obtaining builds of FreeBSD libc for all the target architectures that it supports (and that Zig also supports - see zig targets). Commit those header files to Zig source repository.
  • On FreeBSD, link hello_world.c in verbose mode and observe the link options. Find out what startup files are needed. Anything with gcc in it should be covered by zig's compiler_rt.a and libunwind.a. crtbegin.o and crtend.o are unnecessary. crt1.o, crti.o, etc, these need to be built from source by zig for the target.
  • Look at the build commands FreeBSD libc uses to build the startup files, and extract the minimum necessary source files from FreeBSD libc - patching them if necessary - and include them in Zig source repository and make it so that Zig can build them from source for all the targets supported by both FreeBSD libc and Zig. Follow the example set by glibc, in codegen.cpp and link.cpp.
  • update the wiki article with maintenance instructions for the next person to follow when FreeBSD libc is updated.
@myfreeweb

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2019

observe the link options. Find out what startup files are needed

 "/usr/bin/cc" -cc1 -triple x86_64-unknown-freebsd13.0 -emit-obj -mrelax-all -disable-free -main-file-name basenamewtf.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib/clang/7.0.1 -fdebug-compilation-dir /home/greg/src/localhost -ferror-limit 19 -fmessage-length 284 -fobjc-runtime=gnustep -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/basenamewtf-eff904.o -x c basenamewtf.c -faddrsig
 "/usr/bin/ld" --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 --hash-style=both --enable-new-dtags -o basenamewtf /usr/lib/crt1.o /usr/lib/crti.o /usr/lib/crtbegin.o -L/usr/lib /tmp/basenamewtf-eff904.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/crtend.o /usr/lib/crtn.o

keep in mind the funny build process for crt1 required for the ABI version metadata

crtbegin.o and crtend.o are unnecessary

hm. Do ctors/dtors from shared libraries work without crtbegin/end in the executable?

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 8, 2019

funny build process

I've been staring at glibc code for 7 days now, that "funny build process" looks incredibly clean and easy to understand 🤣

hm. Do ctors/dtors from shared libraries work without crtbegin/end in the executable?

Based on my understanding, crtbegin.o and crtend.o are the "old way" of doing things and aren't necessary with newer C runtime libraries. I haven't tested it yet though.

Another option we have is saying "that's not supported". I think you would have trouble finding someone who thinks shared library constructors/destructors are a good idea.

Finally a third option is to stop fighting it, and support building crtbegin.o and crtend.o. I looked at the code in gcc, and it wouldn't be straightforward. But I have a few ideas to try that could solve it.

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 8, 2019

By the way what ABI does zig say is native for you on freebsd?

zig targets

Which one do you think makes sense to be the native one? I'm trying to understand this triple: x86_64-unknown-freebsd13.0 and I wonder if FreeBSD has a patch to clang to add another ABI?

@daurnimator

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2019

I think you would have trouble finding someone who thinks shared library constructors/destructors are a good idea.

why would they be a bad idea? How else should a shared library e.g. initialise an initialisation mutex?

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 8, 2019

How else should a shared library e.g. initialise an initialisation mutex?

like this

@myfreeweb

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2019

crtbegin.o and crtend.o are the "old way" of doing things and aren't necessary with newer C runtime libraries

oh, right, ctors/dtors are the old way, and the new way is init_array/fini_array. (And FreeBSD on new architectures like aarch64 doesn't support ctors/dtors at all.) I always forget which one is the new one lol. Seems like the _arrays are handled by rtld, so we're fine here!

I'm trying to understand this triple: x86_64-unknown-freebsd13.0 and I wonder if FreeBSD has a patch to clang to add another ABI?

Look at the Triple class in LLVM, everything is parsed with StartsWith and the version is parsed into numbers.

Nothing prevents me from doing

cc -cc1 -triple x86_64-unknown-freebsd69.420 …

:)

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 19, 2019

I started working on making FreeBSD builds available in the llvm8 branch. I ran into a snag. This command:

release/bin/zig build-exe ../example/hello_world/hello_libc.zig --library c

when I run it manually in a freebsd server that I'm renting, works fine. But in the sr.ht freebsd CI, it prints:

/usr/home/build/zig/example/hello_world/hello_libc.zig:1:11: error: C import failed

and nothing else. It's hard to tell what the problem is because as far as I understand the environments should be the same, and it works fine in the environment that I have shell access to.

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 19, 2019

FreeBSD binary builds available on the download page are also blocking on this sr.ht issue.

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 21, 2019

The sr.ht issue is resolved and I have pushed a commit to master branch enabling FreeBSD CI builds to automatically go on the download page when tests pass.

@andrewrk

This comment has been minimized.

Copy link
Member Author

commented Mar 21, 2019

FreeBSD builds are now available on https://ziglang.org/download/. We still don't have all the tests passing yet though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.