Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DragonFly(BSD) #9178

Merged
merged 4 commits into from
May 1, 2020
Merged

Support DragonFly(BSD) #9178

merged 4 commits into from
May 1, 2020

Conversation

mneumann
Copy link
Contributor

@mneumann mneumann commented Apr 25, 2020

  • This work is based on lib_c/x86_64-freebsd, but carefully
    reviewed and modified.

  • Note that the name "DragonFly" and not "DragonFlyBSD" is used in code
    (#ifdef DragonFly) and in LLVM traget triple
    (x86_64-unknown-dragonfly).

  • DragonFly's libc does not expose ntons() and htons() functions. As it
    is x86_64 only, use bswap16 intrisic directly. This is also what
    DragonFly's libc uses internally as a C macro. I did not find a way
    to add ntons() and htons() directly as functions to "lib LibC" so that
    I had to pollute some other code with conditional "if flag?(:dragonfly)"
    code.

  • DragonFly does not support fdatasync(). Use fsync() instead.

  • This does not (yet) run on an unpatched DragonFly system, as
    DragonFly currently limits the main thread's stack to 4MB which is not
    enough for the crystal compiler to compile itself. I have a patch for
    DragonFly's libpthread in the works that uppers the stack size to
    RLIMIT_STACK. With that patch, crystal can successfully boostrap
    itself.

  • A large amount of the test suite passes. Though there are some
    hangs which needs to be fixed in a follow up commit. For instance,
    the spec for IO.pipe() "raises if trying to read to an IO not opened
    for reading" fails, as pipe(2) on DragonFly creates two bidirectional
    endpoints and neither one is special cased for read- or write-only
    operation, meaning, reading from the "write"-end is perfectly legal on
    DragonFly and blocks until data is available via the "read"-end.

  • Cross-compiled from a FreeBSD system. Thanks for making
    cross-compilation that easy!

@mneumann
Copy link
Contributor Author

I run "crystal format" on it :)

@RX14
Copy link
Contributor

RX14 commented Apr 25, 2020

I was helping @Niacat with a NetBSD port recently and I ended up introducing a bsd flag to help avoid the flag?(:freebsd) || flag?(:openbsd) || flag?(:netbsd) mess. Commits are here and here, with some fairly thorough research about what is portable across the BSDs and which parts are bsd-specific.

I haven't reviewed this PR yet, but if you'd like (and you don't have to), we could include the change to add the BSD flag in this PR.

For the record, we did get stuck on a stack overflow in the crystal compiler, which seemed to trigger inside libgc. I'm not sure if that's the same stack limits issue you ran into or not.

@mneumann
Copy link
Contributor Author

@RX14: having a flag?(:bsd) makes sense!

I also got a stack overflow when I let crystal compile itself. But I figured out that DragonFly has a fixed 4MB stack limit for the main thread (under pthreads), whereas FreeBSD uses the process resource limit. Are you sure that this is really a stack overflow, or is crystal just using recursion quite a lot? When I upper the stack limit on DragonFly for the main thread, the crystal compiler can compile itself fine. Did you run into this on NetBSD?

@RX14
Copy link
Contributor

RX14 commented Apr 26, 2020

Did you run into this on NetBSD?

We're not sure, but it's definitely something to look into, thank you.

* This work is based on lib_c/x86_64-freebsd, but carefully
  reviewed and modified.

* Note that the name "DragonFly" and not "DragonFlyBSD" is used in code
  (#ifdef __DragonFLy__) and in LLVM traget triple
  (x86_64-unknown-dragonfly).

* DragonFly's libc does not expose ntons() and htons() functions. As it
  is x86_64 only, use bswap16 intrisic directly. This is also what
  DragonFly's libc uses internally as a C macro.  I did not find a way
  to add ntons() and htons() directly as functions to "lib LibC" so that
  I had to pollute some other code with conditional "if flag?(:dragonfly)"
  code.

* DragonFly does not support fdatasync(). Use fsync() instead.

* This does not (yet) run on an unpatched DragonFly system, as
  DragonFly currently limits the main thread's stack to 4MB which is not
  enough for the crystal compiler to compile itself. I have a patch for
  DragonFly's libpthread in the works that uppers the stack size to
  RLIMIT_STACK. With that patch, crystal can successfully boostrap
  itself.

* A large amount of the test suite passes. Though there are some
  hangs which needs to be fixed in a follow up commit.  For instance,
  the spec for IO.pipe() "raises if trying to read to an IO not opened
  for reading" fails, as pipe(2) on DragonFly creates two bidirectional
  endpoints and neither one is special cased for read- or write-only
  operation, meaning, reading from the "write"-end is perfectly legal on
  DragonFly and blocks until data is available via the "read"-end.

* Cross-compiled from a FreeBSD system. Thanks for making
  cross-compilation that easy!
Note that I did not put Darwin under BSD even though it is based
on BSD.
src/compiler/crystal/codegen/target.cr Outdated Show resolved Hide resolved
src/compiler/crystal/semantic/flags.cr Outdated Show resolved Hide resolved
@straight-shoota straight-shoota added this to the 0.35.0 milestone Apr 28, 2020
This partially reverts commit 1181ab4.

We need to wait for a full bootstrap cycle before being able to use flag?(:bsd).

To not break platforms that do not continously build Crystal, I suggest
to wait for these changes until a new version of Crystal is released.
@mneumann
Copy link
Contributor Author

Did you run into this on NetBSD?

We're not sure, but it's definitely something to look into, thank you.

FYI: I can reproduce a stack overflow on FreeBSD when I set the stack limit to about 4650 kbytes (ulimit -s 4650). A limit of 4700 is just fine, crystal compiles. Interestingly, on FreeBSD I get a rather short stack trace, vs. hundreds of lines on DragonFly.

@straight-shoota straight-shoota merged commit 5d65a7a into crystal-lang:master May 1, 2020
@straight-shoota
Copy link
Member

Dankeschön @mneumann

@mneumann mneumann deleted the support-dragonfly branch July 2, 2020 12:33
mneumann added a commit to mneumann/DragonFlyBSD that referenced this pull request Aug 17, 2020
Prior to this commit, the stack size of the main thread was fixed
at 4 MB which was too low for certain applications.

This commit fixes bootstrapping the lang/crystal compiler (see [1]).

Reviewed-by: dillon

Fixes-issue: #3234

[1]: crystal-lang/crystal#9178
dragonflybot pushed a commit to DragonFlyBSD/DragonFlyBSD that referenced this pull request Aug 17, 2020
Prior to this commit, the stack size of the main thread was fixed
at 4 MB which was too low for certain applications.

This commit fixes bootstrapping the lang/crystal compiler (see [1]).

Reviewed-by: dillon

Fixes-issue: #3234

[1]: crystal-lang/crystal#9178
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants