New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac OS support #55

Closed
phil-opp opened this Issue Nov 23, 2015 · 29 comments

Comments

@phil-opp
Owner

phil-opp commented Nov 23, 2015

Some Mac OS users seem to have tool problems in the first post. So it might be a good idea to create a separate page that lists all needed extra steps (such as updating nasm).

Unfortunately, I have zero Mac experience and can only copy instructions from the post's comments. So if someone got it working, I would really appreciate a list of required commands!

@sptramer

This comment has been minimized.

Show comment
Hide comment
@sptramer

sptramer Nov 24, 2015

Here's what I did to get up and running on OS X:

  • Install Xcode 7.1.1 and Command Line Tools (xcode-select --install once Xcode is installed)
  • Install rust (latest stable? Not quite at the point where features from Nightly might be getting used)
  • Install homebrew: http://brew.sh/
  • Install latest nasm: brew install nasm
  • Install GCC: brew install gcc -- this is only necessary for the binutils compilation, and if you want, you can nuke GCC afterwards.
  • Cross-compile binutils: Just follow the handy instructions provided by you -- although I used /usr/local as the prefix install point just for ease of use.

sptramer commented Nov 24, 2015

Here's what I did to get up and running on OS X:

  • Install Xcode 7.1.1 and Command Line Tools (xcode-select --install once Xcode is installed)
  • Install rust (latest stable? Not quite at the point where features from Nightly might be getting used)
  • Install homebrew: http://brew.sh/
  • Install latest nasm: brew install nasm
  • Install GCC: brew install gcc -- this is only necessary for the binutils compilation, and if you want, you can nuke GCC afterwards.
  • Cross-compile binutils: Just follow the handy instructions provided by you -- although I used /usr/local as the prefix install point just for ease of use.
@sptramer

This comment has been minimized.

Show comment
Hide comment
@sptramer

sptramer Nov 24, 2015

Note on the binutils cross-compile -- looks like it builds into /usr/local/x86_64-elf/ (a change with binutils 2.26.51?) and no longer prefixes the commands automatically. Instead it looks like --program-prefix does the name prefixing. Maybe I just missed the automation before?

Here's the "works for me" binutils config: ../binutils-2.26.51/configure --target=x86_64-elf --prefix=/usr/local --disable-nls --disable-werror --disable-gdb --disable-libdecnumber --disable-readline --disable-sim --program-prefix=x86_64-elf-

sptramer commented Nov 24, 2015

Note on the binutils cross-compile -- looks like it builds into /usr/local/x86_64-elf/ (a change with binutils 2.26.51?) and no longer prefixes the commands automatically. Instead it looks like --program-prefix does the name prefixing. Maybe I just missed the automation before?

Here's the "works for me" binutils config: ../binutils-2.26.51/configure --target=x86_64-elf --prefix=/usr/local --disable-nls --disable-werror --disable-gdb --disable-libdecnumber --disable-readline --disable-sim --program-prefix=x86_64-elf-

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Nov 24, 2015

Owner

Thank you very much! I hope I find some time to create a first draft soon.

Owner

phil-opp commented Nov 24, 2015

Thank you very much! I hope I find some time to create a first draft soon.

@sptramer

This comment has been minimized.

Show comment
Hide comment
@sptramer

sptramer Nov 24, 2015

No problem. There's still one missing piece, which is building the GRUB toolchain. For now there is an (outdated; sketchy?) homebrew tap that can be used, found by a commenter on the blog post:

http://os.phil-opp.com/multiboot-kernel.html#comment-2374018414

Others have said that the OSDev wiki grub building instructions worked:

http://wiki.osdev.org/GRUB_2#Installing_GRUB2_on_Mac_OS_X

sptramer commented Nov 24, 2015

No problem. There's still one missing piece, which is building the GRUB toolchain. For now there is an (outdated; sketchy?) homebrew tap that can be used, found by a commenter on the blog post:

http://os.phil-opp.com/multiboot-kernel.html#comment-2374018414

Others have said that the OSDev wiki grub building instructions worked:

http://wiki.osdev.org/GRUB_2#Installing_GRUB2_on_Mac_OS_X

@jcaudle

This comment has been minimized.

Show comment
Hide comment
@jcaudle

jcaudle Nov 25, 2015

Do you think you'd prefer a list of commands or a PR with how someone could get things set up on a Mac? I was planning on adding a README to my own implementation of the OS, but if you need a writer for the page, I'd be happy to put something together when I get the chance to write out what I did to get things going on my own Mac.

jcaudle commented Nov 25, 2015

Do you think you'd prefer a list of commands or a PR with how someone could get things set up on a Mac? I was planning on adding a README to my own implementation of the OS, but if you need a writer for the page, I'd be happy to put something together when I get the chance to write out what I did to get things going on my own Mac.

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Nov 25, 2015

Owner

@jcaudle That would be really awesome!

Owner

phil-opp commented Nov 25, 2015

@jcaudle That would be really awesome!

@jcaudle

This comment has been minimized.

Show comment
Hide comment
@jcaudle

jcaudle Nov 26, 2015

@sptramer have you been able to link the rust code into your kernel as described in the Setup Rust post? I've been running into a problem where I need to run ranlib, which then doesn't properly add the symbol for rust_main (I think...). If you've gotten past that, I'd be much more able to write something up for future OS builders! 😄

jcaudle commented Nov 26, 2015

@sptramer have you been able to link the rust code into your kernel as described in the Setup Rust post? I've been running into a problem where I need to run ranlib, which then doesn't properly add the symbol for rust_main (I think...). If you've gotten past that, I'd be much more able to write something up for future OS builders! 😄

@sptramer

This comment has been minimized.

Show comment
Hide comment
@sptramer

sptramer Nov 26, 2015

@jcaudle Not yet - I'm still setting up the cross-compilers correctly. It looks like I goofed on the i386 binutils above by putting them in /usr/local -- cross-compiling GCC against these binutils, even setting CC/CXX/CPP/LD to the correct installed gcc version seems to force it to link against the clang assembler, which screws everything up.

I just installed rust from the .pkg distribution they offered. If it turns out that's not quite enough, I'll look into it when I reach that point.

sptramer commented Nov 26, 2015

@jcaudle Not yet - I'm still setting up the cross-compilers correctly. It looks like I goofed on the i386 binutils above by putting them in /usr/local -- cross-compiling GCC against these binutils, even setting CC/CXX/CPP/LD to the correct installed gcc version seems to force it to link against the clang assembler, which screws everything up.

I just installed rust from the .pkg distribution they offered. If it turns out that's not quite enough, I'll look into it when I reach that point.

@sptramer

This comment has been minimized.

Show comment
Hide comment
@sptramer

sptramer Nov 26, 2015

Here's how I was able to eventually get a grub build working:

  • When cross-compiling i386 binutils, make sure that they go into a non-standard location. I used --prefix=/opt/i386-toolchain. The full command line that worked for me was: CC=gcc-5 CXX=g++-5 CPP=cpp-5 LD=gcc-5 ../binutils-2.26.51/configure --target=i386-elf --prefix=/opt/i386-toolchain --disable-nls --disable-werror --disable-gdb --disable-libdecnumber --disable-readline --disable-sim
  • In addition to cross-compiling i386 binutils, you need an i386 GCC for grub: CC=gcc-5 CXX=g++-5 CPP=cpp-5 LD=gcc-5 ../gcc-5.2.0/configure --target=i386-elf --disable-nls --enable-languages=c,c++ --without-headers --prefix=/opt/i386-toolchain.
  • Build objconv for the host architecture, meaning that you can just use gcc-5 as the compiler invoke.
  • Configure grub to use the cross-compiled tools and also the host GCC toolchain: CC=gcc-5 CXX=g++-5 CPP=gcc-5 LD=gcc-5 ../grub/configure --disable-werror TARGET_CC=i386-elf-gcc TARGET_STRIP=i386-elf-strip TARGET_NM=i386-elf-nm TARGET_RANLIB=i386-elf-ranlib --target=i386-elf --prefix=/opt/i386-toolchain
  • NOTE: You may need to upgrade flex for building grub. You want to brew install flex and then brew link --force flex so it replaces the system flex. Then brew unlink flex to clean it up when you're done.

Other brew formulas you'll probably need are qemu and xorriso. At this point I had the bootloader working and showing OK from make run.

sptramer commented Nov 26, 2015

Here's how I was able to eventually get a grub build working:

  • When cross-compiling i386 binutils, make sure that they go into a non-standard location. I used --prefix=/opt/i386-toolchain. The full command line that worked for me was: CC=gcc-5 CXX=g++-5 CPP=cpp-5 LD=gcc-5 ../binutils-2.26.51/configure --target=i386-elf --prefix=/opt/i386-toolchain --disable-nls --disable-werror --disable-gdb --disable-libdecnumber --disable-readline --disable-sim
  • In addition to cross-compiling i386 binutils, you need an i386 GCC for grub: CC=gcc-5 CXX=g++-5 CPP=cpp-5 LD=gcc-5 ../gcc-5.2.0/configure --target=i386-elf --disable-nls --enable-languages=c,c++ --without-headers --prefix=/opt/i386-toolchain.
  • Build objconv for the host architecture, meaning that you can just use gcc-5 as the compiler invoke.
  • Configure grub to use the cross-compiled tools and also the host GCC toolchain: CC=gcc-5 CXX=g++-5 CPP=gcc-5 LD=gcc-5 ../grub/configure --disable-werror TARGET_CC=i386-elf-gcc TARGET_STRIP=i386-elf-strip TARGET_NM=i386-elf-nm TARGET_RANLIB=i386-elf-ranlib --target=i386-elf --prefix=/opt/i386-toolchain
  • NOTE: You may need to upgrade flex for building grub. You want to brew install flex and then brew link --force flex so it replaces the system flex. Then brew unlink flex to clean it up when you're done.

Other brew formulas you'll probably need are qemu and xorriso. At this point I had the bootloader working and showing OK from make run.

@sptramer

This comment has been minimized.

Show comment
Hide comment
@sptramer

sptramer Nov 27, 2015

Well, get ready for the bad news on cross-compiling a bare metal capable rust libcore on OS X:

rust-lang/rust#16259

Seems like the rust team doesn't have the bandwidth to resolve the issue at this time. Cross-compiler support for rust in general seems pretty back-burner, but Darwin -> i686/x86_64 seems like an especially low priority because how how screwed up Apple's linker is. Seems like you can provide your own x86_64 toolchain - but it's complicated as hell.

sptramer commented Nov 27, 2015

Well, get ready for the bad news on cross-compiling a bare metal capable rust libcore on OS X:

rust-lang/rust#16259

Seems like the rust team doesn't have the bandwidth to resolve the issue at this time. Cross-compiler support for rust in general seems pretty back-burner, but Darwin -> i686/x86_64 seems like an especially low priority because how how screwed up Apple's linker is. Seems like you can provide your own x86_64 toolchain - but it's complicated as hell.

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Nov 28, 2015

Owner

This seems to be much more complicated than I thought… Maybe it's easier to use a virtual Linux machine. For example, see Lifepillar's comment.

Ashley Williams's x86-kernel repository contains a Vagrantfile that should make it easy to get a working virtual environment. There are some instructions in the readme, too.

Owner

phil-opp commented Nov 28, 2015

This seems to be much more complicated than I thought… Maybe it's easier to use a virtual Linux machine. For example, see Lifepillar's comment.

Ashley Williams's x86-kernel repository contains a Vagrantfile that should make it easy to get a working virtual environment. There are some instructions in the readme, too.

@phil-opp phil-opp changed the title from Add a `Get started on Mac OS` page to Mac OS support Nov 28, 2015

@jcaudle

This comment has been minimized.

Show comment
Hide comment
@jcaudle

jcaudle Nov 28, 2015

Good call on suggesting the use of vagrant. After the find by @sptramer, I was going to switch to setting up vagrant myself.

jcaudle commented Nov 28, 2015

Good call on suggesting the use of vagrant. After the find by @sptramer, I was going to switch to setting up vagrant myself.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Dec 14, 2015

Contributor

cc @ashleygwilliams, who was mentioned upthread but not in a way that would ping her :)

Contributor

steveklabnik commented Dec 14, 2015

cc @ashleygwilliams, who was mentioned upthread but not in a way that would ping her :)

@ashleygwilliams

This comment has been minimized.

Show comment
Hide comment
@ashleygwilliams

ashleygwilliams Dec 14, 2015

hey @phil-opp would you be interested in a PR with the vagrantfile and the mac docs? i'd me more than happy to contribute

ashleygwilliams commented Dec 14, 2015

hey @phil-opp would you be interested in a PR with the vagrantfile and the mac docs? i'd me more than happy to contribute

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Dec 14, 2015

Owner

That would be awesome!

Owner

phil-opp commented Dec 14, 2015

That would be awesome!

@emkay

This comment has been minimized.

Show comment
Hide comment
@emkay

emkay Jan 10, 2016

FWIW I have a script that is working for me to build the cross compiler tools and compile grub on OSX.

https://gist.github.com/emkay/a1214c753e8c975d95b4

emkay commented Jan 10, 2016

FWIW I have a script that is working for me to build the cross compiler tools and compile grub on OSX.

https://gist.github.com/emkay/a1214c753e8c975d95b4

@jcaudle

This comment has been minimized.

Show comment
Hide comment
@jcaudle

jcaudle Jan 10, 2016

@emkay nice work on the script. I'm curious what you do when you start using rust itself as that's where I decided to look into using a VM.

jcaudle commented Jan 10, 2016

@emkay nice work on the script. I'm curious what you do when you start using rust itself as that's where I decided to look into using a VM.

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Jan 10, 2016

Owner

@emkay this looks pretty great!

@jcaudle the problem with rust was related to cross-compiling libcore, wasn't it?

Owner

phil-opp commented Jan 10, 2016

@emkay this looks pretty great!

@jcaudle the problem with rust was related to cross-compiling libcore, wasn't it?

@jcaudle

This comment has been minimized.

Show comment
Hide comment
@jcaudle

jcaudle Jan 10, 2016

@phil-opp exactly. I basically shelved the project when I saw that, but have considered putting in the "heaps of work" necessary to getting it to work properly, I just haven't taken the time for it yet.

jcaudle commented Jan 10, 2016

@phil-opp exactly. I basically shelved the project when I saw that, but have considered putting in the "heaps of work" necessary to getting it to work properly, I just haven't taken the time for it yet.

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Jan 10, 2016

Owner

@jcaudle Maybe nighly-libcore with the following target json works:

{
  "llvm-target": "x86_64-unknown-none-gnu",
  "target-endian": "little",
  "target-pointer-width": "64",
  "os": "none",
  "arch": "x86_64",
  "data-layout": "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128",
  "pre-link-args": [ "-m64" ],
  "cpu": "x86-64",
  "no-compiler-rt": true,
}
Owner

phil-opp commented Jan 10, 2016

@jcaudle Maybe nighly-libcore with the following target json works:

{
  "llvm-target": "x86_64-unknown-none-gnu",
  "target-endian": "little",
  "target-pointer-width": "64",
  "os": "none",
  "arch": "x86_64",
  "data-layout": "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128",
  "pre-link-args": [ "-m64" ],
  "cpu": "x86-64",
  "no-compiler-rt": true,
}
@emkay

This comment has been minimized.

Show comment
Hide comment
@emkay

emkay Jan 10, 2016

@emkay nice work on the script. I'm curious what you do when you start using rust itself as that's where I decided to look into using a VM.

I haven't gotten to this point yet. I'm following along at intermezzOS so will attempt to cross that bridge when we get there.

emkay commented Jan 10, 2016

@emkay nice work on the script. I'm curious what you do when you start using rust itself as that's where I decided to look into using a VM.

I haven't gotten to this point yet. I'm following along at intermezzOS so will attempt to cross that bridge when we get there.

@n4matti

This comment has been minimized.

Show comment
Hide comment
@n4matti

n4matti Feb 17, 2016

https://gist.github.com/emkay/a1214c753e8c975d95b4

Hi emkay, I tried your script, but it failed for me when it compiled libgcc with "make all-target-libgcc":

In file included from /Users/mathiasbauer/opt/arm-linux-gnueabihf/sys-include/pthread.h:21:0,
from ./gthr-default.h:35,
from ../../../gcc-5.3.0/libgcc/gthr.h:148,
from ../../../gcc-5.3.0/libgcc/libgcov-interface.c:27:
/Users/mathiasbauer/opt/arm-linux-gnueabihf/sys-include/features.h:361:25: fatal error: sys/cdefs.h: No such file or directory

"/Users/mathiasbauer/opt" is my $PREFIX
"sys/cdefs.h" is exactly where it should be, it's in $PREFIX/arm-linux-gnueabihf/sys-include/ and also in $PREFIX/include.
I'm using Yosemite, but I don't expect that this makes a big difference, as it's the cross-gcc that is used here.

Seems that something is missing or different at my system. Do you have an idea?

n4matti commented Feb 17, 2016

https://gist.github.com/emkay/a1214c753e8c975d95b4

Hi emkay, I tried your script, but it failed for me when it compiled libgcc with "make all-target-libgcc":

In file included from /Users/mathiasbauer/opt/arm-linux-gnueabihf/sys-include/pthread.h:21:0,
from ./gthr-default.h:35,
from ../../../gcc-5.3.0/libgcc/gthr.h:148,
from ../../../gcc-5.3.0/libgcc/libgcov-interface.c:27:
/Users/mathiasbauer/opt/arm-linux-gnueabihf/sys-include/features.h:361:25: fatal error: sys/cdefs.h: No such file or directory

"/Users/mathiasbauer/opt" is my $PREFIX
"sys/cdefs.h" is exactly where it should be, it's in $PREFIX/arm-linux-gnueabihf/sys-include/ and also in $PREFIX/include.
I'm using Yosemite, but I don't expect that this makes a big difference, as it's the cross-gcc that is used here.

Seems that something is missing or different at my system. Do you have an idea?

@Others

This comment has been minimized.

Show comment
Hide comment
@Others

Others Oct 16, 2016

I don't think that this is exactly what is being looked for here, but I have a Vagrantfile that works for this project. While it technically doesn't provide macOS support, it lets one follow along the blog posts on macOS. I've put the file in a gist here. To use it you just have to download it into the root directory of the project, and then run vagrant up && vagrant ssh -- -Y. Then once you're ssh'd in, you can cd to /vagrant, and run any command shown in the blog posts. (Obviously this whole process requires having vagrant installed.)

Others commented Oct 16, 2016

I don't think that this is exactly what is being looked for here, but I have a Vagrantfile that works for this project. While it technically doesn't provide macOS support, it lets one follow along the blog posts on macOS. I've put the file in a gist here. To use it you just have to download it into the root directory of the project, and then run vagrant up && vagrant ssh -- -Y. Then once you're ssh'd in, you can cd to /vagrant, and run any command shown in the blog posts. (Obviously this whole process requires having vagrant installed.)

@adeadman

This comment has been minimized.

Show comment
Hide comment
@adeadman

adeadman Dec 24, 2016

After cross-compiling gcc and binutils, and building grub, I'm able to follow along the tutorial no problems. However, since qemu only supports kvm on Linux, and (as far as I can find) no other accelerators on macOS, I'm unable to do the advanced debugging linked in better exception messages by using -enable-kvm. There may be a way to get Qemu to use Intel's HAXM extension, with extra development.

For the meantime, I've created a VirtualBox machine (Type: Other, Version Other/Unknown 64-bit) that can boot the image and reproduce the triple fault expected in "Reproducing the bug in QEMU", but I haven't yet managed to attach a debugger to VirtualBox. This might be a usable alternative to qemu -enable-kvm for Mac users.

adeadman commented Dec 24, 2016

After cross-compiling gcc and binutils, and building grub, I'm able to follow along the tutorial no problems. However, since qemu only supports kvm on Linux, and (as far as I can find) no other accelerators on macOS, I'm unable to do the advanced debugging linked in better exception messages by using -enable-kvm. There may be a way to get Qemu to use Intel's HAXM extension, with extra development.

For the meantime, I've created a VirtualBox machine (Type: Other, Version Other/Unknown 64-bit) that can boot the image and reproduce the triple fault expected in "Reproducing the bug in QEMU", but I haven't yet managed to attach a debugger to VirtualBox. This might be a usable alternative to qemu -enable-kvm for Mac users.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Dec 24, 2016

Contributor

We got a PR to intermezzos that might be helpful: intermezzOS/kernel#101

Contributor

steveklabnik commented Dec 24, 2016

We got a PR to intermezzos that might be helpful: intermezzOS/kernel#101

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Dec 25, 2016

Owner

That's great! Thanks for keeping me updated.

Owner

phil-opp commented Dec 25, 2016

That's great! Thanks for keeping me updated.

@berdon

This comment has been minimized.

Show comment
Hide comment
@berdon

berdon Apr 4, 2017

For anyone who wants to debug via MacOS/gdb, http://austinhanson.com/vscode-gdb-and-debugging-an-os.

berdon commented Apr 4, 2017

For anyone who wants to debug via MacOS/gdb, http://austinhanson.com/vscode-gdb-and-debugging-an-os.

@gil0mendes

This comment has been minimized.

Show comment
Hide comment
@gil0mendes

gil0mendes Apr 4, 2017

Contributor

@berdon That looks amazing, I'm using Radare2 at the moment. We can connect, for example, with QEMU using the following command:

r2 -d gdb://localhost:1234
Contributor

gil0mendes commented Apr 4, 2017

@berdon That looks amazing, I'm using Radare2 at the moment. We can connect, for example, with QEMU using the following command:

r2 -d gdb://localhost:1234
@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Mar 6, 2018

Owner

The work on the second edition (which supports macOS natively) is making good progress, so I'm going to close this issue in favor of #360. Thank you all for helping with this!

Owner

phil-opp commented Mar 6, 2018

The work on the second edition (which supports macOS natively) is making good progress, so I'm going to close this issue in favor of #360. Thank you all for helping with this!

@phil-opp phil-opp closed this Mar 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment