New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port to Android #1442

Merged
merged 1 commit into from Apr 13, 2016

Conversation

Projects
None yet
@modocache
Collaborator

modocache commented Feb 25, 2016

What's in this pull request?

This adds an Android target for the stdlib. It is also the first example of cross-compiling outside of Darwin: a Linux host machine builds for an Android target.

Relevant mailing list discussions:

  1. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151207/000171.html
  2. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151214/000492.html

The Android variant of Swift may be built and tested using the following build-script invocation:

$ utils/build-script \
  -R \                                           # Build in ReleaseAssert mode.
  -T \                                           # Run all tests.
  --android \                                    # Build for Android.
  --android-deploy-device-path /data/local/tmp \ # Temporary directory on the device where Android tests are run.
  --android-ndk ~/android-ndk-r10e \             # Path to an Android NDK.
  --android-ndk-version 21 \
  --android-icu-uc ~/libicu-android/armeabi-v7a/libicuuc.so \
  --android-icu-uc-include ~/libicu-android/armeabi-v7a/icu/source/common \
  --android-icu-i18n ~/libicu-android/armeabi-v7a/libicui18n.so \
  --android-icu-i18n-include ~/libicu-android/armeabi-v7a/icu/source/i18n/

Android builds have the following dependencies, as can be seen in the build script invocation:

  1. An Android NDK of version 21 or greater, available to download here: http://developer.android.com/ndk/downloads/index.html.
  2. A libicu compatible with android-armv7. You may build your own by cloning https://github.com/SwiftAndroid/libiconv-libicu-android and running the build.sh script in that repository.

What's worth discussing about this pull request?

Continuous integration: I'd be thrilled to have this merged into the main Swift repository, but I don't want to dump a bunch of code that no one can maintain. I think CI is a very important part of maintaining a healthy build, but:

  1. An Android-compatible build of libicu is necessary to build for Android.
  2. An Android device (or possibly emulator) is necessary to run the Android tests.

Do either of those sound like something CI servers could be configured with? Or can someone help me think of alternatives--perhaps building libicu as part of the Swift build, as we do with ninja? #1398 includes some pkg-config logic that may be useful here.

FIXMEs: There are quite a few spots labeled "FIXME" that I could use some help with. Feedback welcome!


Thanks a ton to @zhuowei, who deserves the vast majority of the credit. I just tweaked a few things. 馃槉

Show outdated Hide outdated stdlib/private/SwiftPrivateLibcExtras/Subprocess.swift
if result != 0 {
preconditionFailure("execve() failed. errno: \(errno)")
}
_exit(126)

This comment has been minimized.

@weissi

weissi Feb 25, 2016

Collaborator

this has different semantics to posix_spawn. posix_spawn will give you a proper errno value if the spawn failed, this just exits the child process. I.e. spawning ["/bin/zsh", "-c", "exit 126;"] will be indistinguishable for the caller, right? It will trigger the preconditionFailure, no idea what that actually means.

I'd propose to open a separate pipe, called child2parent or so which will be fcntld to FD_CLOEXEC. That means if execve succeeds, it will be closed automatically and that means the parent will read EOF. If execve fails, we just write execve's errno to that pipe so the parent can read it and learns what went wrong in the child. So the parent waits until it can read, if it reads EOF (read returns 0) it knows, the child spawned successfully, if it reads anything else, it knows that this is execve's errno value and can return that to the caller. Does that make sense?

Here some pseudo-C code which will hopefully make clear what I mean:

[...]
} else if (pid == 0) {
    int err = fcntl(child2parent[1], F_SETFD, FD_CLOEXEC); /* This fd will be closed on exec */
    int errno_save = errno;
    if (err) {
       preconditionFailure("fcntl failed, errno: %d", errno_save);
       abort();
    }
    execve(...);
    errno_save = errno;
    /* execve returned, this is an error that we need to communicate back to the parent */
    ssize_t suc_write = write(child2parent[1], &errno_save, sizeof(typeof(errno)));
    errno_save = errno;
    if (suc_write > 0 && suc_write < sizeof(typeof(errno)));
        /* implement write retry ;) */
    } else if (suc_write == 0) {
        /* preconditionFailure("EOF?!"); */
        abort();
    } else if (suc_write < 0) {
        preconditionFailure("write failed, errno %d", errno_save);
        abort();
    }
    close(child2parent[1]);
}
[...]
@weissi

weissi Feb 25, 2016

Collaborator

this has different semantics to posix_spawn. posix_spawn will give you a proper errno value if the spawn failed, this just exits the child process. I.e. spawning ["/bin/zsh", "-c", "exit 126;"] will be indistinguishable for the caller, right? It will trigger the preconditionFailure, no idea what that actually means.

I'd propose to open a separate pipe, called child2parent or so which will be fcntld to FD_CLOEXEC. That means if execve succeeds, it will be closed automatically and that means the parent will read EOF. If execve fails, we just write execve's errno to that pipe so the parent can read it and learns what went wrong in the child. So the parent waits until it can read, if it reads EOF (read returns 0) it knows, the child spawned successfully, if it reads anything else, it knows that this is execve's errno value and can return that to the caller. Does that make sense?

Here some pseudo-C code which will hopefully make clear what I mean:

[...]
} else if (pid == 0) {
    int err = fcntl(child2parent[1], F_SETFD, FD_CLOEXEC); /* This fd will be closed on exec */
    int errno_save = errno;
    if (err) {
       preconditionFailure("fcntl failed, errno: %d", errno_save);
       abort();
    }
    execve(...);
    errno_save = errno;
    /* execve returned, this is an error that we need to communicate back to the parent */
    ssize_t suc_write = write(child2parent[1], &errno_save, sizeof(typeof(errno)));
    errno_save = errno;
    if (suc_write > 0 && suc_write < sizeof(typeof(errno)));
        /* implement write retry ;) */
    } else if (suc_write == 0) {
        /* preconditionFailure("EOF?!"); */
        abort();
    } else if (suc_write < 0) {
        preconditionFailure("write failed, errno %d", errno_save);
        abort();
    }
    close(child2parent[1]);
}
[...]

This comment has been minimized.

@gribozavr

gribozavr Apr 6, 2016

Collaborator

@modocache This great comment by @weissi is still valid, I think.

@gribozavr

gribozavr Apr 6, 2016

Collaborator

@modocache This great comment by @weissi is still valid, I think.

This comment has been minimized.

@modocache

modocache Apr 11, 2016

Collaborator

Finally getting around to this, sorry for the wait. @weissi, could you explain child2parent in greater detail? Would the child process file descriptors be duped to write to the parent pipe in some way? I think I'm missing something.

@modocache

modocache Apr 11, 2016

Collaborator

Finally getting around to this, sorry for the wait. @weissi, could you explain child2parent in greater detail? Would the child process file descriptors be duped to write to the parent pipe in some way? I think I'm missing something.

This comment has been minimized.

@weissi

weissi Apr 11, 2016

Collaborator

Don't worry about the wait, great work on this PR! So the reason we need child2parent is to communicate from child to parent.

There's basically two possible outcomes

  1. everything went fine
  2. there was an error in the child (for example file to execute not found, no permission or what not)

In both cases, it's crucial that the child can tell the parent. The easiest way on UNIX is to just open a pipe (child2parent) in the parent before the fork. Because file descriptors are forked as well, we will have a communication channel from the child to the parent after the fork. child2parent[0] is the reading end, so that's the file descriptor the parent keeps open. And child2parent[1] is the writing end, that's the fd the child keeps open. In other words, if the child writes something to child2parent[1], the parent will receive it on child2parent[0]. So far so good, now we have a communication facility.

If there was an error execveing in the child, we can just write the errno value that it got into child2parent[1] and the parent will know what exactly went wrong. For example if the file to be executed wasn't found, a ENOENT will be transmitted and the parent can fail appropriately.

The seemingly easier case is actually the harder: What to do if there was no error in the child, i.e. the child executed another process. Then execve succeeds and we replaced the process with some other binary. The file descriptor child2parent[1] will now still be open so we could put something in there which tells the parent that everything went fine. However, the program we executed probably doesn't know it's supposed to do that. The standard trick to get around this issue is to set the file descriptor child2parent[1] as FD_CLOEXEC. If FD_CLOEXEC is set, the file descriptor is closed by the OS kernel automatically if execve succeeds. That's perfect for us: If child2parent[1] is closed without anything ever being written to the pipe, the parent can now learn that execve has succeeded which is exactly the missing big.

So the logic in the parent will be

  1. try to read on parent2child[0]
    2a. if read returns 0 (meaning EOF), the parent knows everything went fine, the child successfully executed another program
    2b. if read returns some bytes, we know that something went wrong and the value that we read is the errno of the child that it got when trying to execute the other program. That's awesome as we now know in the parent what went wrong in the child :).

Hope this makes any sense, if not or if there are any questions left, feel free to ask.

@weissi

weissi Apr 11, 2016

Collaborator

Don't worry about the wait, great work on this PR! So the reason we need child2parent is to communicate from child to parent.

There's basically two possible outcomes

  1. everything went fine
  2. there was an error in the child (for example file to execute not found, no permission or what not)

In both cases, it's crucial that the child can tell the parent. The easiest way on UNIX is to just open a pipe (child2parent) in the parent before the fork. Because file descriptors are forked as well, we will have a communication channel from the child to the parent after the fork. child2parent[0] is the reading end, so that's the file descriptor the parent keeps open. And child2parent[1] is the writing end, that's the fd the child keeps open. In other words, if the child writes something to child2parent[1], the parent will receive it on child2parent[0]. So far so good, now we have a communication facility.

If there was an error execveing in the child, we can just write the errno value that it got into child2parent[1] and the parent will know what exactly went wrong. For example if the file to be executed wasn't found, a ENOENT will be transmitted and the parent can fail appropriately.

The seemingly easier case is actually the harder: What to do if there was no error in the child, i.e. the child executed another process. Then execve succeeds and we replaced the process with some other binary. The file descriptor child2parent[1] will now still be open so we could put something in there which tells the parent that everything went fine. However, the program we executed probably doesn't know it's supposed to do that. The standard trick to get around this issue is to set the file descriptor child2parent[1] as FD_CLOEXEC. If FD_CLOEXEC is set, the file descriptor is closed by the OS kernel automatically if execve succeeds. That's perfect for us: If child2parent[1] is closed without anything ever being written to the pipe, the parent can now learn that execve has succeeded which is exactly the missing big.

So the logic in the parent will be

  1. try to read on parent2child[0]
    2a. if read returns 0 (meaning EOF), the parent knows everything went fine, the child successfully executed another program
    2b. if read returns some bytes, we know that something went wrong and the value that we read is the errno of the child that it got when trying to execute the other program. That's awesome as we now know in the parent what went wrong in the child :).

Hope this makes any sense, if not or if there are any questions left, feel free to ask.

This comment has been minimized.

@modocache

modocache Apr 12, 2016

Collaborator

Thank you @weissi, your explanation was incredibly helpful! Thanks for so clearly explaining the problem, it made writing the code much easier. I have a working implementation that I will amend to this commit soon; I'm running the Android test suite from #1714 to confirm it all still works (looks good so far!).

(For those that are interested, there are seven failing tests when the stdlib is compiled for Android. Most are related to assumptions in the test expectations themselves.)

@modocache

modocache Apr 12, 2016

Collaborator

Thank you @weissi, your explanation was incredibly helpful! Thanks for so clearly explaining the problem, it made writing the code much easier. I have a working implementation that I will amend to this commit soon; I'm running the Android test suite from #1714 to confirm it all still works (looks good so far!).

(For those that are interested, there are seven failing tests when the stdlib is compiled for Android. Most are related to assumptions in the test expectations themselves.)

This comment has been minimized.

@modocache

modocache Apr 12, 2016

Collaborator

OK, I've amended changes as per your comments, @weissi! Again, thanks a ton!

@modocache

modocache Apr 12, 2016

Collaborator

OK, I've amended changes as per your comments, @weissi! Again, thanks a ton!

@1ace

This comment has been minimized.

Show comment
Hide comment
@1ace

1ace Feb 25, 2016

First off, thank you for this awesome work!
I haven't read much of it yet, but I just wanted to mention one thing: I think you should try to separate changes in multiple commits as much as possible to make it more digestible. That will help reviewers by letting them focus on related changes before moving on to the next commit.

My personal rule is: if you make sense to be between those two changes (ie. have one but not the other), put them in separate commits. You should obviously order those commits in a way that ensures you don't use features before adding them. For instance, the last commit should be the one adding the new target to utils/build-script once everything else is ready.

1ace commented Feb 25, 2016

First off, thank you for this awesome work!
I haven't read much of it yet, but I just wanted to mention one thing: I think you should try to separate changes in multiple commits as much as possible to make it more digestible. That will help reviewers by letting them focus on related changes before moving on to the next commit.

My personal rule is: if you make sense to be between those two changes (ie. have one but not the other), put them in separate commits. You should obviously order those commits in a way that ensures you don't use features before adding them. For instance, the last commit should be the one adding the new target to utils/build-script once everything else is ready.

Show outdated Hide outdated stdlib/public/Bionic/Glibc.swift
@@ -0,0 +1,215 @@
//===----------------------------------------------------------------------===//

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Please don't duplicate these files. Can we find some way of reusing the Glibc overlay here, adding some #ifs into its source?

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Please don't duplicate these files. Can we find some way of reusing the Glibc overlay here, adding some #ifs into its source?

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Yes, I'll take another look at stdlib/public/Glibc/CMakeLists.txt. I think this should be possible. The difficult part is that since we're building for both Linux and Android at once, I'll need to use variables other than CMAKE_SYSTEM_NAME to determine which modulemap to copy. I'll try working on it some more, thanks!

@modocache

modocache Feb 25, 2016

Collaborator

Yes, I'll take another look at stdlib/public/Glibc/CMakeLists.txt. I think this should be possible. The difficult part is that since we're building for both Linux and Android at once, I'll need to use variables other than CMAKE_SYSTEM_NAME to determine which modulemap to copy. I'll try working on it some more, thanks!

This comment has been minimized.

@modocache

modocache Mar 11, 2016

Collaborator

@gribozavr I've been trying to combine stdlib/public/Bionic and stdlib/public/Glibc.

stdlib/public/Glibc/CMakeLists.txt uses the host system name to either use a Linux or FreeBSD modulemap. I figure I can do something similar to use either a Linux or Android modulemap--but instead of using the host system name (which is always Linux when cross-compiling from Linux to Android), I produce a modulemap for each requested SDK and architecture:

# Unabbreviated source code here: https://gist.github.com/modocache/08e7aefd6c220d7750ac
foreach(SDK ${SWIFT_SDKS})
  foreach(arch ${SWIFT_SDK_${SDK}_ARCHITECTURES})
  string(TOLOWER "${SDK}" sdk)
  set(output_dir "${SWIFTLIB_DIR}/${sdk}/${arch}/glibc")

  # ...

  # Configure the modulemap based on the target. Each platform needs to
  # reference different headers, based on what's available in their glibc.
  set(modulemap_path "${CMAKE_CURRENT_BINARY_DIR}/${sdk}/${arch}/module.map")
  if("${SDK}" STREQUAL "FREEBSD")
    configure_file(module.freebsd.map.in "${modulemap_path}" @ONLY)
  elseif("${SDK}" STREQUAL "ANDROID")
    configure_file(module.android.map.in "${modulemap_path}" @ONLY)
  else()
    configure_file(module.map.in "${modulemap_path}" @ONLY)
  endif()

  # Create the lib/swift SDK/arch subdirectory and copy the configured
  # modulemap there.
  add_custom_command_target(unused_var
    COMMAND
        "${CMAKE_COMMAND}" "-E" "make_directory" "${output_dir}"
    COMMAND
        "${CMAKE_COMMAND}" "-E" "copy_if_different"
        "${modulemap_path}"
        "${output_dir}/module.map"
    CUSTOM_TARGET_NAME "copy_${sdk}_${arch}_glibc_module"
    OUTPUT "${output_dir}/module.map" "${output_dir}"
    DEPENDS "${modulemap_path}"
    COMMENT "Copying Glibc module to ${output_dir}")

  # Install the modulemap. When compiling Glibc, make sure
  # we use the correct module map for the current SDK.
  swift_install_in_component(stdlib
    FILES "${output_dir}/module.map"
    DESTINATION "${output_dir}")
  add_swift_library(swiftGlibc IS_SDK_OVERLAY
    ${sources}
    FILE_DEPENDS "copy_${sdk}_${arch}_glibc_module" "${output_dir}"
    TARGET_SDKS "${SDK}"
    SWIFT_COMPILE_FLAGS "-Xcc" "-fmodule-map-file=${modulemap_path}"
    INSTALL_IN_COMPONENT stdlib-experimental)
  endforeach()
endforeach()

The problem with this approach is that swiftc appears to expect that a Glibc modulemap exists at the following path:

/path/to/built/swift-linux-x86_64/lib/swift/glibc/module.map

Whereas the above CMake file places the module maps at two different paths, in order to avoid them overwriting one another:

/path/to/built/swift-linux-x86_64/lib/swift/linux/x86_64/glibc/module.map
/path/to/built/swift-linux-x86_64/lib/swift/android/armv7/glibc/module.map

This causes the following failure to occur when running tests that import Glibc:

<unknown>:0: error: missing required module 'SwiftGlibc'

Placing either one of these modulemaps at the expected path lib/swift/glibc/module.map gets the tests to pass for that target. That is, copying the Linux modulemap ensures the Linux build succeeds (and Android fails), and copying the Android modulemap causes Android to succeed and Linux to fail.

I'm not sure what the significance of the /path/to/built/swift-linux-x86_64/lib/swift/glibc/module.map path is, or how to hook up the compiler to use the SDK/arch subdirectory path lib/swift/android/armv7/glibc/module.map instead.

Any suggestions? I've been banging my head against this for a while, so any help would be greatly appreciated! 馃檱

@modocache

modocache Mar 11, 2016

Collaborator

@gribozavr I've been trying to combine stdlib/public/Bionic and stdlib/public/Glibc.

stdlib/public/Glibc/CMakeLists.txt uses the host system name to either use a Linux or FreeBSD modulemap. I figure I can do something similar to use either a Linux or Android modulemap--but instead of using the host system name (which is always Linux when cross-compiling from Linux to Android), I produce a modulemap for each requested SDK and architecture:

# Unabbreviated source code here: https://gist.github.com/modocache/08e7aefd6c220d7750ac
foreach(SDK ${SWIFT_SDKS})
  foreach(arch ${SWIFT_SDK_${SDK}_ARCHITECTURES})
  string(TOLOWER "${SDK}" sdk)
  set(output_dir "${SWIFTLIB_DIR}/${sdk}/${arch}/glibc")

  # ...

  # Configure the modulemap based on the target. Each platform needs to
  # reference different headers, based on what's available in their glibc.
  set(modulemap_path "${CMAKE_CURRENT_BINARY_DIR}/${sdk}/${arch}/module.map")
  if("${SDK}" STREQUAL "FREEBSD")
    configure_file(module.freebsd.map.in "${modulemap_path}" @ONLY)
  elseif("${SDK}" STREQUAL "ANDROID")
    configure_file(module.android.map.in "${modulemap_path}" @ONLY)
  else()
    configure_file(module.map.in "${modulemap_path}" @ONLY)
  endif()

  # Create the lib/swift SDK/arch subdirectory and copy the configured
  # modulemap there.
  add_custom_command_target(unused_var
    COMMAND
        "${CMAKE_COMMAND}" "-E" "make_directory" "${output_dir}"
    COMMAND
        "${CMAKE_COMMAND}" "-E" "copy_if_different"
        "${modulemap_path}"
        "${output_dir}/module.map"
    CUSTOM_TARGET_NAME "copy_${sdk}_${arch}_glibc_module"
    OUTPUT "${output_dir}/module.map" "${output_dir}"
    DEPENDS "${modulemap_path}"
    COMMENT "Copying Glibc module to ${output_dir}")

  # Install the modulemap. When compiling Glibc, make sure
  # we use the correct module map for the current SDK.
  swift_install_in_component(stdlib
    FILES "${output_dir}/module.map"
    DESTINATION "${output_dir}")
  add_swift_library(swiftGlibc IS_SDK_OVERLAY
    ${sources}
    FILE_DEPENDS "copy_${sdk}_${arch}_glibc_module" "${output_dir}"
    TARGET_SDKS "${SDK}"
    SWIFT_COMPILE_FLAGS "-Xcc" "-fmodule-map-file=${modulemap_path}"
    INSTALL_IN_COMPONENT stdlib-experimental)
  endforeach()
endforeach()

The problem with this approach is that swiftc appears to expect that a Glibc modulemap exists at the following path:

/path/to/built/swift-linux-x86_64/lib/swift/glibc/module.map

Whereas the above CMake file places the module maps at two different paths, in order to avoid them overwriting one another:

/path/to/built/swift-linux-x86_64/lib/swift/linux/x86_64/glibc/module.map
/path/to/built/swift-linux-x86_64/lib/swift/android/armv7/glibc/module.map

This causes the following failure to occur when running tests that import Glibc:

<unknown>:0: error: missing required module 'SwiftGlibc'

Placing either one of these modulemaps at the expected path lib/swift/glibc/module.map gets the tests to pass for that target. That is, copying the Linux modulemap ensures the Linux build succeeds (and Android fails), and copying the Android modulemap causes Android to succeed and Linux to fail.

I'm not sure what the significance of the /path/to/built/swift-linux-x86_64/lib/swift/glibc/module.map path is, or how to hook up the compiler to use the SDK/arch subdirectory path lib/swift/android/armv7/glibc/module.map instead.

Any suggestions? I've been banging my head against this for a while, so any help would be greatly appreciated! 馃檱

This comment has been minimized.

@modocache

modocache Mar 14, 2016

Collaborator

OK, managed to figure this one out: #1679 -- I'd definitely appreciate feedback on that approach! 馃檱

@modocache

modocache Mar 14, 2016

Collaborator

OK, managed to figure this one out: #1679 -- I'd definitely appreciate feedback on that approach! 馃檱

Show outdated Hide outdated stdlib/public/runtime/Errors.cpp
@@ -132,10 +134,18 @@ reportBacktrace(int *count)
if (count) *count = 0;
return NULL;
}
#if defined(__ANDROID__)

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Why not push the #if all the way up, to where #if !defined(__CYGWIN__) is?

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Why not push the #if all the way up, to where #if !defined(__CYGWIN__) is?

Show outdated Hide outdated stdlib/public/stubs/Stubs.cpp
return strtod(a, b);
}
static float swift_strtof_l(const char* a, char** b, locale_t c) {
return strtof(a, b);

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

This won't implement the right semantics -- the current locale will affect the output on Android.

Also, please use spaces for indentation.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

This won't implement the right semantics -- the current locale will affect the output on Android.

Also, please use spaces for indentation.

This comment has been minimized.

@zhuowei

zhuowei Feb 26, 2016

Contributor

@gribozavr Android's Bionic libc doesn't have support for locales: the only supported locale is the "C" locale. (See https://github.com/android/platform_bionic/blob/master/libc/bionic/locale.cpp#L40)

@zhuowei

zhuowei Feb 26, 2016

Contributor

@gribozavr Android's Bionic libc doesn't have support for locales: the only supported locale is the "C" locale. (See https://github.com/android/platform_bionic/blob/master/libc/bionic/locale.cpp#L40)

This comment has been minimized.

@gribozavr

gribozavr Feb 26, 2016

Collaborator

It says "// We currently support a single locale", so it can change.

You can do the same thing as Cygwin does.

@gribozavr

gribozavr Feb 26, 2016

Collaborator

It says "// We currently support a single locale", so it can change.

You can do the same thing as Cygwin does.

This comment has been minimized.

@zhuowei

zhuowei Feb 27, 2016

Contributor

@gribozavr On api 21 (Android 5.0) uselocale is present, so the real uselocale can be used. However, strtod_l and strtof_l are not present on Android even in the latest version (6.0), and strtold_l is also missing in 5.0 (the version we're targetting). So I don't think locale support is possible at this time.

@zhuowei

zhuowei Feb 27, 2016

Contributor

@gribozavr On api 21 (Android 5.0) uselocale is present, so the real uselocale can be used. However, strtod_l and strtof_l are not present on Android even in the latest version (6.0), and strtold_l is also missing in 5.0 (the version we're targetting). So I don't think locale support is possible at this time.

This comment has been minimized.

@modocache

modocache Mar 10, 2016

Collaborator

Here I've decided to:

  1. Remove the #define uselocale(a), since that function is available on API 21 (which this work is now based upon).
  2. Convert tabs to spaces.
  3. Preserve the definitions of strtod_l and friends here, since they're unavailable on Android.

Does that sound reasonable? Of course we can always improve upon this in the future.

@modocache

modocache Mar 10, 2016

Collaborator

Here I've decided to:

  1. Remove the #define uselocale(a), since that function is available on API 21 (which this work is now based upon).
  2. Convert tabs to spaces.
  3. Preserve the definitions of strtod_l and friends here, since they're unavailable on Android.

Does that sound reasonable? Of course we can always improve upon this in the future.

Show outdated Hide outdated test/IRGen/report_dead_method_call.swift
@@ -5,6 +5,9 @@
// RUN: %target-run %t/a.out p1 p2 2> %t/err3.log; FileCheck %s < %t/err3.log
// REQUIRES: executable_test
// This test correctly outputs "fatal error" on an Android device, but
// the Android test runner interprets the abort as a test failure.

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Could you rewrite it using StdlibUnitest then? It has special provisions for detecting crashes.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Could you rewrite it using StdlibUnitest then? It has special provisions for detecting crashes.

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Oh, good call! Can do. I'll submit this change as a separate pull request.

@modocache

modocache Feb 25, 2016

Collaborator

Oh, good call! Can do. I'll submit this change as a separate pull request.

Show outdated Hide outdated utils/android/adb/commands.py
@@ -0,0 +1,134 @@
from __future__ import print_function

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

License header?

@gribozavr

gribozavr Feb 25, 2016

Collaborator

License header?

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Yes of course, I knew I had forgotten something. I'll add one to all the new Python files.

@modocache

modocache Feb 25, 2016

Collaborator

Yes of course, I knew I had forgotten something. I'll add one to all the new Python files.

@modocache

This comment has been minimized.

Show comment
Hide comment
@modocache

modocache Feb 25, 2016

Collaborator

Thanks for the feedback, @1ace! I agree: this work depends on #1396, #1410, #1426, #1395, #1334, which I've issued separately to make them easier to review. Unfortunately, GitHub falls apart with a dependent stack of commits, such as this pull request would end up being. I think you'll find other ports, such as Cygwin #1108, end up being similarly large.

The problem is that all of these changes are directly related to Android. Take the os(Android) definition, for example: I don't think it would make sense for the Swift maintainers to merge a pull request that added #if os(Android) ... #endif before Swift can even be built for Android. Phabricator would allow users to specify that the commit adding os(Android) is dependent upon the commit that adds Android options to CMakeLists.txt, and so forth. GitHub has no such system of specifying dependencies, and encourages large pull requests like this one.

Of course, if the review unearths more pieces I can split out into separate pull requests, I'll definitely do so! 馃憤

Collaborator

modocache commented Feb 25, 2016

Thanks for the feedback, @1ace! I agree: this work depends on #1396, #1410, #1426, #1395, #1334, which I've issued separately to make them easier to review. Unfortunately, GitHub falls apart with a dependent stack of commits, such as this pull request would end up being. I think you'll find other ports, such as Cygwin #1108, end up being similarly large.

The problem is that all of these changes are directly related to Android. Take the os(Android) definition, for example: I don't think it would make sense for the Swift maintainers to merge a pull request that added #if os(Android) ... #endif before Swift can even be built for Android. Phabricator would allow users to specify that the commit adding os(Android) is dependent upon the commit that adds Android options to CMakeLists.txt, and so forth. GitHub has no such system of specifying dependencies, and encourages large pull requests like this one.

Of course, if the review unearths more pieces I can split out into separate pull requests, I'll definitely do so! 馃憤

Show outdated Hide outdated validation-test/stdlib/CollectionType.swift.gyb
@@ -8,6 +8,9 @@
// <rdar://problem/24357067> The compiler never finishes compiling validation-test/stdlib/CollectionType.swift.gyb in -O
// REQUIRES: swift_test_mode_optimize_none
// FIXME: This test takes too long to complete on Android.
// UNSUPPORTED: OS=linux-androideabi

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Is there a way we could split it up? Maybe something like validation-test/stdlib/Slice?

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Is there a way we could split it up? Maybe something like validation-test/stdlib/Slice?

Show outdated Hide outdated validation-test/stdlib/String.swift
import Glibc
#endif
StringTests.test("lowercaseString") {
// Use setlocale so tolower() is correct on ASCII.
setlocale(LC_ALL, "C")
setlocale(Int32(LC_ALL), "C")

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

I think the right way to fix this is to add an overlay for var LC_ALL or setlocale or both.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

I think the right way to fix this is to add an overlay for var LC_ALL or setlocale or both.

@avaidyam

This comment has been minimized.

Show comment
Hide comment
@avaidyam

avaidyam Feb 25, 2016

@modocache @zhuowei @ephemer Awesome job! I've been watching this since possibly day one. 馃憤 :shipit: 馃槃

@modocache @zhuowei @ephemer Awesome job! I've been watching this since possibly day one. 馃憤 :shipit: 馃槃

Show outdated Hide outdated utils/android/adb_test_runner/main.py
program_name))
return 1
return execute_on_device(executable_path, executable_arguments, True)

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

In our experience, executing tests on device without limiting concurrency will usually oversubscribe the device and will lead to spurious failures (out of memory issues etc.) That's because the number of device cores is typically much smaller than the number of cores on the host where lit is running.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

In our experience, executing tests on device without limiting concurrency will usually oversubscribe the device and will lead to spurious failures (out of memory issues etc.) That's because the number of device cores is typically much smaller than the number of cores on the host where lit is running.

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Ah yes, very true! One hacky solution would be to limit the number of parallel lit tests on Android. I'm sure I could find a way to do this in utils/build-script-impl or test/lit.cfg. I may even be able to query the number of device cores. Is this close to what you had in mind?

@modocache

modocache Feb 25, 2016

Collaborator

Ah yes, very true! One hacky solution would be to limit the number of parallel lit tests on Android. I'm sure I could find a way to do this in utils/build-script-impl or test/lit.cfg. I may even be able to query the number of device cores. Is this close to what you had in mind?

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

That would unnecessarily limit the parallelism for running host (compiler) tests. What I was thinking about is adding a persistent server that would be responsible for device communication, and have %target-run be a thin client that just asks the server to run a certain command. The server would throttle (or possibly load balance over multiple devices!) as appropriate.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

That would unnecessarily limit the parallelism for running host (compiler) tests. What I was thinking about is adding a persistent server that would be responsible for device communication, and have %target-run be a thin client that just asks the server to run a certain command. The server would throttle (or possibly load balance over multiple devices!) as appropriate.

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

This would be awesome. I assume we may use it to handle iOS/tvOS host tests as well some day?

One thought: should I make two pull requests? One to get Android building, and another to get its tests running and passing?

@modocache

modocache Feb 25, 2016

Collaborator

This would be awesome. I assume we may use it to handle iOS/tvOS host tests as well some day?

One thought: should I make two pull requests? One to get Android building, and another to get its tests running and passing?

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Yes, that'd likely make things easier.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Yes, that'd likely make things easier.

Show outdated Hide outdated stdlib/public/runtime/ProtocolConformance.cpp
dlclose(handle);
}
static void android_iterate_libs(void (*callback)(const char*)) {

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Could we factor this code duplication?

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Could we factor this code duplication?

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Absolutely! Will do.

@modocache

modocache Feb 25, 2016

Collaborator

Absolutely! Will do.

This comment has been minimized.

@tinysun212

tinysun212 Feb 27, 2016

Contributor

Just I made #1466, which implemented the idea I mentioned you several days ago in #1108 (Cygwin).
It is factored only for Linux/Cygwin (except APPLE), but I think many lines could be shared with Android.

@tinysun212

tinysun212 Feb 27, 2016

Contributor

Just I made #1466, which implemented the idea I mentioned you several days ago in #1108 (Cygwin).
It is factored only for Linux/Cygwin (except APPLE), but I think many lines could be shared with Android.

Show outdated Hide outdated stdlib/private/StdlibUnittest/StdlibUnittest.swift.gyb
@@ -1089,6 +1092,8 @@ func _getOSVersion() -> OSVersion {
return .Linux
#elseif os(FreeBSD)
return .FreeBSD
#elseif os(Android)
return .Android

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

This needs tests like validation-test/stdlib/StdlibUnittestFreeBSD.swift.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

This needs tests like validation-test/stdlib/StdlibUnittestFreeBSD.swift.

Show outdated Hide outdated lib/Driver/ToolChains.cpp
Arguments.push_back("-target");
Arguments.push_back("armv7-none-linux-androideabi");
const char* ndkhome = getenv("ANDROID_NDK_HOME");

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Is using this environment variable customary for targeting Android? What does gcc and clang do?

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Is using this environment variable customary for targeting Android? What does gcc and clang do?

This comment has been minimized.

@dcow

dcow Feb 26, 2016

This is customary with respect to the Android ndk build scripts. You set this and the build scripts handle constructing the paths given to gcc/clang while compiling. To my knowledge gcc/clang don't handle this themselves in any way.

@dcow

dcow Feb 26, 2016

This is customary with respect to the Android ndk build scripts. You set this and the build scripts handle constructing the paths given to gcc/clang while compiling. To my knowledge gcc/clang don't handle this themselves in any way.

Show outdated Hide outdated lib/Driver/ToolChains.cpp
Arguments.push_back("-rpath");
Arguments.push_back("-Xlinker");
Arguments.push_back(context.Args.MakeArgString(RuntimeLibPath));
if (getTriple().isAndroid()) {

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

I'm not sure this condition is appropriate in a GenericUnix toolchain...

@gribozavr

gribozavr Feb 25, 2016

Collaborator

I'm not sure this condition is appropriate in a GenericUnix toolchain...

This comment has been minimized.

@modocache

modocache Mar 15, 2016

Collaborator

Sorry for the slow turnaround on this one! I'm beginning to better understand the relationship between the Driver and the ToolChain. I'm thinking of modifying Driver::getToolChain such that it returns a toolchains::Android for Android targets. That should help isolate this Android-specific logic.

@modocache

modocache Mar 15, 2016

Collaborator

Sorry for the slow turnaround on this one! I'm beginning to better understand the relationship between the Driver and the ToolChain. I'm thinking of modifying Driver::getToolChain such that it returns a toolchains::Android for Android targets. That should help isolate this Android-specific logic.

Show outdated Hide outdated lib/Driver/ToolChains.cpp
Arguments.push_back("-Xlinker");
Arguments.push_back("-rpath");
Arguments.push_back("-Xlinker");
Arguments.push_back(context.Args.MakeArgString(RuntimeLibPath));

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Changes to the driver need to be accompanied by a driver test.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Changes to the driver need to be accompanied by a driver test.

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Can do. Do you have a suggestion on how I could set these driver flags from within CMake? Is that possible? I'm not thrilled with the getenv() approach here.

@modocache

modocache Feb 25, 2016

Collaborator

Can do. Do you have a suggestion on how I could set these driver flags from within CMake? Is that possible? I'm not thrilled with the getenv() approach here.

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Typically, clang uses -sysroot to specify the path to a copy of the target filesystem. But, I'd recommend looking into what clang actually does for Android, what is the recommended way to invoke it, and trying to replicate that, as long as it makes sense.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

Typically, clang uses -sysroot to specify the path to a copy of the target filesystem. But, I'd recommend looking into what clang actually does for Android, what is the recommended way to invoke it, and trying to replicate that, as long as it makes sense.

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

The CMake function is _add_variant_c_compile_link_flags in cmake/modules/AddSwift.cmake.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

The CMake function is _add_variant_c_compile_link_flags in cmake/modules/AddSwift.cmake.

This comment has been minimized.

@modocache

modocache Feb 25, 2016

Collaborator

Fantastic. I believe I tried that with limited success before, but I'll give it another go and ask questions if it can't get it working. Thanks!

@modocache

modocache Feb 25, 2016

Collaborator

Fantastic. I believe I tried that with limited success before, but I'll give it another go and ask questions if it can't get it working. Thanks!

This comment has been minimized.

@grp

grp Feb 25, 2016

It looks like Android does use --sysroot= for this. I found that referenced here: http://developer.android.com/ndk/guides/standalone_toolchain.html (under "Simple method").

@grp

grp Feb 25, 2016

It looks like Android does use --sysroot= for this. I found that referenced here: http://developer.android.com/ndk/guides/standalone_toolchain.html (under "Simple method").

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

OK. I think --sysroot corresponds to -sdk in the Swift driver. CC @jrose-apple

@gribozavr

gribozavr Feb 25, 2016

Collaborator

OK. I think --sysroot corresponds to -sdk in the Swift driver. CC @jrose-apple

This comment has been minimized.

@@ -105,6 +106,8 @@ std::pair<bool, bool> LangOptions::setTarget(llvm::Triple triple) {
addPlatformConditionValue("os", "watchOS");
else if (triple.isiOS())
addPlatformConditionValue("os", "iOS");
else if (triple.isAndroid())
addPlatformConditionValue("os", "Android");

This comment has been minimized.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

This change needs a test like test/Parse/ConditionalCompilation/x64FreeBSDTarget.swift for each supported triple.

@gribozavr

gribozavr Feb 25, 2016

Collaborator

This change needs a test like test/Parse/ConditionalCompilation/x64FreeBSDTarget.swift for each supported triple.

This comment has been minimized.

@modocache

modocache Mar 14, 2016

Collaborator

Done! I added test/Parse/ConditionalCompilation/armAndroidTarget.swift.

@modocache

modocache Mar 14, 2016

Collaborator

Done! I added test/Parse/ConditionalCompilation/armAndroidTarget.swift.

@gribozavr

This comment has been minimized.

Show comment
Hide comment
@gribozavr

gribozavr Feb 25, 2016

Collaborator

@swift-ci Please smoke test

Collaborator

gribozavr commented Feb 25, 2016

@swift-ci Please smoke test

@gribozavr

This comment has been minimized.

Show comment
Hide comment
@gribozavr

gribozavr Feb 25, 2016

Collaborator

@modocache There seem to be build issues on Linux x86_64, please take a look when you have a moment.

Collaborator

gribozavr commented Feb 25, 2016

@modocache There seem to be build issues on Linux x86_64, please take a look when you have a moment.

@jckarter

This comment has been minimized.

Show comment
Hide comment
@jckarter

jckarter Feb 25, 2016

Member

Looks like <mutex> is missing from the target C++ standard library:

In file included from /home/buildnode/jenkins/workspace/swift-PR-Linux/swift/stdlib/public/runtime/HeapObject.cpp:17:
/home/buildnode/jenkins/workspace/swift-PR-Linux/swift/include/swift/Basic/Lazy.h:19:10: fatal error: 'mutex' file not found
#include <mutex>
         ^
Member

jckarter commented Feb 25, 2016

Looks like <mutex> is missing from the target C++ standard library:

In file included from /home/buildnode/jenkins/workspace/swift-PR-Linux/swift/stdlib/public/runtime/HeapObject.cpp:17:
/home/buildnode/jenkins/workspace/swift-PR-Linux/swift/include/swift/Basic/Lazy.h:19:10: fatal error: 'mutex' file not found
#include <mutex>
         ^
@DCRichards

This comment has been minimized.

Show comment
Hide comment
@DCRichards

DCRichards Feb 25, 2016

this is really cool, I've never seen a 52 file change commit though, good lord!

this is really cool, I've never seen a 52 file change commit though, good lord!

@modocache

This comment has been minimized.

Show comment
Hide comment
@modocache

modocache Feb 25, 2016

Collaborator

I'm working on addressing the feedback--thanks so much for the review, @gribozavr!

In the meantime, if anyone out there wants to help with this pull request, here's the output from the test suite: https://gist.github.com/modocache/48babfa768d495806116. Six tests are failing.

Some of those failures, like IRGen/objc_simd.sil, could be addressed outside of this pull request (for that test specifically, check out some of my notes in SwiftAndroid#16).

Collaborator

modocache commented Feb 25, 2016

I'm working on addressing the feedback--thanks so much for the review, @gribozavr!

In the meantime, if anyone out there wants to help with this pull request, here's the output from the test suite: https://gist.github.com/modocache/48babfa768d495806116. Six tests are failing.

Some of those failures, like IRGen/objc_simd.sil, could be addressed outside of this pull request (for that test specifically, check out some of my notes in SwiftAndroid#16).

@lattner

This comment has been minimized.

Show comment
Hide comment
@lattner

lattner Feb 25, 2016

Collaborator

@modocache This is really awesome to see, thank you for working on it. Per the previous comment, if you can split this up into smaller PR's, that will make it easier to review and merge piecemeal.

Collaborator

lattner commented Feb 25, 2016

@modocache This is really awesome to see, thank you for working on it. Per the previous comment, if you can split this up into smaller PR's, that will make it easier to review and merge piecemeal.

@practicalswift

This comment has been minimized.

Show comment
Hide comment
@practicalswift

practicalswift Feb 25, 2016

Collaborator

@modocache Excellent stuff! Keep those great contributions coming 馃憤

Collaborator

practicalswift commented Feb 25, 2016

@modocache Excellent stuff! Keep those great contributions coming 馃憤

@unsign3d

This comment has been minimized.

Show comment
Hide comment
@unsign3d

unsign3d Feb 25, 2016

Excellent work, I'm looking forword to use it o.o

Excellent work, I'm looking forword to use it o.o

@b3ll

This comment has been minimized.

Show comment
Hide comment
@b3ll

b3ll Feb 26, 2016

馃帀

Keep up the great work @modocache! :D

b3ll commented Feb 26, 2016

馃帀

Keep up the great work @modocache! :D

@pbassut

This comment has been minimized.

Show comment
Hide comment

pbassut commented Feb 26, 2016

Great work! @modocache

@domdomegg

This comment has been minimized.

Show comment
Hide comment

Good work @modocache

@modocache modocache referenced this pull request Feb 28, 2016

Merged

Refactor code duplication in inspecting dylibs #1466

0 of 1 task complete

@at15 at15 referenced this pull request Feb 29, 2016

Closed

Weekly for 2016/03/02 #11

1 of 2 tasks complete
@modocache

This comment has been minimized.

Show comment
Hide comment
@modocache

modocache Mar 1, 2016

Collaborator

For those following this pull request:

  • I'm busy with other stuff this week, but will address feedback (like splitting this up) next week.
  • In the meantime, I'll rebase this work onto apple/swift master each morning (US time), and confirm it all still works for me.
  • If you'd like to help me address feedback, send pull requests to SwiftAndroid/swift's master branch.
  • I don't mind +1 comments, but not everyone likes them. Let's hold off on those! 馃槄
Collaborator

modocache commented Mar 1, 2016

For those following this pull request:

  • I'm busy with other stuff this week, but will address feedback (like splitting this up) next week.
  • In the meantime, I'll rebase this work onto apple/swift master each morning (US time), and confirm it all still works for me.
  • If you'd like to help me address feedback, send pull requests to SwiftAndroid/swift's master branch.
  • I don't mind +1 comments, but not everyone likes them. Let's hold off on those! 馃槄
@JesseScott

This comment has been minimized.

Show comment
Hide comment
@JesseScott

JesseScott Mar 3, 2016

Enough congratulations people! I'm as excited about this as anyone, but either offer to help or stop spamming this issue with 馃憤 's ...

Enough congratulations people! I'm as excited about this as anyone, but either offer to help or stop spamming this issue with 馃憤 's ...

@modocache modocache referenced this pull request Mar 7, 2016

Closed

Enable cross-compiling #1398

@gribozavr

This comment has been minimized.

Show comment
Hide comment
@gribozavr

gribozavr Mar 10, 2016

Collaborator

Here I've decided to:

  1. Remove the #define uselocale(a), since that function is available on API 21 (which this work is now based upon).
  2. Convert tabs to spaces.
  3. Preserve the definitions of strtod_l and friends here, since they're unavailable on Android.

Does that sound reasonable? Of course we can always improve upon this in the future.

SGTM! Thanks, @modocache!

Collaborator

gribozavr commented Mar 10, 2016

Here I've decided to:

  1. Remove the #define uselocale(a), since that function is available on API 21 (which this work is now based upon).
  2. Convert tabs to spaces.
  3. Preserve the definitions of strtod_l and friends here, since they're unavailable on Android.

Does that sound reasonable? Of course we can always improve upon this in the future.

SGTM! Thanks, @modocache!

modocache added a commit to modocache/swift that referenced this pull request Mar 14, 2016

[Glibc] Configure modulemap for target, not host
The current Glibc CMakeLists.txt uses the host machine to determine
which modulemap to use. The same modulemap can't be used for all
platforms because headers are available in different locations on
different platforms.

Using the host machine to determine which modulemap to configure and
place at a specific path in the resource dir is fine, so long as:

1. Only one Glibc is being compiled in a single CMake invocation.
2. The target machine needs the same modulemap as the host.

apple#1442 violates both of these
assumptions: the Glibc module for both Linux and Android is compiled
at the same time, and the Android target can't use the Linux modulemap.

This commit instead uses the target(s) to determine which
modulemap to use. The modulemap is configured and placed in an OS-
and architecture-specific directory in the resource dir. The path to
that modulemap is referenced by the ClangImporter (since it is no
longer at a path that is automatically discovered as an implicit
modulemap).

`lldb-import-test` does not appear to pass a valid target triple to
ClangImporter, so we disable tests that use it outside of OS X.

@modocache modocache referenced this pull request Mar 14, 2016

Merged

[Glibc] Configure modulemap for target, not host #1679

0 of 1 task complete

modocache added a commit to modocache/swift that referenced this pull request Mar 14, 2016

[Glibc] Configure modulemap for target, not host
The current Glibc CMakeLists.txt uses the host machine to determine
which modulemap to use. The same modulemap can't be used for all
platforms because headers are available in different locations on
different platforms.

Using the host machine to determine which modulemap to configure and
place at a specific path in the resource dir is fine, so long as:

1. Only one Glibc is being compiled in a single CMake invocation.
2. The target machine needs the same modulemap as the host.

apple#1442 violates both of these
assumptions: the Glibc module for both Linux and Android is compiled
at the same time, and the Android target can't use the Linux modulemap.

This commit instead uses the target(s) to determine which
modulemap to use. The modulemap is configured and placed in an OS-
and architecture-specific directory in the resource dir. The path to
that modulemap is referenced by the ClangImporter (since it is no
longer at a path that is automatically discovered as an implicit
modulemap).

`lldb-import-test` does not appear to pass a valid target triple to
ClangImporter, so we disable tests that use it outside of OS X.

modocache added a commit to SwiftAndroid/swift that referenced this pull request Mar 14, 2016

[Glibc] Configure modulemap for target, not host
The current Glibc CMakeLists.txt uses the host machine to determine
which modulemap to use. The same modulemap can't be used for all
platforms because headers are available in different locations on
different platforms.

Using the host machine to determine which modulemap to configure and
place at a specific path in the resource dir is fine, so long as:

1. Only one Glibc is being compiled in a single CMake invocation.
2. The target machine needs the same modulemap as the host.

apple#1442 violates both of these
assumptions: the Glibc module for both Linux and Android is compiled
at the same time, and the Android target can't use the Linux modulemap.

This commit instead uses the target(s) to determine which
modulemap to use. The modulemap is configured and placed in an OS-
and architecture-specific directory in the resource dir. The path to
that modulemap is referenced by the ClangImporter (since it is no
longer at a path that is automatically discovered as an implicit
modulemap).

`lldb-import-test` does not appear to pass a valid target triple to
ClangImporter, so we disable tests that use it outside of OS X.

modocache added a commit to SwiftAndroid/swift that referenced this pull request Mar 14, 2016

[Glibc] Configure modulemap for target, not host
The current Glibc CMakeLists.txt uses the host machine to determine
which modulemap to use. The same modulemap can't be used for all
platforms because headers are available in different locations on
different platforms.

Using the host machine to determine which modulemap to configure and
place at a specific path in the resource dir is fine, so long as:

1. Only one Glibc is being compiled in a single CMake invocation.
2. The target machine needs the same modulemap as the host.

apple#1442 violates both of these
assumptions: the Glibc module for both Linux and Android is compiled
at the same time, and the Android target can't use the Linux modulemap.

This commit instead uses the target(s) to determine which
modulemap to use. The modulemap is configured and placed in an OS-
and architecture-specific directory in the resource dir. The path to
that modulemap is referenced by the ClangImporter (since it is no
longer at a path that is automatically discovered as an implicit
modulemap).

`lldb-import-test` does not appear to pass a valid target triple to
ClangImporter, so we disable tests that use it outside of OS X.
@phasuksmit

This comment has been minimized.

Show comment
Hide comment

just 'wow'

@jblorenzo

This comment has been minimized.

Show comment
Hide comment
@jblorenzo

jblorenzo Apr 13, 2016

very promising 馃憤

very promising 馃憤

@GreyEcologist

This comment has been minimized.

Show comment
Hide comment
@GreyEcologist

GreyEcologist Apr 13, 2016

congrats! This is exciting stuff!

congrats! This is exciting stuff!

@ephemer

This comment has been minimized.

Show comment
Hide comment
@ephemer

ephemer Apr 13, 2016

@modocache @zhuowei @jrose-apple @gribozavr thank you all for your awesome work, looking forward to continuing with this

ephemer commented Apr 13, 2016

@modocache @zhuowei @jrose-apple @gribozavr thank you all for your awesome work, looking forward to continuing with this

@rileytestut

This comment has been minimized.

Show comment
Hide comment

Amazing!!

@zakrid

This comment has been minimized.

Show comment
Hide comment
@zakrid

zakrid Apr 13, 2016

Absolutely terrific work from everyone involved!

zakrid commented Apr 13, 2016

Absolutely terrific work from everyone involved!

@freak4pc

This comment has been minimized.

Show comment
Hide comment
@freak4pc

freak4pc Apr 13, 2016

Incredible effort from all! Beautiful work! Looking forward to seeing this move further.

Incredible effort from all! Beautiful work! Looking forward to seeing this move further.

@kleinlieu

This comment has been minimized.

Show comment
Hide comment
@kleinlieu

kleinlieu Apr 13, 2016

Wow, we just witnessed history in the making.

Wow, we just witnessed history in the making.

@zakrid

This comment has been minimized.

Show comment
Hide comment
@zakrid

zakrid Apr 13, 2016

@kleinlieu true! Watch the media/news pick this up in a few days and milk it for weeks 馃槃

zakrid commented Apr 13, 2016

@kleinlieu true! Watch the media/news pick this up in a few days and milk it for weeks 馃槃

@wxyyxc1992

This comment has been minimized.

Show comment
Hide comment
@wxyyxc1992

wxyyxc1992 Apr 14, 2016

Anticipated

Anticipated

@lovemo

This comment has been minimized.

Show comment
Hide comment

lovemo commented Apr 14, 2016

great

@kiban18

This comment has been minimized.

Show comment
Hide comment
@kiban18

kiban18 Apr 14, 2016

Amazing!!

kiban18 commented Apr 14, 2016

Amazing!!

@vapor99

This comment has been minimized.

Show comment
Hide comment
@vapor99

vapor99 Apr 14, 2016

Whoah! Very nice!

vapor99 commented Apr 14, 2016

Whoah! Very nice!

@Ir1d

This comment has been minimized.

Show comment
Hide comment
@Ir1d

Ir1d Apr 14, 2016

Congrats!

Ir1d commented Apr 14, 2016

Congrats!

@wangxuguo

This comment has been minimized.

Show comment
Hide comment

I'm out?

@kewang

This comment has been minimized.

Show comment
Hide comment
@kewang

kewang Apr 14, 2016

Today's Big Thing !!!

  1. Kobe last game
  2. Warriors 73 wins
  3. Swift accepted "Port to Android" PR

kewang commented Apr 14, 2016

Today's Big Thing !!!

  1. Kobe last game
  2. Warriors 73 wins
  3. Swift accepted "Port to Android" PR
@cHaLkdusT

This comment has been minimized.

Show comment
Hide comment
@cHaLkdusT

cHaLkdusT Apr 14, 2016

This is exciting!

This is exciting!

@meeDamian

This comment has been minimized.

Show comment
Hide comment
@meeDamian

meeDamian Apr 14, 2016

Unsubscribbling, because spam. Can someone close, to pipe comments into, comfortably silent, reaction emojis? Also, fantastic work!

Unsubscribbling, because spam. Can someone close, to pipe comments into, comfortably silent, reaction emojis? Also, fantastic work!

@JingweiWang

This comment has been minimized.

Show comment
Hide comment
@JingweiWang

JingweiWang Apr 14, 2016

OMG, I think I must to learn Swift.

OMG, I think I must to learn Swift.

@yichengchen

This comment has been minimized.

Show comment
Hide comment

Exciting锛

@nolim1t

This comment has been minimized.

Show comment
Hide comment

nolim1t commented Apr 14, 2016

Good work @modocache

@consbulaquena

This comment has been minimized.

Show comment
Hide comment
@consbulaquena

consbulaquena Apr 14, 2016

Congratulations!

Congratulations!

@nishantpractoios

This comment has been minimized.

Show comment
Hide comment

Exciting !

@AlwaysTyping

This comment has been minimized.

Show comment
Hide comment

66666666666

@rafaelwkerr

This comment has been minimized.

Show comment
Hide comment

Awesome

@conqueror

This comment has been minimized.

Show comment
Hide comment
@conqueror

conqueror Apr 14, 2016

Great news 馃憤

Great news 馃憤

@shige0501

This comment has been minimized.

Show comment
Hide comment

Good job!

@swiftc-org

This comment has been minimized.

Show comment
Hide comment

Nice work!

@Geno1024

This comment has been minimized.

Show comment
Hide comment
@Geno1024

Geno1024 Apr 15, 2016

Excellent!

Excellent!

@dodola

This comment has been minimized.

Show comment
Hide comment
@dodola

dodola Apr 15, 2016

馃憤 Excellent

dodola commented Apr 15, 2016

馃憤 Excellent

@icepy icepy referenced this pull request Apr 16, 2016

Closed

Swift鐨勬湡寰 #35

@AniOSDeveloper

This comment has been minimized.

Show comment
Hide comment

Good job!

@ray26

This comment has been minimized.

Show comment
Hide comment

ray26 commented Apr 16, 2016

6666

@andrekandore

This comment has been minimized.

Show comment
Hide comment

馃槑 Nice

@justinoboyle

This comment has been minimized.

Show comment
Hide comment

Great work.

@otymartin

This comment has been minimized.

Show comment
Hide comment
@otymartin

otymartin Apr 17, 2016

I just want to put my mark on this馃槑

otymartin commented Apr 17, 2016

I just want to put my mark on this馃槑

@istx25

This comment has been minimized.

Show comment
Hide comment
@istx25

istx25 Apr 18, 2016

Wow! Fantastic news. I'm excited to play around with this. Congrats to everyone involved. Your work is appreciated. 馃憦馃徏馃憦馃徏

istx25 commented Apr 18, 2016

Wow! Fantastic news. I'm excited to play around with this. Congrats to everyone involved. Your work is appreciated. 馃憦馃徏馃憦馃徏

@chenp-fjnu

This comment has been minimized.

Show comment
Hide comment
@chenp-fjnu

chenp-fjnu Apr 19, 2016

It's one very good new for developers, for many companies, they have IOS and Android version for every application, it will reduce the effort. Xamarin is another option, but Linux based system should support swift well. I love Swift. Wish it will become usable for real case soon.

chenp-fjnu commented Apr 19, 2016

It's one very good new for developers, for many companies, they have IOS and Android version for every application, it will reduce the effort. Xamarin is another option, but Linux based system should support swift well. I love Swift. Wish it will become usable for real case soon.

@refinedKing

This comment has been minimized.

Show comment
Hide comment
@refinedKing

refinedKing Apr 19, 2016

Good Job !!!

Good Job !!!

@apple apple locked and limited conversation to collaborators Apr 19, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.