Port of Foundation to Android #622

Merged
merged 25 commits into from Sep 22, 2016

Conversation

Projects
None yet
@johnno1962
Contributor

johnno1962 commented Sep 7, 2016

This PR contains the relatively minor changes to have Foundation compile with the Android NDK. There is an associated pull request with the small change apple/swift#4662 to build-script-impl in the swift project to pass through the required arguments to the ninja scritpt generator. The focus has been on getting the code to compile but preliminary testing of Regular expressions, NSHost, fetching Strings from URLs has been successful so far. There is a known issue with needing to hook thread exit to call JNI method DetachCurrentThread which I’ll have to look at before making a PR on swift-corelibs-libdispatch (resolved, see below).

Issues found and not resolved:
Optimising the creation of a string from zero length data - optimisation patched out for now
Fetching the current Locale language for web fetch request headers - also patched out

Changes tested on Linx and OSX build. Included is a minor change fixing a missing -lcurl when building "util/build-script —foundation" on Linux.

As foundation has binary dependencies on libxml2 and libcurl and it's not been able to automate the libdispatch crosscompile the build is dependent on a pre-built binary & header package downloaded by a series of scripts in the swift-corelibs-foundation/android directory. These binaries were built from the public repos pretty much as-is after some Makefilery. Details are in the README.md in this directory.

@johnno1962 johnno1962 referenced this pull request in apple/swift Sep 7, 2016

Closed

Changes for Android build of Foundation #4662

CoreFoundation/String.subproj/CFString.c
@@ -1264,12 +1264,15 @@ CF_PRIVATE CFStringRef __CFStringCreateImmutableFunnel3(
contentsDeallocator = __CFGetDefaultAllocator();
}
+ #ifndef DEPLOYMENT_TARGET_ANDROID
+ // crashes String.swift, line 51 when creating String from zero length Data

This comment has been minimized.

@modocache

modocache Sep 7, 2016

Collaborator

It'd be great to file a https://bugs.swift.org report to track this crash, and include the report number here. That way, we'll know to remove this #ifdef once the report is fixed.

@modocache

modocache Sep 7, 2016

Collaborator

It'd be great to file a https://bugs.swift.org report to track this crash, and include the report number here. That way, we'll know to remove this #ifdef once the report is fixed.

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

IIRC android's memcpy and memmove are not the same as Darwin. That may be the culprit of this issue.

@phausler

phausler Sep 7, 2016

Member

IIRC android's memcpy and memmove are not the same as Darwin. That may be the culprit of this issue.

Foundation/CGFloat.swift
@@ -611,6 +611,7 @@ public func %=(lhs: inout CGFloat, rhs: CGFloat) {
fatalError("%= is not available.")
}
+#if !os(Android)

This comment has been minimized.

@modocache

modocache Sep 7, 2016

Collaborator

IIRC tgmath is available on Android, and I think the apple/swift tests for tgmath pass when run on Android as well. I must be missing something, but why the #if !os(Android) here? And do the Foundation tests pass this way?

@modocache

modocache Sep 7, 2016

Collaborator

IIRC tgmath is available on Android, and I think the apple/swift tests for tgmath pass when run on Android as well. I must be missing something, but why the #if !os(Android) here? And do the Foundation tests pass this way?

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll have another look at it. There are many type conversions to get right.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll have another look at it. There are many type conversions to get right.

Foundation/NSData.swift
bzero(mutableBytes.advanced(by: range.location), range.length)
+ #endif

This comment has been minimized.

@modocache

modocache Sep 7, 2016

Collaborator

nit-pick: Maybe indent the memset(...) and bzero(...) lines here?

@modocache

modocache Sep 7, 2016

Collaborator

nit-pick: Maybe indent the memset(...) and bzero(...) lines here?

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

hmm. not sure about that!

@johnno1962

johnno1962 Sep 7, 2016

Contributor

hmm. not sure about that!

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

might actually be reasonable to just implement a CF inline shim for bzero; it is used a fair amount.

@phausler

phausler Sep 7, 2016

Member

might actually be reasonable to just implement a CF inline shim for bzero; it is used a fair amount.

This comment has been minimized.

@compnerd

compnerd Sep 7, 2016

Collaborator

Why not change all uses of bzero to memset? bzero is considered deprecated, and POSIX1-2008 removes the specification of bzero entirely.

@compnerd

compnerd Sep 7, 2016

Collaborator

Why not change all uses of bzero to memset? bzero is considered deprecated, and POSIX1-2008 removes the specification of bzero entirely.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll remove bzero altogether.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll remove bzero altogether.

Foundation/NSGeometry.swift
@@ -460,7 +475,7 @@ public func NSIntegralRectWithOptions(_ aRect: NSRect, _ opts: NSAlignmentOption
width = 0
}
-
+#if !os(Android)

This comment has been minimized.

@modocache

modocache Sep 7, 2016

Collaborator

nit-pick: Looks like some trailing whitespace here.

@modocache

modocache Sep 7, 2016

Collaborator

nit-pick: Looks like some trailing whitespace here.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

zapped

@johnno1962

johnno1962 Sep 7, 2016

Contributor

zapped

@modocache

This comment has been minimized.

Show comment
Hide comment
@modocache

modocache Sep 7, 2016

Collaborator

Issues found and not resolved:
Fetching the current Locale language for web fetch request headers - also patched out

I think this might be related?

Collaborator

modocache commented Sep 7, 2016

Issues found and not resolved:
Fetching the current Locale language for web fetch request headers - also patched out

I think this might be related?

Foundation/NSGeometry.swift
+ return floor( Double(value ) )
+}
+private func ceil( value: CGFloat ) -> Double {
+ return floor( Double(value ) )

This comment has been minimized.

@mekjaer

mekjaer Sep 7, 2016

ceil here?

@@ -33,6 +35,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
+#include <netdb.h>

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

These are a global change to both the swift-corelibs, CFLite and Darwin versions, changes to these types of headers come at definite risk. Can this be done without these modifications in CoreFoundation.h? (and perhaps somewhere else instead?)

@phausler

phausler Sep 7, 2016

Member

These are a global change to both the swift-corelibs, CFLite and Darwin versions, changes to these types of headers come at definite risk. Can this be done without these modifications in CoreFoundation.h? (and perhaps somewhere else instead?)

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

Which header is which of the two? I can remove it from the Darwin version and add #if __has_include(). The symbols are used in NSHost.swift.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

Which header is which of the two? I can remove it from the Darwin version and add #if __has_include(). The symbols are used in NSHost.swift.

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

So there are two CoreFoundation.h's one for swift and one for Darwin. this one is the Darwin version.

Having alterations to this header could prove to be a non starter because it would prevent us from merging changes back from CoreFoundation. That being said if the reason is compelling enough for us to change the header for ALL platforms then perhaps it can be done, but I think it would be best to tackle it separately if possible.

@phausler

phausler Sep 7, 2016

Member

So there are two CoreFoundation.h's one for swift and one for Darwin. this one is the Darwin version.

Having alterations to this header could prove to be a non starter because it would prevent us from merging changes back from CoreFoundation. That being said if the reason is compelling enough for us to change the header for ALL platforms then perhaps it can be done, but I think it would be best to tackle it separately if possible.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ve added netdb.h in the copy of this header in the SwiftRuntime directory with a #if __has_include(..)

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ve added netdb.h in the copy of this header in the SwiftRuntime directory with a #if __has_include(..)

#include <syscall.h>
+#else

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

If the previous header does not exist what is to say the sys/syscall version exists? Perhaps that should be gated as well and #error'd on the else claiming that syscall.h is required

@phausler

phausler Sep 7, 2016

Member

If the previous header does not exist what is to say the sys/syscall version exists? Perhaps that should be gated as well and #error'd on the else claiming that syscall.h is required

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

won’t it error anyway? It doesn’t fail silently or anything anti-social. This just improves the odds. I can gate it if you want.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

won’t it error anyway? It doesn’t fail silently or anything anti-social. This just improves the odds. I can gate it if you want.

Foundation/NSAffineTransform.swift
@@ -9,7 +9,7 @@
#if os(OSX) || os(iOS)
import Darwin
-#elseif os(Linux)
+#elseif os(Linux) || os(Android)

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

why is Android named Glibc? shouldn't it be import Bionic?

@phausler

phausler Sep 7, 2016

Member

why is Android named Glibc? shouldn't it be import Bionic?

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

Probably true, but what does that accomplish for us besides making us put another import in our files?

@parkera

parkera Sep 7, 2016

Member

Probably true, but what does that accomplish for us besides making us put another import in our files?

This comment has been minimized.

@phausler

phausler Sep 8, 2016

Member

so perhaps it should be import Libc or System not Darwin then huh?

@phausler

phausler Sep 8, 2016

Member

so perhaps it should be import Libc or System not Darwin then huh?

This comment has been minimized.

@parkera

parkera Sep 12, 2016

Member

Anyone want to file a bug for the Swift project? :)

@parkera

parkera Sep 12, 2016

Member

Anyone want to file a bug for the Swift project? :)

This comment has been minimized.

@modocache

modocache Sep 13, 2016

Collaborator

I started this discussion on the swift-evolution mailing list, but was too lazy to push it forward. @compnerd is now leading the charge. 👍

@modocache

modocache Sep 13, 2016

Collaborator

I started this discussion on the swift-evolution mailing list, but was too lazy to push it forward. @compnerd is now leading the charge. 👍

This comment has been minimized.

@johnno1962

johnno1962 Sep 14, 2016

Contributor

This boilerplate goes away for most users as if they import Foundation as there is an @_exported import of the right C library in NSSwiftRuntime.swift.

@johnno1962

johnno1962 Sep 14, 2016

Contributor

This boilerplate goes away for most users as if they import Foundation as there is an @_exported import of the right C library in NSSwiftRuntime.swift.

Foundation/NSLog.swift
+
+// hook for System.out.println()
+#if os(Android)
+public var debug = {

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

can we name this something else? debug is potentially a very generically named symbol it could be used for storing strings, bools etc that might be something completely different and stomping upon that name might be a bit confusing to developers.

I would expect that NSLog actually hooks to android_log_print so that NSLog's show up in adb

@phausler

phausler Sep 7, 2016

Member

can we name this something else? debug is potentially a very generically named symbol it could be used for storing strings, bools etc that might be something completely different and stomping upon that name might be a bit confusing to developers.

I would expect that NSLog actually hooks to android_log_print so that NSLog's show up in adb

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

This is a temporary convenience. I can rename it or put it in the NSLog class if you like.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

This is a temporary convenience. I can rename it or put it in the NSLog class if you like.

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

I agree with @phausler - please no additional public symbols, and especially not one named debug.

@parkera

parkera Sep 7, 2016

Member

I agree with @phausler - please no additional public symbols, and especially not one named debug.

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

ANLog?

@johnno1962

johnno1962 Sep 8, 2016

Contributor

ANLog?

@@ -624,9 +624,12 @@ fileprivate extension URLSessionTask {
var result = [("Connection", "keep-alive"),
("User-Agent", userAgentString),
]
+ #if !os(Android)
+ // Crashes on Android

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

please file a ticket on this. and perhaps a stack trace.

@phausler

phausler Sep 7, 2016

Member

please file a ticket on this. and perhaps a stack trace.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

ok.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

ok.

This comment has been minimized.

@parkera

parkera Sep 12, 2016

Member

And put the JIRA number in the comment so we can find it later.

@parkera

parkera Sep 12, 2016

Member

And put the JIRA number in the comment so we can find it later.

+ exit
+fi
+
+\cp -v "${BUILD_DIR}/foundation-linux-x86_64/Foundation/libFoundation.so" "${SWIFT_ROOT}/lib/swift/android" &&

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

why is the configuration being passed as a x86_64 triple? shouldn't it be armv7a-linux-androideabi? which would emit the build path as foundation-android-armv7

@phausler

phausler Sep 7, 2016

Member

why is the configuration being passed as a x86_64 triple? shouldn't it be armv7a-linux-androideabi? which would emit the build path as foundation-android-armv7

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ve no idea how build-script-impl works in detail TBH. I may need some help here.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ve no idea how build-script-impl works in detail TBH. I may need some help here.

This comment has been minimized.

@phausler

phausler Sep 9, 2016

Member

So the build-script-impl needs to pass a target triple to the foundation configuration script. If that is not being passed you are building a strange hodgepodge beast of potentially incompatible compiled code.

@phausler

phausler Sep 9, 2016

Member

So the build-script-impl needs to pass a target triple to the foundation configuration script. If that is not being passed you are building a strange hodgepodge beast of potentially incompatible compiled code.

This comment has been minimized.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

Pretty sure this is ok or it wouldn’t even link. https://github.com/SwiftJava/swift/blob/master/utils/build-script-impl#L2269 has been patched to pass in the triple and there is some extra code to make sure that C and Swift get the same one: https://github.com/SwiftJava/swift-corelibs-foundation/blob/master/lib/script.py#L47. The file paths come from build-script-impl

@johnno1962

johnno1962 Sep 9, 2016

Contributor

Pretty sure this is ok or it wouldn’t even link. https://github.com/SwiftJava/swift/blob/master/utils/build-script-impl#L2269 has been patched to pass in the triple and there is some extra code to make sure that C and Swift get the same one: https://github.com/SwiftJava/swift-corelibs-foundation/blob/master/lib/script.py#L47. The file paths come from build-script-impl

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

Hmm it looks like the "error" in the script actually is:
SWIFT_BUILD_PATH="$(build_directory ${host} swift)"
etc.

That is claiming to emit the build directory as the host.

@phausler

phausler Sep 15, 2016

Member

Hmm it looks like the "error" in the script actually is:
SWIFT_BUILD_PATH="$(build_directory ${host} swift)"
etc.

That is claiming to emit the build directory as the host.

This comment has been minimized.

@johnno1962

johnno1962 Sep 16, 2016

Contributor

Trying not to tangle with “build-script-impl”. I understand it’s being refactored to take into account cross compiles apple/swift#4662

@johnno1962

johnno1962 Sep 16, 2016

Contributor

Trying not to tangle with “build-script-impl”. I understand it’s being refactored to take into account cross compiles apple/swift#4662

+
+\cp -v "${BUILD_DIR}/foundation-linux-x86_64/Foundation/Foundation.swift"* "${SWIFT_ROOT}/lib/swift/android/armv7" &&
+
+rsync -arv "${BUILD_DIR}/foundation-linux-x86_64/Foundation/usr/lib/swift/CoreFoundation" "${SWIFT_ROOT}/lib/swift/"

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

how are these packagable with JNI apps?
can they be just dropped in as a pre-built?

@phausler

phausler Sep 7, 2016

Member

how are these packagable with JNI apps?
can they be just dropped in as a pre-built?

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

yes, I have a modified gradle plugin which copies them out of buidl/…/lib/swift/android along with the swift shared lbs.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

yes, I have a modified gradle plugin which copies them out of buidl/…/lib/swift/android along with the swift shared lbs.

@phausler

This comment has been minimized.

Show comment
Hide comment
@phausler

phausler Sep 7, 2016

Member

For the Android port does this need to have a JNI_OnLoad or is that entry point higher?

Member

phausler commented Sep 7, 2016

For the Android port does this need to have a JNI_OnLoad or is that entry point higher?

@johnno1962

This comment has been minimized.

Show comment
Hide comment
Contributor

johnno1962 commented Sep 7, 2016

@phausler

This comment has been minimized.

Show comment
Hide comment
@phausler

phausler Sep 7, 2016

Member

Will that mean we will need a hook in Thread.swift for VM context destruction?

Member

phausler commented Sep 7, 2016

Will that mean we will need a hook in Thread.swift for VM context destruction?

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Sep 7, 2016

Contributor

Absolutely. Android requires you to detach the thread and won’t allow you to do it with anything on the stack, claiming you are "stilll processing". Some help required here also. I haven’t made much headway understanding libdispatch, kqueues, etc. Threads seem to time out after 15 seconds if that sounds familiar crashing the app. Only way to stop it crashing is to keep making requests on the queue.

Contributor

johnno1962 commented Sep 7, 2016

Absolutely. Android requires you to detach the thread and won’t allow you to do it with anything on the stack, claiming you are "stilll processing". Some help required here also. I haven’t made much headway understanding libdispatch, kqueues, etc. Threads seem to time out after 15 seconds if that sounds familiar crashing the app. Only way to stop it crashing is to keep making requests on the queue.

@@ -203,6 +206,7 @@ strlcat(char * dst, const char * src, size_t maxlen) {
}
return dstlen + srclen;
}
+#endif

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

Why is fd_mask defined in an else clause with definitions of strlcpy/strlcat? Are the two related in some way that I am missing?

@parkera

parkera Sep 7, 2016

Member

Why is fd_mask defined in an else clause with definitions of strlcpy/strlcat? Are the two related in some way that I am missing?

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’m just saving a line. I’ll separate them.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’m just saving a line. I’ll separate them.

@@ -14,6 +14,8 @@
*/
+#include <stdarg.h>

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

Nothing should be above the defined macro.

@parkera

parkera Sep 7, 2016

Member

Nothing should be above the defined macro.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

This is the only way it would build for some reason.. I’ll take another look

@johnno1962

johnno1962 Sep 7, 2016

Contributor

This is the only way it would build for some reason.. I’ll take another look

This comment has been minimized.

@johnno1962

johnno1962 Sep 14, 2016

Contributor

This problem has gone away since I completely rebuilt the toolchain.

@johnno1962

johnno1962 Sep 14, 2016

Contributor

This problem has gone away since I completely rebuilt the toolchain.

Foundation/NSData.swift
@@ -930,7 +930,7 @@ open class NSMutableData : NSData {
}
open func resetBytes(in range: NSRange) {
- bzero(mutableBytes.advanced(by: range.location), range.length)
+ memset(mutableBytes.advanced(by: range.location), 0, range.length)

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

What was wrong with bzero here? I think I would rather have a compatibility shim for Android than excise it from our code, if it's missing.

@parkera

parkera Sep 7, 2016

Member

What was wrong with bzero here? I think I would rather have a compatibility shim for Android than excise it from our code, if it's missing.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

Created one in NSSwiftRuntime.swift

@johnno1962

johnno1962 Sep 7, 2016

Contributor

Created one in NSSwiftRuntime.swift

Foundation/NSFileManager.swift
import Glibc
#endif
+#if os(Android)

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

I'd really like any special case for other platforms to at least come with a comment explaining why it's needed and what could be done to elide it in the future. Even better would be to avoid as much platform-specific code as possible, or put it into a central location so we can count on its presence everywhere in the project.

@parkera

parkera Sep 7, 2016

Member

I'd really like any special case for other platforms to at least come with a comment explaining why it's needed and what could be done to elide it in the future. Even better would be to avoid as much platform-specific code as possible, or put it into a central location so we can count on its presence everywhere in the project.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

agreed though this specific case is difficult to avoid/centralise.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

agreed though this specific case is difficult to avoid/centralise.

Foundation/NSFileManager.swift
@@ -279,8 +285,10 @@ open class FileManager : NSObject {
#if os(OSX) || os(iOS)
let ti = (TimeInterval(s.st_mtimespec.tv_sec) - kCFAbsoluteTimeIntervalSince1970) + (1.0e-9 * TimeInterval(s.st_mtimespec.tv_nsec))
-#else
+#elseif !os(Android)

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

I would prefer #elseif os(Android) -- that is, keep the #if in the positive sense.

@parkera

parkera Sep 7, 2016

Member

I would prefer #elseif os(Android) -- that is, keep the #if in the positive sense.

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll change this. I wanted the cases to be in order of likelyhood.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll change this. I wanted the cases to be in order of likelyhood.

Foundation/NSGeometry.swift
+private func +( left: CGFloat.NativeType, right: CGFloat.NativeType ) -> Double {
+ return Double(left) + Double(right)
+}
+#endif

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

No dead code please.

@parkera

parkera Sep 7, 2016

Member

No dead code please.

Foundation/NSGeometry.swift
+private func round( value: CGFloat.NativeType ) -> Double {
+ return round( Double(value) )
+}
+#if false

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

This and the other changes to CGFloat implementations aren't very clear to me. CGFloat should actually be Float on 32-bit and Double on 64-bit.

@parkera

parkera Sep 7, 2016

Member

This and the other changes to CGFloat implementations aren't very clear to me. CGFloat should actually be Float on 32-bit and Double on 64-bit.

This comment has been minimized.

@phausler

phausler Sep 7, 2016

Member

this is likely generating incorrect code since the host OS is leaking into the build product (per my commentary on the triples) where it is building with the flags of 64 bit but emitting 32 bit. Which is probably incorrect.

@phausler

phausler Sep 7, 2016

Member

this is likely generating incorrect code since the host OS is leaking into the build product (per my commentary on the triples) where it is building with the flags of 64 bit but emitting 32 bit. Which is probably incorrect.

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

The generated build.ninja seemed correct. I'm thinking the file paths are a red herring.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

The generated build.ninja seemed correct. I'm thinking the file paths are a red herring.

Foundation/NSGeometry.swift
@@ -459,8 +476,7 @@ public func NSIntegralRectWithOptions(_ aRect: NSRect, _ opts: NSAlignmentOption
if aRect.size.width.native < 0 {
width = 0
}
-
-
+#if !os(Android) // FIXME!!!

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

I think we need to understand what's going on here; I don't really want to see a // FIXME in code. JIRA number, or a fatalError of some kind (like our NSUnimplemented).

@parkera

parkera Sep 7, 2016

Member

I think we need to understand what's going on here; I don't really want to see a // FIXME in code. JIRA number, or a fatalError of some kind (like our NSUnimplemented).

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’m still arguing with the compiler about exactly what is going on in NSGeometry.swift re types. At the moment a signifiant peice of code is effectively commented out.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’m still arguing with the compiler about exactly what is going on in NSGeometry.swift re types. At the moment a signifiant peice of code is effectively commented out.

Foundation/NSTask.swift
@@ -7,6 +7,7 @@
// See http://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
+#if !os(Android)

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

I don't want to comment out whole files for specific platforms like this. Is it impossible to support? If so we should put a fatalError in a useful place (like init).

Either that or we need a better availability strategy for the API (e.g., marking the class as unavailable on android only).

@parkera

parkera Sep 7, 2016

Member

I don't want to comment out whole files for specific platforms like this. Is it impossible to support? If so we should put a fatalError in a useful place (like init).

Either that or we need a better availability strategy for the API (e.g., marking the class as unavailable on android only).

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

This file does not compile at all on Android the way things are. This seemed the least intrusive change

@johnno1962

johnno1962 Sep 7, 2016

Contributor

This file does not compile at all on Android the way things are. This seemed the least intrusive change

+ #else
+ // HACK: Operation Queues not working on Android
+ completion(data, response, nil)
+ #endif

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

What's the plan to make sure dispatch and operation queues are actually working?

@parkera

parkera Sep 7, 2016

Member

What's the plan to make sure dispatch and operation queues are actually working?

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

Operation queue is probably the first thing to look at as it is in Foundation. I have the suspicion that the dispatch thread problem is going to be difficult to resolve considering how deep the hook would need to be. It means that you can not make any call though to Java from a background thread at the moment. There may have to be other strategies such as a barrier of some sort.

Any hints you can provide here would be greatly appreciated. I can’t even detemine the code path. Should I be looking at the libkqueue or libpwq code for where threads for queues are created and time out??

@johnno1962

johnno1962 Sep 7, 2016

Contributor

Operation queue is probably the first thing to look at as it is in Foundation. I have the suspicion that the dispatch thread problem is going to be difficult to resolve considering how deep the hook would need to be. It means that you can not make any call though to Java from a background thread at the moment. There may have to be other strategies such as a barrier of some sort.

Any hints you can provide here would be greatly appreciated. I can’t even detemine the code path. Should I be looking at the libkqueue or libpwq code for where threads for queues are created and time out??

Foundation/NSXMLNode.swift
public static let nodePreserveAll = Options(rawValue: Options([.nodePreserveNamespaceOrder, .nodePreserveAttributeOrder, .nodePreserveEntities, .nodePreservePrefixes, .nodePreserveCDATA, .nodePreserveEmptyElements, .nodePreserveQuotes, .nodePreserveWhitespace, .nodePreserveDTD, .nodePreserveCharacterReferences]).rawValue | UInt(bitPattern: 0xFFF00000))
+ #else
+ public static let nodePreserveAll = Options(rawValue: Options([.nodePreserveNamespaceOrder, .nodePreserveAttributeOrder, .nodePreserveEntities, .nodePreservePrefixes, .nodePreserveCDATA, .nodePreserveEmptyElements, .nodePreserveQuotes, .nodePreserveWhitespace, .nodePreserveDTD, .nodePreserveCharacterReferences]).rawValue | UInt(bitPattern: 0x7FF00000))
+ #endif

This comment has been minimized.

@parkera

parkera Sep 7, 2016

Member

Why is this #if required?

@parkera

parkera Sep 7, 2016

Member

Why is this #if required?

This comment has been minimized.

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll check this. the 0xFFF00000 wouldn’t compile before…. It’s not a valid 32 bit int.
Foundation/NSXMLNode.swift:86:339: error: integer literal '4293918720' overflows when stored into 'Int’
Feels like a compiler oversight. I’ll raise a Jira ticket. There seems to be no way to express more than 31 bit integer literals

@johnno1962

johnno1962 Sep 7, 2016

Contributor

I’ll check this. the 0xFFF00000 wouldn’t compile before…. It’s not a valid 32 bit int.
Foundation/NSXMLNode.swift:86:339: error: integer literal '4293918720' overflows when stored into 'Int’
Feels like a compiler oversight. I’ll raise a Jira ticket. There seems to be no way to express more than 31 bit integer literals

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Sep 8, 2016

Contributor

New commit with responses to comments pushed. The battle with NSGeometry is won and Operation queues work if you make sure -DDEPLOYMENT_ENABLE_LIBDISPATCH is defined when you build. Less common code path does not seem to work. I’ll raise a ticket. Focus turns to the problem of detaching threads if they have interacted with JNI before they exit. Any hint about where to look appreciated.

Contributor

johnno1962 commented Sep 8, 2016

New commit with responses to comments pushed. The battle with NSGeometry is won and Operation queues work if you make sure -DDEPLOYMENT_ENABLE_LIBDISPATCH is defined when you build. Less common code path does not seem to work. I’ll raise a ticket. Focus turns to the problem of detaching threads if they have interacted with JNI before they exit. Any hint about where to look appreciated.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Sep 8, 2016

Contributor

Jira tickets created SR 2587-2590

Contributor

johnno1962 commented Sep 8, 2016

Jira tickets created SR 2587-2590

Foundation/CGFloat.swift
@_transparent
public func acos(_ x: CGFloat) -> CGFloat {
- return CGFloat(acos(x.native))
+ return CGFloat(acos(x.double))

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

What is going on here? Doesn't this change the result of acos(_: CGFloat) for all 32-bit platforms?

@xwu

xwu Sep 8, 2016

Collaborator

What is going on here? Doesn't this change the result of acos(_: CGFloat) for all 32-bit platforms?

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

acos() takes a double whereas CGFloat.native is a Float on 32 bit platforms

@johnno1962

johnno1962 Sep 8, 2016

Contributor

acos() takes a double whereas CGFloat.native is a Float on 32 bit platforms

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

D'oh, of course. So that means these functions never did compile on 32-bit platforms.

I'm guessing you didn't mean to add double as part of the CGFloat public API, though. OTOH, I suppose it ought to be public if you're invoking from an @_transparent function. Would maybe public var _doubleValue be appropriate? And shouldn't it be @_transparent itself?

@xwu

xwu Sep 8, 2016

Collaborator

D'oh, of course. So that means these functions never did compile on 32-bit platforms.

I'm guessing you didn't mean to add double as part of the CGFloat public API, though. OTOH, I suppose it ought to be public if you're invoking from an @_transparent function. Would maybe public var _doubleValue be appropriate? And shouldn't it be @_transparent itself?

This comment has been minimized.

@phausler

phausler Sep 8, 2016

Member

This file was pulled mostly from the overlay which means that we need to keep it in sync as the overlay changes (and for note that builds 32 bit targets so it seems we are still missing something here)

@phausler

phausler Sep 8, 2016

Member

This file was pulled mostly from the overlay which means that we need to keep it in sync as the overlay changes (and for note that builds 32 bit targets so it seems we are still missing something here)

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

Noted. I tried making it internal and got a compier assertion:
SIL verification failed: function_ref inside fragile function cannot reference a private or hidden symbol: (SingleFunction && RefF->isExternalDeclaration()) || RefF->hasValidLinkageForFragileRef()

@johnno1962

johnno1962 Sep 8, 2016

Contributor

Noted. I tried making it internal and got a compier assertion:
SIL verification failed: function_ref inside fragile function cannot reference a private or hidden symbol: (SingleFunction && RefF->isExternalDeclaration()) || RefF->hasValidLinkageForFragileRef()

Foundation/IndexPath.swift
@@ -10,6 +10,8 @@
//
//===----------------------------------------------------------------------===//
+import CoreFoundation

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

This seems to be unused?

@xwu

xwu Sep 8, 2016

Collaborator

This seems to be unused?

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

It’s there for a free() in the code.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

It’s there for a free() in the code.

This comment has been minimized.

@phausler

phausler Sep 8, 2016

Member

free is not defined in CoreFoundation, it is defined in either Darwin or Glibc so the import should be of one of those.

@phausler

phausler Sep 8, 2016

Member

free is not defined in CoreFoundation, it is defined in either Darwin or Glibc so the import should be of one of those.

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

fixed

@johnno1962

johnno1962 Sep 8, 2016

Contributor

fixed

@@ -838,7 +841,7 @@ extension URLSessionTask {
var data = Data(capacity: bodyData!.length)
data.append(Data(bytes: bodyData!.bytes, count: bodyData!.length))
- s.delegateQueue.addOperation {
+ s.delegateQueue.addOperation {

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

Nit: unintentional outdent?

@xwu

xwu Sep 8, 2016

Collaborator

Nit: unintentional outdent?

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

sorted

@johnno1962

johnno1962 Sep 8, 2016

Contributor

sorted

Foundation/CGFloat.swift
@@ -143,6 +143,10 @@ public struct CGFloat {
/// The native value.
public var native: NativeType
+
+ public var double: Double {

This comment has been minimized.

@phausler

phausler Sep 8, 2016

Member

This is an addition to API surface area, can this not be done in the call sites?

@phausler

phausler Sep 8, 2016

Member

This is an addition to API surface area, can this not be done in the call sites?

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

There are an awful lot of call sites.. I’ve made it internal

@johnno1962

johnno1962 Sep 8, 2016

Contributor

There are an awful lot of call sites.. I’ve made it internal

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

@_transparent functions aren't supposed to invoke internal methods. See: https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst

It is doable to do this manually, yes? After all, it's just writing Double(foo.native) instead of foo.double.

@xwu

xwu Sep 8, 2016

Collaborator

@_transparent functions aren't supposed to invoke internal methods. See: https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst

It is doable to do this manually, yes? After all, it's just writing Double(foo.native) instead of foo.double.

This comment has been minimized.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

The original versoin did this. I wanted to centralise this conversion.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

The original versoin did this. I wanted to centralise this conversion.

This comment has been minimized.

@xwu

xwu Sep 9, 2016

Collaborator

Understandable. Having been bitten by a 10x regression when && accidentally lost its @_transparent for a few weeks, my worry here is about the cost of an additional non-@_transparent function call at each of these conversions. Is that silly?

@xwu

xwu Sep 9, 2016

Collaborator

Understandable. Having been bitten by a 10x regression when && accidentally lost its @_transparent for a few weeks, my worry here is about the cost of an additional non-@_transparent function call at each of these conversions. Is that silly?

This comment has been minimized.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

In the end it is @_transparent public

@johnno1962

johnno1962 Sep 9, 2016

Contributor

In the end it is @_transparent public

Foundation/NSLog.swift
+
+// hook for System.out.println()
+#if os(Android)
+public var ANLog = {

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

Does this need to be public? Also, aren't two-letter prefixes reserved by Apple?

@xwu

xwu Sep 8, 2016

Collaborator

Does this need to be public? Also, aren't two-letter prefixes reserved by Apple?

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

Used for debugging. It’s replaced by a closure calling System.out.println() by the app.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

Used for debugging. It’s replaced by a closure calling System.out.println() by the app.

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

Maybe AndroidLog then, since AN is a two-letter prefix? But does this hook have to live in NSLog.swift?

@xwu

xwu Sep 8, 2016

Collaborator

Maybe AndroidLog then, since AN is a two-letter prefix? But does this hook have to live in NSLog.swift?

This comment has been minimized.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

It’s gotta live somewhere! This is the only way to log on Android. Stdout is not captured.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

It’s gotta live somewhere! This is the only way to log on Android. Stdout is not captured.

This comment has been minimized.

@phausler

phausler Sep 9, 2016

Member

My suggestion is to in the backing for NSLog use android_log_print and similarly for the print function.

Technically you could funopen a new descriptor for stdout to pipe logging on Android (I have done that on previous projects that I have worked on and it worked reasonably well for logging)

@phausler

phausler Sep 9, 2016

Member

My suggestion is to in the backing for NSLog use android_log_print and similarly for the print function.

Technically you could funopen a new descriptor for stdout to pipe logging on Android (I have done that on previous projects that I have worked on and it worked reasonably well for logging)

This comment has been minimized.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

Thanks for the pointer. The unpopular ANLog is no more and NSLog now works.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

Thanks for the pointer. The unpopular ANLog is no more and NSLog now works.

Foundation/NSLog.swift
+#if os(Android)
+public var ANLog = {
+ (msg: String) in
+ NSLog( msg )

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

Nit: code style--no spaces inside parens.

@xwu

xwu Sep 8, 2016

Collaborator

Nit: code style--no spaces inside parens.

Foundation/NSSwiftRuntime.swift
@_exported import Glibc
#endif
+#if os(Android) // shim required for bzero
+func bzero( _ ptr: UnsafeMutableRawPointer, _ size: size_t ) {
+ memset( ptr, 0, size )

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

Same nit: no spaces inside parens.

@xwu

xwu Sep 8, 2016

Collaborator

Same nit: no spaces inside parens.

Foundation/NSFileManager.swift
import Glibc
#endif
+#if os(Android) // struct stat.st_mode is UInt32
+func &( left: UInt32, right: mode_t ) -> mode_t {

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

Best to annotate explicitly as internal? Also, same nit about space inside parens.

@xwu

xwu Sep 8, 2016

Collaborator

Best to annotate explicitly as internal? Also, same nit about space inside parens.

This comment has been minimized.

@johnno1962

johnno1962 Sep 9, 2016

Contributor

done

@johnno1962

johnno1962 Sep 9, 2016

Contributor

done

Foundation/NSXMLNode.swift
public static let nodePreserveAll = Options(rawValue: Options([.nodePreserveNamespaceOrder, .nodePreserveAttributeOrder, .nodePreserveEntities, .nodePreservePrefixes, .nodePreserveCDATA, .nodePreserveEmptyElements, .nodePreserveQuotes, .nodePreserveWhitespace, .nodePreserveDTD, .nodePreserveCharacterReferences]).rawValue | UInt(bitPattern: 0xFFF00000))
+ #else
+ //// 0xFFF00000 is not a valid Int literal on 32 bit systems

This comment has been minimized.

@xwu

xwu Sep 8, 2016

Collaborator

If it's not valid on 32-bit systems generally, can we do better than os(Android)?

@xwu

xwu Sep 8, 2016

Collaborator

If it's not valid on 32-bit systems generally, can we do better than os(Android)?

This comment has been minimized.

@johnno1962

johnno1962 Sep 8, 2016

Contributor

Better version on the way

@johnno1962

johnno1962 Sep 8, 2016

Contributor

Better version on the way

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Sep 15, 2016

Contributor

How does it look now?

Contributor

johnno1962 commented Sep 15, 2016

How does it look now?

@@ -25,6 +25,9 @@
#if DEPLOYMENT_TARGET_WINDOWS
#include <process.h>
#endif
+#ifdef __ANDROID__

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

@parkera should we perhaps instead of using macros defined by the compiler use a DEPLOYMENT_TARGET_ANDROID for these?

@phausler

phausler Sep 15, 2016

Member

@parkera should we perhaps instead of using macros defined by the compiler use a DEPLOYMENT_TARGET_ANDROID for these?

This comment has been minimized.

@johnno1962

johnno1962 Sep 16, 2016

Contributor

I can change it back if you like.

@johnno1962

johnno1962 Sep 16, 2016

Contributor

I can change it back if you like.

+ CFStringGetBytes(message, CFRangeMake(0, CFStringGetLength(message)),
+ kCFStringEncodingUTF8, ' ', FALSE, buffer, usedBufLen, &usedBufLen);
+ buffer[usedBufLen] = '\000';
+ __android_log_print(ANDROID_LOG_INFO, "Swift", "%s", buffer);

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

For a tag it should probably be something other than "Swift". the process name would probably be more what users are expecting. Additionally the __android_log_print level enum can map reasonably well onto CFLogLevel.

Note: this is just window dressing imho it just makes the experience better and is not a blocking item for this PR

@phausler

phausler Sep 15, 2016

Member

For a tag it should probably be something other than "Swift". the process name would probably be more what users are expecting. Additionally the __android_log_print level enum can map reasonably well onto CFLogLevel.

Note: this is just window dressing imho it just makes the experience better and is not a blocking item for this PR

This comment has been minimized.

@johnno1962

johnno1962 Sep 15, 2016

Contributor

Mapped the log levels but I couldn’t find a way to get the app name without reaching back to Java through JNI. The process id is recorded:

I/System.out(32441): Swift: Process 1/1
W/Swift (32441): HERE

@johnno1962

johnno1962 Sep 15, 2016

Contributor

Mapped the log levels but I couldn’t find a way to get the app name without reaching back to Java through JNI. The process id is recorded:

I/System.out(32441): Swift: Process 1/1
W/Swift (32441): HERE

CoreFoundation/String.subproj/CFString.c
@@ -1264,12 +1264,16 @@ CF_PRIVATE CFStringRef __CFStringCreateImmutableFunnel3(
contentsDeallocator = __CFGetDefaultAllocator();
}
+ #ifndef __ANDROID__
+ // https://bugs.swift.org/browse/SR-2587
+ // crashes String.swift, line 51 when creating String from zero length Data

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

IIRC bionic's malloc is built with error checking enabled such that free is a bit more pedantic on heap corruption (and has some gnarly behavioral differences)

It would be nice to see a backtrace and values of locals at the crash point in the bug, not everyone who might be versed to fix this has easy access to an Android device and development environment for them ;)

@phausler

phausler Sep 15, 2016

Member

IIRC bionic's malloc is built with error checking enabled such that free is a bit more pedantic on heap corruption (and has some gnarly behavioral differences)

It would be nice to see a backtrace and values of locals at the crash point in the bug, not everyone who might be versed to fix this has easy access to an Android device and development environment for them ;)

This comment has been minimized.

@johnno1962

johnno1962 Sep 16, 2016

Contributor

I think I’ve been able to resolve this problem. Constant strings have a fake isa which wasn’t being linked or relocated correctly so I created an alias in the source. The Locale problem was just a manifestation of the same issue as it also used CONST_STRING_DECL().

#ifdef __ANDROID__
// Avoids crashes on Android
// https://bugs.swift.org/browse/SR-2587
// https://bugs.swift.org/browse/SR-2588
// Seemed to be a linker/relocation? problem.
// CFStrings using CONST_STRING_DECL() were not working
// Applies reference to _NSCFConstantString's isa here
// rather than using a linker option to create an alias.
#define __CFConstantStringClassReference _TMC10Foundation19_NSCFConstantString
#endif
@johnno1962

johnno1962 Sep 16, 2016

Contributor

I think I’ve been able to resolve this problem. Constant strings have a fake isa which wasn’t being linked or relocated correctly so I created an alias in the source. The Locale problem was just a manifestation of the same issue as it also used CONST_STRING_DECL().

#ifdef __ANDROID__
// Avoids crashes on Android
// https://bugs.swift.org/browse/SR-2587
// https://bugs.swift.org/browse/SR-2588
// Seemed to be a linker/relocation? problem.
// CFStrings using CONST_STRING_DECL() were not working
// Applies reference to _NSCFConstantString's isa here
// rather than using a linker option to create an alias.
#define __CFConstantStringClassReference _TMC10Foundation19_NSCFConstantString
#endif
lib/script.py
DSTROOT = """ + Configuration.current.install_directory.absolute() + """
"""
+ swift_triple = triple if triple == "armv7-none-linux-androideabi" else Configuration.current.target.swift_triple

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

It seems like this should actually not be logic here but instead live in lib/target.py instead

@phausler

phausler Sep 15, 2016

Member

It seems like this should actually not be logic here but instead live in lib/target.py instead

This comment has been minimized.

@johnno1962

johnno1962 Sep 15, 2016

Contributor

Moved into target.py

@johnno1962

johnno1962 Sep 15, 2016

Contributor

Moved into target.py

@@ -630,9 +630,12 @@ fileprivate extension URLSessionTask {
var result = [("Connection", "keep-alive"),
("User-Agent", userAgentString),
]
+ #if !os(Android)
+ // Crashes on Android https://bugs.swift.org/browse/SR-2588

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

The bug claims no stack trace, without one the chances of this being fixed is low. Does lldb not work for Android? it should be able to interface with the gdbserver loaded with the apk.

@phausler

phausler Sep 15, 2016

Member

The bug claims no stack trace, without one the chances of this being fixed is low. Does lldb not work for Android? it should be able to interface with the gdbserver loaded with the apk.

This comment has been minimized.

@johnno1962

johnno1962 Sep 15, 2016

Contributor

Not familiar with debugging C on Android. Do you have any pointers - Is this the sort of thing? https://github.com/mapbox/mapbox-gl-native/wiki/Android-debugging-with-remote-GDB. Seems very involved. Tried debugging with Android Studio and it just disconnects when the error comes up so I’m wondering if the same will happen if I do get gdb working.

Connecting to net.zhuoweizhang.swifthello
Connected to the target VM, address: 'localhost:8600', transport: 'socket'
Disconnected from the target VM, address: 'localhost:8600', transport: ‘socket'

And following the instrux above I get:
bash: ./arm-linux-androideabi-gdb: No such file or directory

If anyone wants to TeamView to investiagte this that would be fine.

@johnno1962

johnno1962 Sep 15, 2016

Contributor

Not familiar with debugging C on Android. Do you have any pointers - Is this the sort of thing? https://github.com/mapbox/mapbox-gl-native/wiki/Android-debugging-with-remote-GDB. Seems very involved. Tried debugging with Android Studio and it just disconnects when the error comes up so I’m wondering if the same will happen if I do get gdb working.

Connecting to net.zhuoweizhang.swifthello
Connected to the target VM, address: 'localhost:8600', transport: 'socket'
Disconnected from the target VM, address: 'localhost:8600', transport: ‘socket'

And following the instrux above I get:
bash: ./arm-linux-androideabi-gdb: No such file or directory

If anyone wants to TeamView to investiagte this that would be fine.

This comment has been minimized.

@johnno1962

johnno1962 Sep 16, 2016

Contributor

This is resolved. See above

@johnno1962

johnno1962 Sep 16, 2016

Contributor

This is resolved. See above

+android port of the "icu" libraries in `ANDROID_ICU_UC`
+downloaded from [here](https://github.com/SwiftAndroid/libiconv-libicu-android/releases/download/android-ndk-r12/libiconv-libicu-armeabi-v7a-ubuntu-15.10-ndk-r12.tar.gz).
+The port was tested against api 21 on a Android v5.1.1 LG K4 Phone (Lollipop)
+and requires an Ubuntu 15 host with the Android NDK Gold linker from

This comment has been minimized.

@phausler

phausler Sep 15, 2016

Member

So this won't build as a darwin host? that seems subpar; what would it take to fix that to actually build on a primary development platform of swift?

@phausler

phausler Sep 15, 2016

Member

So this won't build as a darwin host? that seems subpar; what would it take to fix that to actually build on a primary development platform of swift?

This comment has been minimized.

@johnno1962

johnno1962 Sep 15, 2016

Contributor

I’ll look into it. As far as I know the original Android Swift compiler port which this is based on specifies Ubuntu 15. https://github.com/apple/swift/blob/master/docs/Android.md

@johnno1962

johnno1962 Sep 15, 2016

Contributor

I’ll look into it. As far as I know the original Android Swift compiler port which this is based on specifies Ubuntu 15. https://github.com/apple/swift/blob/master/docs/Android.md

@phausler

This comment has been minimized.

Show comment
Hide comment
@phausler

phausler Sep 15, 2016

Member

So this only supports armv7 (no neon right?) and not arm64 yet?
Also it is worth noting that currently the CI for swift does not build anything other than Ubuntu and Darwin based platforms. What can we expect as per CI for this as a target? I would expect that as we iterate there may be portions where the Android target might break due to changes in the other sides of code-branches etc.

Member

phausler commented Sep 15, 2016

So this only supports armv7 (no neon right?) and not arm64 yet?
Also it is worth noting that currently the CI for swift does not build anything other than Ubuntu and Darwin based platforms. What can we expect as per CI for this as a target? I would expect that as we iterate there may be portions where the Android target might break due to changes in the other sides of code-branches etc.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Sep 15, 2016

Contributor

Yes armv7 only as per the compiler though some 32/64 bit wrinkles have been ironed out along the way. We’re a long way from CI for Android though that would be the longer tem goal. Most notably the libdispatch build has an outstanding PR apple/swift-corelibs-libdispatch#162 then there is libxml2 and libcurl to work into a build for Android somehow. It will take some time for a cross compile to bed into the build process. Right now these dependencies are provided as binaries for anyone wanting to work on Foundation for Android itself.

Contributor

johnno1962 commented Sep 15, 2016

Yes armv7 only as per the compiler though some 32/64 bit wrinkles have been ironed out along the way. We’re a long way from CI for Android though that would be the longer tem goal. Most notably the libdispatch build has an outstanding PR apple/swift-corelibs-libdispatch#162 then there is libxml2 and libcurl to work into a build for Android somehow. It will take some time for a cross compile to bed into the build process. Right now these dependencies are provided as binaries for anyone wanting to work on Foundation for Android itself.

@modocache

This comment has been minimized.

Show comment
Hide comment
@modocache

modocache Sep 15, 2016

Collaborator

What can we expect as per CI for this as a target?

I would love to add CI for Android. I had been meaning to email @tkremenek but haven't had the time -- apologies. I'll send that email today.

If Apple isn't interested in supporting an Android target in @swift-ci, I can set up a @swift-android-ci GitHub bot, as well as a few dedicated machines and Android devices, that can be used to test Android.

I would expect that as we iterate there may be portions where the Android target might break due to changes in the other sides of code-branches etc.

Breakages occur (albeit these days pretty rarely) for apple/swift's Android support due to the lack of CI as well. I fix them when they pop up. CI would be ideal, but I this works well enough for now -- there are only a handful of consumers of the Swift stdlib for Android, so hardly anyone notices when it breaks. I imagine this is the same for Linux ARM targets.

So this won't build as a darwin host? that seems subpar; what would it take to fix that to actually build on a primary development platform of swift?

As luck would have it, I started working on cross-compiling the Swift stdlib for Android from a macOS host this morning. I think it's possible, in large part thanks to work like apple/swift#2497 and apple/swift#2560. However, there are still quite a few assumptions baked into Swift's CMake build system: "If my host machine is macOS, then pass the macOS SDK path to all clang++ invocations...", etc. I'm working on fixing these now.

Collaborator

modocache commented Sep 15, 2016

What can we expect as per CI for this as a target?

I would love to add CI for Android. I had been meaning to email @tkremenek but haven't had the time -- apologies. I'll send that email today.

If Apple isn't interested in supporting an Android target in @swift-ci, I can set up a @swift-android-ci GitHub bot, as well as a few dedicated machines and Android devices, that can be used to test Android.

I would expect that as we iterate there may be portions where the Android target might break due to changes in the other sides of code-branches etc.

Breakages occur (albeit these days pretty rarely) for apple/swift's Android support due to the lack of CI as well. I fix them when they pop up. CI would be ideal, but I this works well enough for now -- there are only a handful of consumers of the Swift stdlib for Android, so hardly anyone notices when it breaks. I imagine this is the same for Linux ARM targets.

So this won't build as a darwin host? that seems subpar; what would it take to fix that to actually build on a primary development platform of swift?

As luck would have it, I started working on cross-compiling the Swift stdlib for Android from a macOS host this morning. I think it's possible, in large part thanks to work like apple/swift#2497 and apple/swift#2560. However, there are still quite a few assumptions baked into Swift's CMake build system: "If my host machine is macOS, then pass the macOS SDK path to all clang++ invocations...", etc. I'm working on fixing these now.

@tkremenek

This comment has been minimized.

Show comment
Hide comment
@tkremenek

tkremenek Sep 15, 2016

Member

@modocache The idea we are pursuing is to add another Jenkins master that could be used to add additional machines for different platforms. The second master would be for security reasons, but the intent would be to enable more CI testing on additional platforms the community cares about, including Android. Things are still being investigated which is why it hasn't been communicated on swift-dev yet.

Member

tkremenek commented Sep 15, 2016

@modocache The idea we are pursuing is to add another Jenkins master that could be used to add additional machines for different platforms. The second master would be for security reasons, but the intent would be to enable more CI testing on additional platforms the community cares about, including Android. Things are still being investigated which is why it hasn't been communicated on swift-dev yet.

@modocache

This comment has been minimized.

Show comment
Hide comment
@modocache

modocache Sep 15, 2016

Collaborator

Thanks, @tkremenek! Please loop me in on those talks at whatever stage you think is appropriate. I'd be happy to contribute developer time, hardware, or other resources.

What can we expect as per CI for this as a target?

In the short term, I propose no official CI. If the Android build breaks, its users can follow up with patches. That's how I've been handling Swift's Android support since April. We can add CI for Foundation on Android once the mechanism @tkremenek described is in place.

Collaborator

modocache commented Sep 15, 2016

Thanks, @tkremenek! Please loop me in on those talks at whatever stage you think is appropriate. I'd be happy to contribute developer time, hardware, or other resources.

What can we expect as per CI for this as a target?

In the short term, I propose no official CI. If the Android build breaks, its users can follow up with patches. That's how I've been handling Swift's Android support since April. We can add CI for Foundation on Android once the mechanism @tkremenek described is in place.

@@ -764,7 +767,28 @@ void CFLog(CFLogLevel lev, CFStringRef format, ...) {
#if DEPLOYMENT_RUNTIME_SWIFT
// Temporary as Swift cannot import varag C functions
void CFLog1(CFLogLevel lev, CFStringRef message) {
+#ifdef __ANDROID__
+ android_LogPriority priority = ANDROID_LOG_UNKNOWN;
+ switch ( lev ) {

This comment has been minimized.

@xwu

xwu Sep 16, 2016

Collaborator

Nit (sorry!)--spaces inside parens differs from code style in this file.

@xwu

xwu Sep 16, 2016

Collaborator

Nit (sorry!)--spaces inside parens differs from code style in this file.

+ }
+ CFIndex blen = message ? CFStringGetMaximumSizeForEncoding(CFStringGetLength(message), kCFStringEncodingUTF8) + 1 : 0;
+ char *buf = message ? (char *)malloc(blen) : 0;
+ if ( buf ) {

This comment has been minimized.

@xwu

xwu Sep 16, 2016

Collaborator

Nit--here too, and once more inside the block.

@xwu

xwu Sep 16, 2016

Collaborator

Nit--here too, and once more inside the block.

+they will cause an exception as DetachCurrentThread has not
+been called. To avoid this your app must include the line:
+
+ DispatchGroup.threadCleanupCallback = JNI_DetachCurrentThread

This comment has been minimized.

@xwu

xwu Sep 16, 2016

Collaborator

Not primarily related to this commit, but if a thread cleanup hook could be useful for platforms other than Android, it'd be nice to have a Swift evolution proposal to add it as part of the official public API of the Swift libdispatch overlay.

Otherwise, I'd imagine you'll be asked to prefix the name with an underscore, i.e.: _threadCleanup[Callback|Handler] (and I think "handler" is the terminology used in the overlay).

@xwu