New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite of PR #1103 + addition Android specific patches #1113

Closed
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
7 participants
@johnno1962
Contributor

johnno1962 commented Jul 11, 2017

Hi, This PR is a rewrite of #1103 after discussion that started just after it was merged. This is a restructuring of the Android specific api to provide the path to a certificates authorities file required for libcurl to work. It also includes a few other patches required for foundation to build for Android.

@johnno1962 johnno1962 changed the title from Rewite of PR #1103 after discussion + addition Andoird specific patches to Rewrite of PR #1103 after discussion + addition Andoird specific patches Jul 11, 2017

@johnno1962 johnno1962 changed the title from Rewrite of PR #1103 after discussion + addition Andoird specific patches to Rewrite of PR #1103 + addition Andoird specific patches Jul 11, 2017

@johnno1962 johnno1962 changed the title from Rewrite of PR #1103 + addition Andoird specific patches to Rewrite of PR #1103 + addition Android specific patches Jul 11, 2017

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 11, 2017

Contributor

Ready to roll?

Contributor

johnno1962 commented Jul 11, 2017

Ready to roll?

@alblue

This comment has been minimized.

Show comment
Hide comment
@alblue

alblue Jul 12, 2017

Contributor

@swift-ci please test

Contributor

alblue commented Jul 12, 2017

@swift-ci please test

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 12, 2017

Contributor

Sorry, fix of typo..

Contributor

johnno1962 commented Jul 12, 2017

Sorry, fix of typo..

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 12, 2017

Contributor

@phausler I think this PR is ready now and I’ve just done a final test run. Merge/squish away!

Contributor

johnno1962 commented Jul 12, 2017

@phausler I think this PR is ready now and I’ve just done a final test run. Merge/squish away!

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 14, 2017

Contributor

For reference, the android CACerts are there but are not prepared correctly for OpenSSL 1.0.x to use http://www.javacms.tech/questions/1728116/how-to-make-ssl-peer-verify-work-on-android

Contributor

johnno1962 commented Jul 14, 2017

For reference, the android CACerts are there but are not prepared correctly for OpenSSL 1.0.x to use http://www.javacms.tech/questions/1728116/how-to-make-ssl-peer-verify-work-on-android

@phausler

This comment has been minimized.

Show comment
Hide comment
@phausler

phausler Jul 14, 2017

Member

@swift-ci please test

Member

phausler commented Jul 14, 2017

@swift-ci please test

@phausler

This comment has been minimized.

Show comment
Hide comment
@phausler

phausler Jul 14, 2017

Member

@swift-ci please test

Member

phausler commented Jul 14, 2017

@swift-ci please test

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 15, 2017

Contributor

Did that second "please test" go through?

Contributor

johnno1962 commented Jul 15, 2017

Did that second "please test" go through?

@xwu

This comment has been minimized.

Show comment
Hide comment
@xwu

xwu Jul 15, 2017

Collaborator

@johnno1962 ...Uh, did you mean to invalidate testing for this PR by pushing unrelated changes?

Collaborator

xwu commented Jul 15, 2017

@johnno1962 ...Uh, did you mean to invalidate testing for this PR by pushing unrelated changes?

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 15, 2017

Contributor

Testing didn’t go through anyway for some reason… and I've resynced to master so these are required

Contributor

johnno1962 commented Jul 15, 2017

Testing didn’t go through anyway for some reason… and I've resynced to master so these are required

@pushkarnk

This comment has been minimized.

Show comment
Hide comment
@pushkarnk

pushkarnk Jul 17, 2017

Collaborator

@swift-ci please test

Collaborator

pushkarnk commented Jul 17, 2017

@swift-ci please test

@pushkarnk

This comment has been minimized.

Show comment
Hide comment
@pushkarnk

pushkarnk Jul 17, 2017

Collaborator

@swift-ci please test

Collaborator

pushkarnk commented Jul 17, 2017

@swift-ci please test

@pushkarnk

This comment has been minimized.

Show comment
Hide comment
@pushkarnk

pushkarnk Jul 17, 2017

Collaborator

👆You need to request twice, sometimes 😃

Collaborator

pushkarnk commented Jul 17, 2017

👆You need to request twice, sometimes 😃

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 17, 2017

Contributor

Thanks @pushkarnk, looks good to me now.

Contributor

johnno1962 commented Jul 17, 2017

Thanks @pushkarnk, looks good to me now.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 17, 2017

Contributor

@phausler, would you like me to delete the “android" directory. It’s obsolete now.

Contributor

johnno1962 commented Jul 17, 2017

@phausler, would you like me to delete the “android" directory. It’s obsolete now.

@parkera

This comment has been minimized.

Show comment
Hide comment
@parkera

parkera Jul 18, 2017

Member

Sure, that'd be good.

Member

parkera commented Jul 18, 2017

Sure, that'd be good.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 19, 2017

Contributor

I thnk this is as ready to merge as it will ever be.

Contributor

johnno1962 commented Jul 19, 2017

I thnk this is as ready to merge as it will ever be.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 27, 2017

Contributor

@parkera, I can roll this API back to being an environment variable if it will help get it merged. I’ve had no luck looking for an NDK based solution. I’ve also added code to pass the error message from libcurl through to the NSError thrown when there is a network error.

Contributor

johnno1962 commented Jul 27, 2017

@parkera, I can roll this API back to being an environment variable if it will help get it merged. I’ve had no luck looking for an NDK based solution. I’ve also added code to pass the error message from libcurl through to the NSError thrown when there is a network error.

@parkera

This comment has been minimized.

Show comment
Hide comment
@parkera

parkera Jul 29, 2017

Member

Yah, I would prefer the environment variable for now. We're really cautious about adding API.

Member

parkera commented Jul 29, 2017

Yah, I would prefer the environment variable for now. We're really cautious about adding API.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 29, 2017

Contributor

Thanks for getting back. I’ve reverted to an env var URLSessionCAInfo.

Contributor

johnno1962 commented Jul 29, 2017

Thanks for getting back. I’ve reverted to an env var URLSessionCAInfo.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 29, 2017

Contributor

I agree for an api but for an environment variable I think “UNSAFE_SSL_NOVERIFY” encapsulates that is is a bad idea more than just “NOVERIFY”. If you feel the word UNSAFE already has a conitation in Swift, I could settle for “UNSECURE_SSL_NOVERIFY”. I want to make sure there is a clear disinsentive for people who might be tempted to avoid the problems with pem files etc or cutting and pasting.

Contributor

johnno1962 commented Jul 29, 2017

I agree for an api but for an environment variable I think “UNSAFE_SSL_NOVERIFY” encapsulates that is is a bad idea more than just “NOVERIFY”. If you feel the word UNSAFE already has a conitation in Swift, I could settle for “UNSECURE_SSL_NOVERIFY”. I want to make sure there is a clear disinsentive for people who might be tempted to avoid the problems with pem files etc or cutting and pasting.

@xwu

This comment has been minimized.

Show comment
Hide comment
@xwu

xwu Jul 29, 2017

Collaborator

Insecure, not unsecure; but yes, that's much better IMO. Let's throw in an underscore between "NO" and "VERIFY", a la "GIT_SSL_NO_VERIFY".

Collaborator

xwu commented Jul 29, 2017

Insecure, not unsecure; but yes, that's much better IMO. Let's throw in an underscore between "NO" and "VERIFY", a la "GIT_SSL_NO_VERIFY".

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 29, 2017

Contributor

Cool, “INSECURE_SSL_NO_VERIFY” it is then.

Contributor

johnno1962 commented Jul 29, 2017

Cool, “INSECURE_SSL_NO_VERIFY” it is then.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Jul 29, 2017

Contributor

Good to go my end

Contributor

johnno1962 commented Jul 29, 2017

Good to go my end

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Aug 2, 2017

Contributor

Conflicts resolved, #1122 may also be merged soon

Contributor

johnno1962 commented Aug 2, 2017

Conflicts resolved, #1122 may also be merged soon

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Aug 17, 2017

Contributor

@xwu, I’m minded to change the environment variable name from “URLSessionCAInfo” to “URLSessionCertificateAuthorityInfoFile”. Thoughts?

Contributor

johnno1962 commented Aug 17, 2017

@xwu, I’m minded to change the environment variable name from “URLSessionCAInfo” to “URLSessionCertificateAuthorityInfoFile”. Thoughts?

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Aug 23, 2017

Contributor

Hi @phausler, @parkera, @pushkarnk any chance we could get moving on this PR again please? I’ve updated swift to master my end and made one further change that seemed to be required and renamed the environment variable URLSessionCertificateAuthorityInfoFile so it is self documenting. This PR also deletes the “android” directory files which are out of date and provides a localized description from curl in the NSError thrown when there is a networking failure. Thanks!

Contributor

johnno1962 commented Aug 23, 2017

Hi @phausler, @parkera, @pushkarnk any chance we could get moving on this PR again please? I’ve updated swift to master my end and made one further change that seemed to be required and renamed the environment variable URLSessionCertificateAuthorityInfoFile so it is self documenting. This PR also deletes the “android” directory files which are out of date and provides a localized description from curl in the NSError thrown when there is a networking failure. Thanks!

@xwu

This comment has been minimized.

Show comment
Hide comment
@xwu

xwu Aug 24, 2017

Collaborator

@johnno1962 Can we leave the TimeZone changes out of this PR?

It’s quite another kettle of fish and the changes here aren’t clearly correct (for example: is GMT guaranteed to be equal to UTC for all time going forward? why does “GMT” need to be rewritten to “GMT+00”? should any of this be done in the Swift portion of Foundation or in CF? is simply logging the error acceptable, or should this fatalError on Android? ultimately, a complete solution would be to figure out how to hook up CF to Android’s tzinfo—this should not be insurmountable and may not even be more involved than the workaround.)

Collaborator

xwu commented Aug 24, 2017

@johnno1962 Can we leave the TimeZone changes out of this PR?

It’s quite another kettle of fish and the changes here aren’t clearly correct (for example: is GMT guaranteed to be equal to UTC for all time going forward? why does “GMT” need to be rewritten to “GMT+00”? should any of this be done in the Swift portion of Foundation or in CF? is simply logging the error acceptable, or should this fatalError on Android? ultimately, a complete solution would be to figure out how to hook up CF to Android’s tzinfo—this should not be insurmountable and may not even be more involved than the workaround.)

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Aug 24, 2017

Contributor

No problem. It was mainly to get the tests working. I’ll check again if there is a solution in the NDK.

Contributor

johnno1962 commented Aug 24, 2017

No problem. It was mainly to get the tests working. I’ll check again if there is a solution in the NDK.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Aug 26, 2017

Contributor

@xwu I’ve resubmitted the time zone fix under #1189 in the finish. I’ve looked into the NDK options and they are very limited like they are for any UNIX. On my device there is no /usr/share/zoneinfo/ database but a single file in a proprietary format known only to Java. Answering your reservations one by one UTC and GMT are synonnyms for all intents an purposes in this context and GMT needs to be rewritten to GMT+0000 as this is the only format CFTimeZone.c understands without acces to a database by name. I’d rather not try to alter CFTimeZone.c as it is sourced differently from the swift and swift is a higher level language. An invalid Timezone should be fatal when you are debugging but a foundation library should not crash peoples apps out in the field IMO so I’ve opted for the latter. I’ve introduced a pseudo timezone “System” which people can use to format dates to the timezone of the device.

Contributor

johnno1962 commented Aug 26, 2017

@xwu I’ve resubmitted the time zone fix under #1189 in the finish. I’ve looked into the NDK options and they are very limited like they are for any UNIX. On my device there is no /usr/share/zoneinfo/ database but a single file in a proprietary format known only to Java. Answering your reservations one by one UTC and GMT are synonnyms for all intents an purposes in this context and GMT needs to be rewritten to GMT+0000 as this is the only format CFTimeZone.c understands without acces to a database by name. I’d rather not try to alter CFTimeZone.c as it is sourced differently from the swift and swift is a higher level language. An invalid Timezone should be fatal when you are debugging but a foundation library should not crash peoples apps out in the field IMO so I’ve opted for the latter. I’ve introduced a pseudo timezone “System” which people can use to format dates to the timezone of the device.

@xwu

This comment has been minimized.

Show comment
Hide comment
@xwu

xwu Aug 26, 2017

Collaborator

@johnno1962 Let's discuss the time zone issue on #1189, then.

Collaborator

xwu commented Aug 26, 2017

@johnno1962 Let's discuss the time zone issue on #1189, then.

@amraboelela

This comment has been minimized.

Show comment
Hide comment
@amraboelela

amraboelela Sep 12, 2017

Contributor

Please don't delete the files in android directory as I am working on fixing them. Thanks.

Contributor

amraboelela commented Sep 12, 2017

Please don't delete the files in android directory as I am working on fixing them. Thanks.

@alblue

This comment has been minimized.

Show comment
Hide comment
@alblue

alblue Oct 13, 2017

Contributor

@johnno1962 this PR has become somewhat stale with everything else that's been going on; can we close this one down and then later open a new PR for when the remaining changes can be viewed? I presume you've got the changes in a branch and can call them back when needed. Keeping it open won't help get it reviewed any faster, and rebasing/amending after the work on other PRs isn't likely to be of benefit.

By the way, it's easier for me to go through and review PRs when they're targeted to a specific issue; when there's multiple different potentially different changes overlaid (and many review comments) I tend to get lost. So I appreciate the work you've been doing in splitting them up and drip-feeding them as PRs so we can move things along. Thanks!

Contributor

alblue commented Oct 13, 2017

@johnno1962 this PR has become somewhat stale with everything else that's been going on; can we close this one down and then later open a new PR for when the remaining changes can be viewed? I presume you've got the changes in a branch and can call them back when needed. Keeping it open won't help get it reviewed any faster, and rebasing/amending after the work on other PRs isn't likely to be of benefit.

By the way, it's easier for me to go through and review PRs when they're targeted to a specific issue; when there's multiple different potentially different changes overlaid (and many review comments) I tend to get lost. So I appreciate the work you've been doing in splitting them up and drip-feeding them as PRs so we can move things along. Thanks!

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Oct 13, 2017

Contributor

@alblue, I’m still of the opinion this PR can be rehabilitated once the other PR’s have landed. It’s actually quite minor and I’d like to see it through on principle. Once #1198 has landed I’ll rebase and you’ll see what I mean.

Contributor

johnno1962 commented Oct 13, 2017

@alblue, I’m still of the opinion this PR can be rehabilitated once the other PR’s have landed. It’s actually quite minor and I’d like to see it through on principle. Once #1198 has landed I’ll rebase and you’ll see what I mean.

@alblue

This comment has been minimized.

Show comment
Hide comment
@alblue

alblue Oct 13, 2017

Contributor

@johnno1962 the problem is (from my reviewer's perspective) is that this thread has had 90+ comments on it and covered several different versions of the changes that you want to make, some of which are no longer relevant to the ultimate change set and with commentary that has diverged. So even if this change can be rehabilitated on this PR, it's still going to be difficult to go through and figure out what is still relevant and what isn't.

On the other hand, a new PR rebased onto master as smaller patches is much easier to get through, because they can be reviewed individually. It also helps from a punctuality perspective because more recently created PRs are towards the top of the list of reviews, and this one is languishing down at the bottom (so I have to remember to go and look for it specifically).

Finally, for this particular change set, it was a rebase and split from a prior PR with a number and 'android specific patches' as the title - so it's not really clear to me what this is actually trying to achieve.

So once the other patch set you're working on is changed, then rebasing and taking whatever is outstanding on this PR and creating a new one will make it much easier for me (and others) to help review and move through, especially if the resulting PR is relatively small. More frequent smaller PRs that are targeted to fix a specific issue are better than one large one with a lot of different changes.

Contributor

alblue commented Oct 13, 2017

@johnno1962 the problem is (from my reviewer's perspective) is that this thread has had 90+ comments on it and covered several different versions of the changes that you want to make, some of which are no longer relevant to the ultimate change set and with commentary that has diverged. So even if this change can be rehabilitated on this PR, it's still going to be difficult to go through and figure out what is still relevant and what isn't.

On the other hand, a new PR rebased onto master as smaller patches is much easier to get through, because they can be reviewed individually. It also helps from a punctuality perspective because more recently created PRs are towards the top of the list of reviews, and this one is languishing down at the bottom (so I have to remember to go and look for it specifically).

Finally, for this particular change set, it was a rebase and split from a prior PR with a number and 'android specific patches' as the title - so it's not really clear to me what this is actually trying to achieve.

So once the other patch set you're working on is changed, then rebasing and taking whatever is outstanding on this PR and creating a new one will make it much easier for me (and others) to help review and move through, especially if the resulting PR is relatively small. More frequent smaller PRs that are targeted to fix a specific issue are better than one large one with a lot of different changes.

@amraboelela

Please explain the changes which seems to be affecting non-android environments.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Oct 13, 2017

Contributor

Hi @alblue, chipping away at this rather ill-fated PR I’ve created #1264 which contains the original nub of the original PR. Hopefully we can get this merged without too much drama.

Contributor

johnno1962 commented Oct 13, 2017

Hi @alblue, chipping away at this rather ill-fated PR I’ve created #1264 which contains the original nub of the original PR. Hopefully we can get this merged without too much drama.

@johnno1962

This comment has been minimized.

Show comment
Hide comment
@johnno1962

johnno1962 Oct 13, 2017

Contributor

@alblue et all. This PR has been rebased & tested and can be merged as is. It contains three changes:

  1. The original change to Foundation/NSURLSession/http/EasyHandle.swift that started all this off.
  2. Deleting the out of date scripts in the android directory.
  3. The remainder are the minor changes to have Foundation build for Android.

As item 3. will clash with @amraboelela’s versions there is also PR #1264 which will contains just items 1. & 2. which you could also merge - Take your pick. This PR has been built on OSX and Android where the foundation tests were run to competition (with the last commit applied for android.)

 Executed 1161 tests, with 25 failures (0 unexpected) in 16.021 (16.021) seconds

You can ignore almost all of the above comments as in the end the PR was presented separately and rebased to pretty much just the original changes to Foundation/NSURLSession/http/EasyHandle.swift

Thanks for the drive on merging outstanding PRs this week. It’s been a big help to me keeping sane!

Contributor

johnno1962 commented Oct 13, 2017

@alblue et all. This PR has been rebased & tested and can be merged as is. It contains three changes:

  1. The original change to Foundation/NSURLSession/http/EasyHandle.swift that started all this off.
  2. Deleting the out of date scripts in the android directory.
  3. The remainder are the minor changes to have Foundation build for Android.

As item 3. will clash with @amraboelela’s versions there is also PR #1264 which will contains just items 1. & 2. which you could also merge - Take your pick. This PR has been built on OSX and Android where the foundation tests were run to competition (with the last commit applied for android.)

 Executed 1161 tests, with 25 failures (0 unexpected) in 16.021 (16.021) seconds

You can ignore almost all of the above comments as in the end the PR was presented separately and rebased to pretty much just the original changes to Foundation/NSURLSession/http/EasyHandle.swift

Thanks for the drive on merging outstanding PRs this week. It’s been a big help to me keeping sane!

@alblue

This comment has been minimized.

Show comment
Hide comment
@alblue

alblue Oct 16, 2017

Contributor

Since #1264 has been put through test-and-merge, I'm going to close this down and leave any outstanding fixes to other/newly filed PRs. Thanks for splitting it up and making the process easier for us to review!

Contributor

alblue commented Oct 16, 2017

Since #1264 has been put through test-and-merge, I'm going to close this down and leave any outstanding fixes to other/newly filed PRs. Thanks for splitting it up and making the process easier for us to review!

@alblue alblue closed this Oct 16, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment