Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac App Store Private API Rejection: Electron 5.0.10 #20027

Open
thomasdao opened this issue Aug 29, 2019 · 76 comments
Assignees
Projects

Comments

@thomasdao
Copy link

@thomasdao thomasdao commented Aug 29, 2019

Issue Details

  • Electron Version: 5.0.10

Rejection Email

ITMS-90338: Non-public API usage - The app contains or inherits from non-public classes in Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework: CAContext, CALayerHost, NSAccessibilityRemoteUIElement, NSNextStepFrame, NSThemeFrame, NSURLFileTypeMappings . If method names in your source code match the private Apple APIs listed above, altering your method names will help prevent this app from being flagged in future submissions. In addition, note that one or more of the above APIs may be located in a static library that was included with your app. If so, they must be removed. For further information, visit the Technical Support Information at http://developer.apple.com/support/technical/

@thomasdao

This comment has been minimized.

Copy link
Author

@thomasdao thomasdao commented Aug 29, 2019

I downgrade Electron to 5.0.9 and still get rejection email.

@JustinPierce

This comment has been minimized.

Copy link

@JustinPierce JustinPierce commented Aug 29, 2019

I got this rejection this morning for Electron 5.0.4, and also for 5.0.10. I think something has changed on Apple's end.

@lubo08

This comment was marked as off-topic.

Copy link

@lubo08 lubo08 commented Aug 29, 2019

Big show stopper for me. I need an urgent solution. please help.

@MarshallOfSound

This comment has been minimized.

Copy link
Member

@MarshallOfSound MarshallOfSound commented Aug 29, 2019

List of private APIs detected:

  • _fileport_makefd
  • _fileport_makeport
  • CAContext
  • CALayerHost
  • NSAccessibilityRemoteUIElement
  • NSNextStepFrame
  • NSThemeFrame
  • NSURLFileTypeMappings

Please only comment on this issue if your rejection email has APIs that are not in the list above. If you just comment +1 your comment will be removed. If you are also experiencing this rejection please react to this issue with 👍 to indicate so.

@gaodeng

This comment has been minimized.

Copy link

@gaodeng gaodeng commented Sep 2, 2019

My app has just been approved by apple. It's using electron 6.0.7.
So is this problem only in the 5.x version, or is Apple making adjustments again on the private api detection strategy?

@mytran

This comment has been minimized.

Copy link

@mytran mytran commented Sep 2, 2019

Developer relations responded and stated that they believe that the problem was on issue on their end and they will look into it. I checked tonight and my previously rejected builds are available now in App Store Connect.

@ogi1982

This comment has been minimized.

Copy link

@ogi1982 ogi1982 commented Sep 2, 2019

I just checked as well and my previously rejected build (Electron 4.2.9) is also available on store.

@MarshallOfSound

This comment has been minimized.

Copy link
Member

@MarshallOfSound MarshallOfSound commented Sep 2, 2019

Thanks @gaodeng , @mytran and @ogi1982 for that new information. It sounds like apple got a few reach-outs and either corrected their system or whitelisted the framework temporarily. Still waiting to hear back as to what exactly happened.

I'll leave this open till at least next week where hopefully we'll have more info

@thomasdao

This comment has been minimized.

Copy link
Author

@thomasdao thomasdao commented Sep 2, 2019

I can now upload my build with Electron 5.0.10 to the Store as well. I'll probably leave to @MarshallOfSound to close this ticket :)

@sofianguy sofianguy added this to Unsorted Issues in 5.0.x Sep 18, 2019
@ffflorian

This comment has been minimized.

Copy link

@ffflorian ffflorian commented Oct 30, 2019

My app using Electron 4.2.12 was just rejected because of the following APIs:

CAContext
CALayerHost
NSNextStepFrame
NSThemeFrame
NSURLFileTypeMappings
@gaodeng

This comment has been minimized.

Copy link

@gaodeng gaodeng commented Oct 31, 2019

Electron 「6.1.2」
Your app app links against the following non-public framework(s):

CAContext
CALayerHost
NSAccessibilityRemoteUIElement
NSNextStepFrame
NSThemeFrame
NSURLFileTypeMappings

@iwodoudou

This comment has been minimized.

Copy link

@iwodoudou iwodoudou commented Oct 31, 2019

Electron 5.0.11

Your app uses or references the following non-public APIs:

CAContext
CALayerHost
NSAccessibilityRemoteUIElement
NSNextStepFrame
NSThemeFrame
NSURLFileTypeMappings

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Oct 31, 2019

electron : 6.0.10

Guideline 2.5.1 - Performance - Software Requirements
Your app uses or references the following non-public APIs:

CAContext
CALayerHost
NSAccessibilityRemoteUIElement
NSNextStepFrame
NSThemeFrame
NSURLFileTypeMappings

@aydogankaragoz

This comment has been minimized.

Copy link

@aydogankaragoz aydogankaragoz commented Oct 31, 2019

electron 3.0.2

Your app app links against the following non-public framework(s):

CAContext
CALayerHost
NSURLFileTypeMappings

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Oct 31, 2019

@MarshallOfSound is there anything we can do to help ?
i don't have the skills to fix this my self

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Oct 31, 2019

@zcbenz looks like you have done patches before can you do a new one for these ?

sorry for stressing but i have important update to my app that needs to be deployed

@mytran

This comment has been minimized.

Copy link

@mytran mytran commented Oct 31, 2019

Try appealing and state that you're using Electron and those APIs are internal to Electron:
https://developer.apple.com/contact/app-store/?topic=appeal

@yegor-slate

This comment has been minimized.

Copy link

@yegor-slate yegor-slate commented Nov 1, 2019

Updated to latest electron v7.0.0 and got rejection again.

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Nov 1, 2019

@mytran
don't think do an appeal will help more than maybe once
better try to fix this

i see its a patch file in the code base
patches\chromium\mas_no_private_api.patch

if we somehow can add the APIs there
I have not figure out yet how to do it

and for those that know how to do this.
If they don't have time to fix it maybe we can sponsor them?
time is money :)

If everyone that needs this to be fixed donated some $
I guess it will stack up and maybe will speed up this fix

@gurugeek

This comment has been minimized.

Copy link

@gurugeek gurugeek commented Nov 1, 2019

rejected today electron 6.0.12 and also with 7.0.0

Your app app links against the following non-public framework(s):

CAContext
CALayerHost
NSAccessibilityRemoteUIElement
NSNextStepFrame
NSThemeFrame
NSURLFileTypeMappings

does it work with Electron 5 ?

@gurugeek

This comment has been minimized.

Copy link

@gurugeek gurugeek commented Nov 2, 2019

I informed Apple about my app using electron etc. and received this:

"Hello,

Thank you for providing this information.

Regarding 2.5.1, your app uses or references the following non-public APIs. If you do not have access to your binary or unsure how to remove the APIs in question, please contact your service provider for technical supports."

@ForU

This comment has been minimized.

Copy link

@ForU ForU commented Nov 3, 2019

3.0.0-beta.5 mas version rejected as well for private apis:

CAContext
CALayerHost
NSURLFileTypeMappings

Just a week ago, we just successfully pass apple's audit using the same mas version. I am wondering does the old electron-v3.0.0-beta.5-mas-x64.zip file got rebuild on the download server side or zip file never modified while Apple changes their private api strategy or both? any hints you guys, coz this is really frustrating and annoying.

@JCBsystem

This comment has been minimized.

@alicerunsonfedora

This comment has been minimized.

Copy link

@alicerunsonfedora alicerunsonfedora commented Nov 3, 2019

Also am getting the same issue with 6.0.11 when attempting to build hyperspacedev/hyperspace:

Your app includes a version of an SDK from Electron that violates the App Store Review Guidelines. The version of the Electron SDK you are using in your app attempts to hide the use of private APIs. This is a violation Section 2.5.1 of the App Store Review Guidelines.

Found private class usage:
CAContext
CALayerHost
NSAccessibilityRemoteUIElement
NSNextStepFrame
NSThemeFrame
NSURLFileTypeMappings

I don't know if this is related, but I think this is also causing a crash on the app as well with an "Operation not permitted" error.

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [3221]

Application Specific Information:
dyld: launch, running initializers
/usr/lib/libSystem.B.dylib
Could not set sandbox profile data: Operation not permitted (1)
@thomasdao

This comment has been minimized.

Copy link
Author

@thomasdao thomasdao commented Nov 5, 2019

@JCBsystem how do you scan the build? I used Electron Builder and not too sure which file to scan, I tried with both .app and .pkg file and the command showed error.

What I did was that I downloaded the electron binary file from the link @zcbenz gave, replace that with the local file in node_modules folder then build. I may miss some steps, if you have other suggestions please let me know so I can retry again, thanks!

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Nov 5, 2019

@thomasdao
open a console from 'xxx.app/Contents/Frameworks/Electron\ Framework.framework/Versions/A'
then run
otool -ov Electron\ Framework > ../../../../../../dump.txt
and
strings Electron\ Framework > ../../../../../../dump1.txt

open the dump files and do a search for the api keys

or send me the app and i will test it for you :)

@thomasdao

This comment has been minimized.

Copy link
Author

@thomasdao thomasdao commented Nov 5, 2019

@JCBsystem thanks for the instruction. The dump files are at https://drive.google.com/file/d/1POwnOX_jMcahUi4beBdo7viKjaeey8PY/view?usp=sharing, I think it still contains private API. Probably simply replacing the Electron binary in node_modules does not really remove the private API in final build. If you have any suggestion I can retry submitting to Apple.

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Nov 5, 2019

@thomasdao
yes the api keys is still there
looks like you dident get the test build in your build

dont know how to add the test build correct in yoru MAS build
maybe @MarshallOfSound or @zcbenz can tell you

still think the new build from zcbenz will work

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 6, 2019

@electron/get (via electron-packager or other similar tools) downloads the electron builds from the official repository and caches them in ~/Library/Caches/electron, regardless of what is found in the local node_modules/electron directory. I have replaced the cached Electron.app with the one from @zcbenz and it was picked up for packaging. (note: the patch archive has some issues, in that some folders which should be aliases are included in full and some frameworks are present, which are not usually included in the mas builds of electron).

Running otool on the resulting package does not flag any blacklisted APIs, and the application itself passes a basic smoke test. I have submitted it for Apple testing today, although the official patch of Electron by @zcbenz seems to be very close to official release. At least we could see if they are they are really checking the binaries for the linked APIs or if they look for Electron versions.

Many thanks to @zcbenz for this patch! 👍

@gurugeek

This comment has been minimized.

Copy link

@gurugeek gurugeek commented Nov 6, 2019

@netop thanks for this. Just for the mere mortals JS developers out here what are the steps to be undertaken before rebuilding and checking the package before submission ?
Step 1) download the patched version from @zcbenz at https://drive.google.com/open?id=1RwWd9U-yfpWpn6OhDO1duTXfnRXKIPxL and step 2, 3, 4.. ? :

I think there is a PR for the 5x branch is perhaps better to wait that and install it via NPM ?

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 6, 2019

@gurugeek for sure the best way is to wait for the official Electron patch.

It just happened that our app faced a critical issue which constituted a considerable incentive for trying this out. Plus there is a request in this thread from @zcbenz for someone with an Apple Dev account to test if the patch actually works.

I'll come back here with details once the Apple review team replies.

@gurugeek

This comment has been minimized.

Copy link

@gurugeek gurugeek commented Nov 6, 2019

@netop also willing to try before the official patch. Just couldn't figure it out (yet) so if you know what I need to replace/check let me know.

@sofianguy sofianguy added the 7-1-x label Nov 6, 2019
@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 7, 2019

Well... it still did not fully work. It got past the previous blacklist, but hey are now flagging these:

__CFCopyServerVersionDictionary
__CFCopySystemVersionDictionary
__kCFSystemVersionBuildVersionKey
__kCFSystemVersionProductNameKey
__kCFSystemVersionProductVersionExtraKey
__kCFSystemVersionProductVersionKey

No reference to the fileport_* methods.

I am not sure if those items are present in the @zcbenz build only or if they were also the official Electron build, I will be able to check later. It could be that @zcbenz also targeted dmg in the build, which would also explain the additional frameworks included in the bundle (see this potentially related issue and the links there).

I just wish Apple would just provide a complete API blacklist/whitelist or a tool to check the builds, instead of playing trial-and-error...

@zcbenz

This comment has been minimized.

Copy link
Member

@zcbenz zcbenz commented Nov 7, 2019

@netop Thanks for testing the build, it turns out that I forgot to clear old files before doing new build, and some files from non-MAS build were mixed into the build I uploaded.

I have re-uploaded a clean build, sorry for the trouble.
https://drive.google.com/file/d/1foCvpd2YuD7oEBFvwQwmubtdnS6_VIlN/view?usp=sharing

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 7, 2019

@zcbenz I will retry to package and submit - but could you please grant permission to download the updated build. Thank you so much!

Also, it seems that the symbols come from crashpad_handler - so maybe I can remove that file myself. Otherwise, the frameworks appear to be OK... 🤞

@zcbenz

This comment has been minimized.

Copy link
Member

@zcbenz zcbenz commented Nov 7, 2019

@netop The download link should work now. Apart from the crashpad_handler, you also need to remove Squirrel.framework from the bundle.

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 7, 2019

@zcbenz Thank you so much for the support, I did remove the Squirrel.framework the last time, I just missed crashpad_handler. Since your new build is based on 7.1.0 this time, I will stay with the previous one for convenience and safety, with the only change being the culprit module removed. I can test the 7.1.0 update after that, if the app gets approved.

I am submitting ASAP and will come back with the result once I have it.

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 8, 2019

OK - so the app got accepted and is now Ready for Sale. A big THANK YOU to @zcbenz for the custom build 🙏 .

@ngehlert

This comment has been minimized.

Copy link

@ngehlert ngehlert commented Nov 8, 2019

any time estimates for the official patch?

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Nov 8, 2019

Thank you @zcbenz!

@zcbenz

This comment has been minimized.

Copy link
Member

@zcbenz zcbenz commented Nov 8, 2019

any time estimates for the official patch?

You can follow #20965 on the progress.

Currently as disabling remote layer APIs might drag down performance significantly, we are waiting for responses from Apple to see if it is possible to unblock CAContext and CALayerHost for us, or whether is there a better way of patching to avoid performance hit.

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 8, 2019

How to use the @zcbenz mas patched build (until a cleaner solution is available)

Based on a previous request, this is a description of what to do in order to use the @zcbenz patch with your application packaging tool.

Initial note: our app is built for multiple platforms, one of which being Electron. Because of this, we need a bit more control over the packaging process, therefore are using electron-packager and not electron-builder. The following steps apply to the electron-packager approach, with the 7.0.1 build. However, if you are using builder, it should be easy enough to use the common configuration option electronDist to achieve the same result. Also, using the 7.1.0 build is similar, you should just adapt the instructions accordingly.

This is the sequence of steps I've went through:

  1. Downloaded and cleaned up the Electron.app package from @zcbenz.

    The initial 7.0.1-based package had some additional frameworks and libraries that were not needed for mas (like Squirrel and crash_pad), the updated 7.1.0-based one does not include them any more. However, both have a problem with the zip archive not processing symbolic links (aliases) correctly, and some binaries are included more than one time, making it way larger than it should. You can download the clean version from here (based on the 7.0.1 build) and use it as-is, or you may just use it as a template.

  2. Configure electron-packager to use the specific Electron build and run once

    This is depending on the way you are using the tool. In my case, I am using the API and not the CLI, so I've set the electronVersion option to specify 7.0.1 in the code. Run the packager - it will use @electron/get to download and cache an offical 7.0.1 Electron build (note: you may also use the mirrorOptions configuration, if you want to get more fancy and host the custom build somewhere - but for this temporary fix I did not go that way). Once you have this set up, let the packager create a bundle with the official build - don't upload it to store or anything. Just make sure it uses the specified Electron version.

  3. Find and replace the cached Electron build then run again

    @electron/get stores the cached downloads in the ~/Library/Caches/electron directory. Normally, there should be multiple versions of Electron downloaded there, look for the one that is used for the 7.0.1 macOS App Store package. The subdirectory name you look for should be httpsgithub.comelectronelectronreleasesdownloadv7.0.1electron-v7.0.1-mas-x64.zip and should contain a zip archive of the electron build. Replace that zip archive with the cleaned @zcbenz build you obtained in step 1. Run the packager again - it should skip the official download and use the cached version you've just replaced.

  4. Check the result for private API usage

    Use otool as instructed by @thomasdao. Open a console from xxx.app/Contents/Frameworks/Electron\ Framework.framework/Versions/A then run

    otool -ov Electron\ Framework > ../../../../../../dump.txt
    

    then search the output for the symbols in question, they should not appear

    CAContext
    CALayerHost
    NSAccessibilityRemoteUIElement
    NSNextStepFrame
    NSThemeFrame
    NSURLFileTypeMappings
    

    Alternatively, if you are more comfortable with the GUI tool, you may use MacDependency for listing and searching through symbols imported by your xxx.app-included Electron Framework.

    Note: In my initial attempt I've missed the crash_pad module in the xxx.app\Contents\Frameworks\Electron Framework.framework\Versions\A\Resources" subdirectory, and got flagged for private API usage from it. So you might want to double check for that one and make sure it was removed at step 1.

That's it - you may now submit your app to the store.

@JCBsystem

This comment has been minimized.

Copy link

@JCBsystem JCBsystem commented Nov 9, 2019

any time estimates for the official patch?

You can follow #20965 on the progress.

Currently as disabling remote layer APIs might drag down performance significantly, we are waiting for responses from Apple to see if it is possible to unblock CAContext and CALayerHost for us, or whether is there a better way of patching to avoid performance hit.

is it possible to release this now and when apple response or you find a better way of patching
you do a new update ?
it can take some time before apple response

@thomasdao

This comment has been minimized.

Copy link
Author

@thomasdao thomasdao commented Nov 9, 2019

Would it be possible to release as an alpha or beta version before Apple responds back?

@gurugeek

This comment has been minimized.

Copy link

@gurugeek gurugeek commented Nov 9, 2019

+1 on a temporary version alpha or so that we can easily install with npm and submit :) 💯

@beweinreich

This comment has been minimized.

Copy link

@beweinreich beweinreich commented Nov 9, 2019

Thank you for the awesome directions @netop and the fast work @zcbenz !

Went through the instructions this morning, submitted a build, and was approved this evening! 👏👏 thanks again.

@lefooey

This comment has been minimized.

Copy link

@lefooey lefooey commented Nov 12, 2019

Thank you @zcbenz and @netop ! I submitted a build earlier today and it has been approved!

@manish-patwal

This comment has been minimized.

Copy link

@manish-patwal manish-patwal commented Nov 15, 2019

Using electron-clean version did help removing Private-API's, but the app is lot heavier now. When building it with official(7.0.1) version app size is about 60 MB but with electron-clean the size jumped up to 300 MB.
What can i do on my side to decrease the size?

@netop

This comment has been minimized.

Copy link

@netop netop commented Nov 15, 2019

@manish-patwal Check the above post:

How to use the @zcbenz mas patched build (until a cleaner solution is available)

In the original archive from @zcbenz some items are included twice, instead of using aliases. You can correct those manually or use the download link provided at step #1.

@ivandroid

This comment has been minimized.

Copy link

@ivandroid ivandroid commented Nov 16, 2019

Got approved today! Thanks!

@ncodeyx

This comment has been minimized.

Copy link

@ncodeyx ncodeyx commented Nov 17, 2019

Is there an Electron 6 variant of this fix available? We're having issues loading Flash content with the clean version, see #20744 @netop @zcbenz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
5.0.x
Unsorted Issues
You can’t perform that action at this time.