Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Sparkle feeds for macOS/Windows in-app updates #202

Closed
tonyarnold opened this issue Mar 21, 2019 · 57 comments
Closed

Support Sparkle feeds for macOS/Windows in-app updates #202

tonyarnold opened this issue Mar 21, 2019 · 57 comments

Comments

@tonyarnold
Copy link

@tonyarnold tonyarnold commented Mar 21, 2019

When using HockeyApp, we used to be able to distribute application updates for macOS (using Sparkle).

AppCenter doesn't seem to have any replacement for that, which means I can't use it as a full replacement of Hockey yet.

Please consider adding support for this as soon as it's feasible.

@shpakovski
Copy link

@shpakovski shpakovski commented Mar 27, 2019

One more vote for this. Sparkle support is crucial for most Mac apps. So this feature is very important for the App Center. Thanks.

Loading

@lumaxis
Copy link
Member

@lumaxis lumaxis commented Apr 5, 2019

@tonyarnold @shpakovski The good news is: Sparkle support is already on our public roadmap! We have also mentioned it a couple of times as part of our HockeyApp transition plans, for example.
Unfortunately, we don't have a fixed plan yet of when exactly we will work on it, so keep an eye on our iteration plans.

@tonyarnold In your comment, you had asked about beta vs. production distribution.
My knowledge of Sparkle is fairly limited but there is not really a difference in the distribution mechanism for this, right? It's mainly just which build you ship through the feed.
I don't think we're intending to intentionally limit this to beta distribution but as hinted to earlier, we're in a fairly early planning stage for this, so we'd be happy about any input! 🙂

Loading

@tonyarnold
Copy link
Author

@tonyarnold tonyarnold commented Apr 5, 2019

there is not really a difference in the distribution mechanism for this, right?

That's correct, so really it comes down to you folks deciding what's acceptable use of your service. I'd love to be able to host production builds of our app on AppCenter (it just simplifies things) but there shouldn't really be front-end technical work required to support that if the feeds themselves are publicly accessible.

Loading

@shpakovski
Copy link

@shpakovski shpakovski commented Apr 7, 2019

I don’t think we’re intending to intentionally limit this to beta distribution.

This one is a very important statement. Please do your best to support downloading and Sparkle listing for public builds. This would make your service a full-stack solution for .dmg and .zip distribution. Thanks in advance!

I’d love to be able to host production builds of our app on AppCenter.

Exactly, this would be fantastic.

Loading

@vsanthanam
Copy link

@vsanthanam vsanthanam commented Jul 13, 2019

Any updates on this at all? No status changes in the roadmap -- I really cannot migrate off of HockeyApp without this... I would imagine this is not uncommon for a lot of macOS devs.

Loading

@ghost
Copy link

@ghost ghost commented Jul 22, 2019

If/when you support distribution via AppCenter, will the SUFeedURL that is already out there in our existing macOS releases be mirrored or redirected (https://rink.hockeyapp.net/api/2/apps/APP_ID) for any duration of time? If that link goes dead at some point, or if it won't serve any new versions/releases/updates if we transition to AppCenter, then our users will be stuck on whatever last version they are using.

The later this distribution feature ships in AppCenter, the greater the issue becomes as we won't have time to push out app updates that could redirect our SUFeedURL. I'd love to avoid having to build my own custom replacement beta / production Sparkle distribution workflow if something in AppCenter is already in progress and shipping soonish, but I can't risk leaving my user base behind if it isn't.

Loading

@WCByrne
Copy link

@WCByrne WCByrne commented Aug 6, 2019

@rafifyalda In anticipation of this a while back, to be safe I went ahead and setup a url on my own service that redirects the existing Hockey url. But +1 for MS to keep those Hockey urls alive and redirect them to the new AC feeds.

Also, @lumaxis / AC team, for those that are/were using AC to build and distribute, the recent change that caused AC releases to not sync back to HA is a real drag without feeds in AC. I now have to download the build from AC, re-upload to Hockey and release there. I was really enjoying the ease of releasing directly in AC and automatically getting the updated feed from Hockey. Please consider this regression if there is a chance to get these feeds higher on the priority list 🙏.

Loading

@ghost
Copy link

@ghost ghost commented Aug 7, 2019

@WCByrne: Thanks for the tip/idea, just went ahead and did the same 👍

Loading

@WCByrne
Copy link

@WCByrne WCByrne commented Aug 7, 2019

And it get's worse if using AC builds😞. Now that app center releases don't sync back to Hockey App, not only do you have to upload the app to HA to get the AppCast feed, the symbols from the AC build aren't made available to the build uploaded to HA so crashes that come in later are unsymbolicated 😭.

Loading

@lumaxis
Copy link
Member

@lumaxis lumaxis commented Aug 10, 2019

Hey all! This is of course still firmly in our plans, we just haven't gotten to it yet 😓It will definitely be available before the shutdown though 🙏🏼

@rafifyalda We do have plans to have some APIs from HockeyApp continue working after the shutdown, @derpixeldan can likely comment with more details.

Loading

@lumaxis
Copy link
Member

@lumaxis lumaxis commented Aug 10, 2019

@rafifyalda @WCByrne As a temporary workaround, you can use a post-build script in App Center to automatically upload both the build and symbols to HockeyApp.

I had built one for my own purposes a while ago, you can likely use a slightly adjusted version for your app:

#!/usr/bin/env bash

upload_to_hockeyapp () {
  zip_symbols || return 1

  curl -v \
  -F "status=2" \
  -F "ipa=@$APPCENTER_OUTPUT_DIRECTORY/MyApp.ipa" \
  -F "dsym=@symbols.dsym.zip" \
  -H "X-HockeyAppToken: $2" \
  https://rink.hockeyapp.net/api/2/apps/$1/app_versions/upload
}

zip_symbols () {
  ZIP_FILE_NAME="symbols.dsym.zip"
  SYMBOLS_DIRECTORY="$APPCENTER_OUTPUT_DIRECTORY/../symbols"

  if [ -d $SYMBOLS_DIRECTORY ]; then
    echo "Zipping symbols..."
    zip -rq $ZIP_FILE_NAME $SYMBOLS_DIRECTORY
  else
    echo "Symbols not found."
    return 1
  fi
}

if [ $AGENT_JOBSTATUS != "Succeeded" ]; then
  echo "Build not successful, cancelling upload."
  exit 1
fi

if [[ -n "$HOCKEYAPP_API_TOKEN" && -n "$HOCKEYAPP_APP_ID" ]]; then
  echo "Uploading to HockeyApp"
  upload_to_hockeyapp $HOCKEYAPP_APP_ID $HOCKEYAPP_API_TOKEN
fi

You would need to add HOCKEYAPP_APP_ID and HOCKEYAPP_API_TOKEN as custom environment variables and adjust the curl call to your liking: https://support.hockeyapp.net/kb/api/api-versions#upload-version

Loading

@dipree dipree assigned dipree and unassigned Oddj0b Aug 21, 2019
@dipree dipree changed the title Support for distribution of macOS apps Support Sparkle feeds for macOS/Windows in-app updates Sep 4, 2019
@dipree
Copy link
Member

@dipree dipree commented Sep 19, 2019

@invariant sorry to hear that, we're currently working on it and it should be done soon.

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Sep 21, 2019

@derpixeldan Hi, thanks for the update. Are you considering extending the transition date as it now seems like Sparkle support will be added very close to the planned end date for Hockey. We have a lot of Mac apps that use the HockeySDK and we need to thoroughly test everything once the AppCenter SDK is switched in. I think it's wise to extend the deadline by a few months. Or at least maintain some kind of legacy support until everyone is confident the Sparkle support is working correctly on App Center.

Loading

@dipree
Copy link
Member

@dipree dipree commented Oct 4, 2019

App Center now supports Sparkle feeds. Please read more in the documentation.

Loading

@WCByrne
Copy link

@WCByrne WCByrne commented Oct 4, 2019

Fantastic!

Question on this, what versions make it into the feed? From what I'm seeing it looks like anything released in a public distribution group shows in the feed. What if you have more than one public group?

Loading

@dipree
Copy link
Member

@dipree dipree commented Oct 4, 2019

@WCByrne the five latest public releases, regardless of the group. I’ll clarify that in the docs.

Loading

@shpakovski
Copy link

@shpakovski shpakovski commented Oct 5, 2019

@derpixeldan It works, thanks a lot! If releases are provided by public distribution groups, could you please insert sparkle:releaseNotesLink with a direct URL, so that customers could open a webpage in their browser? Should I create another Issue with this request? Thanks in advance!

Loading

@dipree
Copy link
Member

@dipree dipree commented Oct 7, 2019

@shpakovski yes, please.

Loading

@lukaskubanek
Copy link

@lukaskubanek lukaskubanek commented Oct 10, 2019

Do we need to format it locally and upload as HTML or will that be done server-side in the future?

If you decide to build it server-side in App Center, please make this optional and configurable in the administration, as I'm relying on the Markdown format to do my own processing. Changing the format would break things.

Loading

@ajaykn
Copy link

@ajaykn ajaykn commented Oct 11, 2019

Do we need to format it locally and upload as HTML or will that be done server-side in the future?

Thanks for pointing this, @arrizer .
We have fixed this and you should be able to see the release notes in sparkle feed in html format.

@lukaskubanek, This formats only for sparkle feed. I hope this works for you.

Loading

@lukaskubanek
Copy link

@lukaskubanek lukaskubanek commented Oct 11, 2019

This formats only for sparkle feed. I hope this works for you.

@Ajaykn21 Actually, I was relying on the Markdown within the Sparkle feed. So it broke my preprocessing pipeline. Do you have any other plans on changing the Sparkle feed format in the near future (e.g. #1150)? It's hard to keep pace with these changes if they are not announced in advance and the HockeyApp termination is moving closer...

Loading

@ajaykn
Copy link

@ajaykn ajaykn commented Oct 11, 2019

@lukaskubanek currently in HockeyApp also, we are converting markdown to html and so we are supporting the same in AppCenter feed.
Please let us know if you see any change in the behavior.
Also we are open for suggestions and #1150 is being tracked separately.

Loading

@lukaskubanek
Copy link

@lukaskubanek lukaskubanek commented Oct 13, 2019

@Ajaykn21 It actually makes sense to align the output in the Sparkle feed with HockeyApp. I’ve now updated my processing, so everything is fine again. My point was just about not knowing what might change under our hands. So now I’ll assume it won’t break again and I’ll proceed with the migration. 🤞

Loading

@dipree
Copy link
Member

@dipree dipree commented Oct 15, 2019

Update: Once an app was moved from HockeyApp to App Center, we forward the HockeyApp Sparkle feed to the App Center Sparkle feed. Please note that we don't move the dsa_signature and it's required to add them to the releases in App Center again.

Loading

@dipree dipree closed this Oct 15, 2019
@rishimodha
Copy link

@rishimodha rishimodha commented Oct 15, 2019

@derpixeldan Hi Dan, can you please clarify if the forwarding of the HockeyApp Sparkle feed to the App Center Sparkle feed includes handling for the differences in the app secret between the two platforms. I believe App Center has added dashes '-' in between the app secret at certain intervals.

Loading

@dipree
Copy link
Member

@dipree dipree commented Oct 15, 2019

@rishimodha we're forwarding to the correct Sparkle URL. So yes, of course we're converting the app secret. This way existing installations aren't locked out, make sure to update to the App Center Sparkle feed with any newer version of your app.

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Oct 15, 2019

@derpixeldan - thanks for confirming, just wanted to make sure!

Loading

@invariant
Copy link

@invariant invariant commented Oct 17, 2019

Has anyone got Sparkle forwarding to work?

My feed URLs are:
HA: https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089
AC: https://api.appcenter.ms/v0.1/public/sparkle/apps/58adb6c5-bea2-8ffe-22f3-04f2f383d089

The AC feed is working correctly, but the HA is not forwarding to AC feed. Instead, it's delivering a feed of really old versions for some reason.

Is there anything I have to do to enable to forwarding, @derpixeldan?

Loading

@ghost
Copy link

@ghost ghost commented Oct 17, 2019

@invariant Did you already move your app to AC? If not, check which distribution group you have set the version to in HA.

Releases uploaded to HockeyApp only show up in the App Center Sparkle feed when added to a public distribution group.

Source

Loading

@invariant
Copy link

@invariant invariant commented Oct 17, 2019

@derpixeldan Thank you. Yes, the app has been moved to App Center already and is set to a public distribution group. The Sparkle feed on App Center works correctly. The problem seems to be with the HockeyApp feed forwarding. Basically, I'm not getting any forwarding.

When I "curl" the AC sparkle feed, I get the expected feed containing the latest public release. So the AC feed is working correctly.

But when I curl the HA sparkle feed, I get a different feed. (Weirdly it contains only releases from two years ago.)

I was expecting the feed accessed with the HA sparkle URL to be the same as the AC one (or possible a redirect? I'm not sure which way you implemented it).

The actual URLs in question are in my comment above.

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Oct 21, 2019

@invariant your HA feed is correctly forwarding to the AC feed for me. It is being redirected.

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

@invariant your HA feed is correctly forwarding to the AC feed for me. It is being redirected.

Thank you Rishi. This is really weird, because when I curl the feed I get a 200 OK response, not a redirect.

nick@kvyat:scratch$ curl -v https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089
*   Trying 54.86.96.152...
* TCP_NODELAY set
* Connected to rink.hockeyapp.net (54.86.96.152) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; OU=Microsoft Corporation; CN=*.hockeyapp.net
*  start date: Nov 15 22:51:17 2017 GMT
*  expire date: Nov 15 22:51:17 2019 GMT
*  subjectAltName: host "rink.hockeyapp.net" matched cert's "*.hockeyapp.net"
*  issuer: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; OU=Microsoft IT; CN=Microsoft IT TLS CA 2
*  SSL certificate verify ok.
> GET /api/2/apps/58adb6c5bea28ffe22f304f2f383d089 HTTP/1.1
> Host: rink.hockeyapp.net
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Mon, 21 Oct 2019 09:02:27 GMT
< Content-Type: application/xml; charset=utf-8
< Content-Length: 7231
< Connection: keep-alive

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

Now this gets even weirder. If I paste the feed URL into Firefox I get a 301 redirect. But when I curl in Terminal I get a 200 OK. I feel like I'm being gaslit 😅

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

After some experimenting: The redirect is happening only if the HTTP request has an Accept: header. It can even be a blank header, so long as it is present. The following examples with curl illustrate it.

Running curl https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089 results in 200 OK and old data served.

Whereas curl -H 'Accept: ' https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089 results in the correct 301 redirect as expected.

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Oct 21, 2019

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases: https://api.appcenter.ms/v0.1/public/sparkle/apps/9a440195-a60b-447e-90d4-5b01c63bbdcc

Did you have to make any changes to get the previous releases to show up? I believe we had the app as a public distribution, but where explicitly is this set?

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases:

I'm getting releases in the App Center sparkle feed including my public releases from recent weeks, done on HockeyApp. But the most recent release which I did post-migration in AppCenter, only about 10 minutes ago, is not showing up. (Yet - I was assuming for now it was some kind of cache update delay, or maybe I didn't select the right options.)

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

tested moving over to AC with an app that's no longer in production.

if only I had been so wise .... 🤦‍♂️!

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Oct 21, 2019

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases:

I'm getting releases in the App Center sparkle feed including my public releases from recent weeks, done on HockeyApp. But the most recent release which I did post-migration in AppCenter, only about 10 minutes ago, is not showing up. (Yet - I was assuming for now it was some kind of cache update delay, or maybe I didn't select the right options.)

Did those HA releases appear instantly on the AC sparkle feed or did you have to wait a while? We're not seeing anything!

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases:

I'm getting releases in the App Center sparkle feed including my public releases from recent weeks, done on HockeyApp. But the most recent release which I did post-migration in AppCenter, only about 10 minutes ago, is not showing up. (Yet - I was assuming for now it was some kind of cache update delay, or maybe I didn't select the right options.)

Did those HA releases appear instantly on the AC sparkle feed or did you have to wait a while? We're not seeing anything!

On HA it was always instant (or near-instant).
Edit: I misinterpreted the question. There was no Sparkle feed available on AC at the time I migrated from HA to AC. And since then, I have not added any new releases (until this morning).

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Oct 21, 2019

I figure out what I had done wrong. After moving an app from HA to AC, (even if it was public on HA), I had to go into the app's settings on AC and edit the distribution group to enable "App can be downloaded by anyone with link access".

Screenshot 2019-10-21 at 11 12 16

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

Mini update. When my app was only in the auto-created "All-users-of-app" group, it did not show up in the feed, even though the group was set to public.

I added a brand new Distribution group set to public and released it to that group too, but still the new release did not show up in the feed.

Finally I deleted the "All-users-of ...." group, and at last it appears in the feed.

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

Thought I was nearly there but ..... Now I discover that my apps still won't update. The version of Sparkle my apps have in the wild has Accept: */* in the HTTP header (not sure if more recent version have a different value? I'm using a fairly old version.) Unfortunately, HA is replying 200 OK to this request, instead of redirecting.

curl -H 'Accept: */* ' https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089

Give a 200 OK with very old data.

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

I've now created a new issue #1238 to address this specific redirect bug.

Loading

@rishimodha
Copy link

@rishimodha rishimodha commented Oct 21, 2019

@invariant have you noticed that even releases uploaded via AC have a download URL that points to hockeyapp.net - is this intentional?

Loading

@invariant
Copy link

@invariant invariant commented Oct 21, 2019

@invariant have you noticed that even releases uploaded via AC have a download URL that points to hockeyapp.net - is this intentional?

Yes the download link of my latest release uploaded to AC points to a URL on the hockeyapp.net domain. But it is functional and serves the correct file.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet