New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternatives to Buddybuild? #1330

Closed
rnystrom opened this Issue Jan 2, 2018 · 71 comments

Comments

Projects
None yet
@rnystrom
Member

rnystrom commented Jan 2, 2018

Just as I was getting familiar w/ it....

Existing Free Starter plans and Android app development will be discontinued on March 1, 2018.

We will almost certainly lose support for this project. Time to find a new CI. Any suggestions?

I'm reading this article which offers:

I read about Buildkite and AppCenter on Hacker News.

I'm also considering open source, self-hosted solutions so that something like this doesn't happen again:

@rnystrom rnystrom added the question label Jan 2, 2018

@jamala

This comment has been minimized.

jamala commented Jan 2, 2018

For another self-hosted option, TeamCity(https://www.jetbrains.com/teamcity)?

@diogot

This comment has been minimized.

diogot commented Jan 2, 2018

Usually, CircleCI job queue for open source projects is much less crowded than TravisCI.

@freak4pc

This comment has been minimized.

freak4pc commented Jan 2, 2018

My $0.02 from managing some of the RxSwiftCommunity repos.

Travis is absolute garbage (or has become over time).
Queueing is intolerable and slows down development efforts (waiting 50 minutes for a 90s build is unacceptable) and configuration is relatively annoying.

We're slowly but surely moving most repos away to CirlceCI and have been very happy with it. Queueing is really snappy and fair, and the configuration is relatively a breeze.

Also heard great things about Bitrise in that sense.

@Sherlouk

This comment has been minimized.

Member

Sherlouk commented Jan 2, 2018

Super shame about BB, be interesting to keep an eye on it as I don't think Apple would kill it - but yea

Had experience with an older version of Jenkins, as a tool it's definitely capable but would require quite a lot of maintenance/configuration and in the art of keeping things as open and collaborative probably not the best thing honestly

Also if we go for another "big" provider, we run the same risk of it just being swept up by another firm playing Monopoly

I'd be inclined to try Bitrise, my 2 cents

@ElektrojungeMS

This comment has been minimized.

ElektrojungeMS commented Jan 2, 2018

I’m obviously biased ;) but something like this is not gonna happen with App Center.
Please get in touch in case you’re interested.

@BasThomas

This comment has been minimized.

Collaborator

BasThomas commented Jan 2, 2018

cc @Palleas

Sent with GitHawk

@ijansch

This comment has been minimized.

ijansch commented Jan 2, 2018

Https://Buildozer.io supports both iOS and Android. (disclosure: I’m one of its founders)

@orta

This comment has been minimized.

Contributor

orta commented Jan 3, 2018

We've been using CircleCI on all Artsy iOS projects that require a mac for CI - the OSS queues haven't been an issue like travis'

@sregg

This comment has been minimized.

sregg commented Jan 3, 2018

I spent the whole day trying out Bitrise and App Center. So far, they don't look as easy to use and magical as BuddyBuild did...
I'm happy for the BB team (and proud as I'm in Vancouver as well) but very pissed as a user...
BuddyBuild was one of those services that just work, with almost no configuration needed.

@rizwankce

This comment has been minimized.

Collaborator

rizwankce commented Jan 3, 2018

I really liked the way how Buddybuild works. I have tired Circle CI before but there are some things to notice here is it uses Fastlane for signing and deploy to Test Flight, Automatic Signing cant be used. have to use "Manual"

@rnystrom

This comment has been minimized.

Member

rnystrom commented Jan 3, 2018

I setup some time to chat w/ @TroubleMakerBen, AppCenter looks really nice. Gonna learn more about it and report back!

@yeradis

This comment has been minimized.

yeradis commented Jan 3, 2018

Check https://buildkite.com/ they offer free account for OSS

@benoittgt

This comment has been minimized.

benoittgt commented Jan 3, 2018

Huge fan of bitrise here. We use it a lot with our solution https://www.appaloosa-store.com/

@derpixeldan

This comment has been minimized.

derpixeldan commented Jan 3, 2018

@sregg what exactly wasn't working for you in App Center? I'd be happy to help with anything Build specific.

@sregg

This comment has been minimized.

sregg commented Jan 3, 2018

@derpixeldan for instance, the Slack integration is all manual using webhooks compared to just a checkbox with BB. Also, there's no way to start a build number from a specified number (i.e. my current build number on BB). Finally, signing the iOS app didn't look as straightforward as BB's (I think I only gave them my Apple id username/password and they managed the certificate and provisioning automatically)

@lukekim

This comment has been minimized.

lukekim commented Jan 4, 2018

I am the engineering manager at Microsoft for App Center Build. We have a great team making improvements every week and we are committed to multi-platform support.

@sregg great feedback and I think improvements in all those areas are on our backlog.

Also feel free to DM me on Twitter at https://twitter.com/0xlukekim with any issues/concerns/feedback.

@freak4pc

This comment has been minimized.

freak4pc commented Jan 4, 2018

I have to say I've been very impressed with App Center mainly in the "meta" sense of:

  • The development efforts Microsoft are putting into it.
  • The ownership and care some of the employees show by commenting here.
  • I've seen it in action in AltConf last year and it did seem pretty nice.

Don't have enough experience with it, but seems like a worthy competitor as well :)

@viktorbenei

This comment has been minimized.

viktorbenei commented Jan 4, 2018

Hi everyone, I'm Viktor from https://www.bitrise.io (CTO & Co-founder).

Thank you everyone for the recommendation, it means a lot to our team!

Just wanted to say hi 👋 and ensure you that we're of course listening, feel free to ping us any time via any of our support channels and I'm happy to answer any questions you have here as well!

@sregg

This comment has been minimized.

sregg commented Jan 4, 2018

Haha, I think we can start the bidding war now 😄

@freak4pc

This comment has been minimized.

freak4pc commented Jan 4, 2018

My GitHawk brings all the CIs to the yard 🤓

@castus

This comment has been minimized.

castus commented Jan 6, 2018

There is also https://buddy.works I didn't use their service, so hard to say if they are good. They have a cool name definitely ;P

@richardleggett

This comment has been minimized.

richardleggett commented Jan 7, 2018

I moved over from BuddyBuild to Bitrise (we really want one place for our iOS and Android). It did take some reading of the docs and also some of the step's git repos, but it went pretty smoothly, took about a day alongside doing other things.

@sregg Just to mention we use the "Adjust BuildNumber" step to += 640, as we use it for versionCode and were auto-deploying from BB.

The key differences I found with Bitrise over BuddyBuild (aside from pretty much everything being GUI-based in BB) was to split up the Gradle build into multiple steps. BuddyBuild would (possibly more efficiently) build everything you'd ask for and then pull out the relevant stuff for say deployment emails or publishing to Play Store, with Bitrise you have a couple of choices that I could see: 1. split it up into multiple Build/Gradle steps, e.g. for UI/Android Tests, for your test builds [2x3=6 variants for us], one for your App/Play Store deployment artefacts, with some cleanup steps between (e.g. I change the deploy folder after to prevent emails going out where filters didn't suffice)... or 2. be comfortable with bash script and have a script step that splits up the pipe-map ENV variables so they can be used more easily in later steps.

It would also be nice to have more examples, setting the slack messages to include something like the ones that BB sent by default for e.g. it's nice to be able to customise but generally we just want to be back to writing code.

The other would be features we weren't using, tester management (we use Play Store Alpha and TestFlight), and shake for crash / crash logger (we prefer Firebase).

One of the features that is really cool is being able to not just view and edit your workflow as a .yml file, but download and run that locally with the CLI.

To be honest it's a lot more work than we were used to, that's the point in paying monthly right? But it's pretty much a one-time thing and the bonus is extra customisability. All in all though it's doing the job very well, and for a good price. I'm happy with this switch. I'm sure CircleCI is good too (we use that for our back-end).

@orta

This comment has been minimized.

Contributor

orta commented Jan 7, 2018

It's also worth noting, bitrise's entire infra is OSS - https://github.com/bitrise-io/bitrise.io

@viktorbenei

This comment has been minimized.

viktorbenei commented Jan 8, 2018

Thanks @richardleggett for the feedback, I'll discuss this with the team!

Especially the Slack one - I think that's long overdue now to have a "fancier" (more useful) default message there, instead of requiring you to create your "dream" msg right after dropping in the step the first time. Flexibility is important, but so is the default value / setup experience (and speed). You can tweak the message after that anyway, so there's no issue with making the default more verbose.

Gradle: will discuss it with the tooling team as well, thanks for highlighting!

@rnystrom

This comment has been minimized.

Member

rnystrom commented Jan 19, 2018

I've been kicking the can on this (slowly increasing my anxiety). Wanted to link to this blog post exploring alternatives in case there's any more info there.

I want to spend some time w/ @orta and @KrauseFx while in town to discuss my "north star" vision for automating this project (beyond just CI). Will report back once I muster the energy to actually work on this.

@BasThomas

This comment has been minimized.

Collaborator

BasThomas commented Jan 19, 2018

Thanks for the update on this @rnystrom. I feel you. 😕

@viktorbenei

This comment has been minimized.

viktorbenei commented Jan 20, 2018

@rnystrom thanks for the blog post and sorry to hear you had troubles with setting up the distribution on bitrise. Not sure if you saw it but we now have built in Auto Provisioning for code signing, which once configured can manage the the iOS signing files for you automatically: https://blog.bitrise.io/ios-auto-provision-step

Anyway, just wanted to say thanks for trying out bitrise, and to let you know we're always happy to help, in case you'd give another try to bitrise. Feel free to ping me anywhere, e.g. on our Slack (http://chat.bitrise.io).

@Sherlouk

This comment has been minimized.

Member

Sherlouk commented Jan 20, 2018

@viktorbenei For what it's worth, that isn't Ryan's blog post!

@viktorbenei

This comment has been minimized.

viktorbenei commented Jan 20, 2018

Ouch, my bad, it's still quite early in the morning here 😅 Sorry gents and thanks @Sherlouk !

@dkhamsing

This comment has been minimized.

Collaborator

dkhamsing commented Feb 11, 2018

I don’t think app center supports per commit [ci skip] like syntax

@ElektrojungeMS

This comment has been minimized.

ElektrojungeMS commented Feb 12, 2018

@dkhamsing unfortunately not (yet).

@jogu

This comment has been minimized.

jogu commented Feb 16, 2018

One real plus for buddybuild that I loved was the way they show before/after/diffs in the test results if an FBSnapshotTestCase unit test produced an unexpected image.

Does anyone know if any of these other systems have a similar feature?

@dkhamsing

This comment has been minimized.

Collaborator

dkhamsing commented Feb 18, 2018

Does App Center support building from a pull request?? I’m very confused cc @TroubleMakerBen

@rnystrom

This comment has been minimized.

Member

rnystrom commented Feb 18, 2018

@dkhamsing it does!

edit: never mind 🙊

Sent with GitHawk

@ElektrojungeMS

This comment has been minimized.

ElektrojungeMS commented Feb 19, 2018

@dkhamsing App Center supports build on PUSH, but not build on PR (build on merge), yet.

@dkhamsing

This comment has been minimized.

Collaborator

dkhamsing commented Feb 19, 2018

Ah I see. Thx Ryan, Ben 😊

Sent with GitHawk

@acristescu

This comment has been minimized.

acristescu commented Feb 22, 2018

After reading this thread it does seem to be a two-horse race: Bitrise and App Center. Yet nobody has touched on the subject of UI tests: I loved how in BB, with just a few clicks, you could run espresso tests on a virtual device. Does any of the two platforms under discussion support this?

@dkhamsing

This comment has been minimized.

Collaborator

dkhamsing commented Feb 22, 2018

@ElektrojungeMS

This comment has been minimized.

ElektrojungeMS commented Feb 22, 2018

@dkhamsing App Center Test actually runs UI Tests on physical devices – we have a few thousand. Nope, you can't see pictures ;)

@cbedoy

This comment has been minimized.

cbedoy commented Feb 22, 2018

We're actually trying to identify a new CI solution.

  • AppCenter: similar to bb but doesn't provide PR support, I think is more focused to management people, also the logs doesn't provide a stack if any task fails.

  • Bitrise: Very configurable, offers many open "steps" like codecoverage, deploy, sign, unitTest, UITest, build, deliver, clean and custom, cuz you have the ability to configure it, just got a little confused with .yaml files, you can trigger steps given Push, PR, etc

  • Nevercode Very configurable, you can select between gradlew task per branch, build PR, no free plan.

I think at least Bitrise offers many features that we can exploit!

@acristescu

This comment has been minimized.

acristescu commented Feb 22, 2018

From the link posted above about testing in App Center

  1. Review the core concepts
    Understanding the core concepts of the Test Cloud experience improve ease of use, navigation, and communications with support. It is recommended to become familiar with these concepts before running your first tests.

What the hell... I don't want to review any concepts, core or otherwise, I just want it to work with 2 clicks like it did on BB :( I don't have 10 hours to sink into making this work, I'm a coder not a devops engineer...

@dkhamsing

This comment has been minimized.

Collaborator

dkhamsing commented Feb 22, 2018

Yea the App Center documentation could be streamlined

Sent with GitHawk

@viktorbenei

This comment has been minimized.

viktorbenei commented Feb 22, 2018

@acristescu

After reading this thread it does seem to be a two-horse race: Bitrise and App Center. Yet nobody has touched on the subject of UI tests: I loved how in BB, with just a few clicks, you could run espresso tests on a virtual device. Does any of the two platforms under discussion support this?

Not (yet) a single click but also more powerful in a couple of ways: https://blog.bitrise.io/introducing-solid-and-snappy-virtual-device-testing-for-android

We're working on making the setup easier (that's why it's still "beta", not because of lacking functionality ;)).

@ElektrojungeMS

This comment has been minimized.

ElektrojungeMS commented Feb 22, 2018

@acristescu @dkhamsing We are aware! Keep the feedback coming.

@acristescu

This comment has been minimized.

acristescu commented Feb 22, 2018

@viktorbenei I'm going to try but oh my god that art style is off-putting. I remembered why I stayed away from Bitrise in the past... smiling clouds hugging each other? Sharks with lasers strapped to them?!? Green buttons over a vibrant purple background? Perhaps I'm just showing my age here, but how could I recommend this tool to a client that is... say... a bank?

@viktorbenei

This comment has been minimized.

viktorbenei commented Feb 22, 2018

No problem @acristescu , I definitely see your point, honest feedback is always welcome and a design update is already in the works ;)

@acristescu

This comment has been minimized.

acristescu commented Feb 24, 2018

I decided to try them both with a simple repo ( https://github.com/acristescu/GreenfieldTemplate ) and see where I get to. So far I've tried App Center and hit a few snags:

  • solved (it couldn't find the gradle build tools!), you have to manually add the google() repository to your project's gradle
  • it restarted the build number from 1 whereas I already released on play store a 42, google is just going to reject the build!)
  • I couldn't find how to test on a virtual device for free, only on real devices with a 30 day free trial.
  • I'm not sure it ran the unit tests because
##[warning]No test result files matching /Users/vsts/agent/2.127.0/work/1/s/**/build/test-results/TEST-*.xml were found, so publishing JUnit test results is being skipped.

Not sure what's that about...

@acristescu

This comment has been minimized.

acristescu commented Feb 24, 2018

Bitrise: On the plus side the setup was just as painless although I think that if I need to change anything I need to bring up an yml file and fiddle with it (update: found a thing called workflow editor. it looks scary but powerful). Snags:

  • it never asked me which variant to build and picked the wrong one. I wanted prodRelease but for whatever reason it decided to build exactly the other two mockDebug and prodDebug. Can't find where to change that, but I'm pretty sure there must be one.
  • build took longer, 4 mins instead of 2 mins 16 for app center. This is perhaps because of the above problem?
  • don't see any mention of junit tests anywhere in the logs. I doubt it ran them. Not obvious how to add them, perhaps somewhere in workflow editor? (update tinkered with the Workflow Editor for about 10 minutes and found it. Bonus points for allowing me the liberty to pick which test target to run)
  • not sure what build ID it used, how do I even see that?
@dkhamsing

This comment has been minimized.

Collaborator

dkhamsing commented Feb 24, 2018

Thx for posting these

Sent with GitHawk

@acristescu

This comment has been minimized.

acristescu commented Feb 24, 2018

image

@ranterle

This comment has been minimized.

ranterle commented Feb 24, 2018

Thanks @acristescu for the detailed feedback, we greatly appreciate it. Especially on the warning for JUnit test report files in App Center, this is not affecting your actual test run and should be fixed soon.
Keep it coming!

@acristescu

This comment has been minimized.

acristescu commented Feb 24, 2018

It took me two hours but I managed to convince app center to upload to google play. However, I cannot seem to convince it to do this automatically, I had to download the signed APK from the app center and then upload it back up into the deploy/stores section (!) to make it work. Seems awefully convoluted, what am I doing wrong?

PS: To add insult to injury, BuddyBuild deployed several times in the same timespan since I forgot to disable it in the beginning and it just works automatically without any human intervention...

@viktorbenei

This comment has been minimized.

viktorbenei commented Feb 26, 2018

Hi @acristescu ,

Re: #1330 (comment)

it never asked me which variant to build and picked the wrong one. I wanted prodRelease but for whatever reason it decided to build exactly the other two mockDebug and prodDebug. Can't find where to change that, but I'm pretty sure there must be one.

Indeed our current scanner will add a Gradle Runner step, with assembleDebug configured for the base workflow. We realise that this might not be straightforward enough, but in short if you want to build prodRelease then the gradle task is assembleProdRelease. If you want to run lint then the gradle task is lint. You can do all of these with the Gradle Runner step, in fact gradle can handle multiple tasks so to run lint and then assembleProdRelease you can also specify this as the task: lint assembleProdRelease which will do both.

We're working on new steps & new scanner default configs which will make this easier, with more specific steps (e.g. a "Lint" step which runs the gradle lint task, instead of requiring you to set that task in the "Gradle Runner" step) 😉

build took longer, 4 mins instead of 2 mins 16 for app center. This is perhaps because of the above problem?

Indeed that seems to be the case, as assembleDebug most likely generates 2 separate APKs / variants in your case instead of the single "ProdRelease" one.

don't see any mention of junit tests anywhere in the logs. I doubt it ran them.

Specify test as the gradle task input of the Gradle Runner step, that will run your tests - or add the Gradle Unit Test step which is configured to run that gradle task by default.

not sure what build ID it used, how do I even see that?

If you mean whether we set the build number to the bitrise.io build number: by default we don't, you can do that by adding the Change Android versionCode and versionName step for example.

Thanks again for your feedback, we're listening & already scheduled to improve these points of the setup experience! ;)

@jamesone

This comment has been minimized.

jamesone commented Mar 20, 2018

Awesome discussion. I've been finding it hard to find a BuddyBuild alternative that supports Carthage.

I looked into Nevercode, they only support cocoapods.

I believe the App center supports Carthage.

Any others?

@cbedoy

This comment has been minimized.

cbedoy commented Mar 21, 2018

@jamesone

I think the best option for you could be Bitrise, they provide a platform like pipelines bitbucket, also you can configure based on your needs using Steps.

Actually we moved from bb to bitrise, we're using Android and iOS and everything is perfect!

@jamesone

This comment has been minimized.

jamesone commented Mar 21, 2018

Awesome @cbedoy What did you do about the installable builds that buddybuild provides for all your branches? Does Bitrise have an integration OR support for this?

@cbedoy

This comment has been minimized.

cbedoy commented Mar 21, 2018

You can trigger workflows (many steps) when you push, create a PR or tag.

Also you can schedule buils per branch.

You should check:

https://devcenter.bitrise.io/bitrise-cli/workflows/
https://devcenter.bitrise.io/bitrise-cli/steps/

When you undertand how bitrise works can create worksflows based of what you need, i.e I want a workflow where just running unitTesting if someone create a PR or I want a workflow where build and generate the .ipa when master was tagged.

Bitrise is something like docker images where you can chose a thirdparty steps for run unitTest, CodeCoverage, or Archive and deploy.

@jamesone

This comment has been minimized.

jamesone commented Mar 21, 2018

Awesome man! Sounds really interesting. I'll look into it.

@rnystrom rnystrom closed this Aug 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment