New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Pipelines を調査する #446

Open
m-tmatma opened this Issue Sep 15, 2018 · 36 comments

Comments

Projects
None yet
6 participants
@m-tmatma
Member

m-tmatma commented Sep 15, 2018

Azure Pipelines を調査する

https://azure.microsoft.com/ja-jp/services/devops/pipelines/

オープン ソース用に 10 個の無料の並列ジョブ (時間制限なし) を実行可能な、Linux、macOS、
および Windows 用のクラウドホステッド パイプラインを入手

とある。

調べること

@KENCHjp

This comment has been minimized.

Member

KENCHjp commented Sep 15, 2018

オンラインでコンパイルできるってことですよねぇ。
コンパイラ(例えばDelphi)とか持ち込めるのかなぁ。。。

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

ちなみに 2018年9月13日に発表されたばかりみたいです。
https://www.publickey1.jp/blog/18/cicdazure_pipeline130.html

2018年9月10日に発表されたみたいです。
https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

10 個の無料の並列ジョブ が魅力ですね。

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

appveyor に Azure Pipelines と同程度あるいはそれ以上の同時タスクをサポートしてほしいと送ってみた。
appveyor/ci#2641

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

オンラインでコンパイルできるってことですよねぇ。

Windows に対応している Cloud の CI として appveyor を使っていましたが、
採用当時、windows に対応しているが、appveyor しか選択肢がありませんでしたが、
他に候補ができたということです。

説明欄に調査項目を追加しました。

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

https://github.com/MicrosoftDocs?utf8=%E2%9C%93&q=pipelines&type=&language=
で検索すると pipeline の YAML の書き方のサンプルプロジェクトを検索できる。

以下は ASP.NET Core project using Azure Pipelines のサンプル
https://github.com/MicrosoftDocs/pipelines-dotnet-core/blob/master/azure-pipelines.yml

c++ は見当たらないけどサポートしてないのかな?

以下関連ドキュメント
https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started-yaml?view=vsts
https://docs.microsoft.com/en-us/azure/devops/pipelines/build/ci-public?view=vsts&tabs=github

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

sakura editor には関係ないけど iOS のビルドもできそう。
https://github.com/MicrosoftDocs/pipelines-xcode

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 15, 2018

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 16, 2018

c++ は見当たらないけどサポートしてないのかな?

https://docs.microsoft.com/en-us/azure/devops/pipelines/apps/windows/cpp?view=vsts
に cpp に関する記述がある

https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started-designer?view=vsts&tabs=new-nav
にリンクが張ってあって Web から設定する説明がある。

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 16, 2018

appveyor は使うの簡単だったけど、 Azure Pipelines は使うの難しそう。

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 16, 2018

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 16, 2018

https://docs.microsoft.com/en-us/azure/devops/pipelines/index?view=vsts
にドキュメントのトップページがある。

各ページの下に Feedback という項目があり、直接 GitHub 経由で Feedback を送ることが
できるみたい。

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 16, 2018

成果物の保存期間の調査

https://docs.microsoft.com/en-us/azure/devops/pipelines/policies/retention?view=vsts
に記載があって、どれぐらいの期間保存するのかを確認方法が説明されているが、
具体的にどれぐらいかは記載されていない。

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 17, 2018

Hello. I'm a product manager for Azure Pipelines. If there's anything I can do to help, let me know. (I'm sorry for chiming in using English. I don't know Japanese.)

@k-takata

This comment has been minimized.

Member

k-takata commented Sep 17, 2018

@vtbassmatt Thank you for your comment.
Currently we use AppVeyor in this project, and now we are investigating if we can use Azure Pipelines instead (or parallelly). We think that 10 parallel jobs for OSS projects is attractive, but using Azure Pipelines looks more difficult than AppVeyor.

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 17, 2018

I would be grateful to know which part is more difficult. We can make product changes or documentation improvements.

Azure Pipelines has a LOT of options and flexibility, which makes it challenging/overwhelming to get started with. Did you see our templates repository? That might be a good place to start - we have templates for most languages and platforms.

@beru

This comment has been minimized.

Contributor

beru commented Sep 17, 2018

@vtbassmatt

Azure Pipelines has a LOT of options and flexibility, which makes it challenging/overwhelming to get started with. Did you see our templates repository? That might be a good place to start - we have templates for most languages and platforms.

I don't see a template for Windows classic desktop application which would be of use to us.

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 17, 2018

@vtbassmatt

We have the following issues to solve.

  • We need to run Set-WinSystemLocale ja-JP and Restart-Computer with Powershell to change system locale before starting builds.
  • We need to specifiy build settings by a configuration file like appveyor.yml so that anyone can request to change build settings and review them.
  • We need the following tools on build environments.
    • Visual Studio 2017 Community
    • Inno Setup Unicode (innosetup-5.5.9-unicode.exe or innosetup-5.6.1-unicode.exe)
    • HTML Help Compiler
    • Python 2.7.x (including pip)
    • 7Zip
    • (if possible) MingW
  • We use custom batch files to build all artifacts.
  • We need an organization acoount for CI so that project members can control it.
  • Anonymous users can download any aritifacts including for master, branches, and PR from CI.

Do you have any user support repository for Azure Pipeline on GitHub like https://github.com/appveyor/ci ?

appveyor's document is very easy to understand.
It explains what tools are installed on the build images, what environment variables are available,
and how we write appveyor.yml.

@beru

This comment has been minimized.

Contributor

beru commented Sep 17, 2018

@vtbassmatt

I'd like to see pipelines-cplusplus sample app repository which isn't available yet.

I see 10 repositories on this query result page.
https://github.com/MicrosoftDocs/?utf8=%E2%9C%93&q=pipeline&type=&language=

But I see only 5 programming languages listed on the documentation.
https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started-yaml?view=vsts#get-the-sample-code
The other 5 missing repositories must be weeping...

@beru

This comment has been minimized.

Contributor

beru commented Sep 17, 2018

@vtbassmatt

Can you please provide a link to https://azure.microsoft.com/en-us/services/devops/pipelines/ on this page?
I had to manually type the address which was quite challenging for me. The screenshots seem really helpful but due to the lack of the link, an average person might not start the procedure.

@beru

This comment has been minimized.

Contributor

beru commented Sep 17, 2018

@vtbassmatt

I didn't know the template named .NET Desktop can also be used for Windows classic desktop solutions.
Disguised name!

image

@beru beru added CI research labels Sep 18, 2018

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 19, 2018

Thanks for all the feedback!

  • I have submitted the PR to add a link to the signup page. Really silly oversight on our part.
  • Feedback noted about building Windows desktop apps. I'll see what we can do.
  • Good idea about having a C++ sample app!
  • We're YAML based like AppVeyor, and you can easily run batch scripts as part of the build.
  • I need to check our set of pre-installed tools; for any that we're missing, we can get those into the image. VS2017 and Python 2.7 with pip are definitely available.
  • You can add whoever you want to your Azure Pipelines organization.
  • We currently have a bug with uploading artifacts from a PR fork build. However, uploading from any non-fork build works, and anyone on the internet can access those artifacts.

Unfortunately, the one thing I haven't found a workaround for is changing the system locale. I don't believe you can reboot our hosted agents. We support private agents (which you can configure with any software or settings you want), but then you lose the value of having everything cloud-hosted.

I'm still looking into this, though, and maybe someone on my team will have a suggestion.

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 19, 2018

Oh - I missed two things.

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 19, 2018

@vtbassmatt

When I tried Azure Pipelines, I was asked to log in by Microsoft Account.

If it is possible to log in by GitHub Account like appveyor, let me how.
If not, please support it (including GitHub organization account).

I think most of GitHub users want to log in by GitHub accounts instead of Microsoft Accounts.

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 19, 2018

For now you have to use a Microsoft Account. We want to support GitHub identities but don't have a timeframe yet.

@beru

This comment has been minimized.

Contributor

beru commented Sep 24, 2018

@vtbassmatt I'm wondering if importers and migration guides from AppVeyor or Travis or Jenkins are available to us.

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 25, 2018

@beru we're writing some of that up. The person responsible for it is at a conference all week but I think he plans to get started next week.

Unfortunately we don't have a good workaround for needing to reboot the agents. It's been added to our backlog. If you're interested in still trying Azure Pipelines, you could set up self-hosted Windows agents that are already configured in ja-JP, so no reboot would be required.

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 25, 2018

@vtbassmatt

Unfortunately we don't have a good workaround for needing to reboot the agents.

@berryzplus removed the necessity of rebooting agents at #482 by setting /execution-charset:shift_jis.
So we don't need a workaround any more.

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 25, 2018

Oh, great! When I get a few minutes, I'll see if I can port over your Appveyor config.

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Sep 27, 2018

I took a run at this, but I don't know enough about building C++ apps or this code in particular to be successful. There are lots of hardcoded references to AppVeyor which I couldn't easily extract.

I'm still happy to help by answering questions and getting you information, though!

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Sep 27, 2018

@vtbassmatt

There are lots of hardcoded references to AppVeyor which I couldn't easily extract.

We use some appveyor environment variables in https://www.appveyor.com/docs/environment-variables/

  • APPVEYOR_ACCOUNT_NAME
  • APPVEYOR_BUILD_NUMBER
  • APPVEYOR_REPO_COMMIT
  • APPVEYOR_REPO_TAG_NAME
  • APPVEYOR_URL
  • APPVEYOR_BUILD_VERSION
  • APPVEYOR_PROJECT_SLUG
  • APPVEYOR_PULL_REQUEST_NUMBER
  • APPVEYOR_PULL_REQUEST_HEAD_COMMIT
  • APPVEYOR_REPO_NAME
  • APPVEYOR_REPO_PROVIDER

Do you have alternative variables on Azure Pipelines?

Note: we support building this software locally. So you can build it without these appveyor-specific variables for an experiment.

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Nov 6, 2018

GitHub 連携始めたみたい。(It seems that the integration between GitHub and Azure Pipelines has alredy started)
https://github.com/marketplace/azure-pipelines
https://blog.github.com/2018-09-10-azure-pipelines-now-available-in-github-marketplace/

@m-tmatma

This comment has been minimized.

Member

m-tmatma commented Nov 21, 2018

@vtbassmatt

For now you have to use a Microsoft Account. We want to support GitHub identities but don't have a timeframe yet.

Do you have the timeframe now?

@vtbassmatt

This comment has been minimized.

vtbassmatt commented Nov 22, 2018

Nothing to announce right now.

@berryzplus

This comment has been minimized.

Contributor

berryzplus commented Nov 22, 2018

さっき Microsoft Azure のアカウント登録してきました。
https://azure.microsoft.com/ja-jp/features/azure-portal/
個人だとSMS認証とクレジットカードの登録がいるんですね。
他で使うアテがない場合、アカウント登録の敷居は高いように思いました。

GitHubアカウントを連携して使えるように・・・というか、
organizationで共用できるようになるまでは少しかかりそうかな?と思っています。

そろそろ少しずつ試験運用をしてみてもいいかな~と思い始めました。
個人アカウントを用意できる人で「色々やってみる」を試験運用と言ってます。
結果は できるだけここに書く というスタンスでまずはやってみようってことです。

@vtbassmatt

Nothing to announce right now.

thank you for your reply.
we want to use the Azule-pipeline as a user-group named SAKURA editor organization on GitHub.
we would like to start trial anyway, to learn about that on a personal basis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment