New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CircleCI PoC #294

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
4 participants
@levlaz
Contributor

levlaz commented Nov 2, 2017

This is what running the build on CircleCI might look like. If you look at my fork there are a couple cool things.

https://circleci.com/gh/levlaz/devguide/5

  1. The entire build (including linkcheck) ran in under 1 minute.
  2. We are able to store the build output as artifacts.

screen shot 2017-11-01 at 7 23 02 pm

This allows us to make a nice report for "broken" things in linkcheck here: https://3-108794746-gh.circle-artifacts.com/0/home/circleci/project/_build/linkcheck/broken.txt

And we can see the entire build HTML which could be useful for previewing the state of the project at a specific commit: https://3-108794746-gh.circle-artifacts.com/0/home/circleci/project/_build/html/index.html

*note: linkcheck does not fail the build even though it has failures because the command that is run is make linkcheck || true which forces an exit code of 0.

fix #292

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Nov 2, 2017

Member

The issue with Circle CI is we get a single builder for the entire Python organization. That means your 1 minute build time won't start in 1 minute if another project starts using it. While with Travis we have I think 20 builders across the organization.

Member

brettcannon commented Nov 2, 2017

The issue with Circle CI is we get a single builder for the entire Python organization. That means your 1 minute build time won't start in 1 minute if another project starts using it. While with Travis we have I think 20 builders across the organization.

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Nov 2, 2017

Member

I noticed at #292 (comment) you mention how you work at Circle, so obviously I just explained something to you which you already knew. 😄

Member

brettcannon commented Nov 2, 2017

I noticed at #292 (comment) you mention how you work at Circle, so obviously I just explained something to you which you already knew. 😄

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Nov 2, 2017

Member

If you want to discuss opportunities for the Python organization to use Circle then feel free to email core-workflow@python.org to start up a conversation (or you can email me personally if you want me to guess at what initial hurdles there may be).

Member

brettcannon commented Nov 2, 2017

If you want to discuss opportunities for the Python organization to use Circle then feel free to email core-workflow@python.org to start up a conversation (or you can email me personally if you want me to guess at what initial hurdles there may be).

@levlaz

This comment has been minimized.

Show comment
Hide comment
@levlaz

levlaz Nov 2, 2017

Contributor
Contributor

levlaz commented Nov 2, 2017

@levlaz

This comment has been minimized.

Show comment
Hide comment
@levlaz

levlaz Nov 2, 2017

Contributor

Hey Brett,

Cool! My previous comment was just to say that by "default" any FOSS project will get 4 builders for free. Which is cool.

I am 99% certain that we could make something work with the python org though so I'll be in touch over that email.

Thank you!

Contributor

levlaz commented Nov 2, 2017

Hey Brett,

Cool! My previous comment was just to say that by "default" any FOSS project will get 4 builders for free. Which is cool.

I am 99% certain that we could make something work with the python org though so I'll be in touch over that email.

Thank you!

@levlaz

This comment has been minimized.

Show comment
Hide comment
@levlaz

levlaz Nov 9, 2017

Contributor

I reached out to the core-workflow team via the email that Brett provided. I suppose we can put this "on hold" while we continue the conversation over email.

Contributor

levlaz commented Nov 9, 2017

I reached out to the core-workflow team via the email that Brett provided. I suppose we can put this "on hold" while we continue the conversation over email.

- checkout
- restore_cache:
key: dependency-cache-{{ checksum "requirements.txt" }}

This comment has been minimized.

@merwok

merwok Nov 14, 2017

Contributor

This means we lose the whole virtualenv when a dependency is added or changed. In my projects I have keys like project-v1 and I bump the number when I change cache paths, base image or CI tools.

@merwok

merwok Nov 14, 2017

Contributor

This means we lose the whole virtualenv when a dependency is added or changed. In my projects I have keys like project-v1 and I bump the number when I change cache paths, base image or CI tools.

This comment has been minimized.

@levlaz

levlaz Nov 15, 2017

Contributor

Hey,

This means we lose the whole virtualenv when a dependency is added or changed.

I am not quite sure I understand what you mean. If you add a dependency, pip is still being ran so it will always install all dependencies. If it is found in the cache it will be "instant", if not it will be downloaded.

In my projects I have keys like project-v1 and I bump the number when I change cache paths

I am not quite sure I understand the difference between this and your example. The key can be any arbitrary thing. I just named it dependency-cache. In addition you can have multiple cache keys which increases the chance of a cache being hit.

@levlaz

levlaz Nov 15, 2017

Contributor

Hey,

This means we lose the whole virtualenv when a dependency is added or changed.

I am not quite sure I understand what you mean. If you add a dependency, pip is still being ran so it will always install all dependencies. If it is found in the cache it will be "instant", if not it will be downloaded.

In my projects I have keys like project-v1 and I bump the number when I change cache paths

I am not quite sure I understand the difference between this and your example. The key can be any arbitrary thing. I just named it dependency-cache. In addition you can have multiple cache keys which increases the chance of a cache being hit.

This comment has been minimized.

@merwok

merwok Nov 15, 2017

Contributor

I mean the previous cache won’t be restored, so pip will have to install all dependencies fresh. In contrast, a key without requirements file checksum will cause the cache to be restored, and then pip will install only the one new dependency.

@merwok

merwok Nov 15, 2017

Contributor

I mean the previous cache won’t be restored, so pip will have to install all dependencies fresh. In contrast, a key without requirements file checksum will cause the cache to be restored, and then pip will install only the one new dependency.

@willingc

This comment has been minimized.

Show comment
Hide comment
@willingc

willingc Mar 25, 2018

Collaborator

Hi all,

Thanks to @levlaz for the Proof of Concept PR.

The conversation about CircleCI and workflow has moved to the core-workflow mailing list.

As an FYI, I have been happily using CircleCI alongside Travis on JupyterHub development for the past 2 years.

Currently, I'm going to close this Proof of Concept PR, and after additional discussion on the core-workflow mailing list, we can submit a formal PR, if needed.

Thanks. - Carol ☀️

Collaborator

willingc commented Mar 25, 2018

Hi all,

Thanks to @levlaz for the Proof of Concept PR.

The conversation about CircleCI and workflow has moved to the core-workflow mailing list.

As an FYI, I have been happily using CircleCI alongside Travis on JupyterHub development for the past 2 years.

Currently, I'm going to close this Proof of Concept PR, and after additional discussion on the core-workflow mailing list, we can submit a formal PR, if needed.

Thanks. - Carol ☀️

@willingc willingc closed this Mar 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment