Skip to content

Test Doc Builds #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 34 commits into from
Sep 9, 2020
Merged

Test Doc Builds #165

merged 34 commits into from
Sep 9, 2020

Conversation

jeff-hernandez
Copy link
Contributor

Closes #159 by adding integration tests for doc builds.

@codecov
Copy link

codecov bot commented Aug 27, 2020

Codecov Report

Merging #165 into main will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #165   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           28        28           
  Lines         1254      1254           
=========================================
  Hits          1254      1254           

@jeff-hernandez
Copy link
Contributor Author

The docs take ~12 minutes to build on CircleCI. Locally, it takes about a minute. Not sure if this is a problem.

@jeff-hernandez jeff-hernandez added documentation Improvements or additions to documentation devops labels Aug 27, 2020
@jeff-hernandez jeff-hernandez requested a review from rwedge August 27, 2020 17:40
Copy link
Contributor

@rwedge rwedge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good, only comment is if you want warnings to be treated as errors by the docs build, there's a flag you need to set

@jeff-hernandez
Copy link
Contributor Author

jeff-hernandez commented Aug 28, 2020

Hmm, after updating the doc builds to treat warnings as errors, a context deadline exceeded problem occurs on CircleCI. I don't get any errors when building the docs locally.

Solved by increasing the no output timeout (f4e4a1d).

@jeff-hernandez
Copy link
Contributor Author

jeff-hernandez commented Aug 28, 2020

Found this on CircleCI Support:

In CircleCI, a command will be killed if a certain period of time has passed with no output. By default, this is 10 minutes. This is designed to prevent errors in builds from hanging using a large number of credits unintentionally.

@rwedge
Copy link
Contributor

rwedge commented Aug 28, 2020

I reran a previous commit on CircleCi and also got a timeout, so the warnings as errors may not be the issue. It seems that a lot of time is being spent building the page on the fan degradation example

@jeff-hernandez
Copy link
Contributor Author

jeff-hernandez commented Aug 28, 2020

On Read the Docs, the latest build with the notebooks took ~6 minutes. On CircleCI, it took ~20 minutes.

@jeff-hernandez
Copy link
Contributor Author

I'm thinking about using a smaller sample of data in the notebooks to speed up the doc builds on CircleCI .

@rwedge
Copy link
Contributor

rwedge commented Sep 1, 2020

What if we tried using a time-based timeout for the EvalML search instead of number of pipelines?

@jeff-hernandez
Copy link
Contributor Author

@rwedge sounds good, will try the time-based search.

@jeff-hernandez
Copy link
Contributor Author

jeff-hernandez commented Sep 1, 2020

I didn't notice a difference in the doc builds after pinning the EvalML version (a14f2bd) or using a time-based search (50dfe51).

@jeff-hernandez
Copy link
Contributor Author

The latest build on CircleCI with an additional notebook took ~37 minutes.

@jeff-hernandez
Copy link
Contributor Author

The latest build using smaller samples of data took ~28 minutes on CircleCI.

@jeff-hernandez
Copy link
Contributor Author

After increasing the resource class of the executor, the latest build took ~11 minutes.

@jeff-hernandez
Copy link
Contributor Author

jeff-hernandez commented Sep 2, 2020

I tried running the notebooks before the doc build at runtime. The build took ~7 minutes. Then after minor changes in the makefile (c9cf833, 62a4c20), the build took ~14 minutes. Not sure why this is.

@jeff-hernandez
Copy link
Contributor Author

After limiting the search by model type, the build took a minute!

Copy link
Contributor

@rwedge rwedge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@jeff-hernandez jeff-hernandez merged commit b0c8963 into main Sep 9, 2020
@jeff-hernandez jeff-hernandez deleted the test_doc_build branch September 9, 2020 19:57
@jeff-hernandez jeff-hernandez mentioned this pull request Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integration Test for Doc Builds
2 participants