Skip to content
This repository has been archived by the owner on Nov 22, 2022. It is now read-only.

Remove and mock doc dependencies because readthedocs is OOM #983

Closed
wants to merge 1 commit into from

Conversation

snisarg
Copy link
Contributor

@snisarg snisarg commented Sep 16, 2019

ReadTheDocs has limited memory to import docs. We can mock some of the modules that we are not really using for documentation purposes so as the keep the process light-weight and not OOM ReadTheDocs, keeping room for future modules as well.

Note that this still fails because of a new, untest error introduced. Trying to keep 1 diff to do one thing only, the fix for the error right now will be on top of this diff.

Motivation and Context

How Has This Been Tested

$ make html

...
Creating file source/configs/pytext.task.new_task.rst
Creating file source/configs/pytext.task.disjoint_multitask.rst
Creating file source/configs/pytext.task.tasks.rst
Creating file source/configs/pytext.config.doc_classification.rst
Creating file source/configs/pytext.config.pytext_config.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 376 source files that are out of date
updating environment: 0 added, 376 changed, 0 removed
reading sources... [ 11%] configs/pytext.data.bert_tensorizer.BERTTensorizer.Config .. configs/pyreading sources... [ 16%] configs/pytext.data.packed_lm_data.PackedLMData.Config .. configs/pytexreading sources... [ 22%] configs/pytext.data.tensorizers .. configs/pytext.data.tensorizers.Vocareading sources... [ 27%] configs/pytext.data.tensorizers.VocabFileConfig .. configs/pytext.loss.reading sources... [ 33%] configs/pytext.loss.loss.KLDivergenceCELoss.Config .. configs/pytext.mereading sources... [ 38%] configs/pytext.metric_reporters.metric_reporter.MetricReporter.Config .reading sources... [ 44%] configs/pytext.models.decoders.decoder_base .. configs/pytext.models.doreading sources... [ 55%] configs/pytext.models.joint_model.IntentSlotModel.Config .. configs/pytreading sources... [ 61%] configs/pytext.models.output_layers.doc_regression_output_layer .. confreading sources... [ 66%] configs/pytext.models.qna.bert_squad_qa.BertSquadQAModel.Config .. confreading sources... [ 72%] configs/pytext.models.representations.biseqcnn .. configs/pytext.modelsreading sources... [ 77%] configs/pytext.models.representations.pooling.MeanPool.Config .. configreading sources... [ 83%] configs/pytext.models.semantic_parsers.rnng.rnng_parser .. configs/pytereading sources... [ 88%] configs/pytext.optimizer.optimizers .. configs/pytext.optimizer.sparsifreading sources... [ 94%] configs/pytext.optimizer.swa .. configs/pytext.task.tasks.MaskedLMTask.reading sources... [100%] configs/pytext.task.tasks.NewBertClassificationTask.Config .. configs/pytext.trainers.trainer.TrainerBase.Config
waiting for workers...
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [  5%] configs/pytext.config .. configs/pytext.data.compositional_data_handler.writing output... [ 11%] configs/pytext.data.data .. configs/pytext.data.sources.data_source.Rootwriting output... [ 17%] configs/pytext.data.sources.data_source.RowShardedDataSource.Config .. cwriting output... [ 23%] configs/pytext.data.tensorizers.GazetteerTensorizer.Config .. configs/pywriting output... [ 29%] configs/pytext.exporters .. configs/pytext.metric_reporters.classificatiwriting output... [ 35%] configs/pytext.metric_reporters.compositional_metric_reporter .. configswriting output... [ 41%] configs/pytext.models.bert_classification_models.BertModelInput .. confiwriting output... [ 47%] configs/pytext.models.doc_model.DocModel_Deprecated.Config .. configs/pywriting output... [ 52%] configs/pytext.models.ensembles.bagging_intent_slot_ensemble.BaggingIntentSlotEnsembleModel.Config .. configs/pytext.models.output_layers.doc_classification_output_layerwriting output... [ 58%] configs/pytext.models.output_layers.doc_classification_output_layer.Multwriting output... [ 64%] configs/pytext.models.qna.bert_squad_qa.BertSquadQAModel.Config .. confiwriting output... [ 70%] configs/pytext.models.representations.contextual_intent_slot_rep .. confwriting output... [ 76%] configs/pytext.models.representations.pure_doc_attention.PureDocAttentiowriting output... [ 82%] configs/pytext.models.semantic_parsers.rnng.rnng_parser.RNNGParser_Deprewriting output... [ 94%] configs/pytext.task.task.TaskBase.Config .. configs/pytext.trainers.ensewriting output... [100%] configs/pytext.trainers.hogwild_trainer .. index                        
waiting for workers...
generating indices...  genindex py-modindexdone
highlighting module code... [ 69%] pytext.models.representations.huggingface_bert_sentence_encodehighlighting module code... [ 75%] pytext.models.representations.sparse_transformer_sentence_encohighlighting module code... [ 76%] pytext.models.representations.transformer_sentence_encoder_bashighlighting module code... [100%] typing                                                        
writing additional pages...  search/private/tmp/venv1/lib/python3.6/site-packages/sphinx_rtd_theme/search.html:20: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
  {{ super() }}
done
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

The HTML pages are in build/html.

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Sep 16, 2019
@snisarg snisarg force-pushed the readthedoc_issue branch 3 times, most recently from 55d9473 to 84ee029 Compare September 17, 2019 03:48
ReadTheDocs has limited memory to import docs. We can mock some of the modules that we are not really using for documentation purposes so as the keep the process light-weight and not OOM ReadTheDocs, keeping room for future modules as well.
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@snisarg has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@snisarg merged this pull request in 8c80be9.

@snisarg snisarg deleted the readthedoc_issue branch September 21, 2019 02:12
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants