docs: initial POC of Jessica's fabric doc generator #2023
Conversation
|
Hey @mhamilton723 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
update fabric channel
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report
@@ Coverage Diff @@
## master #2023 +/- ##
==========================================
- Coverage 87.07% 85.05% -2.03%
==========================================
Files 306 306
Lines 16063 16063
Branches 852 852
==========================================
- Hits 13987 13662 -325
- Misses 2076 2401 +325 |
…23/SynapseML into rebase-fabric-channel
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
tools/docgen/docgen/channels.py
Outdated
| html = markdown.markdown(md, extensions=["markdown.extensions.tables", "markdown.extensions.fenced_code"]) | ||
| parsed_html = BeautifulSoup(html) | ||
| # Download images and place them in media directory while updating their links | ||
| parsed_html = self._download_and_replace_images(parsed_html, resources, output_img_dir, os.path.dirname(output_file), None, False) |
There was a problem hiding this comment.
Looks like this is in both branches of if state ent
There was a problem hiding this comment.
They are slightly different. The only line that's the same is parsed_html = BeautifulSoup(html) is this OK?
There was a problem hiding this comment.
the extra formatting steps seem like they would be no-ops on the first branch. We also want to remove useless style and cell output metadata in both branches. In this case they can be safely combined right?
There was a problem hiding this comment.
Sure. Extra style and output metadata are not likely to be in a rst file, but it won't hurt to combine.
tools/docgen/docgen/channels.py
Outdated
| # Download images and place them in media directory while updating their links | ||
| parsed_html = self._download_and_replace_images(parsed_html, resources, output_img_dir, os.path.dirname(output_file), None, False) | ||
| # Remove StatementMeta | ||
| for element in parsed_html.find_all(text=re.compile("StatementMeta\(.*?Available\)")): |
There was a problem hiding this comment.
These two can be put in both branches of it statement right, also statement meta check should be pushed upstream to the actual notebooks when possible because we don’t want them there either
There was a problem hiding this comment.
This is for Sempy doc. We do not have that in our notebooks and they have that line in some of their example. Just want to put it there to have a cleaner output for their samples.
There was a problem hiding this comment.
yes makes sense, but this would be a no-op for us (And would be helpful if our docs had those by mistake)
There was a problem hiding this comment.
Adding a warning... do you want to it be an error?
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary by GPT-4
The changes in this commit include:
- Adding new dependencies to
environment.ymlfor mistletoe, pypandoc, markdownify, and traitlets. - Creating a new README.md file for the doc generating pipeline onboarding for the Fabric channel.
- Updating the
channels.pyfile to include a new class calledFabricChannelwith various methods for processing input files, downloading and replacing images, validating metadata, generating metadata headers, reading RST files, converting to markdown links, and more. - Modifying the
core.pyfile to update the process method with an index parameter. - Updating the
manifest.yamlfile with new metadata for various notebooks related to FabricChannel. - Updating the
setup.pyfile to include new dependencies like pypandoc, markdownify, and traitlets.
These changes are mainly focused on adding support for a new Fabric channel in the doc generating pipeline and updating related files with necessary modifications and dependencies.
Suggestions
No suggestions are needed as the changes in this PR seem to be well implemented and organized.
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
…23/SynapseML into rebase-fabric-channel
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
* docs: initial POC of Jessica's fabric doc generator * update fabric channel * update fabric channel - rst file * update fabric channel * update fabric channel * add readme, resolve conflict * add install requires * update fabric channel * format channel * add back WebsiteChannel * formatting docgen * Update tools/docgen/docgen/core.py * Update tools/docgen/docgen/core.py * fix index issue * raise warning for if statementmeta in notebookcell output --------- Co-authored-by: Jessica Wang <jessiwang@microsoft.com> Co-authored-by: JessicaXYWang <108437381+JessicaXYWang@users.noreply.github.com>
Related Issues/PRs
#xxx
What changes are proposed in this pull request?
Briefly describe the changes included in this Pull Request.
How is this patch tested?
Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.yarn run startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.