New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HelpSystem should consume markdown natively #3954

Open
SteveL-MSFT opened this Issue Jun 6, 2017 · 17 comments

Comments

5 participants
@SteveL-MSFT
Member

SteveL-MSFT commented Jun 6, 2017

HelpSystem currently relies on MAML based help. We have platyPS which converts markdown to MAML. The HelpSystem should understand markdown natively removing the conversion step.

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 5, 2017

Collaborator

If we'll directly read and parse Markdown files we have to rewrite all help subsystem which currently full based on xml/maml. If we'll rewrite the help sybsystem maybe use a lightweight DB engine like MongoDB? In the case we'll parse once with Update-Help and then import help info in a DB.

Collaborator

iSazonov commented Sep 5, 2017

If we'll directly read and parse Markdown files we have to rewrite all help subsystem which currently full based on xml/maml. If we'll rewrite the help sybsystem maybe use a lightweight DB engine like MongoDB? In the case we'll parse once with Update-Help and then import help info in a DB.

@SteveL-MSFT

This comment has been minimized.

Show comment
Hide comment
@SteveL-MSFT

SteveL-MSFT Sep 5, 2017

Member

@iSazonov using a local noSQL DB like MongoDB may make sense. First step is to separate out the HelpSystem.

Once we can consume markdown natively, we can do some interesting stuff with rendering any markdown (not just Help) on the console

Member

SteveL-MSFT commented Sep 5, 2017

@iSazonov using a local noSQL DB like MongoDB may make sense. First step is to separate out the HelpSystem.

Once we can consume markdown natively, we can do some interesting stuff with rendering any markdown (not just Help) on the console

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 5, 2017

Collaborator

So do we need to implement a parser and a console renderer?

Collaborator

iSazonov commented Sep 5, 2017

So do we need to implement a parser and a console renderer?

@SteveL-MSFT

This comment has been minimized.

Show comment
Hide comment
@SteveL-MSFT

SteveL-MSFT Sep 5, 2017

Member

Would be best if we can use some existing OSS implementation

Member

SteveL-MSFT commented Sep 5, 2017

Would be best if we can use some existing OSS implementation

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 5, 2017

Collaborator

I see many parser implementations (PlatyPS?) but can not find console renderer.

Collaborator

iSazonov commented Sep 5, 2017

I see many parser implementations (PlatyPS?) but can not find console renderer.

@SteveL-MSFT SteveL-MSFT added this to the 6.1.0 milestone Sep 5, 2017

@SteveL-MSFT

This comment has been minimized.

Show comment
Hide comment
@SteveL-MSFT

SteveL-MSFT Sep 5, 2017

Member

Reusing PlatyPS. We will probably have to write a console renderer.

Member

SteveL-MSFT commented Sep 5, 2017

Reusing PlatyPS. We will probably have to write a console renderer.

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Oct 9, 2017

Collaborator

Interesting JavaScript implementation https://github.com/linuxenko/lessmd

Collaborator

iSazonov commented Oct 9, 2017

Interesting JavaScript implementation https://github.com/linuxenko/lessmd

@vors

This comment has been minimized.

Show comment
Hide comment
@vors
Collaborator

vors commented Jan 27, 2018

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Jan 28, 2018

Collaborator

We have RFC to move HelpSystem out PowerShell Engine. So we should solve that we implement first.

Collaborator

iSazonov commented Jan 28, 2018

We have RFC to move HelpSystem out PowerShell Engine. So we should solve that we implement first.

@MaximoTrinidad

This comment has been minimized.

Show comment
Hide comment
@MaximoTrinidad

MaximoTrinidad Jan 28, 2018

Just curious!

This is not going to impact Windows PowerShell Help system?
As, Windows PowerShell is complete. This is only for PSCore6. Right!!

Also, is this also going to impact the the online Microsoft Doc information?

:)

MaximoTrinidad commented Jan 28, 2018

Just curious!

This is not going to impact Windows PowerShell Help system?
As, Windows PowerShell is complete. This is only for PSCore6. Right!!

Also, is this also going to impact the the online Microsoft Doc information?

:)

@kvprasoon

This comment has been minimized.

Show comment
Hide comment
@kvprasoon

kvprasoon Jan 28, 2018

Contributor

IMO, this will be for PowerShell Core and PowerShell cmdlet docs are already using markdown .

Contributor

kvprasoon commented Jan 28, 2018

IMO, this will be for PowerShell Core and PowerShell cmdlet docs are already using markdown .

@MaximoTrinidad

This comment has been minimized.

Show comment
Hide comment
@MaximoTrinidad

MaximoTrinidad commented Jan 28, 2018

@kvprasoon, Thanks!

@SteveL-MSFT

This comment has been minimized.

Show comment
Hide comment
@SteveL-MSFT

SteveL-MSFT Jan 30, 2018

Member

@MaximoTrinidad correct, this change is only for PSCore6 which means for Windows PowerShell we'll continue to use platyPS to convert the markdown to MAML if that help is updated.

Member

SteveL-MSFT commented Jan 30, 2018

@MaximoTrinidad correct, this change is only for PSCore6 which means for Windows PowerShell we'll continue to use platyPS to convert the markdown to MAML if that help is updated.

@vors

This comment has been minimized.

Show comment
Hide comment
@vors

vors Feb 1, 2018

Collaborator

One thought about smoothing the transition: when we will have HelpSystem in a separate module, we can add part of platyPS there, that can convert platyPS markdown to the tradition help objects on the fly. That way, users can stop shipping maml xmls earlier and package consumers can still get it in the traditional form.

Collaborator

vors commented Feb 1, 2018

One thought about smoothing the transition: when we will have HelpSystem in a separate module, we can add part of platyPS there, that can convert platyPS markdown to the tradition help objects on the fly. That way, users can stop shipping maml xmls earlier and package consumers can still get it in the traditional form.

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Feb 1, 2018

Collaborator

In short from discussion with @vors

  • In any case we need ported console markdown renderer.
    Perhaps we could support pluggable markdown renderer modules (ex. show HTML) (maybe as part of a host implementation)

  • In any case we should parse mardown help files in order to provide the ability to search and filter.
    It does not require use platyPS.
    Preffered way is direct using markdig (BSD-like license) which is fast and extensible.

  • The parsing in turn implies the existence of a scheme.
    PlatyPS uses a complex scheme. Really we should simplify it because we don't need MAML anymore.
    Actually we have enough the markdown scheme which has Section.
    In other words we can realize to start something like
    Get-MarkdownHelp -Name CmdletName -Section Description

  • To ensure backward compatibility we need to add an option to convert existing Help (xml) files to the markdown on the fly.
    To do this, we will need integrate platyPS package (yet it doesn't exist).

  • To avoid repeated markdown parsing and maml/xml-markdown conversions, we could use a cache based on noSQL (mongoDB?).
    Update-MarkdownHelp cmdlet could update not only the source files but also the cache.

  • Given that this cache will contain a list of cmdlets by modules,
    this cache could be used to speed up the search for cmdlets for module autoloading.

  • This is all very similar to the roadmap for creating a new MarkdownHelpSystem that is independent of the current HelpSystem but backward compatible.

Perhaps we can not waste time and not separate the current system into a separate module.
We can work on the new system right now. The main difficulty is the lack of ported console markdown renderer.
We should consider to use https://github.com/rprichard/winpty although this may require a refactoring of the console output code.

Collaborator

iSazonov commented Feb 1, 2018

In short from discussion with @vors

  • In any case we need ported console markdown renderer.
    Perhaps we could support pluggable markdown renderer modules (ex. show HTML) (maybe as part of a host implementation)

  • In any case we should parse mardown help files in order to provide the ability to search and filter.
    It does not require use platyPS.
    Preffered way is direct using markdig (BSD-like license) which is fast and extensible.

  • The parsing in turn implies the existence of a scheme.
    PlatyPS uses a complex scheme. Really we should simplify it because we don't need MAML anymore.
    Actually we have enough the markdown scheme which has Section.
    In other words we can realize to start something like
    Get-MarkdownHelp -Name CmdletName -Section Description

  • To ensure backward compatibility we need to add an option to convert existing Help (xml) files to the markdown on the fly.
    To do this, we will need integrate platyPS package (yet it doesn't exist).

  • To avoid repeated markdown parsing and maml/xml-markdown conversions, we could use a cache based on noSQL (mongoDB?).
    Update-MarkdownHelp cmdlet could update not only the source files but also the cache.

  • Given that this cache will contain a list of cmdlets by modules,
    this cache could be used to speed up the search for cmdlets for module autoloading.

  • This is all very similar to the roadmap for creating a new MarkdownHelpSystem that is independent of the current HelpSystem but backward compatible.

Perhaps we can not waste time and not separate the current system into a separate module.
We can work on the new system right now. The main difficulty is the lack of ported console markdown renderer.
We should consider to use https://github.com/rprichard/winpty although this may require a refactoring of the console output code.

@SteveL-MSFT

This comment has been minimized.

Show comment
Hide comment
@SteveL-MSFT

SteveL-MSFT Feb 1, 2018

Member

Great discussion!

cc @daxian-dbw @adityapatwardhan who has been looking into this

Member

SteveL-MSFT commented Feb 1, 2018

Great discussion!

cc @daxian-dbw @adityapatwardhan who has been looking into this

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Feb 2, 2018

Collaborator

Add about using winpty.

Collaborator

iSazonov commented Feb 2, 2018

Add about using winpty.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment