[Feature request] Tag support (again) [with arguments and explained in depth)

[noraj]

Tag support [Feature request]

Feature idea

Adding tags (keywords) in frontmatter (meta-data) will generate a page referencing all pages containing this page.

Example:

In frontmatter you add (in YAML for example):

---
title: My Document
summary: A brief description of my document.
authors:
    - Waylan Limberg
    - Tom Christie
date: 2018-07-10
some_url: https://example.com*
tags:
    - dev
    - python
---

So it will create a endpoint http://mkdocs.org/tags/dev and http://mkdocs.org/tags/python referencing, respectively, all pages containing the dev tag and the python tag.

Then on the page of the documentation where there is such a tags frontmatter, the page will contain some tag link #dev and #python respectively referencing the previously mentioned endpoints.

Related issues

There was already #759 and #1411.

In the old #759 issue, waylan and d0ugal said this feature was out of scope right now.
The feature request was severely lacking of argumentation and why this must be a core feature.
In this issue, waylan also said this feature would be better as a plugin (because he didn't see the need/use case for it to be in core) but this was before version 1.0 and the API was not existing yet. So at this time it was not even possible to make a plugin.
This issue was forgotten.

In the more recent #1411, plugin through the API is now possible and waylan said that frontmatter (meta-data) will allow a plugin to retrieve tags.
Bit he still suggest to create a plugin and don't see the need for tag support to be part of MkDocs. He also suggest it can be a theme dependant feature.

Why this would be a great feature?

I good real life example will be better to understand than pages of theoretical argumentation.

Imagine I want to build a hacking documentation with a tree like that (for, let's say, my OSCP exam preparation):

- Admin # Machines from the Admin network, containing a write-up for each machine
  - Machine 1 # tags: [eop,windows]
  - Machine 2 # tags: [network,linux]
  - ...
- Cheat sheets # for various topics
  - Elevation of Privilege # tags: [eop]
  - Metasploit # tags: [exploit]
  - MS SQL Server # tags: [mssql,windows]
  - Network # tags: [network]
  - PWK/OSCP Specific
  - Pivoting # tags: [pivoting,network]
  - ...
- Connectivity
  - Machines
  - PWK Lab
  - Remote desktop
  - ssh-hostkey # tags: [ssh]
- Dev # Machines from the Dev network, containing a write-up for each machine
  - Machine 1
  - Machine 2
  - ...
- Exam
  - Exam specific
  - ...
- IT # Machines from the IT network, containing a write-up for each machine
  - Machine 1
  - Machine 2
  - ...
- Public # Machines from the Public network, containing a write-up for each machine
  - Machine 1
  - Machine 2
  - ...
- Tools
  - Nmap # tags: [network,scanner]
  - John the ripper # tags: [cracking,password,hash]
  - gcc # tags: [compilation,exploit]
  - dnsrecon # tags: [osint,dns]
  - dnsenum # tags: [osint,dns]
  - cmsmap # tags: [cms,web]
  - ncat # tags: [backdoor,network,scanner]
  - recon-ng tags: [osint]
  - snmpwalk # tags: [snmp,network]
  - snmpbrute # tags: [snmp,network]
  - onesixtyone # tags: [snmp,network]
  - ...

So I want to be able to search for a keyword (tag) like snmp and see all machines write-up where I attacked snmp, all cheat sheets explaining snmp stuff, all tools I can use to exploit snmp, etc.

Without tag support this is impossible.

One could say: "But there is already the search feature parsing teh content of pages" yeah but this is automatic and introduce a lot of false positive (content containing a keywork but not really about it) and false negative (content not containing teh keywork but actually talking about it) when the tag feature will be manual so there will be only exact matches.

Why this shouldn't be theme dependant?

First of all this mean you will have to choose between:

1. A nice looking theme you love but which doesn't not implement tag support
2. An ugly theme you have but with the cool tag support

This will also mean that each theme creator will have to implement its own tag support, resulting in:

1. Much work for theme creators as they have to implement this feature
2. Duplicate efforts: instead of working on the theme they will have to work on features and the feature will be implemented several times
3. Incompatibility between:

• Themes with tags support and theme without tags support
• Themes with tags support between them as each theme will have its own way top implement it
• In general a full theme inconsistency

And finally why would a theme creator will implement a feature not provided by the core API?

And theme should be themes (CSS) not providing half the feature of the product.

Why this shouldn't be plugin based?

By making a plugin for providing this feature:

• Themes don't add supports for dozens and dozens of plugins, but tags support requires a theme handling it (to place the tag link on the page and special theme for the tag list page, and maybe a tag cloud on the sidebar) else it won't be usable
• Plugin quality is more likely to be less good than the core
• Plugin will be maintained by a third party, so it won't necessarily be update each time and quickly after MkDocs update. It may even never be maintained.

So in the end very few themes will support tags as a theme creator can't provide support for all plugins.
Imagine you want to create a Hexo theme, you definitely won't add support for the currently 295 existing hexo plugins.

Conclusion

Saying that it should be a plugin or theme dependant is killing the feature:

• The community is not big enough so that someone skilled already created a plugin or a theme
• The problems mentioned above (so if someone in the feature creates a plugin/theme either nobody will use it or lot of people will use some but great compatibility issues will arise)

So now, I wish everybody as understand the need of tag support (every static site generator has support for it so why no documentation static site generator has it?) and this feature to be part of the core. (I also hope you won't insta-close this issue as a wontfix).

[waylan]

As a core feature it will never happen as the core devs have no need for the feature and no interest in building/maintaining it. As a third party plugin, people who actually care about the feature can build/maintain it and not be tied to the much slower release cycle of the core. Seems to me like this is better as a plugin. If I wanted the feature, I would want it as a plugin.

I suggest building the plugin and if it becomes wildly popular then we will consider adding it to the core.

[froschdesign]

@noraj
I also see no need to add this feature to the core because the tag support may help in your use case but they are not typical for a documentation.

• Plugin quality is more likely to be less good than the core
• Plugin will be maintained by a third party, so it won't necessarily be update each time and quickly after MkDocs update. It may even never be maintained.

I do not think that can be good arguments for anything because these points will only put third party developers in a bad light. (I do not think that is your intention.)

---

@waylan
I think that can always be a plugin, even if interest for this feature is growing. It can be added to MkDocs organisation here on GitHub, but there is no need to extend the core for it.

An interesting point which I can see: extending the plugin system for a better and simpler usage of template blocks or template inheritance in general. Which allows to use a template or template block of the plugin without a manual copying of files to the custom directory by the user. A plugin author can also provide templates or template blocks with his plugin.

[noraj]

I'm actually using this VSCode plugin to have YAML frontmatter support for my markdown notes. I looked at all OSS documentation staticgens like MkDocs, Sphinx, Daux, SkyDocs, Docsify, Docute, Couscous, Docusaurus, Orchid. But none have a tag support. Paradoxically nearly all generic staticgen (blog, website, etc.) have it. So I was wondering why staticgen targeting documentation will remove such a handy feature. Either blog post or documentation pages could take advantages of such a feature.
I'm quite disappointed that this vision is not shared by you guys.
I don't have either time (already contributing to and maintaining to many OSS repo) or the skills (I'm coding in ruby not python so I'm not used to it; I would also need to learn the plugin API syntax and the code architecture and also develop a theme supporting the plugin) to develop such plugin only to be able to share my private notes with tags.
So if you don't mind, I'd like to keep this issue open so maybe one day someone wanting to make a MkDocs plugin will read this.

[froschdesign]

@noraj

I looked at all OSS documentation staticgens like MkDocs, Sphinx, Daux, SkyDocs, Docsify, Docute, Couscous, Docusaurus, Orchid. But none have a tag support.

See my previous comment:

the tag support may help in your use case but they are not typical for a documentation.

#1828 (comment)

Paradoxically nearly all generic staticgen (blog, website, etc.) have it.

A website or blog is a different system than a documentation, therefore the requirements are different.

So I was wondering why staticgen targeting documentation will remove such a handy feature.

Nothing can be removed if it was not there before. 😉

---

Based on your initial comment, I think the search is your problem, not the lack of tag support. To improve the search results or fix existings problem, the issue should be reported at the issue tracker of lunr.js.

[noraj]

@froschdesign Ultimately tags index with a dedicated page is not required, having tags displayed is a nice to have but not mandatory, but at least lunr.js should take into account tags from the frontmatter and give them an heavy ratio (more than the the content) to help referencing. You may be more aligned with this idea.
And there is nothing to change with lunr.js as you just have to tell it to take into account a new field.

Auto-quote:

One could say: "But there is already the search feature parsing teh content of pages" yeah but this is automatic and introduce a lot of false positive (content containing a keywork but not really about it) and false negative (content not containing teh keywork but actually talking about it) when the tag feature will be manual so there will be only exact matches.

[froschdesign]

@noraj

Ultimately tags index with a dedicated page is not required, having tags displayed is a nice to have but not mandatory, but at least lunr.js should take into account tags from the frontmatter...

Then you should look at the following issue #1243

[noraj]

I also see no need to add this feature to the core because the tag support may help in your use case but they are not typical for a documentation.

An argument for why tags display is useful for any documentation:

• If you document tools: displaying tags you instantly know on which categories the tool belongs, not displaying tags you have either to write a sentence telling This tool is useful for XXX and allow to do XXX where XXX could just be tags or either stored tool pages in category sections of teh documentation but this mean you can only put them into one category not several which is quite annoying.
• For a generic dev project documentation this would allow to tag pages with dev, user, admin, install, usage, or any specific tags.

No project needs that or is using that? mozilla is using tags in its documentation for the HTTP protoco or JavaScript langl: https://developer.mozilla.org/en-US/docs/Web/HTTP / https://developer.mozilla.org/en-US/docs/Web/JavaScript

Or Joomla:

Or docker:

Or any big serious documentation project that is thousands or hundred of thousands pages.

[noraj]

Then you should look at the following issue #1243

Yeah but I think the index page and tags displays matters a lot. I was saying it was no mandatory for my little documentation as you seemed no wanting to implement that, but it I would really love it and it would greatly improve my documentation.

[froschdesign]

@noraj

No project needs that or is using that? mozilla is using tags...

Please keep in mind: I'm not against tags in documentations, but I see no need to add the tag support to the core of MkDocs. This also applies to a lot of other feature requests.
I think the plugin system itself can be improved and then the plugin ecosystem will grow too.

---

For voting your own comments: 👍 😉

[noraj]

Please keep in mind: I'm not against tags in documentations, but I see no need to add the tag support to the core of MkDocs. This also applies to a lot of other feature requests.

Saying it can exist as a plugin will end in the feature never existing. And as plugin are unofficial they are unsupported and not guarantee to works or to be maintained. That's why I'm lobbying for such an essential feature to be in core. Also theme are compliant with the core but not with all plugins.

[noraj]

Core vs feature: http://www.stateoftheevolution.com/core-vs-plugins

Plugin when:

1. Features that are useful only to a minority of users; but tags are useful for any project
2. Features that are experimental; N/A
3. Interfaces to proprietary systems (like twitter for example); N/A
4. Giving options when there are competing solutions (like wysiwyg editors). N/A

So it should be core.

Also plugin have to be compatible with mkdocs versions, need to be manually installed, need some dependencies that won't be package with mkdocs, and may conflicts with other plugins.

the majority of users faint when they have to think about "installing", "compatibility", "dependencies", let alone "conflicts".

@florimondmanca Please give me a strong argument why it should be a plugin ?

---

@waylan Please, could you read it all my comments and arguments (now there are more and better arguments about why it is needed and why it need to be a core feature), and give your final statement: "Yes I will add it to the roadmap to implement this in the future" or "no I don't want to develop this feature".

[froschdesign]

Saying it can exist as a plugin will end in the feature never existing.
...
So it should be core.

It's easy: convince @waylan and pay for it. Just like any other open source project.

[noraj]

Saying it can exist as a plugin will end in the feature never existing.
...
So it should be core.

That's a big shortcut and ignoring the dozen of others I arguments I gave about compatibility, installation, faint of users, dependencies, conflicts, support, guarantee.

pay for it.

That's not the free libre and open-source philosophy. Close source it and sell it if the goal is earning money. I know how "feature requests" feels, when people asks for features but nobody help you, pay you and you're doing the development and maintaining it on your free personal time. But that's how FLOSS projects works: you dev tools without any returns but other people dev other tools without any returns that you can use for free.
You can't tell me "make a PR or a donation if you wan't it to happens". I'm also dev and maitaining toos of other FLOSS projects without asking for anything, and I can't know any project architecture and programming languages to implement myself the features I'd like to see.

[facelessuser]

Mkdocs isn't motivated by money for features, but yeah sometimes people are told to create a PR if they want a feature, because sometimes it isn't in the interest of the maintainers to add a feature, but they aren't against a feature. Sometimes though, a PR will be rejected as it doesn't align with the vision of the the project, and if you want it, you'd have to fork or create a plugin.

That is how open source works.

[noraj]

That is how open source works.

Unfortunately yes, resulting in tons of forks and divided work and effort.

[facelessuser]

Unfortunately yes, resulting in tons of forks and divided work and effort.

I wouldn't want it either way. The projects I maintain and choose to volunteer to help at do not fund my time spent on them. They offer value for free, and because of that, are allowed to be biased in their approach.

[noraj]

I would have developed it if mkdocs was in ruby by I'm not programming in python.

[facelessuser]

You can port it to Ruby if you like.

[noraj]

You can port it to Ruby if you like.

Please read the code of conduct https://www.pypa.io/en/latest/code-of-conduct/

Trolling or insulting/derogatory comments

[waylan]

I'm locking this discussion to let everyone cool down. For the record, I've stated my position: This should be implemented as a plugin. I will only reconsider if such a plugin becomes wildly popular.