Export and Import a Glossary Index Source for reuse

[christophcemper]

What's your user story?

Story 1:

As a analyst of large multi component projects I would like to
generate a glossary-reference-index
for other projects to import, so they can use the Glossary in their source
and create links to the place where the original glossary is published

related Story 2:

As a domain expert I would like to publish my authoritative glossary on the web myspecialknowhow.com/glossary
and give other projects and people an easy way to use the term definitions
and create links back to my glossary on their source.

less but still related Story 3:

As an analyst
I would like to have externally defined Glossary terms linked
with my document having term and term definition in the Link Title,
but the actual link go out to URLs I pre-defined to go to authoritative sources Wikipedia, Investopedia, Merriam-Webster Medical Dictionary, etc.

How would you like the system behave to satisfy your needs? How would you like it not?

glossarify-md

• shall provide the option to generate an glossary-source-file to contain at least the terms, term-aliases, term definitions (first line), an URL or Partial URL to be prefixed. Perhaps stored as YAML, JSON or XML
• shall provide the option to add an additional glossary not only from a directory, but from such a precompiled glossary-source-file.
• shall generate a link for each term that match with term or term-alias defined in the glossary-source-file, add term definition as link title and the external URL as link target
• configuration could have a change look like the first two examples with source elements

  "glossaries": [
    {
      "source": "./include/myspecialknowhow-glossary.yaml",
      "termHint": ""
    },
    {
      "source": "./ext-includes/*.yaml",
      "termHint": ""
    },
    {
      "file": "./en/glossary/**/*.md",
      "termHint": ""
    },
    {
      "file": "./en/glossary-custom/**/*.md",
      "termHint": ""
    },
    {
      "file": "./en/requirements/**/*.md",
      "termHint": ""
    },
    {
      "file": "./en/design/**/*.md",
      "termHint": ""
    }
  ],

Note: on the format specification I am unsure, as research has shown quite a lot of different initiatives over the years, including some ISO standards. (I have not found yet, if there's a dominant syntax,language def to use)

Rationale:

Currently I clone relevant dependencies as sub modules, which does the job as in the requirements example for one.
Example:

myproject
├── glossary  <= git submodule 1
│   ├── en
│   └── de
├── component1  <= git submodule 2
│   ├── en
│   └── de
├── component2  <= git submodule 3
│   ├── en
│   └── de
└── spec
    ├── en
    └── de

However this leads to duplication of the "common" glossaries across projects. And while this option is a good workaround for a local documentation base, it does not allow the second Story 2 use case.

myproject
├── includes                                     <= here I include the generated glossary source files
         - allIknow-glossary.yaml
         - component1-glossary.yaml
         - component2-glossary.yaml
├── en
└── de

Glossary Project  <= Published on allIknow.com/glossary/en and allIknow.com/glossary/de
└── glossary
    ├── en
    ├── de
    └── export       <= here the projects provide a generated export glossary source file to use per revision

Component 1 Project <= Published on component1.com/en and component1.com/de
└── component1
    ├── en
    ├── de
    └── export       <= here the projects provide a generated export glossary source file to use per revision

Component 2 Project  <= Published on component2.at/en and component2.at/de
└── component2
    ├── en
    ├── de
    └── export        <= here the projects provide a generated export glossary source file to use per revision

The generates Glossary source files of each component could also be published e.g. on component2.at/en/component2-glossary-source.de.yaml for the German glossary terms of Component 2 Project

Do you know examples of systems that have a similar feature you need?

https://github.com/itsallcode/openfasttrace

OpenFastTrace does a great job for requirements tracing.
It has an export/import function for the requirement outputs of a project, that can be included in other projects.

We use it to generate the requirements from one project as XML and import all requirements from the stack of inherited projects.

A similar functionality would allow us to export the terms and definitions per project using glossarify-md.

Additional context

Only generating links to glossary documents helps avoid redundancy and allows to reference external sources.

If the glossary-index-format is something easy to read and still ensure consistency in the editor then it may be even possible to create a list Glossary terms that should link out to say Wikipedia or another trusted source on the web.

[about-code]

This is just a quick answer and we may be able to brake this story down in multiple requirements / issues

REQ-1: Structured output/export format for glossary terms

1. glossary terms found in glossary markdown files should be exported in a structured format (JSON, YAML,...)

REQ-2: Structured input format for glossary terms

1. the structured format should be consumable as an alternative glossary input format (alternatively to markdown)
2. terms in the input format may come with a (short) description
3. glossarify-md should compile a glossary Markdown file from terms in the input file unless they refer to an external definition

REQ-3 External Definitions

1. terms found in documents should be linked to an external definition when listed in the input file with an external URL
2. glossarify-md does not need to compile a glossary Markdown file from terms in the input file if they have an external definition
3. glossarify-md does not need to crawl short descriptions itself

I've long been thinking about doing something based on JSON-LD and I think this one could make me work on it.

[about-code]

Hello @christophcemper . It took me a while but eventually there's an import and export functionality in glossarify-md@next.

Exporting a glossary with terms in german

{
  "glossaries": [{
    "uri": "http://allIknow.com/glossary/de#",
    "file": "./glossary.md",
    "export": {
      "file": "./glossary.json"
    }
  }],
  "i18n": {
     "locale": "de"
  }
}

While I stated in REQ-3.2 that glossarify does not need to compile a Markdown file from a JSON import it was easier to implement so, eventually. So by default when importing terms any occurrences will be linked to the markdown file generated from the import.

There are two options by which you can choose to link to an external authoritative source, using a terms URI:

• "showUris": true will make the URI visible within the generated glossary.md file (also possible: a markdown string with a placeholder ${uri}"
• "linkUris": true will cause any term occurrences be linked to the external source rather than to the glossary of the markdown book.

So a config on the importing side could look like:

{
  "glossaries": [{
    "import": {
      "file": "./glossary.json"
    },
    "file": "./glossary.md",
    "linkUris": true,
    "showUris": true
  }]
}

Would be glad to hear your feedback.

[christophcemper]

Thank you very much for the ping and the new feature! Fantastic!

I noticed some weeks ago that you were heavily working on it and going to lenghts I thought about but didn't suggest, so I was excited.

I am swamped in some other projects, including actually the complex Spec project - so that "pain" is still there and I cannot wait to roll this out to the project and give you extensive feedback!

Thanks again, expect detailled feedback in the next weeks when this is all in place - cannot wait for it myself!

[about-code]

I am closing this since v6.0.0 is out now featuring imports and exports. However, I am still interested in your feedback so feel free to keep on commenting on this issue.