From 2a25a8e36596022965f5b709f7e65cf372893585 Mon Sep 17 00:00:00 2001 From: Novan Allanadi <25704935+nofun97@users.noreply.github.com> Date: Thu, 25 Jun 2020 17:20:35 +1000 Subject: [PATCH 1/4] fixes source file generation issue --- README.md | 214 +----------------- .../ApplicationPackage/index.html | 20 +- demo/html/FirstDivision/index.html | 6 +- .../SecondDivision/MegaDatabase/index.html | 12 +- demo/html/SecondDivision/index.html | 4 +- .../ThirdDivision/ServerPackage/index.html | 36 +-- demo/html/ThirdDivision/foo/index.html | 8 +- demo/html/ThirdDivision/index.html | 4 +- .../html/ThirdDivision/simpleredoc/index.html | 12 +- demo/html/index.html | 7 +- .../ApplicationPackage/README.md | 22 +- .../ApplicationPackage/integration.svg | 12 +- demo/markdown/FirstDivision/README.md | 4 +- demo/markdown/FirstDivision/integration.svg | 12 +- .../markdown/FirstDivision/integrationepa.svg | 26 +-- demo/markdown/README.md | 1 + .../SecondDivision/MegaDatabase/README.md | 12 +- .../MegaDatabase/integration.svg | 8 +- demo/markdown/SecondDivision/README.md | 4 +- demo/markdown/SecondDivision/integration.svg | 8 +- demo/markdown/ThirdDivision/README.md | 4 +- .../ThirdDivision/ServerPackage/README.md | 36 +-- .../ServerPackage/integration.svg | 12 +- demo/markdown/ThirdDivision/foo/README.md | 8 +- demo/markdown/ThirdDivision/integration.svg | 12 +- .../ThirdDivision/simpleredoc/README.md | 12 +- demo/markdown/integration.svg | 12 +- .../ApplicationPackage/index.html | 20 +- docs/FirstDivision/index.html | 6 +- docs/SecondDivision/MegaDatabase/index.html | 12 +- docs/SecondDivision/index.html | 4 +- docs/ThirdDivision/ServerPackage/index.html | 36 +-- docs/ThirdDivision/foo/index.html | 8 +- docs/ThirdDivision/index.html | 4 +- docs/ThirdDivision/simpleredoc/index.html | 12 +- docs/index.html | 7 +- pkg/catalog/generator.go | 31 ++- pkg/catalog/generator_test.go | 24 ++ pkg/catalog/spec.go | 11 + pkg/catalog/spec_test.go | 8 + 40 files changed, 297 insertions(+), 414 deletions(-) diff --git a/README.md b/README.md index 0c7b628..9b2031b 100644 --- a/README.md +++ b/README.md @@ -1,210 +1,16 @@ -# sysl-catalog -A markdown/html + Diagram generator for sysl specifications -## Objective -The objective of sysl-catalog is to create the most seamless experience for developers to document their API behaviour, as well as creating a standardised way multiple teams can create documentation whilst gaining mutual benefit from already existing documentation. -## Background -Let’s say that a team wants some diagrams to represent how their services interact with other services. First, the team needs to choose what format to use, then the team needs to decide on where the docs are going to be hosted, and how often they should be updated. + | [Chat with us]() | [New bug or feature request]() +# test.sysl -Let’s say that this team (Team A) chooses plantuml to create sequence, data model, and integration diagrams for their service. They choose to generate to a `docs` directory on their single repository and use some proto plugins to automate their markdown generation: -- Protoc-gen-doc to generate a digest of what how their protos are structured -- Protoc-gen-uml to generate plantuml diagrams to generate diagrams for the needed diagrams -- Manually written markdown to describe how their services interact with different services -- Manually written sequence diagrams to describe which dependencies are called +| Package | +----| +[App1](App1/README.md)| +[App2](App2/README.md)| -This works fine; the team has somewhat automated their documentation workflow, with some manual parts. +## Integration Diagram + -A couple of months pass and now there’s another team which relies on team A’s service heavily. They are releasing soon and need to create release documentation; so they decide to use the same method that Team A is using. +## End Point Analysis Integration Diagram + -Now there’s a problem; there are two teams with two separate sets of documentation. Some of it is manual and some of it is automated. This can cause problems for multiple reasons; -1. Manually written parts might need to be repeated and might fall out of sync across the two sources -2. If manually written documentation isn't repeated then the representation of their dependency is limited by a hyperlink to Team A's documentation without fully integrating -3. The decoupling of documentation and code means that the documentation is most likely going to fall out of date -4. Because there’s no persistence between team A’s documentation and Team B; if team A make a very specific change it likely won’t show in team B’s documentation - -This is what sysl-catalog is trying to solve - -sysl-catalog uses the sysl language as an intermediary between different formats to be able to generate different views of how services work - -### What is sysl? -[Sysl](http://sysl.io/) is a “system specification language”; think of it like swagger or protos, but a much higher level, and with the ability to represent not only types, applications and endpoints, but interactions between those applications and endpoints; it plans to define what the code does itself. - -### What does sysl-catalog do? - -Sysl-catalog is just a static site generator. - -Sysl-catalog parses a sysl file (with the .sysl extension) and represents it in a visual form; -It can represent endpoints (in sequence diagrams) request/response types or database tables, as well as integration diagrams. - -It uses go's `text/template` to do this, and if any addition is needed to be made, custom templates can be used (see `templates` for examples) - -## Installation - -#### go -```bash -go get -u -v github.com/anz-bank/sysl-catalog -``` - -#### docker -- only recommended for `--serve` mode -```bash -docker run --rm -p 6900:6900 -v $(pwd):/usr/:ro anzbank/sysl-catalog:latest input.sysl --serve -``` - -#### docker-compose -- see (demos)[demo/protos] for full example that includes .proto files -```yaml -version: '3.8' - -services: - plantuml-server: - image: plantuml/plantuml-server:tomcat-v1.2020.13 - ports: - - 8080:8080 - protoc-gen-sysl: - image: anzbank/protoc-gen-sysl:latest - volumes: - - ./:/usr/files - working_dir: /usr/ - entrypoint: ["protoc", "--sysl_out=.", "files/simple.proto"] - sysl-catalog: - image: anzbank/sysl-catalog:latest - volumes: - - ./:/usr/ - environment: - - SYSL_PLANTUML=http://plantuml-server:8080 - entrypoint: ["sysl-catalog", "-o", "docs", "project.sysl", "--imageDest", "docs/images"] - depends_on: - - plantuml-server - - protoc-gen-sysl -``` - - -## How to use -1. Set up environment -`export SYSL_PLANTUML=http://www.plantuml.com/plantuml` - -2. Run - -```bashs -sysl-catalog -o -``` -- You can optionally specify the `--type=html` if you want to generate html instead of markdown, which is useful for use with github pages, which you can see a demo of with this repo [here](https://anz-bank.github.io/sysl-catalog/) - -3. That's it (basically!) - - This will generate markdown with integration diagrams + sequence diagrams + data model diagrams as seen in [demo/markdown/README.md](demo/markdown/README.md) or see html generation at [demo/html/index.html](demo/html/index.html). - - -## Server Mode -sysl-catalog comes with a `serve` mode which will serve on port `:6900` by default - -```bash -sysl-catalog --serve -``` -This will start a server and filewatchers to watch the input file and its directories recursively, and any changes will automatically show: -![example gif](resources/example.gif) - -## Requirements -In [demo/markdown/README.md](demo/markdown/README.md) we have an example with a couple of interesting parts: - - -1. `@package` attribute must be specified: -- This will create a markdown page for `ApplicationPackage` as seen in [demo/markdown/ApplicationPackage/README.md](demo/markdown/ApplicationPackage/README.md). - Currently the package name is not inferred from the application name (`MobileApp`), so this needs to be added (`ApplicationPackage`). -``` -MobileApp: - @package = "ApplicationPackage" - Login(input <: Server.Request): - Server <- Authenticate - return ok <: MegaDatabase.Empty -``` - -2. Application names might need to be prefixed to parameter types if the type is defined in another application, since defined parameters are under scope of the application it is defined in: -```diff -MobileApp: - @package = "ApplicationPackage" -+ Login(input <: Server.Request): -- Login(input <: Request): - Server <- Authenticate - return ok <: MegaDatabase.Empty -``` - -3. Add `~ignore` to applications/projects that are to be ignored in the markdown creation -```diff -ThisAppShouldntShow[~ignore]: - NotMySystem: - ... -# Or ignore only specific endpoints -ThisAppShouldShow[~ignore]: - NotMySystem[~ignore]: - ... -``` - -## CLI options - -#### Output default Markdown -`sysl-catalog -o=docs/ filename.sysl` - -#### Output default HTML -`sysl-catalog -o=docs/ --type=html filename.sysl` - -#### Run with custom templates -- With this the first template will be executed first, then the second -`sysl-catalog --templates=, filename.sysl` - -#### Run in server mode -`sysl-catalog --serve filename.sysl` -![server mode](resources/server.png) - -#### Generate Redoc files -`sysl-catalog --redoc filename.sysl` -This generates a [Redoc](https://github.com/Redocly/redoc) page that serves the original .json or .yaml OpenAPI spec on Github. Currently only supports spec files located in the same repo, and must be run in a git repo (so that the remote url can be retrieved using `git`). - -#### Run in server mode without css/rendered images -- good for rendering raw markdown - -`sysl-catalog --serve --noCSS filename.sysl` -![server mode raw](resources/standard-template.png) - -#### Run server with custom template -`sysl-catalog --serve --templates=, filename.sysl` - -![server mode raw](resources/custom-template.png) - -- See templates/ for custom template examples - -## Command Details -```bash -$ sysl-catalog --help -usage: sysl-catalog [] - -Flags: - --help Show context-sensitive help (also try --help-long and --help-man). - --plantuml=PLANTUML plantuml service to use - -p, --port=":6900" Port to serve on - --type="markdown" Type of output - -o, --output=OUTPUT OutputDir directory to generate to - -v, --verbose Verbose logs - --templates=TEMPLATES custom templates to use, separated by a comma - --outputFileName="" output file name for pages; {{.Title}} - --serve Start a http server and preview documentation - --noCSS disable adding css to served html - --disableLiveReload diable live reload - --noImages don't create images - --embed Embed images instead of creating svgs - --mermaid use mermaid diagrams where possible - --redoc generate redoc for specs imported from openapi. Must be run on a git repo. - --imageDest=IMAGEDEST Optional image directory destination (can be outside output) - -Args: - input sysl file to generate documentation for -``` - -## Screenshots -![resources/project_view.png](resources/project_view.png) -*project_view* - -![resources/package_view.png](resources/package_view.png) -*package_view* diff --git a/demo/html/FirstDivision/ApplicationPackage/index.html b/demo/html/FirstDivision/ApplicationPackage/index.html index 60b7107..f43a88a 100644 --- a/demo/html/FirstDivision/ApplicationPackage/index.html +++ b/demo/html/FirstDivision/ApplicationPackage/index.html @@ -2,11 +2,11 @@ -Sysl Catalog +Sysl Catalog
-

Back

+

Back | Chat with us | New bug or feature request

ApplicationPackage

Integration Diagram

@@ -23,22 +23,22 @@

Application Index

MobileApp Another -../../../../demo/simple2.sysl +https:/github.com/anz-bank/sysl-catalog/blob/master/demo/simple2.sysl MobileApp Login -../../../../demo/simple2.sysl +https:/github.com/anz-bank/sysl-catalog/blob/master/demo/simple2.sysl MobileApp2 SecondEndpoint -../../../../demo/simple2.sysl +https:/github.com/anz-bank/sysl-catalog/blob/master/demo/simple2.sysl MobileApp2 ThirdEndpoint -../../../../demo/simple2.sysl +https:/github.com/anz-bank/sysl-catalog/blob/master/demo/simple2.sysl @@ -57,7 +57,7 @@

Application MobileApp

  • this is a comment for MobileApp
-

MobileApp Another

+

MobileApp Another

hello, world

Sequence Diagram @@ -75,7 +75,7 @@

Response types


-

MobileApp Login

+

MobileApp Login

hello, gamers

Sequence Diagram @@ -97,7 +97,7 @@

Application MobileApp2

  • this is a comment for MobileApp
-

MobileApp2 SecondEndpoint

+

MobileApp2 SecondEndpoint

SecondEndpoint, comment

Sequence Diagram @@ -115,7 +115,7 @@

Response types


-

MobileApp2 ThirdEndpoint

+

MobileApp2 ThirdEndpoint

ThirdEndpoint comment

Sequence Diagram diff --git a/demo/html/FirstDivision/index.html b/demo/html/FirstDivision/index.html index 56b8adb..9cc9056 100644 --- a/demo/html/FirstDivision/index.html +++ b/demo/html/FirstDivision/index.html @@ -2,11 +2,11 @@ -Sysl Catalog +Sysl Catalog
-

Back

+

Back | Chat with us | New bug or feature request

FirstDivision

@@ -23,7 +23,7 @@

FirstDivision

Integration Diagram

End Point Analysis Integration Diagram

- +