Skip to content

Alipsa/munin

Repository files navigation

Munin

This is a report server for reports created in either R based on Renjin or Groovy . It is an application on Spring Boot. The name comes from the one of Odin's ravens who he sent out every day to scout the world and bring him back reports.

example report

Overview

Munin is a reporting server that can run and display reports, created in Renjin R or Groovy, on the web.

Currently, it supports:

  • R reports where the R code returns html
  • mdr Reports (markdown with support for r code, similar to rmd - more on that further down).
  • Groovy reports where the Groovy code returns html
  • gmd reports (markdown with support for groovy code - more on that further down)

Ride supports the Munin R and mdr report formats natively, so you can use Ride to create and edit Munin reports.

Gade support the Munin Groovy and gmd report formats natively, so you can use Ride to create and edit Munin reports.

Parameterized reports

When publishing a report you can optionally add report parameters in the form of html form content, e.g:

<div class="form-group">
<label for="firstName">First name</label>
<input id="firstName" name="firstName">
</div>

<div class="form-group">
<select name="dataSet">
   <option value="mtcars">Motor Trend Car Road Tests</option>
   <option value="iris">3 species of iris flowers</option>
</select>
</div>

If you want numeric values, you need to convert the parameter to a number.

Note that, in order to be able to schedule a parameterized report, you must provide default parameters in the R code e.g. by using exists(). Let's say the parameter is the name of the dataset to use i.e.

<div class="form-group">
<select name="dataSet">
   <option value="mtcars">Motor Trend Car Road Tests</option>
   <option value="iris">3 species of iris flowers</option>
</select>
</div>

Then you can provide a default value for it as follows:

if (!exists("dataSet")) {
  dataSet <- "iris"
}

or the equivalent for groovy:

if (binding.hasVariable('dataSet')) {
  dataSet = se.alipsa.groovy.datasets.Dataset.iris()
}

Styling

Bootstrap is available, so you can use bootstrap classes to style the form. If you are using the htmlcreator package, the html.add() functions takes a list of attributes as an optional parameter. This way you can add attributes such as id and class like this:

html.add(mtcars, htmlattr=list(id = "mtcars-table", class="table table-striped"))

for groovy (using the gmd library) it is

html = new se.alipsa.groovy.gmd.Html()
html.add(se.alipsa.groovy.datasets.Dataset.mtcars(), ["id":"mtcars-table", "class": "table table-striped"])

You can either upload a common stylesheet (using the "common resources" button) that you can reference in your reports e.g.

# import the uploaded stylesheet mystyle.css
html.add("<link rel='stylesheet' href='/common/resources/mystyle.css' />")

or, to put it in the head section (should only be needed if your viewers have very old browsers):

# import the uploaded stylesheet mystyle.css
html.add('
  <script>   
    const cssLink = document.createElement("link");
    cssLink.href = "/common/resources/mystyle.css";
    cssLink.rel="stylesheet";
    document.head.appendChild(cssLink);
  </script>
')

...and you can of course also add stylesheets inline, e.g.

# add a style class to adjust the font size of table text:
html.add("
  <style>
    .table-font-size {
      font-size: 14px;
    }
  </style>
")

then for R:

# reference the class together with some bootstrap classes when rendering a table:
html.add(mtcars, htmlattr=list(class="table table-striped table-font-size"))

... and for groovy

// reference the class together with some bootstrap classes when rendering a table:
html.add(mtcars, ["class": "table table-striped table-font-size"])

Installing

There are a few different ways to install Munin.

  1. Simple:

    • Download the munin-[version].jar file from https://github.com/perNyfelt/munin/releases/latest
    • Copy the jar to a directory of your choice
    • create an application-prod.properties file and override whatever default config you need
    • run the application with java -Dspring.profiles.active=prod -jar munin-[version]-exec.jar See production config for info on how to make it a service.
  2. Customized: This is appropriate if you want to do more involved customization.

    • Create a new maven (of Gradle or whatever) project and set munin as the parent project:
    <parent>
        <artifactId>munin</artifactId>
        <groupId>se.alipsa</groupId>
        <version>1.2.1</version>
    </parent>
  3. Customized alternative: Fork the project on github and make any changes you want. Create the executable jar with mvn clean package and copy it from your target dir.

Demo

The release jar is in "demo" mode meaning it comes with a few user accounts preinstalled, and uses a file base h2 database for persistence. You start it by simply doing java -jar munin-[version]-exec.jar. The application will be available on http://localhost:8088

See here for some screenshots.

Admin

The default admin user name / password is admin / adminpwd. If you log in as admin you will see an "Administration" button on the main page (http://localhost:8088). There are three predefined roles:

  • Viewer: someone who can only view reports. There is one predefined: test / testpwd
  • Analyst: someone who can view / add / edit and schedule reports. There is one predefined: analyst /analystpwd
  • Admin: someone who can do user/role administration. There is one predefined: admin / adminpwd

Sample reports

There a few example reports that might help you get going which you can download/copy and publish to the Munin server:

Production config

You can do any customization by adding an application-prod.properties file next to the jar. Then start the server with -Dspring.profiles.active=prod set e.g. java -Dspring.profiles.active=prod -jar munin-[version]-exec.jar This will override any default config with your specific config.

application-prod.properties variables

See application.properties for settings to override. Typically, you will change the following

Web port

Set the property server.port to something else e.g. server.port=8080 to listen for web requests on port 8080 instead of the default 8088.

Database

The database stores the reports and user config. Default config is a file based H2 database (jdbc:h2:file:./munindb;DATABASE_TO_LOWER=TRUE) To change the underlying database config, set the spring.datasource.xxx parameters as you see fit.

There is a braking change between the h2 database used prior to munin version 1.2.0 (i.e. h2 1.4.x) and the h2 version used in version from 1.20 and later (h2 version 2.1.x). To upgrade the database you need to export the old database to SQL and then import in the new version. Essentially you need to the following steps

  1. Download the 1.4.200 h2 from https://repo1.maven.org/maven2/com/h2database/h2/1.4.200/h2-1.4.200.jar
  2. Backup using java -cp h2-1.4.200.jar org.h2.tools.Script -url jdbc:h2:file:./munindb -user sa -script test.zip -options compression zip
  3. rename ./munindb.* to ./oldmunindb.*
  4. Download 2.210 from https://repo1.maven.org/maven2/com/h2database/h2/2.1.210/h2-2.1.210.jar
  5. Restore using java -cp h2-2.1.210.jar org.h2.tools.RunScript -url jdbc:h2:file:./munindb -user sa -script test.zip -options compression zip FROM_1X

See the Upgrade, Backup, and Restore section of the h2 documentation for details. There is also a simple database migration script you can use (edit credentials etc. appropriately first). For 2.1 to 2.2 upgrade. use the upgradeH2.sh script instead.

Note that if you want another database other than H2, you need to make sure spring boot can access the jdbc driver jar. This can be done by setting the loader.path, e.g:

  1. create a lib folder where your spring boot jar resides
  2. copy the additional jar to the lib folder
  3. add the path to the folder when starting spring boot: java -Dloader.path=file:lib/ -Dspring.profiles.active=prod -jar munin-[version]-exec.jar

Mail

Mail is used to email passwords when users are created as well as mailing out scheduled reports. Set spring.mail.xxx properties (host, port, username, password) as suitable for your mail server. The "from" address is controlled by the property munin.email.from

Monitoring

Perhaps not something you would typically override, but you likely want to set up some kind of integration with whatever health monitoring tool you are using at your business.

Actuator is included with default settings which means that a network monitoring tool can check for availability by querying http://your-host-name:8088/actuator/health which will return the json string {"status":"UP"} if everything is normal.

Run Munin as a service

To run munin as a service, create a service starter script and make it run as a Linux service. Essentially:

  1. Edit the service starter script
  2. Copy the script to /etc/systemd/system
  3. start the service sudo systemctl start munin.service
  4. Make sure the service is running systemctl is-active munin.service

For a Windows service see winsw. Essentially you do:

  1. Take WinSW.NET4.exe (or WinSW.NETCore31.x64.exe if you do not have .NET installed) from the distribution, and rename it to munin-windows.exe.
  2. Edit munin-windows.xml to match your file locations.
  3. Place those two files side by side.
  4. Run munin-windows.exe install to install the service.
  5. Run munin-windows.exe start to start the service.

See the Spring documentation for more info.

Other configuration tasks

The first thing you should probably do after setting up a database and providing the necessary config overrides is to change / remove the three predefined users using the admin interface mentioned in the demo section above. If you want to keep the admin user, begin by assigning your email to it and then log out and reset the password - a new password will then be emailed to you.

Integration

Munin provides a REST api for integration with other applications. It is described here.

Reusing code

You might notice that you have code snippets that you want to centralize and reuse. The standard approach to that is to formulate those code snippets into functions that you include in a package.

An alternative way is to upload the R code into the common resource area and source it from the report e.g.

source(paste0(muninBaseUrl, "/common/resources/utils.R"))

Version history

2.0.0, in progress

  • Add support for Groovy and GMD reports.
  • Upgrade spring boot (2.7.2 -> 3.1.2)
  • Upgrade thymeleaf (5 to 6)
  • Upgrade boostrap (5.2.0 -> 5.3.1)
  • Upgrade jQuery (3.6.0 -> 3.7.0)
  • Upgrade liquibase (4.15.0 -> 4.23.1)
  • Upgrade mdr (1.5.1 -> 1.5.2)
  • Upgrade webjars (0.45 -> 0.47)
  • Upgrade h2 (2.1.214 -> 2.2.220)
  • Upgrade cron-utils (9.2.1 -> 9.2.0)
  • Require Java 17
  • Add parameters to the Preprocessor so that e.g. = expressions works

1.2.1 (2022-Aug-16)

  • Upgrade dependencies for bootstrap (5.1.3 -> 5.2.0), cron-utils (9.1.6 -> 9.2.0), liquibase (4.9.1 -> 4.15.0), h2 (2.1.210 -> 2.1.214), spring-boot (2.6.6 -> 2.7.2)

1.2.0 (2022-Jan-22)

  • Upgrade the h2 version from 1.4.200 to 2.1.210 due to security vulnerability issues with the previous versions. Migration is needed.
  • Added a migrateDb.sh script to facilitate h2 migration
  • Upgrade liquibase core from 4.7.0 to 4.7.1

1.1.7 (2022-Jan-21)

  • upgrade spring boot, mdr, plugins etc.
  • Add some example to docs

1.1.6 (2021-Dec-08)

  • Add breadcrumbs for improved navigation
  • Cleanup alignments and fix some bootstrap 4 to 5 changes
  • Add info dialog for report types
  • upgrade spring-boot, liquibase, junit, htmlcreator, mdr2html

1.1.5 (2021-Oct-17)

  • Bump versions for jquery, spring boot, cronutils, liquibase, spotbugs-annotations, junit, commons-collections bootstrap, webjars-locator
  • Load javascripts at the end of the body instead of in the header - should give a slight perceived speed increase
  • Add preview to common files section

1.1.4 (2021-Mar-16)

Fix regression bug introduced in 1.1.3 where jquery update was not handled properly in the header.

1.1.3 (2021-Mar-13)

  • Allow public access to shared resources (/common)
  • Add muninBaseUrl variable to the global R env so it can be referred to in reports.
  • Upgrade jquery

1.1.2 (2021-Feb-17)

  • add docs on how to add external css (two ways, in body or in head using js)
  • cleanup some link tags in the headers (use xhtml style)
  • Add mdr output to the mdr example in the README.md
  • Add restapi for integration with other tools (mainly to be able to add built in support for creating/editing Munin reports in Ride).
  • convert unmanaged R reports to character vector if it returned something else.
  • upgrade dependencies for junit, liquibase, mdr2html, and spotbugs, htmlcreator, spring-boot, bootstrap

1.1.1 (2021-Jan-13)

  • Add syntax highlighting for code blocks in mdr reports.
  • Fix bug that prevented "normal" syntax highlighted blocks from working properly
  • Go to group page instead of back to index after edit report

1.1.0 (2021-Jan-08)

  • Change to deploy the original jar to central instead of the repackaged one.
  • Improve documentation
  • Add support for the mdr (markdown with r) format
  • Add report groups (folders that go one level deep only)

1.0.1 (2020-Dec-26)

Add support for adding/removing external content (e.g. images and css) which can be referenced from the reports.

1.0.0

Basic functionality for admin and view/add/edit/delete/schedule reports.

3:rd party dependencies

Renjin

  • R interpreter for the JVM; used to execute reports
  • License: GNU General Public License v2.0

Spring boot

  • The application server framework providing most of the functionality
  • License: Apache License 2.0

renjin-spring-boot-starter

  • Integration of Renjin with Spring boot
  • License: MIT

htmlcreator

  • renjin extension (package) to create html from R objects (data.frames, images etc)
  • License: MIT

mdr2html

  • renjin extension (package) to create html from mdr files / content
  • License: MIT

commons-io

  • Various IO stuff
  • License: Apache License 2.0

commons-collections4

  • Used to handle various collection transformations
  • License: Apache License 2.0

cron-utils

  • Used to parse cron expressions to provide "plain english" descriptions for them.
  • License: Apache License 2.0

webjars-locator

  • Simplifies web resource locations
  • License: MIT

webjars bootstrap

  • Pretty web pages
  • License: Apache License 2.0

webjars jquery

  • More convenient javascripts
  • License: MIT

webjars codemirror

  • Syntax highlighting for R and html code
  • License: MIT

highlightjs

  • Syntax highlighting for mdr reports
  • License: BSD 3-Clause

See the pom.xml for more details...