Website analytics, embed Javascript config?

[r3cgm]

I'm asking this question a little prematurely so please pardon that I haven't gone through all the code.

One additional feature I'd like to take advantage of is website tracking. I could go the oldschool route and send the madness logs through Splunk or even parse manually to get # of visitors and popular pages etc. I'm hoping to use madness to (among other things) publish a professional portfolio, and for that I'd like some basic website analytics.

It has been several years since I've set one of these up but I think for example Google Analytics just gives you a simple javascript embed. I did notice that madness is already including other javascript like jquery.

Is there a way to leverage themes or the madness config itself to specify a javascript include? Just wanted to ask before digging in and proposing a PR with new code to support this feature, assuming it would be of interest more generally.

[DannyBen]

Well - medness was primarily designed for internal / local use. It does not contain analytics integration by design.

That said, it is quite easy to customize all aspects of the HTML and CSS. To add Google Analytics (or any other), you can do this:

1. Create a theme folder in your document folder

Run madness create theme _theme

This will copy the original theme from madness to ./_theme

2. Edit the layout to include another "partial" HTML

Edit _theme/views/layout.slim and add this immediately after head:

html
  head
    == slim :'_analytics'
    # ... the rest of the file 

3. Create the Analytics partial

Create a new file in _theme/views/analytics.slim with this content:

- if ENV['GOOGLE_ANALYTICS']
  javascript:
    (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

    ga('create', '#{ENV['GOOGLE_ANALYTICS']}', 'auto');
    ga('send', 'pageview');

4. Instruct madness to use this theme

Either by running madness --theme _theme or adding theme: _theme in the config file.

You can create a new config with madness create config.

5. Set the environment variable

Now, on any environment that contains the environment variable GOOGLE_ANALYTICS (it should include the Google Analytics ID for the new analytics - G-xxxxxxxxx), you will have analytics enabled.

---

Of course, if you do not need the environment variable, or you prefer putting this code directly inside the layout file instead of having a partial, you can do that.

You can use this approach to add anything you want - the only think you need to know, is that madness uses slim, so any HTML you may want to include, you need to do it in slim.

Does this cover your need?

[DannyBen]

Thinking about it - maybe we should revisit the "no analytics by design" statement, and just add it as a config?

[r3cgm]

Again, thanks for all the detailed and timely help. And to be clear, this is in no way intended to be used for any commercial purposes. But it is a replacement for my legacy WordPress blog, which featured photography, woodworking, and coding articles. WordPress had many problems including the amount of overhead maintaining it, that it was an absolute bear trying to export from the WordPress hosted service to my own self-hosted, and that it was just fundamentally way too complex for the simple needs that I had.

I know this doesn't seem necessarily relevant for this discussion but I wanted to mention just to set context for what is motivating the desire for analytics in the first place. Madness may not have been intended for internal use and not use analytics by design, but I would humbly submit that it is a perfect platform for the public sharing of knowledge. Better than a blog, better than a wiki, and having all the content itself as raw markdown files on disk means it is futureproof and I can even leave a backup on a thumb drive for Next of Kin in the far future. All I had to do was hook up Dropbox so that I could sync the content itself from my local Mac to a headless hosted Linux box, and now all my local edits with vim, Obsidian, or Typora literally take 3-4 seconds to automatically show up on the hosted website. I am very pleased with this solution.

My overall approach with madness was to create a Public instance with all the notes, historical archives, photos, and articles that I wanted the world to see. And a Private instance with details that are just meant for personal consumption. This is working really well in practice. I have two madness containers on two different ports, and the Private one is password-protected.

So in terms of analytics I have two needs: 1. a Google Analytics style system to see who was interested in which public pages, and 2. a log to provide an audit trail against the Private instance in case I misconfigured the password feature some day and there was unauthorized access. For the purposes here we don't need to worry about #2, but in truth I am probably interested in long-term storage of logs from both instances. I have reservations about Google Analytics itself because I know a lot of people (myself included) use selective JavaScript loading via Ghostery, so we filter out the JS include itself and that prevents the pageloads from being counted in their stats. I wouldn't count GA as an accurate representation of actual user traffic. I'd rather rely on the logfiles themselves as the source of truth because that is an accurate count of what was actually downloaded.

Yesterday I went back to GA to set up a new account and was floored with how bad the system has gotten. They said I had to switch to a new version called GA4 and they helpfully provided extensive links to hours and hours of documentation I could familiarize myself with to make the jump. This is the same problem WordPress had. When you say you have "hours and hours" of tutorials to use the system, that's not a good thing, that's showing how poor the user interface has become to your product that you require users to spend that much time trying to figure things out. :)

So I started looking for alternatives and found a couple at least that started with the pitch "Are you frustrated by the 80 options for reporting on the GA homepage? Looking for something simpler?" Bottom line, I'm trying to find something better and long-lasting than the unmanageable behemoth GA has become.

Again, I know this is potentially slightly off topic, but it is shaping how I want to use madness, and I'll bet the same would be for others as well. I guess at this point I'd rather ditch GA entirely and go with a combination of custom local log parsing (does madness emit in Apache-compatible weblogs? if so then I could use a bunch of off-the-shelf stuff to do it), and maybe explore some of the alternative GA-like systems that still operate via inclusion of a JS snippet, in which case all the info you'd supplied about theme inclusion still applies.

All that said, I don't know if that answers your question about the "no analytics by design" and whether to include as a config option. For my case at least it's a little complicated and I'm still trying to figure out some of these details. I definitely can understand you not wanting to support a bunch of analytics stuff in madness itself, but I do feel like the two use cases I'm highlighting could be shared as others, and some basic way to support them would be appreciated.

George

[DannyBen]

Well:

this is in no way intended to be used for any commercial purposes

You can use madness in commercial context, as it is MIT licensed. I know you probably meant you are not using it for "heavy duty production" purposes, but just thought I'd clear this point as well.

Bottom line, I'm trying to find something better and long-lasting than the unmanageable behemoth GA has become.

I totally agree. Google Analytics was always convoluted and overly complex, and the new version does not improve upon it, it just makes it even worse. This is a perfect point adding to my "no analytics by design" choice - since Google Analytics is no longer the immediate "go to" solution.

does madness emit in Apache-compatible weblogs?

Madness uses puma and Sinatra, so it emits whatever you see in the stdout. Although - similarly to the analytics topic - logging is not a top priority for madness, I guess it should not be difficult to add some configuration for it, if there is a clear use case for it.

I could use a bunch of off-the-shelf stuff to do it), and maybe explore some of the alternative GA-like systems that still operate via inclusion of a JS snippet, in which case all the info you'd supplied about theme inclusion still applies.

Yes. The approach of exposing the theme and customizing it as you see fit seems most appropriate for you.
The only disadvantage of such an approach is that you diverge from any changes madness may have in its built in templates - but these changes are rare.

All that said, I don't know if that answers your question about the "no analytics by design"

It absolutely does. I will avoid adding analytics for the time being. Adding Google Analytics can open the door for more requests to add other analytics systems, which is undesirable as I am an extremely lazy developer... :)

---

Thanks for sharing your journey.
Let me know how you progress and if there is anything that you need help with.

[DannyBen]

@r3cgm - just to make sure you haven't missed it in your search for the optimal tool for your use case - have you seen Retype? For documentation that is intended for public consumption, it might be more suitable.

[r3cgm]

Re: customizing theme and diverging from updated templates, I can live with that. Believe me, I've used more than one project over the years where every so often I had to bring up the current and new configs side by side in an editor and reconcile my overrides against a new config file. Not a big price to pay.

Retype looks interesting. I had not seen that before. It really does look like a great project, but it suffers from the same limitation that prevented me from using MKDocs too, which was that I didn't want to pre-render a website from Markdown and have it live side-by-side the original site, I literally wanted to have the Markdown files themselves and nothing else needed, and leave the rendering dynamically for when the document needs to be read, and do it with either a web process or a local editor/reader. I don't know how you view madness vs. Retype but in my opinion the madness method (method to the madness?) is the far preferable method.

It's kind of like in the old days when they used to say you should only SSL-encrypt the pages that absolutely needed it and leave everything else unencrypted, because otherwise it would be so much computation to the webserver to process all that SSL it would never keep up! Fast forward a few years and that's really not a concern anymore. Processors are fast enough to keep up. And I'm not being dismissive of the performance cost of dynamic rendering at scale; my team manages hundreds of thousands of cores. And if it took 320,000 cores to render an unencrypted site vs. 360,000 cores to render w/TLS, you can probably assert that's money well spent. YMMV.

Same thing for madness vs. pre-rendering a site a la Retype or mkdocs. You could claim "it's so much more efficient to pre-render the site to cached html and then serve those pages over and over again, than it is to re-render ondemand the Markdown > html over and over again." And you wouldn't be wrong. But the overhead and hassle factor of having to maintain a pre-rendered website / provide disk for / fix permissions with / deal with delays in conversion to / ... it just doesn't justify the added complexity IMO. I mean, what truly makes this whole setup magical is that we completely avoid tool lock-in. You know I like madness; that's obvious right? But even if madness went away we'd still be left with ASCII Markdown files, readable directly with any standard text editor. I'll bet we could beam a megabyte of Markdown toward Vega and any intelligence living there could reverse engineer Markdown in a day. The format is that easy!

All that said, there is a significant limitation with an approach like madness that I'm not going to pretend isn't there. With other systems like Retype you can have this idea of components and widgets and do fancy stuff like emojis and file download dialog boxes and panel expanders etc. It would be really cool to have that kind of capability in madness too, but unless I'm misunderstanding something you couldn't do that unless you either hacked up Markdown beyond its original specifications to support proprietary syntax for widgets, or created a whole new DSL / macro system for substituting these things. But in the end if you're willing to forgo widgets, then you achieve future proof content that is maximally portable. That's the tradeoff. And I'm happy with the results and consequences, for now at least.

I guess it's possible however that once I start really transferring all my content to this system I might start to feel like I'm bumping up against the limitations more and more. One limitation I can already see looming is with photography and thumbnails. Once upon a time I'd written my own Perl/CGI thumbnail tool that would generate them on-the-fly (in madness style) from the full sized jpegs, rather than necessarily pre-rendering them in advance to save on computation. Even 20 years ago I felt this same way about the overhead of pre-rendered sites. :)

The Markdown controls for specifying image display size as a fraction of the original (either in pixels or percent) is lean to non-existent. So I'm imagining now I'll have to write a script to generate a page of thumbnails embedded in Markdown, and then click out to other Markdown pages where maybe it's one page per full-sized image. Not sure yet. That would have the advantage of keeping the file format 100% compatible Markdown, at the cost of dependency on an external helper tool to do some of the thumbnail render heavy lifting.

Maybe that's what we need: a repo of interesting scripts that can key off SOME SYNTAX SOMEHOW and provide widget-like functionality that would still do its best to adhere to render the widgets while sticking to Markdown pure format. Kind of a tall order, but I could imagine how this could work for image thumbnails.

[DannyBen]

I didn't want to pre-render a website from Markdown and have it live side-by-side the original site

Agree. This is one of the reasons I made madness.

we completely avoid tool lock-in. You know I like madness; that's obvious right? But even if madness went away we'd still be left with ASCII Markdown files, readable directly with any standard text editor

Heh :) - well, it is an interesting point you touched. One of the things that annoy me a little in other markdown renderers, is the fact that they "upgrade" the markdown syntax to support all sorts of additional custom components. This makes the markdown not portable, and less human-readable. So, yes - having madness use a standard markdown so that your documents remain portable is also one of the objectives.

All that said, there is a significant limitation with an approach like madness that I'm not going to pretend isn't there. With other systems like Retype you can have this idea of components and widgets and do fancy stuff like emojis and file download

I wrote the previous section before I read this comment of yours. I have migrated some of my sites from GitBook to Retype. Both use widgets, using a different syntax. It was a bad experience. The advantage of being able to show nicely formatted panels is ruined by the reduced portability and readability. It is unlikely madness will ever have non-custom markdown widgets. It is a design choice, not an oversight.

And if I never see another emoji ever again, it will be too soon.... I think the iconization of everything got a little out of hand.