🌶 Ethereum.org Website's External URLs 🌶

                                                                                                 

Test/Demo repo for implementing the content pages' link references parameterization.

Parameterize references to external links, using some sort of variable injection logic to replace these in the content md files before building; creating a single source of truth for references to external URLs used across the website, and simplifying maintenance tasks

🚀 Installation & Usage

### Get your copy

1. fork the repo
2. clone your fork

git clone git@github.com:[your_github_handle]/ethorg-markdown-preprocessor.git && cd ethorg-markdown-preprocessor

3. configure your fork

git remote add upstream https://github.com/juanchax/ethorg-markdown-preprocessor.git

4. ensure your fork is up-to-date

git checkout dev
git fetch upstream
git merge upstream/dev

5. install dependencies
6. create a new branch and get your hands dirty(?)

git checkout -b new_branch_name

🎮 Use it!

This is an isolated implementation of a preprocessor for Ethereum.org Website's content Markdown files. The folder structure matches and/or mimics the ethorg's folder structure.

ethorg-markdown-preprocessor
├── public
│   └── content
│       └── index.md
├── src
│   └── lib
│       ├── md
│       │   ├── compile.ts
│       │   ├── data.ts
│       │   ├── export.ts
│       │   ├── import.ts
│       │   └── preprocess.ts
│       ├── constants.ts
│       └── variables.ts
├── README.md
├── index.ts
├── package-lock.json
└── package.json

🔦 Make & Break - Files ref.

• index.ts - File that contains the calls to pick up a Markdown file and pre-process it. Creates an output file in the same path prepending compiled- to the filename.
• index.md - Sample Markdown file, cloned directly from my fork of ethereum-org-website
• constants.ts - File reference since constant base URLs are referenced in the variables file.
• variables.ts - File containing the variable refs in use in markdown content files . Note: the uppermost variables defined in the file are already defined in the original, just copied here for reference+usage.
• export.ts - Utility file containing the logic to export the processed md contents to a file; in the actual implementation the content us just fed to the subsequent build steps.
• import.ts, compile.ts, data.ts - Minimal logic to mimic ethereum-org-website structure and execution.
• preprocess.ts - WiP This is where the (RegEx) magic happens!

🔧 Tools

Here are some tools that are useful (or even cool) that were used while building this:

• Coffee. LOTS of coffee.
• RegEx101 - Build, test, explore, save & share custom RegEx
• Tree - Neat little tool to ogenerate a Markdown friendly structure of your repo. Just install brew install tree and run `tree --gitignore --dirsfirst on the root of your project folder

💣 Issue Description

Currently, the content pages contain a multitude of hardcoded link references, which of course are duplicated on the translated versions as well.

Causing:

• There's no single source of truth
• brittle approach
• Difficult to know if all links are synched (as in: all have the same url)
• Updating dead links is a laborious task to say the least

📌 Background

I was omw to tackle some GFI on GitHub when I noticed a few Issues that were related to link references, more specifically the OATs reference on the Contributing page

Needless to say, I went down the rabbit hole 🐇 and soon enough I managed to get a clearer idea of what was up.. link references are hardcoded across the website. Not sure if it was always the case or not, but it feels it might not be the best approach.

Here's a bird's eye view of the relevant items:

Issue	Description
Dead links around site #14823	A neat report of all dead links found across website pages
Fixed broken link on Contributing Page #14605	A pr that updated /public/content/index.md only
Old link for article about OATs on Contributing page #14200	An open issue for the same link reference (More on OATs)
Bug report #14711	Same issue, different translated page

But rest assured, many more are coming 😵!

💡 Parameterized Link References

The idea is to apply a modularity mindset and parameterize references to external links, using some sort of variable injection in the content md files; thus creating a single source of truth for references to external URLs used across the website, and simplifying maintenance tasks:

• link references updated
• content pages consistent
• periodically check and update dead link references
• reduced duplication, improved reusability

Implementation Considerations

• Ease of Merging - Opt for an implementation with the least amount of code refactoring
• Consistency - Reuse as many existing classes, functions, structure & architecture as possible
• Reusability - Keep code as reusable as possible, ask for feedback to polish the solution
• Modularity - Break down the preprocess logic into clear bits that just do one thing

ⓘ Implementation Approach

The proposed approach is just my first iteration on the root issue, there's a few items to review/discuss before merging, detailed below.

That being said, let's jump right into it.

🤖 Implementation Details

🗺 Implementation Design

• /src/lib/variables - Append new link reference variables, exported as an object containing for ease of import + processing

• /src/lib/md/preprocess.ts - Contains variable preprocessing logic for the md files

• /src/lib/md/compile.ts- Swap declaration of preprocessMarkdown() for the implementation of the same in preprocess.ts

📋 Task Overview

1. Extract references to external URLs
2. Create easy-to-maintain external-link-reference variables for each item in #1
3. Replace harddoced references with {placeholders} for their variables in all content md files
4. Add necessary processing logic to replace {placeholders} with the corresponding value

🧀 Naming Overview

• EXT_ + REF_NAME - Naming convention for new link references variables appended to the variables.ts file. Variables not individually exported.

• EXT_URLS - Exported object to be imported for preprocessing content markdown files, making it easy to include / exclude existing variables: { EXT_REF_ONE, EXT_REF_TWO }

• preprocessMd() - Moved and renamed function previously known as preprocessMarkdown() 🕺

• {EXT_REF_NAME} - Naming convention used for placeholders in the content md files. EXT_REF_NAME matches the variable name contained in the variables.ts file.

  Code has been structured to accommodate a couple different naming conventions for the placeholders: {EXT_REF_NAME} | ${EXT_REF_NAME} | {{EXT_REF_NAME}}

Implementation Status                                                                           

Repo	Task	Status
ethorg-markdown-preprocessor	Implement link ref variables in variables.ts	Done
ethorg-markdown-preprocessor	Implement Markdown preprocessing logic	Done
ethorg-markdown-preprocessor	Implement link ref placeholders in /contributing/index.md	Done
ethorg-markdown-preprocessor	Move md preprocessing from compile.ts to preprocess.ts file	Done
ethorg-markdown-preprocessor	Test preprocessing variable resolution	Done
ethorg-markdown-preprocessor	Test preprocessing heading anchors escaping	Done
-----------	-----------	-----------
ethereum-org-website	Implement link ref variables in variables.ts	Done
ethereum-org-website	Implement Markdown preprocessing logic	Done
ethereum-org-website	Implement link ref placeholders in /contributing/index.md	Done
ethereum-org-website	Move md preprocessing from compile.ts to preprocess.ts file	Done

Next steps

• Implement in forked repo test, and build locally to ensure nothing breaks.

• Test implementation

• Create pull request

• Link related issues <-> PR

Waiting on PR Aproval

If everything looks good, and Ethereum.org Community Admins approve the refactor:

• Create a complete list of md files

• Replace hardcoded urls with variable placeholders

• Create any additional variables needed in 'variables.ts'

• Test some more, double, triple check (?)

• Create new pull request

• Link previous PR related issues <-> PR