Skip to content

substrate-system/hyperstream

Repository files navigation

hyperstream

tests types module semantic versioning Common Changelog install size license

Use CSS selectors & HTML as a template language.

A re-implementation of the classic @substack module, using Web Streams for compatibility with browsers, Cloudflare Workers, and Deno.

Contents

Install

npm i -S @substrate-system/hyperstream

Browser Example App

A browser demo is in /example_browser.

npm start

Example

Take some template HTML, and transform it using CSS selectors.

Stream a template file through hyperstream, inject stream values by selector, then stream the transformed output into result.html:

import hyperstream from '@substrate-system/hyperstream'
import { S } from '@substrate-system/stream'
import { toFileSink } from '@substrate-system/stream/node'
import { open, type FileHandle } from 'node:fs/promises'

function toByteStream (fh:FileHandle):ReadableStream<Uint8Array> {
    return fh.readableWebStream({ type: 'bytes' }) as ReadableStream<Uint8Array>
}

async function run ():Promise<void> {
    const template = await open('./template.html', 'r')
    const nav = await open('./partials/nav.html', 'r')
    const footer = await open('./partials/footer.html', 'r')
    const result = await open('./result.html', 'w')

    try {
        const hs = hyperstream({
            '#main-nav': toByteStream(nav),
            '#main-footer': toByteStream(footer),
            '#build-time': S.from([
                new TextEncoder().encode(new Date().toISOString())
            ]).toStream()
        })

        await toByteStream(template)
            .pipeThrough(hs.transform)
            .pipeTo(toFileSink(result))
    } finally {
        await Promise.allSettled([
            template.close(),
            nav.close(),
            footer.close(),
            result.close()
        ])
    }
}

await run()

Strings

Process HTML with string replacements:

import { fromString } from '@substrate-system/hyperstream'

const result = await fromString(
    `<html>
        <head><title id="title"></title></head>
        <body>
            <div class="content"></div>
        </body>
    </html>`,
    {
        '#title': 'Hello World',
        '.content': { _html: '<p>This is the content</p>' }
    }
)

console.log(result)

Output:

<html>
    <head><title id="title">Hello World</title></head>
    <body>
        <div class="content"><p>This is the content</p></div>
    </body>
</html>

TransformStream API

Use the TransformStream interface for streaming processing:

import hyperstream from '@substrate-system/hyperstream'
import { S } from '@substrate-system/stream'

const hs = hyperstream({
    '#title': 'Hello World',
    '.content': { _html: '<p>This is the content</p>' }
})

// Create a readable stream from a string
const encoder = new TextEncoder()
const input = S.from([
    encoder.encode('<html><head><title id="title"></title></head><body><div class="content"></div></body></html>')
]).toStream()

// Pipe through hyperstream and consume the result
const decoder = new TextDecoder()
const output = input.pipeThrough(hs.transform)
const chunks = await S(output).toArray()
const totalLength = chunks.reduce((sum, chunk) => sum + chunk.length, 0)
const bytes = new Uint8Array(totalLength)
let offset = 0
for (const chunk of chunks) {
    bytes.set(chunk, offset)
    offset += chunk.length
}
const result = decoder.decode(bytes)

console.log(result)

Append and prepend

Use _appendHtml, _prependHtml, _append (text), or _prepend (text) to add content before or after existing content:

import { fromString } from '@substrate-system/hyperstream'

const result = await fromString(
    '<ul class="list"><li>First</li></ul><span class="greeting">World</span>',
    {
        '.list': { _appendHtml: '<li>New item</li>' },
        '.greeting': { _prepend: 'Hello, ' }
    }
)

console.log(result)

Output:

<ul class="list"><li>First</li><li>New item</li></ul><span class="greeting">Hello, World</span>

Streams

Pass a ReadableStream as the value to insert streamed content:

import hyperstream from '@substrate-system/hyperstream'
import fs from 'node:fs'
import { S } from '@substrate-system/stream'

// Helper to convert a file to a ReadableStream
function fileToStream(path: string): ReadableStream<Uint8Array> {
    const content = fs.readFileSync(path)
    return S.from([new Uint8Array(content)]).toStream()
}

const hs = hyperstream({
    '#a': fileToStream('./content-a.html'),
    '#b': fileToStream('./content-b.html')
})

// Process template
const template = fileToStream('./template.html')
const output = template.pipeThrough(hs.transform)
const chunks = await S(output).toArray()
const totalLength = chunks.reduce((sum, chunk) => sum + chunk.length, 0)
const bytes = new Uint8Array(totalLength)
let offset = 0
for (const chunk of chunks) {
    bytes.set(chunk, offset)
    offset += chunk.length
}

const decoder = new TextDecoder()
console.log(decoder.decode(bytes))

Attribute manipulation

Set attributes directly, or use append/prepend to modify existing values:

import { fromString } from '@substrate-system/hyperstream'

const result = await fromString(
    '<input><button class="btn">Click</button><a>Link</a>',
    {
        'input': { value: 'default', placeholder: 'Enter text...' },
        '.btn': { class: { append: ' active' } },
        'a': { href: 'https://example.com' }
    }
)

console.log(result)

Output:

<input value="default" placeholder="Enter text...">
<button class="btn active">Click</button>
<a href="https://example.com">Link</a>

Transform functions

Pass a function to transform the existing content:

import { fromString } from '@substrate-system/hyperstream'

const result = await fromString(
    '<span class="count">41</span><span class="upper">hello</span>',
    {
        '.count': (html) => String(parseInt(html) + 1),
        '.upper': (html) => html.toUpperCase()
    }
)

console.log(result)

Output:

<span class="count">42</span><span class="upper">HELLO</span>

API

hyperstream(config)

Create a Hyperstream instance with the given configuration.

Returns an object with:

  • transform: A TransformStream<Uint8Array, Uint8Array> for piping
  • readable: The readable side of the transform
  • writable: The writable side of the transform

fromString(html, config)

Convenience function to process HTML from a string.

Returns a Promise<string> with the processed HTML.

createHyperstream(config)

Create a raw TransformStream<Uint8Array, Uint8Array>.

processHyperstream(input, config)

Process a ReadableStream<Uint8Array> and return a Promise<Uint8Array>.

Configuration

The config object maps CSS selectors to values:

  • String/Number: Replace element content
  • { _html: string }: Replace content with raw HTML
  • { _text: string }: Replace content with HTML-escaped text
  • { _appendHtml: string }: Append raw HTML to content
  • { _prependHtml: string }: Prepend raw HTML to content
  • { _append: string }: Append HTML-escaped text
  • { _prepend: string }: Prepend HTML-escaped text
  • { attr: value }: Set attribute value
  • { attr: { append: string } }: Append to attribute
  • { attr: { prepend: string } }: Prepend to attribute
  • ReadableStream: Replace content with stream output
  • (html) => string: Transform existing content
  • null: Skip this selector