NIP-XX Decentralized Web Hosting on Nostr

[studiokaiji]

Decentralized Web Hosting on Nostr

By recording HTML, CSS, and JS on the Nostr relay, it becomes possible to create a decentralized web hosting solution that eliminates the need for centralized servers. Web servers or Nostr clients retrieve these recorded data, transform them into appropriate forms, and deliver them.

Reasons for Hosting on Nostr

• Tamper-resistant through public key-based integrity
• Fault tolerance through deployment on multiple relays
• Resistance to blocking due to the distribution of web servers or clients
• Faster retrieval speed compared to IPFS's DHT

Proposed Approach

Each HTML, CSS, and JS file is assigned a kind for identification.

• HTML: kind: 5392
• CSS: kind: 5393
• JS: kind: 5394

The "content" field contains the content of the file. However, internal links (href, src, etc.) referenced within should be replaced with event IDs.

Example: <link rel="stylesheet" href="066b7ca0b167f0adad5c6d619ab1177050423e3979e83b8dfa069992533bdcf5">

Implementation on Web Server or Client

Access events using /e/{event_id}. Since event IDs are specified for each internal link, opening an HTML file enables automatic retrieval of data from this endpoint.

Implementation Example (Golang)

r.GET("/e/:idHex", func(ctx *gin.Context) {
	id := ctx.Param("idHex")

	// Fetch data from nostr pool
	ev := pool.QuerySingle(ctx, allRelays, nostr.Filter{
		Kinds: []int{consts.KindWebhostHTML, consts.KindWebhostCSS, consts.KindWebhostJS, consts.KindWebhostPicture},
		IDs:   []string{id},
	})

	if ev != nil {
		// Return data with content-type adapted to kind
		switch ev.Kind {
		case consts.KindWebhostHTML:
			ctx.Data(http.StatusOK, "text/html", []byte(ev.Content))
		case consts.KindWebhostCSS:
			ctx.Data(http.StatusOK, "text/css", []byte(ev.Content))
		case consts.KindWebhostJS:
			ctx.Data(http.StatusOK, "text/javascript", []byte(ev.Content))
		default:
			ctx.String(http.StatusNotFound, http.StatusText(http.StatusNotFound))
		}
	} else {
		ctx.String(http.StatusNotFound, http.StatusText(http.StatusNotFound))
	}

	return
})

Replaceable Decentralized Web Hosting

Additionally, this proposal can be extended to incorporate the NIP-33 based decentralized web hosting specification. This allows tracking of website data with a single identifier, keeping URL paths immutable.

Following the NIP-33 specification, the "kind" would be as follows.

• HTML: kind: 35392
• CSS: kind: 35393
• JS: kind: 35394

Identifiers must be included within the "d" tag.

Example

{
	...,
	"kind": 35392,
	"tags": [["e", "hostr-lp"]]
}

Moreover, internal links within the "content" should be assigned NIP-33 identifiers instead of event IDs.

Identifier Example: [html_identifier]-[filepath(replace '/' with a character that can be included in the URL path e.g. '_')]

Link Example: <link rel="stylesheet" href="hostr-lp-_assets_index-ab834f60.css">

Implementation on Web Server or Client

Events can be accessed through /p/{npub_or_hex}/d/{d_tag}.

Implementation Example (Golang)

r.GET("/p/:pubKey/d/:dTag", func(ctx *gin.Context) {
	pubKey := ctx.Param("pubKey")
	// decode npub
	if pubKey[0:4] == "npub" {
		_, v, err := nip19.Decode(pubKey)
		if err != nil {
			ctx.String(http.StatusBadRequest, "Invalid npub")
			return
		}
		pubKey = v.(string)
	}
	// Add authors filter
	authors := []string{pubKey}

	// Add #d tag to filter
	dTag := ctx.Param("dTag")
	tags := nostr.TagMap{}
	tags["d"] = []string{dTag}

	// Fetch data from pool
	ev := pool.QuerySingle(ctx, allRelays, nostr.Filter{
		Kinds: []int{
			consts.KindWebhostReplaceableHTML,
			consts.KindWebhostReplaceableCSS,
			consts.KindWebhostReplaceableJS,
		},
		Authors: authors,
		Tags:    tags,
	})
	if ev != nil {
		// Return data with content-type adapted to kind
		switch ev.Kind {
		case consts.KindWebhostReplaceableHTML:
			ctx.Data(http.StatusOK, "text/html", []byte(ev.Content))
		case consts.KindWebhostReplaceableCSS:
			ctx.Data(http.StatusOK, "text/css", []byte(ev.Content))
		case consts.KindWebhostReplaceableJS:
			ctx.Data(http.StatusOK, "text/javascript", []byte(ev.Content))
		default:
			ctx.String(http.StatusNotFound, http.StatusText(http.StatusNotFound))
		}
	} else {
		ctx.String(http.StatusNotFound, http.StatusText(http.StatusNotFound))
	}

	return
})

Web Server Implementation Vulnerabilities

The current web server implementation allows access to websites within a single domain. While this reduces the implementation complexity on the server side and provides resilience against blocking, it is not suitable for use with domain-based authorization systems (such as NIP-07). For instance, if signing is permitted for Nostr clients on the web hosting relay, it would grant permission for all web pages hosted on that relay, making it vulnerable to spam postings.

Implementation

Repository: https://github.com/studiokaiji/nostr-webhost

Example Implementation: https://h.hostr.cc/p/a5a44e2a531efcc86491c4b9a3fa67daee8f60c9d2757a12eed95d98c5d6fc42/d/hostr-lp

[mattn]

Event id is obtained by following structure. So if you change the content, IDs must be changed.

[
  0,
  <pubkey, as a lowercase hex string>,
  <created_at, as a number>,
  <kind, as a number>,
  <tags, as an array of arrays of non-null strings>,
  <content, as a string>
]

If you provide an original ID system (hostr-id) for hostr different from the ID on nostr, and put the hostr-id in note1, you may be able to access /e/hostr-id. If you rewrite content, you will want to rewrite note1 content (hostr-id should be written), so I think you should provide a kind in the range 10000 <= n < 20000.

[studiokaiji]

A note with kind: 10000 <= n < 20000 is a unique note for pubkey and kind. When you record the hostr id in note1, does that mean you write the identifiers of multiple websites in one note?

[AsaiToshiya]

My implementation (AsaiToshiya/brostr) uses NIP-21 nostr:nevent1... for links.

<link rel="stylesheet" href="nostr:nevent1qqsqv6mu5zck0u9d44wx6cv6kythq5zz8cuhn6pm3haqdxvj2vaaeagvsfd05">

NIP-19 is also fine, I think this is more unified and has more informative.

[vitorpamplona]

Super nice!

Replaceable Decentralized Web Hosting

The replaceable events idea only work for things the pubkey controls. It doesn't make sense for pubkey A's website to reference a pubkey B that has a jQuery in a replaceable event because that event can be independently changed to a scam script and it will be immediately live on pubkey A's website. Replaceable events should be used only to link files within the pubkey and, in that use case, the d-tag can then be a stringified file directory+name (my_site/src/index.css) .

Events can be accessed through /p/{npub_or_hex}/d/{event_id}.

should be something like /a/{hex}/d/{d_tag}. Don't do OR in a spec (unnecessary complications), and event_ID is not replaceable. It will change every new version. The d-tag is stable across versions.

Event id is obtained by following structure...

I couldn't find where this proposal is changing the hash of event.id.

kind: 10000 <= n < 20000

It doesn't make much sense because these types of events only offer 1 event per event type. There is no d-tag to differentiate them.

My implementation (AsaiToshiya/brostr) uses NIP-21 nostr:nevent1... for links.

This makes more sense than hexes because it includes information that helps the webserver find the event in multiple relays.

[studiokaiji]

My implementation (AsaiToshiya/brostr) uses NIP-21 nostr:nevent1... for links.

I think using NIP-21 is a great idea and should definitely be adopted.

The replaceable events idea only work for things the pubkey controls.

Implementing replaceable events without affecting other sites is required.

How about assigning public keys or similar to subdomains? While this will increase the implementation cost for web server administrators, who would need to specify subdomains using a wildcard, it can minimize the impact on other sites.

should be something like /a/{hex}/d/{d_tag}.

I don't think NIP-19 is a must, but from a usability perspective, it's worth considering thoroughly on the web server side. Therefore, I think it's okay to include it in the specifications. What do you think?

[mattn]

The root problem is how to replace HTML content posted at a specific URL with different content that keep the same URL. Thus, you can't use note1 for the URL at least, I think. Right?

[alexgleason]

Here's a cool thing. Deno is similar to Node, except you can import modules by URL. With this, it's possible to import JavaScript files hosted on Nostr directly into Deno code.

So, you could use this not just for hosting websites, but for hosting libraries.

Now, is it a good idea? Probably not, at least not yet. If web browsers (and Deno) added native support for Nostr URIs it would be more appealing. Even then I'm still not sure. It's experimental, and we're waiting for the gem to be uncovered.

[vitorpamplona]

The root problem is how to replace HTML content posted at a specific URL with different content that keep the same URL. Thus, you can use note1 for the URL at least, I think. Right?

You can use a note1 or nevent1 with a link to the replaceable event. It's the same if you use /a/{pub_key_hex}/d/{d_tag}. It's just a different format.

[mattn]

It's just a different format.

make sense.

[AsaiToshiya]

My implementation (AsaiToshiya/brostr) uses NIP-21 nostr:nevent1... for links.

<link rel="stylesheet" href="nostr:nevent1qqsqv6mu5zck0u9d44wx6cv6kythq5zz8cuhn6pm3haqdxvj2vaaeagvsfd05">

nostr:nevent1... also used to link to other events.

[frbitten]

I didn't read it very carefully, so there may have been something I missed.
But why use an event for each file type? Wouldn't it be better to have a single kind with a TAG specifying the data format? Well, that's how it works for the numerous formats that exist and are currently used.

[vitorpamplona]

Mostly because tags are case sensitive and we have had issues in other events (hashtags are a classic problem) when filtering for events of a certain type in all possible character cases. :(

[studiokaiji]

I agree that inserting the Content-Type into tags can make filtering more difficult, but such cases may be rare. Because most of the time, data is retrieved using the event id or d tag. I apologize if I've overlooked anything🙇‍♀.

[AsaiToshiya]

Something like a search engine using NIP-50 would be one use case.

[jiftechnify]

I'm with @frbitten on using tags to indicate file types instead of assigning kinds for each possible file type, especially if you have a vision for extending this proposal to allow hosting of arbitrary file types.

The inconvenience of filtering is not a strong reason to use separate kinds for different file types, since there is no need for the ability to filter files by content type for web hosting, I think.

We may make use of the m-tag defined in NIP-94 to indicate the content type of the content in MIME type format.

[AsaiToshiya]

I'm with @frbitten on using tags to indicate file types instead of assigning kinds for each possible file type, especially if you have a vision for extending this proposal to allow hosting of arbitrary file types.

I agree with the use of tags.

since there is no need for the ability to filter files by content type for web hosting

Search engine and other may want to search for only hosted HTML.

We may make use of the m-tag defined in NIP-94 to indicate the content type of the content in MIME type format.

I am concerned that MIME types are case sensitive.

[vitorpamplona]

I agree that inserting the Content-Type into tags can make filtering more difficult, but such cases may be rare. Because most of the time, data is retrieved using the event id or d tag. I apologize if I've overlooked anything🙇‍♀.

For a web browser application, sure.

But this isn't just for web browsers. I can do a js minifier Data Vending Machine and for that, I need to filter only JS and process them. Or maybe I do a JS to web assembly compiler. In both cases, one can injest the event type and create a new event (on a new type) with the resulting compilation.

Different kinds also help storage systems easily pick what they want to store, which kinds they allow on their relays.

An important distinction is between media and text Mine types. If we merge them as one type, relays will need to say things like "we support nip-xx/kind-Y, but only text ones in the x, y, z mime types". It becomes messy.

Also, there is no shortage of event kinds. We can create as many as we want.

[studiokaiji]

I see. Being able to specify a 'kind' to differentiate the files that the relay accepts seems good, considering the versatility of this system.

[SnowCait]

How about adding rule to m tag? It can be filtered by #m.
m tag MUST be lower case.

The allow-list is essentially the same between kind and m tag unless define separated NIPs as NIP-11 supported_nips.

[frbitten]

I agree that inserting the Content-Type into tags can make filtering more difficult, but such cases may be rare. Because most of the time, data is retrieved using the event id or d tag. I apologize if I've overlooked anything🙇‍♀.

For a web browser application, sure.

But this isn't just for web browsers. I can do a js minifier Data Vending Machine and for that, I need to filter only JS and process them. Or maybe I do a JS to web assembly compiler. In both cases, one can injest the event type and create a new event (on a new type) with the resulting compilation.

Different kinds also help storage systems easily pick what they want to store, which kinds they allow on their relays.

An important distinction is between media and text Mine types. If we merge them as one type, relays will need to say things like "we support nip-xx/kind-Y, but only text ones in the x, y, z mime types". It becomes messy.

Also, there is no shortage of event kinds. We can create as many as we want.

Ok. It's a valid reason. I don't know if it's enough, but I understand the option. So I suggest that NIP defines a range of allowed kinds and a list of each type and its respective kind. As we have NIPs that define kinds 10000, 20000, 30000, etc. So to avoid that in the future we have numbers of random kinds and having to make chains of IFs to support. It's a simple way to bypass relays that aren't interested in this feature.

It might be interesting to use the NIP-94 in this idea. So I can make a "site" that uses part hosted on the relay and part hosted in other ways via NIP-94 that contains the external URL.

[AsaiToshiya]

How about adding rule to m tag? It can be filtered by #m. m tag MUST be lower case.

The allow-list is essentially the same between kind and m tag unless define separated NIPs as NIP-11 supported_nips.

@vitorpamplona What do you think about this?

But this isn't just for web browsers. I can do a js minifier Data Vending Machine and for that, I need to filter only JS and process them. Or maybe I do a JS to web assembly compiler. In both cases, one can injest the event type and create a new event (on a new type) with the resulting compilation.

We can keep unchanged naddr1... by using m tag.

[AsaiToshiya]

I see. Being able to specify a 'kind' to differentiate the files that the relay accepts seems good, considering the versatility of this system.

That said, I think it is somewhat clear what MIME types this NIP supports.

[studiokaiji]

Here's a cool thing. Deno is similar to Node, except you can import modules by URL. With this, it's possible to import JavaScript files hosted on Nostr directly into Deno code.

As in this example, if there is a relay that only hosts JavaScript, and if filtering is possible with the "kind" parameter, there is no need for additional feature implementation in the relay. However, defining "kind" for each Content-Type is not practical, so some level of selection or an alternative solution is necessary, I believe.

[vitorpamplona]

We might not need a new kind for every mime/type. We just new kinds when there is a chance relays and clients can use that information to simplify their work. We can have these 3 types + the NIP-94/95/96/97 types (with the mime tag) for now. Maybe there are other mime types worth considering for their own kind, but we can leave that decision for later when the need arises.

[frbitten]

text files would not need to use NIP-95 or some variation. I think there could be a NIP to define a kind or tag to inform the format of what is in "content", it would be a generic solution for several uses.

[AsaiToshiya]

The problem with using m tag has been resolved.

I prefer m tag because it will be one kind for one purpose, but I'll go along with the consensus.

[AsaiToshiya]

Sorry, the link above to this is wrong.

[vitorpamplona]

If we have one or two serious implementers of this idea, we should move it to a Draft PR.

[studiokaiji]

Based on the discussions we've had here, I'd like to propose the following changes and simultaneously work on their implementation:

1. Switch to a URL format that does not use NIP-19 (npub).
2. Modify the <d> tags for internal links in replaceable events.
3. Utilize references using NIP-21 (nevent) (limited to non-replaceable ones).

Regarding the ongoing discussion about how to identify the MIME type of files, whether to use the m tag or differentiate by kind, both approaches have their merits. However, it seems that the specification of differentiating by kind has no significant drawbacks other than allowing multiple kind for a single purpose. Therefore, I'd like to keep it as it is for now.

Do you think it's okay to proceed with the above changes? If there are any further points to discuss or any aspects that we might have missed, I'd appreciate your input.

[vitorpamplona]

What's this URL format switch? I am not sure what it is about.

[studiokaiji]

should be something like /a/{hex}/d/{d_tag}. Don't do OR in a spec (unnecessary complications), and event_ID is not replaceable. It will change every new version. The d-tag is stable across versions.

Based on your suggestion not to include OR in the specification, I'm considering changing it from /p/{npub_or_hex} to /p/{author_hex} or /a/{author_hex}.

[vitorpamplona]

Nice! Keep in mind that for a, the "hex" should be kind:author-hex:dtag and not the event id.

[AsaiToshiya]

1. Switch to a URL format that does not use NIP-19 (npub).

Various implementations are possible, but must we follow the format of those URL?

3. Utilize references using NIP-21 (nevent) (limited to non-replaceable ones).

We can also use naddr for replaceable events.

Regarding the ongoing discussion about how to identify the MIME type of files, whether to use the m tag or differentiate by kind, both approaches have their merits. However, it seems that the specification of differentiating by kind has no significant drawbacks other than allowing multiple kind for a single purpose. Therefore, I'd like to keep it as it is for now.

OK, but what do you think about the second sentence in #742 (comment)?

[studiokaiji]

Various implementations are possible, but must we follow the format of those URL?

There's no need for that. This is done to remove unnecessary complexity. In fact, my implementation also allows routing on npub.

We can also use naddr for replaceable events.

Agree.

OK, but what do you think about the second sentence in #742 (comment)?

We can keep unchanged naddr1... by using m tag.

My understanding is insufficient, so is it possible for you to provide additional explanation? I'm sorry. My opinion is that it's better to use kind, which allows filtering by file type without making any changes to the relay.

[AsaiToshiya]

My understanding is insufficient, so is it possible for you to provide additional explanation? I'm sorry. My opinion is that it's better to use kind, which allows filtering by file type without making any changes to the relay.

If we use kind, kind may inevitably change according to the result of the DVM, then we can't use naddr. But I feel like I think too much.

I agree to use kind.

[sunetraalex]

Hi, I'd like to add one thing.

To be able to create complete installable PWA we also need to support JSON files for the web manifest alongside the other formats (HTML, CSS and JS).

From the NIP it seems to not be included, is it support somehow? Do we need to add it as a kind to the list?

[AsaiToshiya]

PR: #811

[alexgleason]

I posted my thoughts about Deno + this NIP here: denoland/deno#22779

It's been burning in my brain.

Do you think kind 5394 events should be shared between all variants of JavaScript, including ESM, CJS, IIFE, TypeScript source files, etc? Or should they be separate event kinds?

Even among what the browser will accept, there are various formats, and it's still changing. Browsers now accept ESM when they didn't before, and they may accept TypeScript tomorrow. So I am tempted to have them all use the same kind, since runtimes that support them will treat them the same.

[AsaiToshiya]

Our consensus for now is separate event kinds for each MIME type. So, I think at least TypeScript needs to be use another kind.