Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIP-068 Blinded Nostr Assets #250

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kinakuta-co
Copy link

@kinakuta-co kinakuta-co commented Feb 13, 2023

NIP-068 Blinded Nostr Assets

Centralized file storage (e.g. Amazon) is great for availability and speed but susceptible to censorship.

Decentralized file hosting (for images, etc.) is a difficult problem to solve.

Nostr thrives on simplicity, so I'd like to suggest a way of utilizing centralized file storage defensively so that hosts are less likely to censor or deplatform content.

An image (or any file) can be encrypted with a password (maybe AES-256-CBC), given a random filename, and uploaded as a BLOB to a centralized host. The note that references the image URI will also contain the information necessary to decrypt the image. When a client loads a note referencing an encrypted image, it will fetch the BLOB and decrypt it locally to display it.

Although this increases client load, it prevents the host from assessing the stored content without having knowledge of the particular note that holds the decryption information.

This NIP is for defining how a file is encrypted and stored and how it is referenced, retrieved and decrypted in the client so that all clients can consistently recognize and handle blinded assets.

The ubiquitous availability of cheap programmatic cloud hosting can be seen as a kind of decentralized mesh we can use to our advantage as long as the content hosted is uncensorable. If the host can't see the content they won't have a reason to censor it. Unless hosts begin to ban encrypted content, this seems fairly robust.

If this made Amazon upset and they began to siphon notes from relays to index their blinded assets, we have another tool we can use: zaps #224

Consequently, if a lightning paywall is put up to prevent an angry host from indexing encrypted content, this also serves as a great generalized way for users to monetize their content.

Simply: if enabled, you must zap a note to receive the decryption information to decrypt the media.

There are many ways this paywall could be implemented. I'm looking to the nostr community for ideas.

AES-256-CBC encryption references

RFC

@barkyq
Copy link
Contributor

barkyq commented Feb 14, 2023

Cool idea. The part of the zaps is nice, but probably could be a composable design (i.e., two nips: some notes require a zap to read the content, and this NIP defines a certain way to encode an image URL + extra decryption info into the content of a note).

I think there is an interesting discussion to be had about how to get "image upload" to work smoothly (and censorship resistantly) on nostr. Do you have any thoughts about this and how it could interact with your proposal?

@fiatjaf
Copy link
Member

fiatjaf commented Feb 14, 2023

I don't understand the paywall part, but the first part is quite ingenuous, I like it. That would also give plausible deniability to image hosts and they wouldn't have to care about what kind of content they are hosting.

@cameri
Copy link
Member

cameri commented Feb 15, 2023

I like this!

@mikedilger
Copy link
Contributor

I like the idea. Is this done as a tag with URL + AES key? Is the AES key in the tag for everybody with the nostr note?

I like AES for this since it is (AFAIK) low CPU/power on phones, compared to verifying Schnorr signatures (as I recently learned).

@staab
Copy link
Member

staab commented Feb 15, 2023

I like this a lot as well.

@kinakuta-co
Copy link
Author

I think there is an interesting discussion to be had about how to get "image upload" to work smoothly (and censorship resistantly) on nostr. Do you have any thoughts about this and how it could interact with your proposal?

Yes. I envision non-technical people deploying filedump servers quickly with prebuilt recipes and deploy buttons like this:

The recipe would create an endpoint that the owner could publish in a note to share with others.

The filedump endpoint would accept BLOB file uploads only. No cleartext data would be allowed to ensure maximum deniability. This means the file is encrypted before it reaches the filedump.

There could be a special note kind to publish a filedump endpoint. When a client sees a note of this kind, it would automatically add the filedump to its local list of filedumps.

When a user uploads a file through their client, the client chooses a filedump at random, encrypts the file, changes the filename to a random string with at least 128 bits of entropy, and POSTs the file to it. In the note referencing the file, the link could reside in the content and take a form such as:

file<initialization vector>ivk<key>https://<filedump>/<filename>

UUIDs may NOT be used for the filename as the form is too easy to recognize from the host's perspective. Beyond this, any sufficiently random filename should do, and more variety in the format of filenames will make filedumps harder to detect.

This NIP could recommend that clients encountering a file like this in note content can find the iv and key to decrypt the file in that form.

AES-CBC is recommended, but 128 bit keys could possibly be used instead of 256. Here is an example of this in code: https://www.npmjs.com/package/aes-js#cbc---cipher-block-chaining-recommended

To summarize:

  • anyone (and everyone) should run a filedump
  • the more filedumps that exist, the less censorable they are, and the cheaper each one is to run
  • filedumps only accept encrypted BLOBs with no header information

Other ideas:

Alternatively, the file string could be moved into a tag and a #[0] reference could be left in the note content. However, making the file indexable may have tradeoffs. An angry host looking to index content for censorship could benefit from the file being indexed in a tag. I'm not sure if this would make a measurable difference. I'm also not sure of the benefits of putting the file into a tag.

I was trying to brainstorm how a broken file link might be repaired within the nostr protocol, and how a file could be made redundant with voluntary subsequent filedump uploads that link together. It would be interesting if the filedump announcement kind also contained the vendor (Google, Amazon, Microsoft, etc.) and the client could pick 3 filedumps from different vendors to upload to at the same time, while somehow linking those references for durability. Further ideas would be welcome.

Potential problems:

  • people upload files that are too large
  • non-nostr usage/abuse
  • image reference durability

Many problems can be solved by paywalls, but filedumps are stronger against tyranny when they are plentiful and free. Filedumps can permeate every cloud host like an unstoppable virus of freedom spread by nostr.

Another interesting consideration: hosts make more money with more filedumps. Incentives. If they can't easily see the content it is less of a liability for them. Only in extreme cases would a host actually attempt to scry a filedump (like if a nation-state compelled them) but I think that we should always conceptually treat the host as hostile and plan for the worst.

Other NIPs needed:

  • kind for publishing a filedump, clients handling filedump notes

@kinakuta-co
Copy link
Author

Every client could have a "Create Filedump" recipe button and encourage every nostr user to make one. With thousands of filedumps to choose from, everyone's hosting costs would be miniscule. The odds of you uploading to the same filedump twice would be near zero. This might be a great way to bootstrap blinded nostr assets into the nostr ecosystem.

@kinakuta-co
Copy link
Author

I've written a condensed summary of this NIP here: note1reehmfcw40kd6hasp6d9zxdzewp93yx0fysl8khnu8gqrg2pqn4q4z4dqk

Feel free to discuss here or on nostr.

Summary below:


Instead of trying to solve the difficult problem of decentralized file storage for nostr images and assets, I propose a way of making cheap, plentiful centralized cloud storage more censorship resistant.

Introducing NIP-68: Nostr Blinded Assets - #250

Concept:

A "filedump" is a simple cloud server that accepts anonymous file uploads on the condition that the file is an encrypted BLOB.

When a filedump is created, a kind 1000 event should be published with a content containing the filedump address. When a client sees this note, it will automatically add the filedump to its internal list of filedumps.

When the user uploads a file in their client, here is what happens:

  • the client will choose a filedump at random
  • the client will encrypt the file with AES-CBC
  • the client will change the filename to any valid non-UUID name with at least 128 bits of entropy
  • the client will POST (upload) the file to the filedump
  • the client will put the decryption key and iv directly into the note in the following form:

fileivkhttps:///

When a client encounters a note referencing a file in this form:

  • GET the file
  • decrypt it
  • display the file with the note as normal

Key features:

  • Very little impact on UX. The average nostr user will not know the difference between a normal linked photo and a blinded asset except that blinded assets may take a few extra milliseconds to decrypt and display.
  • The file is encrypted on the server but can be seen (decrypted) by anyone who sees the note because the decryption info is stored publicly in the note's content.
  • The cloud provider will be unable to know what the file contains unless they have knowledge of the note that references it and bothers to decrypt it.
  • Users will have no relationship to a single filedump. They use a random filedump for every file they upload. This makes it very hard to deplatform a specific user's files.
  • The owner of the filedump has deniability for the content they are storing.
  • All nostr users will be able to spin up a filedump with no technical knowledge thanks to one-click deployment recipes supported by most major cloud service providers. Ideally there will be tens of thousands of filedumps to choose from but even several dozen would work well.
  • The more filedumps there are to choose from, the less content each will store and serve. This reduces the average operation costs for all filedump owners.

Click-to-deploy resources for different platforms:

Ideas to explore:

File durability and repair. NIP-68 could define a method of referencing multiple copies of a blinded asset across multiple filedumps such that the file can still be accessed even if the primary filedump is taken down by the host. Additional copies could be added to repair the asset's redundancy. Filedump announcements could contain the cloud provider (Microsoft, Google, Amazon, etc.) and clients could upload a single file to at least 3 different providers, making it very difficult to coordinate censorship.

Filedump protection. Anonymous uploads could be abused for non-nostr purposes. Require a pubkey, signature, POW, or zap to upload? If there are enough filedumps, each one could limit each user to only 1 lifetime upload and any given user would still likely never have trouble uploading.

Let me know your thoughts on this proposal.

🌴⚔️🌴

@v0l
Copy link
Member

v0l commented Feb 22, 2023

Client side encryption/decryption is supported in https://void.cat here are a few notes from running client side encryption for a few years:

  1. Browsers are bad at file encryption and decryption because the entire file is loaded into RAM, so sharing large files is a problem unless you do some fancy tricks
  2. Encrypted files are not immune to censorship, file encryption used to be mandatory on void.cat but that didn't stop me from receiving numerous DMCA takedown notices.

Managing encrypted files is hard and i dont think that encryption will really help with censorship resistance, if somebody wants to censor you they still can.

That being said there is some value in having encrypted files... privacy from the hoster. It may also be useful to send encrypted file links via DM, then you have end-to-end encrypted file sharing.

@telamon
Copy link

telamon commented Mar 26, 2023

If deepweb emergence within nostr is inevitable,
Why not use the brightnet/xor-pad approach and incentivize equal value and distribution of encrypted chunks?

(edit: This might be a horrible/nuclear idea, i honestly don't know and wouldn't be asking if i knew)

Copy link
Collaborator

@Semisol Semisol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a NIP, sadly. Could be an issue though, and I like the idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants