Add server-side functionality (e.g. short share links, collection of community-made shaders)

[etareduction]

What's maintainers opinion on features in title?

[Lutymane]

@etareduction ye, this sounds like a good idea! Although more thinking is needed.

1. Should there be a separate button to publish the shader?
2. Should we store file change history, when the author hits "publish" after it's been published. Or perhaps publish it as a new shader with the previous published revision being its parent.
3. I suppose there also should be some parent/children relation when the shader is edited. Also would be cool to have a tree which would span from the original shader to all its published descendants.

Feel free to share your thoughts

[etareduction]

I think there should be separate button to share which creates a short link to the current revision. About the gallery feature: it sounds fine to just create a separate entry for each publish.

We could have an edit option which would track the parent-child relations right away, but i think displaying this history and trees in the UI could get tricky, so I'd rather plan that out as a separate feature.

BTW, how do you host the project? I might want to contribute some backend implementation.

[Lutymane]

@etareduction I use Apache server, but it shouldn't be a problem to deploy anything else.

[Lutymane]

For subsequent publishes and parent/child relations we could actually store git diff changes instead of duplicating entire file

[etareduction]

For subsequent publishes and parent/child relations we could actually store git diff changes instead of duplicating entire file

But you would still have to request two rows from the DB and transfer both over the network to the client. I'm not sure if space savings on the DB side worth it. We might just as well gzip a file before storing it or something.

[etareduction]

I use Apache server, but it shouldn't be a problem to deploy anything else.

So is it running on your VPS/VDS? If so, i could make a docker image for deployment. What backend technologies are you comfortable maintaining? And would you like to do a monorepo here or create another repo for the backend?

Edit: Just saw you're doing PHP for the backend. I have no idea about it but i can do pretty much anything else like Go, Node, .NET or JVM.

[Lutymane]

@etareduction

So is it running on your VPS/VDS?

Yep. Although I think it's worth deploying it on a separate server, because mine is kinda overloaded already 😅 Docker image sounds very nice. I mainly used PHP, but other languages are fine, whatever is most modern and convenient.

The idea with compressing files is cool. We better use Brotli since it's supported everywhere.

I think we'd have some meta field for shaders, which would store parent ID and author, creation date, whatever else.

[etareduction]

That's what i think about initial data structure. Not sure if we'd want author entity to have any authentication-related fields. If edits would create revisions instead of editing in-place we could mitigate the necessity to sign-up which i think is convenient.

erDiagram
    shared_shader {
        cuid id
        text shader
    }

    author {
        cuid id
        text name
    }

    published_shader {
        cuid        id
        cuid        parent_id
        cuid        author_id
        text        shader
        timestamptz created_at
        int         stars
    }

    author           ||--|{ published_shader : author_id
    published_shader }o--|| published_shader : parent_id

Loading

[Lutymane]

@etareduction do you think it's worth having two different tables for shared and published? I'd suggest to have a single shaders table and include a flag published instead, so it'd also be possible to unpublish it, as well as publish already shared one. Also it needs a title field too.

Regarding authorization, I've been thinking about utilizing digital signatures (ed25519), so we'd store pub keys and signatures themselves. So there's no need to use any password/username stuff

[etareduction]

@Lutymane I think it might have worked in a desktop app environment. But on web - having to upload private/public keys for authorization sounds actually less convenient than password login or OAuth.

Ability to publish/unpublish shared shader would require authorization (to prove you are the one who shared it). And even if we're going to implement it, being able to share short link without login is still good. Like tailwind playground for example.

[Lutymane]

@etareduction My idea for the keys was to allow people to store only the key itself (as they'd do with password, we can even use password type field, so it's autosaved into passwords manager), plus on the initial publish, we could offer users to generate them a key, or they could use their own. So basically in author field there will be the public key. So it's really no different to username/password.

So there's no login at all, it all comes down to sending a signed shader, so it works for both sharing and publishing. And it's easier to maintain, because there are no sessions or whatever, we just check if the signature valid.

In case of simple sharing, we could allow not to sign the shader as well.

I hope I explained it well 😄

Also for sharing/publishing we could perhaps hash (with BLAKE3) the shader code and generate the link for that hash, so it wouldn't be possible to generate/publish the same shader multiple times

[etareduction]

And it's easier to maintain, because there are no sessions or whatever, we just check if the signature valid.

Without sessions user would have to enter his key each time he publishes, whereas with simple login and JWT's it's only required once. But i see how that's easier to develop, yeah.

Also for sharing/publishing we could perhaps hash (with BLAKE3) the shader code and generate the link for that hash, so it wouldn't be possible to generate/publish the same shader multiple times

Yeah, i agree, we would need some terms of deduplication like that.

[Lutymane]

@etareduction

enter his key each time he publishes

Not unless we just save it into local storage and provide the user with this option. Whether he wants to save it for further use, or enter each time. Plus add a function to "Settings" to delete the key from storage (and notify user about it if they decide to store it)

Another thought for deduplication is perhaps removing comments/whitespace before hashing, to further improve duplication handling?

[Lutymane]

Tbh, we could just make an input field in settings to place the key there, so it'd be used for publishing. Also it should be encrypted before writing to local storage with some local app key inlined in code

[etareduction]

Not unless we just save it into local storage and provide the user with this option. Whether he wants to save it for further use, or enter each time. Plus add a function to "Settings" to delete the key from storage (and notify user about it if they decide to store it)

As far as i know, any browser extension can read local storage. So we probably shouldn't store credentials in it. Only http only cookies are safe, and those could be only read on server.

Another thought for deduplication is perhaps removing comments/whitespace before hashing, to further improve duplication handling?

Not sure about that one. Maybe reformatting/adding explanations and republishing/sharing is a valid usecase?

Also it should be encrypted before writing to local storage with some local app key inlined in code

Extensions are able to read page's JS code too.

[Lutymane]

@etareduction

browser extension can read local storage

Indeed. But this vector of attack is pointless (who needs to develop a specific extension to steal a key for signing shaders? ._. ) Our app is not something of critical importance to care about such things? And I believe we don't even need the encryption, as I proposed before.

reformatting/adding explanations and republishing/sharing is a valid usecase

True

[Lutymane]

Although I found this cool approach with storing unexportable keys in IDB: https://github.com/fission-codes/keystore-idb
But it basically locks you to a specific machine with a specific key, so you can't use your own or reexport.

[etareduction]

@Lutymane

But this vector of attack is pointless (who needs to develop a specific extension to steal a key for signing shaders? ._. )

Yeah right. Still, I'd better go with something like Github OAuth as it is both secure and convenient. But we could try the keys approach.

[Lutymane]

@etareduction

Github OAuth

Hmm, actually, I agree using GH OAuth would be probably the best 😄 I forgot about spamming, so having a valid GH account would make it harder to do so. And it's also possible to ban users based on their account. Something we couldn't do with keys, since they'd just generate new ones.

[Lutymane]

@etareduction
Got an idea, how about using the same ID for shader revisions? So if a user publishes same shader multiple times, instead of assigning a unique ID every time, just keep the ID and append a revision number. Like https://glsl.app#s0m3iD#2, so ID is s0m3iD and revision is 2

[etareduction]

@etareduction Got an idea, how about using the same ID for shader revisions? So if a user publishes same shader multiple times, instead of assigning a unique ID every time, just keep the ID and append a revision number. Like https://glsl.app#s0m3iD#2, so ID is s0m3iD and revision is 2

What should happen when someone creates revision of a revision?

[etareduction]

@Lutymane

Btw, do you prefer consuming REST or GraphQL on a client? I personally like gql for it's tooling a little more, but it's probably overkill for this project.

[Lutymane]

@etareduction I never tried GraphQL, so let's stick to it, so I could finally learn it 😄

Regarding revisions, I think it should just be limited to a single author. So if a user reshares or republishes a shader, revision increases while keeping the same ID, but if another user uses that shared shader and shares/publishes, it gets a new ID already, since author is different. But it will still have its parent to be that previous shader of specific revision.

[Lutymane]

Also regarding the backend code, I can't decide what to choose, turn this into monorepo on a single branch, or make an orphan branch for the backend

[etareduction]

Also regarding the backend code, I can't decide what to choose, turn this into monorepo on a single branch, or make an orphan branch for the backend

Yeah i think we can start with monorepo and then split if it gets messy