Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server-provided client-side scripting #5393

Open
octacian opened this issue Mar 14, 2017 · 125 comments

Comments

@octacian
Copy link
Contributor

commented Mar 14, 2017

The title says it all. What are the issues with doing this? Would security be an issue?

I would see allowing server-side mods to inject temporary client-side mods as being very useful, e.g. allowing a protection mod to make things more smooth for players. If the protection mod client-side portion were to be manually installed by the user, they could modify it so that they could get around protection requiring the protection mod to have to keep server side validation. Instead, if the server could inject the client side mod at load, this wouldn't be an issue.

@nerzhul

This comment has been minimized.

Copy link
Member

commented Mar 14, 2017

We are thinking about it but it's a very very low priority, the more important is to have pure client features at this moments, mod store, etc

@nerzhul nerzhul referenced this issue Mar 14, 2017

Closed

[CSM] Second Roadmap #5394

15 of 27 tasks complete
@raymoo

This comment has been minimized.

Copy link
Contributor

commented Mar 14, 2017

The server should keep server-side validation in either case since an attacker could ignore the mod and send digs anyway if they make a modified version of Minetest. For protection, the client-side mod would be just for prediction.

@octacian

This comment has been minimized.

Copy link
Contributor Author

commented Mar 15, 2017

This is true. However, being able to inject the clientmod would still be very useful. One could also make it harder to avoid using a clientmod by forcing the client to provide a list of mods to the server. Though the client could be modified to provide a bad list, it makes things a bit harder.

@BluebirdGreycoat

This comment has been minimized.

Copy link
Contributor

commented Mar 16, 2017

I do not understand why this is considered low priority. Is this not the exact reason for having client-side mods in the first place? The ability to inject code into the client isn't even really a big thing -- I mean, websites do it to your browser all the time. If Minetest servers cannot do the same thing to Minetest clients despite supposedly being able to support it, then a huge opportunity for enhanced content delivery is lost. Minetest might as well not have client-side mods at all, if this important feature is left out.

Edit (need to clarify something -- and I've probably been too harsh):

The chief problem I see with supporting client-side mods, but not letting the server upload the mod code dynamically, is that this would require all players who want a given client-side mod to download it manually, and hope its communication channel/version is compatible with its counterpart mod that runs on the server. But in my experience, mod code frequently changes. I think it would be better if the client got a fresh version from the server every time, so that the user doesn't have to worry about versioning, compatibility, etc., and server operators who also manage all their own mod code can update their mods without having to ask their players to download a new version.

@nerzhul

This comment has been minimized.

Copy link
Member

commented Mar 16, 2017

@BluebirdGreycoat before making server sending code, you don't think we need API calls to run code ?
Also sending code should be very very very controled, and needs to change the current client to server connection to accept CSM and non CSM client, it's not easy. First have calls to call, second enhance them, permit to run code from client and at least allow server to send code, with very very careful protocol

@BluebirdGreycoat

This comment has been minimized.

Copy link
Contributor

commented Mar 16, 2017

@nerzhul I understand there are huge complexities (esp. security) that need to be overcome before this can even be possible. However, I have difficulty understanding (going just by the title of this issue) why this would be considered as anything other than the end goal of the effort toward client-side mods.

@celeron55

This comment has been minimized.

Copy link
Member

commented Apr 1, 2017

I think this is very important. It's in line with Minetest 0.4's original goal of server-defined games.

@rubenwardy

This comment has been minimized.

Copy link
Member

commented Apr 1, 2017

I agree that this is very important. For me, server mods being able to make GUIs better is the best thing about csm

Maybe we do need to add new client functions first, but it shouldn't be "very very low priority" just a more medium/long term goal

@nerzhul

This comment has been minimized.

Copy link
Member

commented Apr 1, 2017

for GUI better, personnaly please justify which feature you need exactly, because we already send formspec to client and it sends events to server, in 99% cases it's normal because you need to handle server side things

@raymoo

This comment has been minimized.

Copy link
Contributor

commented Apr 1, 2017

@nerzhul
With client-side Lua code, you could have a GUI system that allows for custom GUI elements with custom behavior. It will also decrease the response time of GUIs on a laggy connection.

An example for the first thing: I want to make a mod where players can design their own spells, which are defined as some kind of tree of spell shapes and spell effects. Players can fit these together graphically similar to programming in Scratch. For example, an "area of effect" spell shape node would have slots for spell effects.

An example for the second thing: Same example as the first, but assuming I don't have custom GUI elements so I use some system other than drag-and-drop. I don't want to have to wait for a roundtrip to the server between every click if there is 200ms+ latency. I can send the server the completed spell once the player decides to save it, because there's no need to validate the intermediate steps as long as the finished product, which is what will actually affect the world, is validated.

In the same line of examples, it would also allow the player to save spells for use on any server with the mod. The server would just have to validate the spells when imported from the client.

@Ferk

This comment has been minimized.

Copy link
Contributor

commented Apr 17, 2017

for GUI better, personnaly please justify which feature you need exactly, because we already send formspec to client and it sends events to server

Creative GUI, for once, is slow on heavy servers.
Client-side scripting would make viable new features in the formspecs like autocompletion in fields, and updating the formspec on the fly as you type (to search for nodes) without having to communicate heavily with the server.

And I'm sure creative wouldn't be the only server mod that could benefit from injecting client-side GUI.
There's even a mod that adds a tetris arcade game made with formspecs.

That's not counting things like preventing showing the "undo lag" from protection mods (yes, you still need the server check, but this would reduce traffic and be much more usable).
Or having an ambient sounds / weather effects server mod that actually uses sounds provided by the server and applies in specific circumstances for that server but whose logic for checking if the circumstances apply and running the effects is executed client-side.

Honestly, most of the good uses of client-side scripting involve server mod dependencies.
IMHO, having "clientmods/" has much less uses (and more mis-uses by cheaters) than having client scripts provided by server mods. Personally I would have rather only done the latter.

@sofar sofar removed the Low priority label Apr 18, 2017

@sofar

This comment has been minimized.

Copy link
Member

commented Apr 18, 2017

I'm removing the "low priority" label because as people listed, it's not low priority.

However, just because this is not low priority, it doesn't mean that other things don't need to go in first.

@rubenwardy

This comment has been minimized.

Copy link
Member

commented Apr 29, 2017

@jastevenson303

This comment has been minimized.

Copy link
Contributor

commented Oct 4, 2017

Phew! Sorry about the dupe. Thanks for this issue, can't wait until this is possible. Here's what I wrote:

Right now we have Server-Side Mods (SSM), mod channels, and Client-Side Mods (CSM). However, in order for this to be entirely practical for the server's needs, it must know that the client has connected with the appropriate CSM loaded.

I suggest a setting, csm_autodownloads, default false, that when enabled will allow the client to download the necessary CSM from the server. The server will then know it can use mod channels to interact between SSM and CSM.

<+sfan5> csm is sandboxed, so servers sending lua code should not be a problem
<+Calinou> jas_: client-side mods can't be sent from server to client, that would be too insecure
@Ferk

This comment has been minimized.

Copy link
Contributor

commented Oct 4, 2017

<+sfan5> csm is sandboxed, so servers sending lua code should not be a problem
<+Calinou> jas_: client-side mods can't be sent from server to client, that would be too insecure

Aren't these two comments contradicting each other?
Is there a log of the full conversation? was there any agreement?

@paramat

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

http://irc.minetest.net/minetest-hub/2017-10-04#i_5097521
Calinou is wrong, we intend to add server-sent CSM once it can be made secure enough.
Server-sent CSM was the original plan documented in the dev wiki and is the more useful aspect of CSM.

@Ezhh

This comment has been minimized.

Copy link
Member

commented Jul 29, 2018

what is the wrong approach ? having code running on client ?

I wrote what "the wrong approach" was in past tense and spelled it out as "the complete inability to control or restrict" CSM. I even acknowledged that restrictions have been added. It's simply that it leaves many of us with something we still don't want and are still concerned by. If it's not finished (SSCSM) why should it stay?

You've been asked when you will work on this, and instead of answering you are just blaming "the community". This attitude doesn't help anyone.

In world of warcraft ...

MT is not World of Warcraft.

The argument (you seem to be making??) is also ridiculous. If you imagine a situation where people will use CSM mods because without them they can't "perform" to the same level as others, that is really just another way in which client provided CSM is bad.

... fair play ...

I don't know why you keep mentioning this. Server owners get to decide what it fair on their servers. Not you. Not players.

and sorry it's not the SSCSM approach on 95% of the servers.

What does this mean? 100% of servers don't have SSCSM because it hasn't happened.

@rubenwardy

This comment has been minimized.

Copy link
Member

commented Jul 29, 2018

No one wanted or asked for client side modding, and it wasn't on the roadmap. What people asked for, and what was on the roadmap, was client side scripting - an amazing new expansion which would allow games and mods to take advantage of graphics and low latency controls, to do stuff that wasn't possible before

@nerzhul

This comment has been minimized.

Copy link
Member

commented Jul 29, 2018

client side scripting & modding are the same thing. Did you expect to have script without mods ?
Also having graphics in lua modding with the current design is just the worst design we can do, except if we want to have low performing graphics.

@rubenwardy

This comment has been minimized.

Copy link
Member

commented Jul 29, 2018

client side scripting & modding are the same thing. Did you expect to have script without mods ?

Yes. You can send the scripts from the server and store in a virtual file system and load directly. No need to search for mods locally. As in the original proposal

Also having graphics in lua modding with the current design is just the worst design we can do, except if we want to have low performing graphics.

That's not what I meant

@paramat

This comment has been minimized.

Copy link
Member

commented Sep 26, 2018

red-001 seems to be uninterested in completing SSCSM. It would be good if nerzhul had some help and feedback, but core dev time is very limited at the moment and we're struggling with the basic essentials, obviously we have to prioritise. No core dev has stated an intention to work on SSCSM, so it does seem unlikely the SSCSM work will be completed in the short/medium term. We can't have CPCSM being a feature for years, and becoming a standalone feature by accident, that wasn't even intended by the CSM devs.

This permit to have a default restricted CSM which is a good preview for server owners and players, permitting to have some interesting mod appearing using it, before sending anything from server.

That's unnecessary and there's no benefit. If CPCSM is needed as a stepping stone in CSM development, which in a way it is i agree, that should be worked on by the CSM devs outside of a stable release. Features are not released until they are finished.
This has been a mistake as most know, however of course we are all to blame for not keeping an eye on the development and objecting earlier, so i can understand this has been hard on the CSM devs, i feel guilty about this.

@paramat

This comment has been minimized.

Copy link
Member

commented Sep 26, 2018

I can't see any misunderstanding, it's all been discussed in depth so how things are is very obvious.
If someone has the wrong idea we'll correct them.
People know we have restrictions in place and the defaults are set for maximum protection. The issues people now have with CSM are unrelated to that.

then there is no sense to totally disable CSM here. If a anoying user want to be really anoying he just has to recompile MT disabling the restrictions and that's it.

Yes, but this argument can also be used to state that the 'flavor' restrictions are powerless, and that CSM has made altering your client in order to cheat much easier to do (which it has).

My argument for disabling CPCSM for 5.0.0 is based on other things anyway, not the hope it would prevent cheating, which i agree it won't.

@jastevenson303

This comment has been minimized.

Copy link
Contributor

commented Nov 26, 2018

nerzhul commented on Mar 14, 2017
We are thinking about it but it's a very very low priority, the more important is to have pure client features at this moments, mod store, etc

There very first reply. Wow. It's all in perspective now, they had to make the CSM first! You cannot have server-sent anything, if it's not there, I guess. But after some of it was established, this whole server-sent idea went by the wayside. When it was brought up again, it was met with resistance by those who'd worked so hard to make CSM in the first place, and others who'd come to suspect CSM of being a security risk.

I think this is low priority, after all, and can be done and made and supported by any non-CSM author.

@ClobberXD

This comment has been minimized.

Copy link
Contributor

commented Nov 26, 2018

I'd like to try my hand at this. If I'm not capable of finishing this... well, I could at least learn a thing or two. But where do I start? What is actually needed here? A list of things to-do (in depth) would be very helpful for potential contributors. Thanks.

@paramat

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

The discussion on IRC -hub channel on this day http://irc.minetest.net/minetest-hub/2018-12-09 makes nerzhul's attitudes to CSM very clear.
For me personally, these attitudes are even more concerning than i had previously suspected, and somewhat explain why CSM development happened the way it did. The rest of us core devs are also to blame for allowing this to happen.

@paramat

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

jastevenson303

There very first reply. Wow. It's all in perspective now, they had to make the CSM first! You cannot have server-sent anything, if it's not there, I guess.

Well, see the replies immediately after which made it clear, by celeron55 too, that server-provided is indeed high priority. Nerzhul's attitude was (and still is) very bad: "thinking about it", "very very low priority", no wonder we never got server-sent CSM. He even added the 'low priority' label to the issue.

@paramat

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

Anyway, the news is that nerzhul has stated he will work on server-provided CSM after release of MT 5.0 So we have finally got the commitment we needed.

If no-one had committed before MT 5.0 release i was considering suggesting that all CSM be removed before MT 5.0 release. Because: If we had decided to remove CSM after release anyone could just use an older 5.x client and still use CSM. The network compatibility breakage of MT 5.0 would have been an excellent way to prevent clients using CSM.

@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

@celeron55 question is a very good question and i want to enhance it to ensure the debate is correct.

What is the problematic feature for server owners which cause problems for them in the CSM ? It seems it's the only issue which triggers all this debate. If we need control on some part, target it.

@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

and no the network compat breakage cannot prevent CSM, you are totally wrong, because CSM is purely client side (except mod channels) then you can prevent servers from detecting client, like any C++ rogue code.

@jastevenson303

This comment has been minimized.

Copy link
Contributor

commented Dec 11, 2018

@ClobberXD

This comment has been minimized.

Copy link
Contributor

commented Dec 11, 2018

Yes, I was just asking for a clearer roadmap with respect to SSCSM. One that makes potential contributors want to work on it. :)

@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

the problem with MT roadmap is because it's a free software with various contributors, we can push a roadmap for a dev cycle but generally it's not done like this, there are variations.
When 5.0.0 is in feature freeze i will ask for core devs and community what are the needs (except SSCSM we already know that), and after we will see in the coredev team what should happen.
I also think we should do this with MTG
Last point, i think we really should reduce release interval, a release each 3 month should be sufficient, and we shoul really continue to refactor & increase the unittests code coverage to ensure code has sufficient quality and reduce regressions.

@jastevenson303

This comment has been minimized.

Copy link
Contributor

commented Dec 11, 2018

I will tell you, @ClobberXD, first I'm sorry I misspelled your name! :)

Second, I think if you can somehow make it so the server can send a file to the client, we'll have made much progress. Optionally, the client should be prompted that a file has arrived, and be given the choice of rejecting it, viewing it, or installing it.

Beyond that, I do not know.

@ClobberXD

This comment has been minimized.

Copy link
Contributor

commented Dec 11, 2018

first I'm sorry I misspelled your name! :)

Ha, that's alright :)

I discussed the file-sending part with nerzhul a couple of days ago. See http://irc.minetest.net/minetest-hub/2018-12-09#i_5453620. Pasting the relevant chat here, just for the record:

 ANAND  | Would it be possible to package code as serialized text, along with metadata
          that the client can use to re-create the mod folder?
nerzhul | it's the idea but it's not as easy

I'll just attempt the mod-sending first. Security is the tougher part.

@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

@jastevenson303 imagine you ask for 1000 files yeah
in SSCSM context it's not correct. And also i really think we need both CPCSM and SSCSM, maybe having 2 lua states client side will be the option to ensure we have proper separation of each model

@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 11, 2018

@ClobberXD wait for 5.0.0 to be released and we can discuss that feature together if you want to code it, and define a proper exchange format, with the correct unittests

@ClobberXD

This comment has been minimized.

Copy link
Contributor

commented Dec 11, 2018

Been waiting for a long time, but I understand. I can continue waiting, no problem... :)

@paramat

This comment has been minimized.

Copy link
Member

commented Dec 18, 2018

nerzhul wrote:

and no the network compat breakage cannot prevent CSM, you are totally wrong, because CSM is purely client side (except mod channels) then you can prevent servers from detecting client, like any C++ rogue code.

Ok, but a 0.4.x client won't work on a 5.0.0 server, so only a 5.0.0 client can be used, which would not have the CSM code.
Obviously a 5.0.0 client could be hacked to have CSM code (so i know absolute prevention is impossible) but that would not be easy to do.

I'm comparing the complete removal of CSM before or after 5.0.0 release, if it was removed afterwards in 5.1.0 then a 5.0.0 client could easily be used by anyone, without hacking, to continue using CSM.
So, the network breakage would make using CSM much more difficult.

This is rather theoretical now anyway because it looks more likely CSM will be completed. But a while back it looked possible all CSM was going to be removed. If that was going to happen it would be best to do it before 5.0.0 not after.

@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 18, 2018

what scenario i want:

  • audit the API to ensure problematic calls for server owners are under restrictions (it's server owner role to list which API can be problematic for them, most of them has already been identified)
  • ensure we have a restriction to disable the client lua stack entierely from server side if needed (you mentioned it but i can't remember if it's possible or need some different handling). This is not bypassable except if you recompile a rogue client (as difficult to bypass as if you hardcode the CSM disable but you keep the possibility to make player use it)
  • have the good CSM restriction flag by default (all restrictions except disable CSM entierelement sounds reasonable and permit server owners to remove it if they want, but permit some others to use it and start to think about CSM mods they can use in 5.1 and push to clients)
  • release 5.0.0 with new restrictions (if new restrictions found)
  • do the 5.1.0 (short dev cycle, when SSCSM is ready and no major waited feature, feature freeze + release)
@nerzhul

This comment has been minimized.

Copy link
Member

commented Dec 19, 2018

I started #8002 technical discussion about this need for the next roadmap.

@nerzhul nerzhul modified the milestones: SSCSM, 5.1.0 Dec 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.