New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script Definition Template #28

Closed
ligee opened this Issue Jun 21, 2016 · 34 comments

Comments

@ligee
Collaborator

ligee commented Jun 21, 2016

initial proposal for Script Definition Template

*Open issues *

  • Controlling access to host app's APIs
@bsideup

This comment has been minimized.

Show comment
Hide comment
@bsideup

bsideup Jun 21, 2016

Alternatively, Kotlin might use @file:org.jetbrains.kotlin.gradle.GradleScript() notation.

Pros:

  • Less typing and visual noise
  • Parameterization with: @file:foo.bar.MyScript(version=1.1)

Cons:

  • Instead of one, generic annotation, we have multiple of them - harder to find one in the compiler
  • "Can I define multiple script annotations on the same file?"-like questions

bsideup commented Jun 21, 2016

Alternatively, Kotlin might use @file:org.jetbrains.kotlin.gradle.GradleScript() notation.

Pros:

  • Less typing and visual noise
  • Parameterization with: @file:foo.bar.MyScript(version=1.1)

Cons:

  • Instead of one, generic annotation, we have multiple of them - harder to find one in the compiler
  • "Can I define multiple script annotations on the same file?"-like questions
@bsideup

This comment has been minimized.

Show comment
Hide comment
@bsideup

bsideup Jun 21, 2016

Just to note - there should be a way to mark script as Serializable (Spark-like use cases)

bsideup commented Jun 21, 2016

Just to note - there should be a way to mark script as Serializable (Spark-like use cases)

@bsideup

This comment has been minimized.

Show comment
Hide comment
@bsideup

bsideup Jun 21, 2016

Dependency resolver should be able to resolve other scripts as dependencies:

@file:include("./shared/code.kts")

bsideup commented Jun 21, 2016

Dependency resolver should be able to resolve other scripts as dependencies:

@file:include("./shared/code.kts")
@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Jun 21, 2016

Contributor

Alternatively, Kotlin might use @file:org.jetbrains.kotlin.gradle.GradleScript() notation.

@bsideup this is only possible if GradleScript is an annotation class, and that prohibits many things, e.g. extending other classes.

Contributor

abreslav commented Jun 21, 2016

Alternatively, Kotlin might use @file:org.jetbrains.kotlin.gradle.GradleScript() notation.

@bsideup this is only possible if GradleScript is an annotation class, and that prohibits many things, e.g. extending other classes.

@orangy

This comment has been minimized.

Show comment
Hide comment
@orangy

orangy Jun 21, 2016

file:include is good catch, but shouldn't be called include I think, because it's not actually textual inclusion.

orangy commented Jun 21, 2016

file:include is good catch, but shouldn't be called include I think, because it's not actually textual inclusion.

@bsideup

This comment has been minimized.

Show comment
Hide comment
@bsideup

bsideup Jun 21, 2016

Naming is not my strongest skill :)

bsideup commented Jun 21, 2016

Naming is not my strongest skill :)

@ligee

This comment has been minimized.

Show comment
Hide comment
@ligee

ligee Jun 29, 2016

Collaborator

@bsideup: about serialization: do you see any problem in simply marking script template class as Serializable? It will be actually the base class for the generated script class.

Collaborator

ligee commented Jun 29, 2016

@bsideup: about serialization: do you see any problem in simply marking script template class as Serializable? It will be actually the base class for the generated script class.

udalov referenced this issue Jun 30, 2016

Update script definition template document:
- place AcceptedAnnotation annotation on the resolve method
- replace scriptFile and annotations params of resolve with ScriptContent interface, that also allow content-based access to the file,
   remove obsolete relevant item from Further Ideas section
 - add javaHome and scripts params to Dependencies interface
@elifarley

This comment has been minimized.

Show comment
Hide comment
@elifarley

elifarley Jul 18, 2016

file:include is good catch, but shouldn't be called include I think, because it's not actually textual inclusion.

Maybe
@file:import("./shared/code.kts")
(or importscript)

elifarley commented Jul 18, 2016

file:include is good catch, but shouldn't be called include I think, because it's not actually textual inclusion.

Maybe
@file:import("./shared/code.kts")
(or importscript)

@Jire

This comment has been minimized.

Show comment
Hide comment
@Jire

Jire Aug 16, 2016

What about support for calling scripts and their functions from regular Kotlin code? And the other way around; what about scripts using regular Kotlin code?

Jire commented Aug 16, 2016

What about support for calling scripts and their functions from regular Kotlin code? And the other way around; what about scripts using regular Kotlin code?

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Aug 16, 2016

Contributor

@Jire

What about support for calling scripts and their functions from regular Kotlin code?

Script is compiled to a class that can be used by other code.

And the other way around; what about scripts using regular Kotlin code?

Of course, scripts can use other code

Contributor

abreslav commented Aug 16, 2016

@Jire

What about support for calling scripts and their functions from regular Kotlin code?

Script is compiled to a class that can be used by other code.

And the other way around; what about scripts using regular Kotlin code?

Of course, scripts can use other code

@Jire

This comment has been minimized.

Show comment
Hide comment
@Jire

Jire Aug 16, 2016

@abreslav Wow perfect!! Very excited to see this in action.

Jire commented Aug 16, 2016

@abreslav Wow perfect!! Very excited to see this in action.

@jonninja

This comment has been minimized.

Show comment
Hide comment
@jonninja

jonninja Aug 18, 2016

If scripts can call other code (a good thing), will there be a way to limit which packages can be imported? This would allow us to prevent authors from writing code that we'd prefer to limit.

jonninja commented Aug 18, 2016

If scripts can call other code (a good thing), will there be a way to limit which packages can be imported? This would allow us to prevent authors from writing code that we'd prefer to limit.

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Aug 18, 2016

Contributor

You have internal for that.

Contributor

cypressious commented Aug 18, 2016

You have internal for that.

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Aug 18, 2016

Contributor

If scripts can call other code (a good thing), will there be a way to limit which packages can be imported? This would allow us to prevent authors from writing code that we'd prefer to limit.

I don't see why this needs to be limited to scripts only. Seconding @cypressious on this: visibility modifiers can be used.

Contributor

abreslav commented Aug 18, 2016

If scripts can call other code (a good thing), will there be a way to limit which packages can be imported? This would allow us to prevent authors from writing code that we'd prefer to limit.

I don't see why this needs to be limited to scripts only. Seconding @cypressious on this: visibility modifiers can be used.

@Cypher121

This comment has been minimized.

Show comment
Hide comment
@Cypher121

Cypher121 Aug 19, 2016

I fail to see how internal and other modifiers cover that question. It's more about restricting what functionality can script invoke: for example, if I'm using scripts for some sort of plugin system and don't want to run them as full-privilege code, there's no way to restrict them from accessing system, fs, net or other sensitive classes by using visibility, unless it's somehow possible to decrease it for that specific script without altering the original code.

Although there may be one with security managers, but that's still worth looking into.

Cypher121 commented Aug 19, 2016

I fail to see how internal and other modifiers cover that question. It's more about restricting what functionality can script invoke: for example, if I'm using scripts for some sort of plugin system and don't want to run them as full-privilege code, there's no way to restrict them from accessing system, fs, net or other sensitive classes by using visibility, unless it's somehow possible to decrease it for that specific script without altering the original code.

Although there may be one with security managers, but that's still worth looking into.

@ryleykimmel

This comment has been minimized.

Show comment
Hide comment
@ryleykimmel

ryleykimmel Aug 20, 2016

Will Kotlin scripts have a type or other identifier (other than its file extension ".kts") that will distinguish it from Kotlin classes?

ryleykimmel commented Aug 20, 2016

Will Kotlin scripts have a type or other identifier (other than its file extension ".kts") that will distinguish it from Kotlin classes?

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Aug 20, 2016

Contributor

@ryleykimmel Yes, there will be a way to see if a class or instance represents a script.

Contributor

abreslav commented Aug 20, 2016

@ryleykimmel Yes, there will be a way to see if a class or instance represents a script.

@ryleykimmel

This comment has been minimized.

Show comment
Hide comment
@ryleykimmel

ryleykimmel Aug 20, 2016

@abreslav Is it present in the 1.1 preview?

ryleykimmel commented Aug 20, 2016

@abreslav Is it present in the 1.1 preview?

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Aug 22, 2016

Contributor

Can't check the code quickly ATM. CC @ligee

Contributor

abreslav commented Aug 22, 2016

Can't check the code quickly ATM. CC @ligee

@jonninja

This comment has been minimized.

Show comment
Hide comment
@jonninja

jonninja Aug 25, 2016

@abreslav It is different from internal. As @Cypher121 describes, this is about limiting what a script writer has access to. We have a tool that we provide that is currently configured with XML. The XML is a mess, and it would be really beautiful to allow the configuration as a kotlin script. However, I don't want configuration writers to do all kinds of things... I want to limit the library functionality so they don't do dangerous things.

I guess a most basic implementation would be an attribute of ScriptDefinitionTemplate that defined the allowed packages as a regex or other patterns. Another option would be to have an attribute that, if true, didn't allow for importing... that is, only 'ScriptImplicitImports' would be allowed. This might solve the problem just as well.

jonninja commented Aug 25, 2016

@abreslav It is different from internal. As @Cypher121 describes, this is about limiting what a script writer has access to. We have a tool that we provide that is currently configured with XML. The XML is a mess, and it would be really beautiful to allow the configuration as a kotlin script. However, I don't want configuration writers to do all kinds of things... I want to limit the library functionality so they don't do dangerous things.

I guess a most basic implementation would be an attribute of ScriptDefinitionTemplate that defined the allowed packages as a regex or other patterns. Another option would be to have an attribute that, if true, didn't allow for importing... that is, only 'ScriptImplicitImports' would be allowed. This might solve the problem just as well.

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Aug 30, 2016

Contributor

this is about limiting what a script writer has access to

@jonninja The important part of your request is that compile-time safety will likely not be good enough for it: if the script puts a bunch of bytes together and loads them as a class that actually accesses bad APIs, no static checks in Kotlin will mitigate that. So, the runtime-support is required, and there we have mostly ClassLoaders and SecurityManager.

Whether we should do something in the script templates to facilitate such configuration, or just rely on what JRE and JSR 223 provide, is a good question. // CC @ligee

Contributor

abreslav commented Aug 30, 2016

this is about limiting what a script writer has access to

@jonninja The important part of your request is that compile-time safety will likely not be good enough for it: if the script puts a bunch of bytes together and loads them as a class that actually accesses bad APIs, no static checks in Kotlin will mitigate that. So, the runtime-support is required, and there we have mostly ClassLoaders and SecurityManager.

Whether we should do something in the script templates to facilitate such configuration, or just rely on what JRE and JSR 223 provide, is a good question. // CC @ligee

@jonninja

This comment has been minimized.

Show comment
Hide comment
@jonninja

jonninja Aug 31, 2016

That makes sense. Certainly from the use cases I'm considering, this isn't much of an issue, as the access limitations are used to guide script writers. I just want a way to make clear what is supported functionality.

jonninja commented Aug 31, 2016

That makes sense. Certainly from the use cases I'm considering, this isn't much of an issue, as the access limitations are used to guide script writers. I just want a way to make clear what is supported functionality.

@ligee

This comment has been minimized.

Show comment
Hide comment
@ligee

ligee Aug 31, 2016

Collaborator

@ryleykimmel the proper reflection for script classes is not yet part of 1.1, and even not fully designed yet. See https://youtrack.jetbrains.com/issue/KT-13382

Collaborator

ligee commented Aug 31, 2016

@ryleykimmel the proper reflection for script classes is not yet part of 1.1, and even not fully designed yet. See https://youtrack.jetbrains.com/issue/KT-13382

@ligee

This comment has been minimized.

Show comment
Hide comment
@ligee

ligee Sep 1, 2016

Collaborator

@jonninja you may (to the certain extent) control the runtime security using SecurityManager, since script execution is in your hands. You may also control script's classpath with DependenciesResolver.
Deeper restrictions will need to be thought through carefully, because even limiting imports is sort of language level change. And then logically the next requirement would be to define which language constructs are allowed in a particular script, like with groovy's SecureASTCustomizer. Which looks scary for me. :)
But we'll keep that in mind, thank you for the suggestion.

Collaborator

ligee commented Sep 1, 2016

@jonninja you may (to the certain extent) control the runtime security using SecurityManager, since script execution is in your hands. You may also control script's classpath with DependenciesResolver.
Deeper restrictions will need to be thought through carefully, because even limiting imports is sort of language level change. And then logically the next requirement would be to define which language constructs are allowed in a particular script, like with groovy's SecureASTCustomizer. Which looks scary for me. :)
But we'll keep that in mind, thank you for the suggestion.

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Sep 5, 2016

Contributor

@ligee I think SecureASTCustomizer is far from what we are looking for here. Controlling access to APIs is very relevant when Kotlin is used for customization/scripting inside a bigger app

Contributor

abreslav commented Sep 5, 2016

@ligee I think SecureASTCustomizer is far from what we are looking for here. Controlling access to APIs is very relevant when Kotlin is used for customization/scripting inside a bigger app

@DaanDeMeyer

This comment has been minimized.

Show comment
Hide comment
@DaanDeMeyer

DaanDeMeyer Sep 9, 2016

Could the @file:ScriptTemplate annotation be extended to allow pointing to an url from which an IDE could download the script template?

Along with support in Intellij for editing single script files without creating a project, this would provide users with an intuitive way of using Intellij as a script editor without having to download separate script definitions or deal with lots of project files.

DaanDeMeyer commented Sep 9, 2016

Could the @file:ScriptTemplate annotation be extended to allow pointing to an url from which an IDE could download the script template?

Along with support in Intellij for editing single script files without creating a project, this would provide users with an intuitive way of using Intellij as a script editor without having to download separate script definitions or deal with lots of project files.

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Sep 13, 2016

Contributor

@DaanDeMeyer Interesting use case. @ligee What do you think?

Contributor

abreslav commented Sep 13, 2016

@DaanDeMeyer Interesting use case. @ligee What do you think?

@Cypher121

This comment has been minimized.

Show comment
Hide comment
@Cypher121

Cypher121 Sep 13, 2016

Since template is just a kt/class file, wouldn't pointing to remote URL be a security problem?

Cypher121 commented Sep 13, 2016

Since template is just a kt/class file, wouldn't pointing to remote URL be a security problem?

@ilya-g

This comment has been minimized.

Show comment
Hide comment
@ilya-g

ilya-g Sep 26, 2016

Member

Is this proposal somehow related to the recently introduced kotlin.script.StandardScriptTemplate in kotlin-runtime?

Member

ilya-g commented Sep 26, 2016

Is this proposal somehow related to the recently introduced kotlin.script.StandardScriptTemplate in kotlin-runtime?

@abreslav abreslav added this to the 1.1 milestone Oct 10, 2016

@voddan

This comment has been minimized.

Show comment
Hide comment
@voddan

voddan Nov 30, 2016

Contributor

Do other languages have such a mechanism? What would be the closes analogs?

Contributor

voddan commented Nov 30, 2016

Do other languages have such a mechanism? What would be the closes analogs?

@raniejade

This comment has been minimized.

Show comment
Hide comment
@raniejade

raniejade Dec 3, 2016

Are there any plans for scripts to have an output? (return something)

raniejade commented Dec 3, 2016

Are there any plans for scripts to have an output? (return something)

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Dec 3, 2016

Contributor
Contributor

cypressious commented Dec 3, 2016

@mkobit

This comment has been minimized.

Show comment
Hide comment
@mkobit

mkobit Dec 14, 2016

Could the @file:ScriptTemplate annotation be extended to allow pointing to an url from which an IDE could download the script template?

I really like the idea of this. It seems like it would be useful to be able to have some notion of script template resolution without having to download additional plugins, pull down and activate something (like how GDSL), or having to configure IntelliJ to properly understand those files.

The way I generally (and others at my work) use IntelliJ is for importing Gradle projects and letting IntelliJ figure out all the modules and do all the dependency resolution magic. We use Jenkins 2.0 and the Jenkins pipeline functionality which requires a Jenkinsfile Groovy script in the base of the process. It is difficult for others to develop against the Jenkinsfile because of the complexities and not having code documentation. Jenkins vends a GDSL but the workflow is confusing and too many steps for a lot of the developers at my work.

The idea that a script template can simply be pointed to a url and get a lot of the development features seems like it would be useful.

mkobit commented Dec 14, 2016

Could the @file:ScriptTemplate annotation be extended to allow pointing to an url from which an IDE could download the script template?

I really like the idea of this. It seems like it would be useful to be able to have some notion of script template resolution without having to download additional plugins, pull down and activate something (like how GDSL), or having to configure IntelliJ to properly understand those files.

The way I generally (and others at my work) use IntelliJ is for importing Gradle projects and letting IntelliJ figure out all the modules and do all the dependency resolution magic. We use Jenkins 2.0 and the Jenkins pipeline functionality which requires a Jenkinsfile Groovy script in the base of the process. It is difficult for others to develop against the Jenkinsfile because of the complexities and not having code documentation. Jenkins vends a GDSL but the workflow is confusing and too many steps for a lot of the developers at my work.

The idea that a script template can simply be pointed to a url and get a lot of the development features seems like it would be useful.

@abreslav abreslav moved this from KEEP Pending to Implemented in KEEP Kotlin 1.1 Jan 27, 2017

@dzharkov

This comment has been minimized.

Show comment
Hide comment
@dzharkov

dzharkov Feb 14, 2018

Collaborator

Closed in favor of #75

Collaborator

dzharkov commented Feb 14, 2018

Closed in favor of #75

@dzharkov dzharkov closed this Feb 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment