Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

suggest: `valueof` operator to eval expression at compile time #804

Open
xp44mm opened this issue Nov 6, 2019 · 69 comments
Open

suggest: `valueof` operator to eval expression at compile time #804

xp44mm opened this issue Nov 6, 2019 · 69 comments
Labels

Comments

@xp44mm
Copy link

@xp44mm xp44mm commented Nov 6, 2019

fsharp have added nameof operator in recent version. it is just a syntax sugar. but people like it. I propose we should be to do further more.

for example:

let name = "abc"
let length = 3 // value of name.length

maybe, we can do this:

let name = "abc"
let length = valueof name.Length

this code will be translated into previous code at compile time. it will reduce people a lot of typos. and more readable code.

maybe, we can get an inference:

purity function that take arguments that are all constant return constant result.

@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 6, 2019

You can already do this, Length is a valid property on a string, but with a capital L.

let name = "abc"
let length = name.Length  // gives 3

And because it's a let binding, if it's part of a module, it's only calculated once.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 6, 2019

@abelbraaksma
if the length lie in a function that will be call n times, then length will be calculated n times.

but nameof or valueof will be calculated once at compile time. it's will be calculated zero times at any runtime.

@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 6, 2019

Ah, you mean you want some way of writing expressions that evaluate to literals (aka constants), in other words, you want to introduce a macro language, something like C++ has?

Maybe you can clarify that in your original text,and add use cases and more examples that are resolved (take a look at the template). It may also help to emphasize the constant nature, like: let <Literal> len = valueof x.Length or something.

It could be powerful, but I wouldn't be surprised if there's already a suggestion hanging around with a similar idea.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 6, 2019

fsharp should be able to inference that some values are constants actually in logical. it will be reduced a lot of boilerplate design pattern. for example, memoize, curray ...

there are lot of use cases, i can proivde a case. excel sheet have address of range represent A1, and parse it need some constant. so:

///excel sheet max column address
let maxLetters = "XFD" //letter maxColumn

/// [|x;d;f|] index of  a..z
let maxValues = [|23; 5; 3|]

/// d + f * 26 + x * 26 * 26
let maxColumns = 16384 //position maxLetters

/// xfd.length = 3
let maxLength = 3

Debug.Assert((maxLength = maxLetters.Length), "maxLength", maxLetters.Length.ToString())
@xp44mm xp44mm changed the title suggest: valueof operator suggest: `valueof` operator just like `nameof` Nov 7, 2019
@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

In F#, const variables are marked with [<Literal>].
To my knowledge there's only partial support for C++ constexpr-style values, because objects like arrays are always run-time objects, and cannot be declared [<Literal>]

Adding support for real constexpr-style compile-time evaluation would be a big effort.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

@yatli
[<Literal>]并不是重点,有更好,没有也无伤大雅。

it's no important, infer out [<Literal>] not very much improvement.

编译时推算全局不变量确实是一个艰巨的任务。能够推算一些简单的计算,比如1+1,就很高兴了。

compile-time evaluation some let binding as global immutable contant indeed is a big effort. if be able to recongnize very simple, 1+1, that it is very good .

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

> let (+) x y = printfn "Hi~"; x - y
let a = 1 + 1
let b = 3 + 2;;
Hi~
Hi~
val ( + ) : x:int -> y:int -> int
val a : int = 0
val b : int = 1

> let [<Literal>] a' = 1 + 1;;

  let [<Literal>] a' = 1 + 1
  ---------------------^^^^^

stdin(13,22): error FS0267: This is not a valid constant expression or custom attribute value

[<Literal>] cannot be inferred here.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

@Happypig375

I think @xp44mm would want something like this:

[<ConstExpr>]
let ( + ) x y = x - y

[<Literal>]
let a = 1 + 1

In a constexpr function, all bindings need to be literal.

As long as we keep printf not [<Literal>] we will be fine. :)

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

What about "".Length? Should that be made a [<Literal>]? That is a part of the .NET BCL, we would have to make a whitelist here.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

Strings are a little bit special already because of the private implementations. A string is an object, has a ton of constructors, and yet it's not mutable as arrays/lists, and could be literals.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

#804 (comment)
There are signs of arrays-as-literals too.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

What do you mean? Arrays can only be static readonly at best.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

If we want to support constexpr for any type, a potential way is to keep the initialization bytecode in the assembly -- so when another assembly references it, it can run the initializer again at compile time..

Sounds like, this can be done in a language neutral way, something like an IL injector.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

@xp44mm recommend to update the title to something more truly reflects the intent. :)

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

One of the C# discussions on this topic:

dotnet/roslyn#15079

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

@yatli @Happypig375

i just want to F# recongnize some let binding virtually is literal. is NO relationship to [<Literal>]

let abc = "abc"
let chars= abc.ToCharArray()
let len = abc.Length

the compiler will transpile it to as:

let abc = "abc"
let chars= [|'a';'b';'c'|]
let len = 3
@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

Isn't that the current state? The let binding is evaluated only once. You can put it in global scope to ensure that.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

@Happypig375
no! compiler will replace the source code with literal. the len in the file .dll or .exe is 3, instead of abc.Lengh, it need not to calculate even once.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

How will you know that Length will not modify state?

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

@Happypig375
abc is a literal "abc", so it Length property abc.Length should be literal 3 also. this issue just for performence.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

> let r = System.Random()
type System.String with
    member _.Length' = r.Next()
let a = "123".Length'
let b = "123".Length';;
val r : System.Random
type String with
  member Length' : int
val a : int = 20296113
val b : int = 36580259

abc is a literal "abc", but its Length' property abc.Length' is not constant.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

@Happypig375
abc.Length property is constant ==| abc.AnyProperty is constant.
abc.SomeProperty is not constant ==| abc.Length is not constant.

the property or function need is pure function. purity function that take arguments that are all constant return constant result.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

You cannot determine whether string.Length is a pure function. There is no indication of this.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

no effect itself, && no call impure function, is pure function.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

How can this be checked?

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

first, provide some basic pure function as seed.
and no effect itself, && just call known pure function, is pure function.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

some basic pure function as seed

Which?

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

operator, arith +-*/, Math operator, relation operator, string operator, compare operator. maybe so. Excel function most in.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

Operators only?

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

right! a different attribute.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

Back on topic, how would you classify "pure functions"? If string.Length is one, then what is the rule?

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

length is pure function, because it is char array's count, it call +. and without effect itself.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

How can the F# compiler know this?

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

array.map, array.reduce, is pure function.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

  • Seed functions (those we mark as [<ConstExpr>], or white-listed)
  • Rule of transitivity
@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

What will be in the whitelist?

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

array.map, array.reduce, is pure function.

Will this be the result of attributes or transitivity?

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

I don't think Array.map is constexpr at all.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

i think whilelist can very long. when you find some rule, then to shorten whilelist.

@yatli

This comment has been minimized.

Copy link

@yatli yatli commented Nov 12, 2019

Another approach is that, transitivity is only a constraint within a function (to check one declared [<ConstExpr>] is really pure or not)

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

amost all of Excel function is pure function. can in whilelist.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

@yatli
manual mark pure function is good idea.

@Happypig375

This comment has been minimized.

Copy link
Contributor

@Happypig375 Happypig375 commented Nov 12, 2019

Then the code in the first post will not work. Please update.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 12, 2019

the excel function random, now row, column, address, caller is effect function. just several.

@xp44mm xp44mm changed the title suggest: `valueof` operator just like `nameof` suggest: `valueof` operator to eval expression at compile time Nov 12, 2019
@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 12, 2019

amost all of Excel function is pure function. can in whilelist

There's nothing in F# that should be compared to Excel. Excel is not even a programming language.

The problem in this discussion is one of definition. I think the OP wants auto-magic constants, where a let binding 'transpiles' to a constant.

But this can never, ever work, because a constant in the .NET world is replaced by its value when consumed. That means, if a.dll has a constant, and b.dll consumes this, and we now change the value in a.dll, it will not be reflected in b.dll until you recompile it.

Therefore, magically changing existing let bindings into constants is a no-go.

There is, however, something to say for constant evaluation of expressions, provided that the compiler can infer this. This isn't as hard as it looks and there's already a proposal somewhere for F#. We can just use Literal for it. Or a special operator or function that takes an expression and evaluates it at compile time (cannot be valueof, as that is not reserved).

array.map, array.reduce, is pure function.

They are not. They can change state by virtue of the mapper or folder function you use. Besides, their output is never constant (remember that compile time constants can only ever be the built in value types that are not structs, and strings; not even a decimal can be costant, for instance).

I agree that to pre-calculate one time evaluations can be valuable, but if they are expensive calculations, you should reconsider your design. If they aren't expensive, the benefit is minimal, as all given examples from the OP are already valid code. I like the idea of an expression that forces compile time evaluation, but by virtue of the limits of CLR, this will have a very limited set of allowed functions (and let's be fair, if you know beforehand that the length of a string in your code is constant for eternity, you can micro optimize yourself by replacing it with a constant, I think we need better use cases. Here's one: current time, which could be stored as ticks as int64).

@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 12, 2019

It took me a moment to find it, but the proposal for allowing expressions in literals already exists and is approved. It just needs someone to work on it: #539

It's not exactly the same as the OP asks, but it goes a long way and could in time be expanded.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 13, 2019

@abelbraaksma
regex is another use case: it can be precompiled to DFA, and never changed if regex no changed.

i think literal more than constant, literal include constant you metioned and also include tuple literal list literal, optional literal, any immutable data struct's literal. even any can be serialized data. for example array literal json literal.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 13, 2019

But this can never, ever work, because a constant in the .NET world is replaced by its value when consumed. That means, if a.dll has a constant, and b.dll consumes this, and we now change the value in a.dll, it will not be reflected in b.dll until you recompile it.

this do not matter. because the length value be infered or evaluated form "abc" literal, if length be updated in a.dll. then "abc" must be updated also in a.dll, so anyway, the b.dll must be recompiled.

@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 13, 2019

i think literal more than constant, literal include constant you metioned and also include tuple literal list ....

Actually, no. In F#, a literal, declared with the [<Literal>] attribute, or inlined as a literal expression, is a constant. I mean, what other languages like C#, VB, C++ call constants, is called a Literal in F#.

And constants, or literals, by their very definition, cannot be reference types (the exception being strings and byte arrays). So your example of regex won't work, not even if you allow calculations of function calls in literals that are to be evaluated at compile time. The only operators presently allowed are ||| on numbers and + on strings. But have a look at the proposal on expanding the set of allowed operators, as I mentioned before. It would be a good starting point.

To get what you want with regexes, you can actually use the methods available from System.Text.RegularExpressions, like CompileToAssembly.

Wanting to allow a "constant" like that, i.e., a whole object, or in this case an actual assembly, to be pre-compiled and stored in your DLL or EXE would mean that somehow the compiler needs to serialize and deserialize the data that you you pre-computed. That's far from trivial, there are books written on that subject alone, and likely not going to happen (but I'm not the one who decides, if there's a reasonable proposal and it gets traction, and you're willing to implement it, they may accept it).

An alternative you might consider is using a RESX file. Create an auxiliary project that can do the pre-calculations that you want. Store them in a (binary) file. Read this in into the resx file as resources and access them directly from your code just like with C# projects. That is the proper way to go about resources (or "constants") that extend the realm of the expressive power or usefulness of Literal bindings in F#.

About regexes in particular, a good tool that I use is this Regular Expression Library Builder, which can do exactly what you suggest and create "constant", that is, DLL assemblies, of your compiled regular expressions.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 13, 2019

@abelbraaksma
thank you! actually i can work aroud it, the simplest is:

let len = 3 // abc.Length

or i can test it:

Assert.equal(len,abc.Length)

maybe i can just keep it no changed:

let len = abc.Length
@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 13, 2019

this do not matter. because the length value be infered form "abc", if length be updated in a.dll. then "abc" must be updated also in a.dll, so anyway, the b.dll must be recompiled.

I'm not sure you understand how literals, or constants work. I don't understand what you mean with "infer" here. Let's say you have:

// a.dll
let [<Literal>] SomeString = "John"

// b.dll
let [<Literal>] LengthSomeString = SomeString.Length

// c.dll
let [<Literal>] DoubleLength = LengthSomeString * 2

If one company created a.dll another one b.dll and another one c.dll, if the first changes the name, and you have some code depending on this length, then everything falls apart (for instance, say you have code that iterates over all characters for i=0 to LengthSomeString do ... this will horribly fail).

This can also happen with actual constants and is a well documented fact and cause of confusion for C# developers. Allowing arbitrary expressions is potentially going to wreak havoc.

I still think there's value in an expression like valueof taking an argument that is evaluated at compile time, but it should be severely restricted. A current workaround, btw, is using generated files or something like Fody for weaving, or Cecil, which all allow much of what you are after and more. But each comes with their own caveats.

@cartermp

This comment has been minimized.

Copy link
Member

@cartermp cartermp commented Nov 13, 2019

I would think the function could only apply to things that you could also apply the [<Literal>] attribute to. Would that be unexpected, since it's so restrictive?

@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 13, 2019

@cartermp, I think it should be allowed everywhere where we allow literals to appear (like "a string", 123y etc). Technically that's the same as bindings marked with Literal, as they simply get replaced inline, as opposed to referenced.

A large part of the discussion was about the confusion of what the OP wanted, which frankly, I'm still unsure about, but taken as a compile time evaluation of a valid F# expression that yields a .NET Framework allowed literal, I think it has merit.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 13, 2019

@abelbraaksma
yes! dependent on constant in another assembly, and fixed it to result value is dangous. maybe outdated. so i must sure another assembly is never absolutely changed for example constant PI. compiler don't recongnize this dangous situation. so, must manual mark need evaluation of expression at compile time.

@xp44mm

This comment has been minimized.

Copy link
Author

@xp44mm xp44mm commented Nov 13, 2019

@abelbraaksma
regex is reference type in implement, but it is some immutable data in logic. so can resolve it in advance.

another use case: reflection is some immutable data, it represent source code, never changed except source code changed. it can be resolved in advance at compiler.

@abelbraaksma

This comment has been minimized.

Copy link

@abelbraaksma abelbraaksma commented Nov 13, 2019

regex is reference type in implement, but it is some immutable data in logic. so can resolve it in advance.

Sorry, still not sure what you mean, as this is simply not possible given the current state of the art of compilers for the .NET platform. So either you mean something that I don't understand (examples, please), or I'm afraid you may not understand what the purpose, usages and limits of literals are, which is fine, but then I suggest you read the article I referenced earlier. The types there are the types any literal is limited to by the .NET Framework. So whatever your proposal, it has to be within those limits, or you would have to make your suggestion to Roslyn in stead.

reflection is some immutable data,

Nope, reflection is not immutable. That's why people use reflection: they only know the type signatures at runtime. For instance because the versions of the methods they inspect can be different, or because they use pluggable libraries.

You could do it once, serialize it (MethodInfo is serializable) and read this back in, but that only makes sense if you expect libraries to never be updated. But then again, serializing and deserializing is already possible, just like with regexes, you don't need extra language features for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.