-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
jvm: make call* variadic in the unsafe interface #135
Conversation
Have you checked that the MethodID is actually cached? By looking at the code, it looks to me like the internal calls |
I had missed that part of the changes in #133, but we can do the same here. You changed |
28105d2
to
9269909
Compare
@facundominguez PTAL. I rewrote the patch with the following differences:
As noted in the commit message, the error messages are pretty much as good as they get. I shied away from using anything fancy like closed type families, for fear that the implementation detail would leak in error messages. The price to pay is more verbose instance implementations, but we have Template Haskell to write the boilerplate for us. |
`call` previously took a list of `JValue`'s. Using the dictionaries embedded in each element, it could construct a `MethodID`. However, this `MethodID` was not cached as well as one would hope, say when multiple calls are made to the same method at the same argument types but different values. Another problem was that all arguments to the method had to be packed into a list and coerced into a `JValue` before being passed to the method. This was syntactic overhead. The solution is to make `call`, `new` and `callStatic` variadic, using the same GADT solution as for variadic `printf`. Using ad hoc private type classes per function, we make them variadic up to 32 arguments. ```haskell call obj "frobnicate" x y z ``` We no longer need to coerce `x`, `y`, `z`, nor pack them into a list. Furthermore, this solution leads to decent error messages. The private type class should never appear in error messages. It's hard to think of any example where the error message would be something else than "could not deduce `Coercible a` ...". Which is good enough, since it's telling us that you can pass anything you want to `call`, but it must be coercible.
I would expect New, Call and CallStatic to appear in error messages of
I don't have a problem with error messages of this sort, after all, I was already fine with the class JNIArguments in #133. As @goldfirere suggested, this might be possible to improve with TypeError TH might not be possible to use in the safe interface yet. Last time I tried, TH didn't allow linear arrows. Lastly, is this going to work with the safe interface, where we have an abstract monad instead of IO? That is the case that motivated @goldfirere to propose a closed type family. |
Why so? In your example, the error message is:
Which is exactly what it ought to be, in my view, because you're asking for the function to return something in the wrong monad. The type classes you mention are internal details that the user should, as much as possible, not have to worry about.
I tried this, and it works. Even with an abstract monad, GHC does not see overlap in my tests. I kept things in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took a look through. It looks like this new approach works well. I hope you don't mind my small suggestions!
@@ -155,3 +160,20 @@ getStaticFieldAsJValue retsing cname fname = do | |||
SPrim "double" -> JDouble <$> getStaticDoubleField klass field | |||
SVoid -> fail "getStaticField cannot yield an object of type void" | |||
_ -> JObject <$> getStaticObjectField klass field | |||
|
|||
mkVariadic :: TH.TypeQ -> (TH.TypeQ -> TH.TypeQ -> [TH.PatQ] -> TH.ExpQ -> TH.ExpQ -> TH.DecsQ) -> TH.DecsQ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know the normal documentation standards in this code, but I would want more documentation here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's an internal function that's only part of a separate module due to the stage restriction. Haddock won't help document the arguments of the continuation, but we can do that with a regular comment.
@@ -155,3 +160,20 @@ getStaticFieldAsJValue retsing cname fname = do | |||
SPrim "double" -> JDouble <$> getStaticDoubleField klass field | |||
SVoid -> fail "getStaticField cannot yield an object of type void" | |||
_ -> JObject <$> getStaticObjectField klass field | |||
|
|||
mkVariadic :: TH.TypeQ -> (TH.TypeQ -> TH.TypeQ -> [TH.PatQ] -> TH.ExpQ -> TH.ExpQ -> TH.DecsQ) -> TH.DecsQ | |||
mkVariadic retty k = fmap concat $ for [0..32 :: Int] $ \n -> do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should 32 be a defined constant somewhere? Presumably, this number is the maximum number of arguments supported in variadic notation, and perhaps clients will want to query it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed inline-java
(which is a package downstream of jvm
) currently duplicates the same constant (with a comment pointing here). I'll export a constant.
mkVariadic :: TH.TypeQ -> (TH.TypeQ -> TH.TypeQ -> [TH.PatQ] -> TH.ExpQ -> TH.ExpQ -> TH.DecsQ) -> TH.DecsQ | ||
mkVariadic retty k = fmap concat $ for [0..32 :: Int] $ \n -> do | ||
let -- Coercible type class and Ty associated type defined in downstream module. | ||
coercible = TH.conT (TH.mkName "Coercible") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This approach is quite fragile -- especially given that there may reasonably be the wrong Coercible
in scope. It is possible to concretely specify the defining module, etc., for a name, even before it's in scope. See the mk_name_...
functions in https://github.com/goldfirere/singletons/blob/master/src/Data/Singletons/Names.hs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As noted above, mkVariadic
should only be used in Language.Java.Unsafe
, not anywhere else (and certainly not by the user). That said, if there is something more robust than mkName
(which is only necessary because of stage restriction collateral damage), might as well. I noticed mkNameG_tc
and friends in the template-haskell API docs, but they're not documented. What does it do, and how does it differ from mkName
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those functions make names tagged with a package and a module; mkName
makes a name that unhygienically refers to whatever is in scope. I don't think mkNameG_tc
was really meant to be exported from TH, but it works well in this scenario.
-- | Inject a value (of primitive or reference type) to a 'JValue'. This | ||
-- datatype is useful for e.g. passing arguments as a list of homogeneous type. | ||
-- Synonym for 'coerce'. | ||
jvalue :: (ty ~ Ty a, Coercible a) => a -> JValue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why have the ty
type variable here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is preexisting code that was moved earlier in the file because of the intervening splice leading to scoping errors. So I left it as-is. You're right that ty
here serves no purpose. Probably a holdover from the past.
-- | Get the Java class of an object or anything 'Coercible' to one. | ||
classOf | ||
:: forall a sym. (Ty a ~ 'Class sym, Coercible a, KnownSymbol sym) | ||
=> a | ||
-> JNI.String | ||
classOf x = JNI.fromChars (symbolVal (Proxy :: Proxy sym)) `const` coerce x | ||
|
||
-- Unexported type classes. | ||
class New r where |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might want to consider exporting the class (but not the method). This will improve the Haddocks (where the class won't need to be written qualified) and clean up error messages that might happen to mention this class. I agree that it would be nice to avoid such error messages, but I'm sure some will slip through. The documentation should explain that it's an internal class, exported only for Haddocking and error messages. Ditto for other internal classes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the class does end up leaking in error messages, then it would be better to export, I agree. The downside is that the documentation will end up bloated with 32 * 3 = 96
instances that aren't very interesting to the user. That's why I was trying to keep the class completely unexported. Interestingly, I find it pretty hard to get error messages that mention them. I haven't managed so far.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, so when the return type is wrong, type errors frequently arise elsewhere before the compiler complains about lack of instances of class Call
. But not always. So indeed, Call
is leaking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And, regardless, it will appear if a user says :t call
in GHCi. Haskellers who see a variadic function will know that some cleverness lurks nearby, and I don't think they'll be too scared off. There might be a way to convince Haddock not to include the instances. On the other hand, seeing the instances will make it very clear what constraints will need to be satisfied by the arguments to call
, if a user wants to take the time to piece all of this together.
Using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@goldfirere thanks for the comments.
@@ -155,3 +160,20 @@ getStaticFieldAsJValue retsing cname fname = do | |||
SPrim "double" -> JDouble <$> getStaticDoubleField klass field | |||
SVoid -> fail "getStaticField cannot yield an object of type void" | |||
_ -> JObject <$> getStaticObjectField klass field | |||
|
|||
mkVariadic :: TH.TypeQ -> (TH.TypeQ -> TH.TypeQ -> [TH.PatQ] -> TH.ExpQ -> TH.ExpQ -> TH.DecsQ) -> TH.DecsQ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's an internal function that's only part of a separate module due to the stage restriction. Haddock won't help document the arguments of the continuation, but we can do that with a regular comment.
@@ -155,3 +160,20 @@ getStaticFieldAsJValue retsing cname fname = do | |||
SPrim "double" -> JDouble <$> getStaticDoubleField klass field | |||
SVoid -> fail "getStaticField cannot yield an object of type void" | |||
_ -> JObject <$> getStaticObjectField klass field | |||
|
|||
mkVariadic :: TH.TypeQ -> (TH.TypeQ -> TH.TypeQ -> [TH.PatQ] -> TH.ExpQ -> TH.ExpQ -> TH.DecsQ) -> TH.DecsQ | |||
mkVariadic retty k = fmap concat $ for [0..32 :: Int] $ \n -> do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed inline-java
(which is a package downstream of jvm
) currently duplicates the same constant (with a comment pointing here). I'll export a constant.
mkVariadic :: TH.TypeQ -> (TH.TypeQ -> TH.TypeQ -> [TH.PatQ] -> TH.ExpQ -> TH.ExpQ -> TH.DecsQ) -> TH.DecsQ | ||
mkVariadic retty k = fmap concat $ for [0..32 :: Int] $ \n -> do | ||
let -- Coercible type class and Ty associated type defined in downstream module. | ||
coercible = TH.conT (TH.mkName "Coercible") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As noted above, mkVariadic
should only be used in Language.Java.Unsafe
, not anywhere else (and certainly not by the user). That said, if there is something more robust than mkName
(which is only necessary because of stage restriction collateral damage), might as well. I noticed mkNameG_tc
and friends in the template-haskell API docs, but they're not documented. What does it do, and how does it differ from mkName
?
-- | Inject a value (of primitive or reference type) to a 'JValue'. This | ||
-- datatype is useful for e.g. passing arguments as a list of homogeneous type. | ||
-- Synonym for 'coerce'. | ||
jvalue :: (ty ~ Ty a, Coercible a) => a -> JValue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is preexisting code that was moved earlier in the file because of the intervening splice leading to scoping errors. So I left it as-is. You're right that ty
here serves no purpose. Probably a holdover from the past.
-- | Get the Java class of an object or anything 'Coercible' to one. | ||
classOf | ||
:: forall a sym. (Ty a ~ 'Class sym, Coercible a, KnownSymbol sym) | ||
=> a | ||
-> JNI.String | ||
classOf x = JNI.fromChars (symbolVal (Proxy :: Proxy sym)) `const` coerce x | ||
|
||
-- Unexported type classes. | ||
class New r where |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the class does end up leaking in error messages, then it would be better to export, I agree. The downside is that the documentation will end up bloated with 32 * 3 = 96
instances that aren't very interesting to the user. That's why I was trying to keep the class completely unexported. Interestingly, I find it pretty hard to get error messages that mention them. I haven't managed so far.
This is fine for the unsafe interface. Orthogonal or not, the linear interface uses an abstract monad, and we need to solve caching of MethodIDs there as well for our benchmarks. |
This PR is breaking inline-java when built with the linear-types-enabled GHC.
|
That would be great. I don't feel confident about (or have much time for) hacking on the linear variant. I expect those modules can use similar type classes to the unsafe interface, with or without @goldfirere's Meanwhile, I believe it's possible to hide the variadic type classes from all error messages using a catchall (and yes, overlapping) instance with |
Variadic functions all have a constraint, which appears in their type signature. In an effort to further hide this constraint, we replace it in each case with a generic `Variadic "foo" r` constraint, where `foo` is the name of the variadic function. That way we can explain in a single place what this one constraint means throughout the module. This constraint should only appear in Haddock. In principle, it should never appear in error messages.
Using TypeError, we can replace the error message that GHC provides to the user when a constraint is not satisfied. We can in particular arrange for such a custom message to be displayed whenever GHC would instead talk about potentially private constraints that mean nothing to the user. This requires a catchall instance that overlaps with all others. The overlap is benign, because the overlapping instances are for private constraints that are never exported. Also, the error instance doesn't have any operational meaning. It's just to throw an error at the user when the program doesn't type check anyways.
@facundominguez @goldfirere I added catchall instances with With this change, I believe that it's not necessary to expose the private classes in the Haddocks. We now mark all variadic functions in a uniform way using a single |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Documentation can be improved, but I can contribute that.
I think we can go with this approach. The only question I'd like discussed a bit, is whether we can have a more descriptive and/or decoupled Variadic type class/family.
@@ -319,7 +327,7 @@ $(mkVariadic [t| J ('Class $(TH.varT (TH.mkName "sym"))) |] $ \ctx typ pats args | |||
-- @ | |||
-- | |||
-- You can pass any number of 'Coercible' arguments to the constructor. | |||
new :: New r => r | |||
new :: Variadic "new" r => r |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe the type of new
would be more informative if the return type was present in the context.
new :: (Coercible a (J ('Class class)), Variadic "new" r (IO a)) => r
-- or
new :: (Coercible a (J ('Class class)), Variadic "new" r) => r (IO a)
One could also wish for the constraints of the arguments to be explicit:
new :: (Coercible a (J ('Class class)), Variadic "new" Coercible r (IO a)) => r
-- or
new :: (Coercible a (J ('Class class)), Variadic "new" r ) => r Coercible (IO a)
As it is now, the documentation of the functions that use Variadic
and Variadic
itself need to compensate for this opaqueness.
Furthermore, would it be possible to make a standalone version of Variadic, that is not specific to inline-java?
Adding a parameter to describe the "fixed" arguments, it could be possible to subsume New, Call and CallStatic with a single type class which is instantiated with specific parameters on each case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A challenge with variadic programming is that Haskell considers functions to be just like all other data. But a variadic function can't return a function, or else we don't know how many parameters there are. That's why having a result in IO
works easily, but have a general monad does not work as easily. Still, I agree that this could be generalized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Variadic
itself isn't exported, so the documentation for it won't help much. What we have is an explanation of the Variadic
constraint at the module level, and in the documentation of each function.
A standalone version of Variadic
would be very nice. But it would have to look different than it does now. Indexing with a symbol allows us to use the exact same name as the function: Variadic "foo"
says foo
is variadic. But if Variadic
were to escape the scope of the current module, then we'd have to be careful that no other module exports a variadic function also called foo
. The fix is to use an empty datatype instead of a symbol as the index.
I'm okay with generalizing Variadic
with the extra indices that you suggest, especially if Variadic
is to be spun out, though I'd keep r :: *
as in your first example. Because it will be hard to have an r :: Constraint -> * -> *
be anything other than a data/newtype.
Keep in mind that even with extra indices, Variadic
is still a special case of variadic functions. The prototypical one is printf
, where one would normally want the freedom to specify arbitrary serializers/formats for each argument, rather than be constrained to canonical serializers given by type class instances (like the non-type-class-based one does). The inline-java case is a special case.
@@ -161,14 +162,18 @@ getStaticFieldAsJValue retsing cname fname = do | |||
SVoid -> fail "getStaticField cannot yield an object of type void" | |||
_ -> JObject <$> getStaticObjectField klass field | |||
|
|||
-- | The maximum supported number of arguments to variadic functions. | |||
maxVariadicArgs :: Int | |||
maxVariadicArgs = 32 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nitpick: Is it hard to get the constant used at the type level in the TypeError
message that refers to it? 😈
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, you could have
type MaxVariadicArgs = 32
maxVariadicArgs :: Int
maxVariadicArgs = fromInteger $ natVal @MaxVariadicArgs Proxy
Gee, it would be nice to have dependent types... :)
I'm adapting @goldfirere last proposal to work with a finite amount of instances and no TH.
I would try something similar on top of this PR, unless I'm missing something essential. |
hm, when the monad is abstract the Compiling this in @goldfirere program (either with or without my modifications):
yields
In #133, the original signature ( |
Could we please proceed incrementally? The safe interface is wholly separate from the unsafe interface. The quick way forward is to simply adapt the safe interface to the changes in |
Works for me. |
There you go. #138 is a PR over this one to simplify the type class instance business. |
Introduce another implementation of Variadic
This makes simpler explaining what the meaning of VariadicIO f b.
, [| fromString $(TH.stringE ("io.tweag.inlinejava." ++ mangle thismod)) |] | ||
, [| fromString $(TH.stringE mname) |] | ||
] ++ map TH.varE thnames') | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code is used both by the linear and the non-linear interfaces. Looks like we will have to abstract the way in which arguments are passed.
This allows the unsafe and the safe interface to adjust the generation to each case.
I'm waiting for CI to merge. |
This PR is mutually exclusive with #133.
call
previously took a list ofJValue
's. Using the dictionariesembedded in each element, it could construct a
MethodID
. However, thisMethodID
was not cached as well as one would hope, say when multiplecalls are made to the same method at the same argument types but
different values. Another problem was that all arguments to the method
had to be packed into a list and coerced into a
JValue
before beingpassed to the method. This was syntactic overhead. The solution is to
make
call
,new
andcallStatic
variadic, using the same GADTsolution as for variadic
printf
.We now pass an extra spec argument to
call
. Like the format string inprintf
, this argument determines how many more arguments need to bepassed to
call
. Example:We no longer need to coerce
x
,y
,z
, nor pack them into a list.Furthermore, this solution leads to decent error messages if arguments
are missing.