Olegs/expose native objects

[wvxvw]

The changes roughly fall into these categories:

• New Prolog type term.Native
  This required some (very few) modifications in term package to deal with caching of terms (perhaps it would be better to let each kind of term to implement a method which returns something that may be cached?) Another question here: I treated native nil as if it was a non-instantiated Prolog variable. It was useful in some of my code, but I'm not sure it's the correct way to go about it.

• Serialization and deserialization of Go objects into Prolog objects
  I'm not entirely confident about correctness of all of my code. (Go reflection is a huge mess...) But for the instances I tried, it seems to work. There are some tests for decoding and encoding, but they don't cover 100% of possible cases. There are also some todos in that general area related to integer overflow that may happen when converting between these types. It wasn't a high priority problem for me, but if need be, I can fix that.

• help and apropos predicates
  I found that for interactive use with a large database these are invaluable (I used them with my own code because I couldn't remember the exact names or the number / order of arguments). But, really this is optional feature, however it required one small change in machine struct. I also used a regular hash-table rather than persistent one because these are intended for interactive use, where concurrency isn't an issue. Again, I can redo it using the persistent hash-table to be consistent with the rest of the code.

• Pretty printing of strings and lists
  Instead of printing strings like this '.'(23, '.'(24, ...) now it prints it "ab...". This makes the printing a little slower, but I think it's worth the effort. Especially since most printing happens interactively anyways. The printing code doesn't handle recursive terms (I'm not really sure if this is something that's even supported in Golog in general). Anyways, if need be, I can fix it to also avoid looping indefinitely over recursive structures.

[mndrix]

I finished reading all the changes. First off, I can tell that this took a lot of effort. Thanks for being willing to contribute it.

Can you isolate the pretty printing changes in one commit, the apropos/help changes in a second commit and send me a pull request for those two? If so, I should be able to merge it right away.

I love the idea of easy conversion between Go types and Prolog terms. The biggest question in my mind is how to represent a Go struct in idiomatic Prolog. Let's consider this type for conversation:

type Person struct {
    Name string
    Age int
}

The code in this pull request represents the Go value Person{Name:"Bob", Age: 27} with the following term:

person(name("Bob"),age(27))

I'm I were to represent that same value in idiomatic Prolog, without any thought for Go, I'd probably use person("Bob",27) with some accessor predicates like:

person_name(person(N,_),N).
person_age(person(_,A),A).

Losing the name wrapper around each Prolog argument makes it harder to convert from Prolog back into Go, but I think it gives us a more natural representation in Prolog.

Ideally, I think I'd like something along these lines. You may add field tags like golog:"foo,7" to assist with the conversion to/from Prolog. That uses the Prolog name foo and puts it in the 7th position in a compound term. Using the type above, and being explicit about the tags, we'd have:

type Person struct {
    Name string `golog:"name,1"`
    Age int `golog:"age,2"
}

Then something like term.NewCompound(val interface{}) *term.Compound as a convenience for converting Go values into Prolog terms. Along with term.Decode(t term.Term, dst interface{}) to converting Prolog terms into Go values. Both of these functions are probably built on top of more general Marshal and Unmarshal interfaces. Since all these functions are about Prolog terms, not their execution, I think they belong in the term package.

I'd also want to be able to do something like m := golog.NewMachine().Consult(&Person{}) to automatically add person_name/2 and person_age/2 definitions to my machine. It'd be especially cool if methods on *Person were converted into foreign predicates which called the method. Anyway, these helper predicates make it easy to work with Go values in Prolog in a natural way. If the Go types change, the Prolog helpers automatically change too so I don't have to rewrite my Prolog code.

If you're willing to refactor your native object support in this direction, I think we can get it to a point to merge as well.

[wvxvw]

Hi, It shouldn't be a problem to separate the pretty printing and the help/apropos changes. I might even do it today, or if not, then over the weekend. Wrt. serialization, here are some thoughts: I didn't actually represent Go structs like person(name("Bob"), age(27)), it was person([name("Bob"), age(27)]), so the query to extract a value would usually look more like this: person_name(person(X), Name) :- member(X, name(Name)). In the project I worked on, Prolog was also used to represent configuration, and it would've been inconvenient to have to specify all the values (while there may have been defaults). It makes field value extraction slower and "kind of" nondeterministic, but speed wasn't a concern for me, and if someone was to write a configuration with multiple values for the same field--so bad for them. Say, I'm going to write the code for accessing field values, do you think it might be still better to have fields as a list rather than tuple? Re' autogeneration of methods... it sounds doable, but Go's reflection is a minefield... I will try, but it's hard to tell if it is in fact doable before I can get it to work (or fail to). Also, when writing foreign predicates, I discovered that, at least for me, there were two useful patterns: (1) for a predicate foo/2, if the Go implementation could return error, I'd also create foo/1 which would fail if error was instantiated by foo/2. The problem with this, however, was that the way to figure out if foo/2 instantiated the error was very invasive (I needed to try to cast the result to []term.Term and then look for the variable I thought would be the error). (2) In a similar way, whenever there was a predicate taking context.Context as its first argument, I'd have one without, substituting context.Background() for the first argument. This is easier because I don't need to examine the values returned, but it's not a very common case, I think. So, say, I succeed in writing some generic code to invoke methods defined on Go object, do you think it's interesting / worthwhile to generate these extra predicates with less arguments?

…

On Wed, Mar 29, 2017 at 9:06 PM, Michael Hendricks ***@***.***> wrote: I finished reading all the changes. First off, I can tell that this took a lot of effort. Thanks for being willing to contribute it. Can you isolate the pretty printing changes in one commit, the apropos/help changes in a second commit and send me a pull request for those two? If so, I should be able to merge it right away. I love the idea of easy conversion between Go types and Prolog terms. The biggest question in my mind is how to represent a Go struct in idiomatic Prolog. Let's consider this type for conversation: type Person struct { Name string Age int } The code in this pull request represents the Go value Person{Name:"Bob", Age: 27} with the following term: person(name("Bob"),age(27)) I'm I were to represent that same value in idiomatic Prolog, without any thought for Go, I'd probably use person("Bob",27) with some accessor predicates like: person_name(person(N,_),N). person_age(person(_,A),A). Losing the name wrapper around each Prolog argument makes it harder to convert from Prolog back into Go, but I think it gives us a more natural representation in Prolog. Ideally, I think I'd like something along these lines. You may add field tags like golog:"foo,7" to assist with the conversion to/from Prolog. That uses the Prolog name foo and puts it in the 7th position in a compound term. Using the type above, and being explicit about the tags, we'd have: type Person struct { Name string `golog:"name,1"` Age int `golog:"age,2" } Then something like term.NewCompound(val interface{}) *term.Compound as a convenience for converting Go values into Prolog terms. Along with term.Decode(t term.Term, dst interface{}) to converting Prolog terms into Go values. Both of these functions are probably built on top of more general Marshal and Unmarshal interfaces. Since all these functions are about Prolog terms, not their execution, I think they belong in the term package. I'd also want to be able to do something like m := golog.NewMachine().Consult(&Person{}) to automatically add person_name/2 and person_age/2 definitions to my machine. It'd be especially cool if methods on *Person were converted into foreign predicates which called the method. Anyway, these helper predicates make it easy to work with Go values in Prolog in a natural way. If the Go types change, the Prolog helpers automatically change too so I don't have to rewrite my Prolog code. If you're willing to refactor your native object support in this direction, I think we can get it to a point to merge as well. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[mndrix]

I've cherry picked the help/apropos and pretty printing commits. I also pushed a follow up commit to fix test failures related to pretty printing.

It'd be nice if help(printf/1) worked so that users don't have to wrap the predicate indicator in a string. I figured that was a minor UI issue at this point, so I didn't want it to hold back the changes entirely.

I didn't actually represent Go structs like person(name("Bob"),
age(27)), it was person([name("Bob"), age(27)])

Sorry. I missed the list wrapper during my review.

do you think it
might be still better to have fields as a list rather than tuple?

I prefer to work with plain compound terms in Prolog, but SWI-Prolog uses option lists in some places. At this point, I'm not sure we know enough about how Golog users prefer to translate their Go types into Prolog values. Without that kind of feedback, I'm hesitant to merge anything yet.

[wvxvw]

Sorry to re-open this thread: I've went on to try to add predicates to map Go methods to Prolog, and there are several conceptual problems with that. I just wanted to leave these findings here for if this discussion will surface again later:

• The generated name may clash with an existing predicate.
• Machine.Consult() isn't probably the best way to do this because if a Go object implements io.Reader, Consult() will not encode it, but instead will try to read it. Also, strings can be encoded by native.Encoder too, so that would be a problem as well.
• You may want to encode / decode Go objects of the same type more often than you'd want to create accessors for them, because what you are interested in is in getting their values into the interpreter. A simple example would be a Go slice containing multiple instances of the same kind of object--obviously, you wouldn't want to go through the same process for each element of a slice, for example.
• Related to the bullet-point above: would it be necessary to generate accessors / other methods for types used / embedded in the Go object? If the type is embedded, it is likely to have methods with the same names as the embedding type: should these be treated like if they were the same method, or different methods?
• Making sure that the signature of the generated predicate is called with compatible arguments is hard in a case when some arguments aren't fully instantiated. Also, related: should the presence of extra fields on Prolog side count as incompatibility with Go side?

---

A little more info on generating methods: it is hard to know if the method is a mutator, and if so, if it should instantiate the variables in the arguments. Suppose this example:

X = x([a(A)]), x_test(X).

Now, suppose the Go code was this:

type X struct {
    A int
}
func (x *X) Test() {
    x.A = 42
}

Or, maybe slightly different:

type X struct {
    A int
}
func (x *X) Test() {
   fmt.Printf("calling Test")
}

Thus, without knowing whether the Go code modified X, we cannot tell if in Prolog code we need to instantiate A... Or, maybe we could... but this would require of us to maintain some sort of mapping between Prolog variables and corresponding Go values, and then verifying if any of them changed after calling Go method... which is becoming very tricky...