Proposal for clarifying vocabs

[tcstewar]

As discussed in #784 and as has come up a number of times when using Vocabularies, there is a lot of black magic going on. The particular issue in #784 is that it can be very unclear as to which vocab is being used where when parsing Actions. This proposal is meant to make things more explicit and give the user control over this as needed.

Consider the Effect memory = vision*~BLUE*NOUN. This can be interpreted as "take the blue thing I am looking at and store it in memory bound with NOUN". So if vision is currently BLUE*SQUARE+RED*TRIANGLE I should end up with SQUARE*NOUN stored in memory. If memory and vision have the same Vocabulary, then this is fine. But what happens if they have different vocabularies? Right now, what it does is use vision's vocab for BLUE and NOUN, and then finds a linear transform from vision's vocab to memory's vocab (using the transform_to function). In this particular case, it makes sense to look up BLUE in vision's vocab, but it's kind of weird to look up NOUN in vision's vocab, since that's really a memory thing. In #784 there was some discussion of checking to see if vision is marked as a read-only vocab, and if it is a read-only vocab but NOUN doesn't exist in it, then we should try memory's vocab, but we couldn't find a clean set of logic to do this.

Instead, let's force people to be explicit when converting between vocabs. This means I would instead write memory=((vision*~BLUE).convert(memory))*NOUN. Now we are explicit about BLUE being in vision's vocab and NOUN being in memory's vocab.

Notes:

• There would no longer be automatic conversions between vocabs. If you do memory=vision and those use different vocabs, that is now an error. You'd have to do memory=vision.convert(memory)
• Similarly, you can only add things that have the same vocab
• Similarly, you can only convolve things that have the same vocab. So instead of a=b*c which can do some pretty wacky transformations when a b and c all have different vocabs. Instead, you'd have to do a=b.convert(a)*c.convert(a) or possibly a=(b.convert(c)*c).convert(a) depending on which one you wanted.
• This doesn't change any of the simple examples at all, since this only ever comes up if you have multiple vocabs.
• A different term rather than convert might be nice. @arvoelke suggested to. Any other ideas?

Thoughts?

[jgosmann]

A different term rather than convert might be nice. @arvoelke suggested to. Any other ideas?

tr (short for transform), but I like to more

In general: 👍

[xchoo]

Hmmmm. Would it still throw an error if the modules are of the same dimensionality?
EDIT: Nvm. It should throw an error.

[xchoo]

Also, it's not clear what this should do:

vocab = Vocabulary(32)
vocab.parse('A+B+C+D')
am_vocab = vocab.create_subset(['A', 'B'])

with spa.SPA() as model:
    model.mem = spa.Buffer(32)
    model.am = spa.AssociativeMemory(am_vocab)

    cortical_actions = spa.Actions('mem = am')
    model.cortical = spa.Cortical(cortical_actions)

Should it fail? (Because both the vocabularies are different)
Also, if we replace the rule with:

cortical_actions = spa.Actions('mem = am.to(mem)')

What should that do? (It should be an identity transform, but the transform_to code won't give you one, and that'll add noise into the system - possibly reducing the effectiveness of the Assoc Mem?)

[xchoo]

Other words for transform: (i just looked up the synonyms)

change, alter, convert, metamorphose, transfigure, transmute, mutate; revolutionize, overhaul; remodel, reshape, redo, reconstruct, rebuild, reorganize, rearrange, rework, renew, revamp, remake, retool, transmogrify, morph

I like morph. 😄

[bjkomer]

I like it!

A couple other options for syntax that come to mind:

This conversion stuff reminds me a lot of type casting, so something along the lines of a = (a)b*(a)c instead of a = b.convert(a)*c.convert(a) and a = (a)b*c instead of a = (b*c).convert(a) could make sense, and reduce the amount of characters needed. Maybe have a different style of braces like a = {a}b*{a}c to avoid potential confusion.
Another possibility is to us some operator for this function. a = a.b*a.c or a = b->a*c->a

That being said, I still like to, but I haven't used vocabs much at all yet, so it's hard to say

[jgosmann]

Maybe we need another operation for the associative memory which changes the associated vocabulary but without applying any transform (and the dimensionality of input and output vocab have to match in this case)? In C++ casting terms we would have a reinterpret_cast in addition to the normal static_cast/dynamic_cast.

But I don't like cryptic the C/C++ casting syntax.

[xchoo]

I kinda like the dot syntax. It opens up the possibility to do this:

a = b * c.VERB

i.e. convolve b with VERB from c's vocab and put it in a.
(assuming a, b, and c are all the same dimensionality). But maybe that's a can of worms we don't want to open. haha

[tcstewar]

(assuming a, b, and c are all the same dimensionality). But maybe that's a can of worms we don't want to open. haha

I'd very much prefer if we kept with only two cases: are the vocabs the same or are they different. I think things get nedlessly much more complicated if we handle three cases: are the vocabs the same, are they different but have the same dimensionality, and are they different.

[tcstewar]

For subvocabs, I'm completely not sure. My instinct is that a subvocab is just the same as the normal vocab, but with a subset of the terms, so we should just treat it as matching as far as this logic is concerned. That way the only things the subvocab does is give a restricted set of terms for display or for tranform_to. So @xchoo 's example above would work fine and not require a to. That covers all the cases I can think of, butI've never seriously used subvocabs.

[tcstewar]

Oh, and one quick comment for people brainstorming different syntax possibilities: I really really really want it to stay as valid Python syntax. So a = (a)b*(a)c wouldn't work but a = a(b*c) would work. As would a = a.b * a.c or a = b.a * c.a

[Seanny123]

+1 for casting with vocabs

[tcstewar]

+1 for casting with vocabs

what do you mean by this? Do you mean the idea in general or do you mean the specific C syntax of a=(a)(b*c)

[Seanny123]

Sorry, I meant a=a(b*c)

[jgosmann]

a(b * c) looks like it could mean a * (b * c)

[xchoo]

I agree.

In regards to the dot format, this: a.(b * c) is not valid python, right? (i.e. the dot format might not be viable either). My vote is still for morph(). 😄

[tcstewar]

a.(b * c) is not valid python, but (b * c).a is valid

[bjkomer]

a(b * c) looks like it could mean a * (b * c)

What about a[b * c]? It's valid syntax and you could think of it like b*c is an element within the vocab of a

[xchoo]

What about a[b * c]? It's valid syntax and you could think of it like b*c is an element within the vocab of a

Hmmm. I could get on board with that!

[xchoo]

Further thinking about it. (b * c).a actually makes sense-issssssh? The transform is an outer product, and you compute the resulting vector by doing a dot product (matrix dot product, i.e. matrix multiply) with the transform.

[tcstewar]

I could see a[b*c] working.... but I also kinda like (b*c).a and (b*c).to(a). Here's the original example done in the different ways:

memory = NOUN * (vision * ~BLUE).convert(memory)
memory = NOUN * memory[vision * ~BLUE]
memory = NOUN * (vision * ~BLUE).memory
memory = NOUN * (vision * ~BLUE).to(memory)

[tcstewar]

Ah, one more constraint to add to the syntax. It is sometimes useful to be a little more explicit about the rules of the conversion from one vocab to another. For example, I might want to only consider certain terms, or I might want it to only work for terms that already exist in both vocabs (or in either vocab). So I might want something like

memory = NOUN * (vision * ~BLUE).to(memory, terms=[CIRCLE, TRIANGLE])

or

memory = NOUN * (vision * ~BLUE).to(memory, require_common_terms=True)

That consideration leans me back away from a = a[b*c] toward a = (b*c).to(a).

[xchoo]

the .to() syntax is also consistent with the function call format we have been using for dot()

[jgosmann]

memory = NOUN * (vision * ~BLUE).memory

looks like an attribute with the name memory as accessed.

I still like .to()/.morph()/.whatever() best and it gives us probably the most flexibility for extensions to do the transform in different ways. It is also consistent with regular intuitions from Python syntax. If one considers the semantic pointers objects, it is a function call converting the pointer to a different vocabulary. In contrast, most of the other examples look like regular Python syntax, but are conceptually quite different.

[tcstewar]

I still like .to()/.morph()/.whatever() best and it gives us probably the most flexibility for extensions to do the transform in different ways. It is also consistent with regular intuitions from Python syntax.

Yup, that fits with my thoughts too.

Here's a fifth option to consider:

memory = NOUN * (vision * ~BLUE).transform_to(memory)

It's longer to type, but more readable, I think. And I think it'll be pretty rare for people to do a = b.transform_to(a) * c.transform_to(a) but even then it might be worth the extra verbosity....

[tcstewar]

(Note: one reason I like transform_to() is that it gives a bit of a hint that it really is just a transform matrix being generated, while convert() or to() are a bit more ambiguous as to what they actually do)

[xchoo]

It's longer to type, but more readable, I think. And I think it'll be pretty rare for people to do a = b.transform_to(a) * c.transform_to(a) but even then it might be worth the extra verbosity....

I don't think it's as rare as you think. 😉 The moment you start messing around with wanting different vocabularies for different parts of your model, which is not uncommon (it's in Spaun, Dan used it in his models, Eric is using it in stuff he's doing at telluride, and even folks at the summer school did this too).

[xchoo]

That said, I'm still okay with .transform_to. or trfm_to (that's how i abbreviate 'transform' haha)

[bjkomer]

If it's something that will be used a lot, especially in one line, I'd prefer something shorter like to()

When I see trfm_to for some reason the first thing I think of is 'terraform to'

[Seanny123]

I've changed my mind and decided that I also enjoy the .to option, although
I'm also fine with more verbosity.

On Wednesday, July 15, 2015, Brent Komer notifications@github.com wrote:

If it's something that will be used a lot, especially in one line, I'd
prefer something shorter like to()

When I see trfm_to for some reason the first thing I think of is
'terraform to'

—
Reply to this email directly or view it on GitHub
#789 (comment).

[drasmuss]

I think I like transform_to the best. That's the only one that I think someone might be able to get an intuition for just by looking at the statement. And even if using different vocabularies is (somewhat) common, what matters is how many times you connect between those vocabularies (and want to apply the transform, meaning that the vocabularies are different dimensions). And I think that is rare enough that we can afford to be a little verbose.

[hunse]

I'm also curious to know when this will be used. The reason I ran into problems is that I had a cleanup memory a with a limited vocabulary, and I tried to do b = a * THING. The default was to look up THING in a's vocab, and it wasn't there, causing an error. However, I'm not sure it would be any better to look up THING in b's vocab, because what if b was a cleanup instead and you wanted to do b = a * ~THING. It would be nice for these examples just to work, without having to use transform_to (assuming everything is the same dimensionality).

It's not clear to me what is the point of Buffer's having vocabs different from the default vocabs in the model. It seems to me that if everything ran through the default vocabs, then as long as a and b are the same size in the above examples, we can just look up THING in the default vocab, and things would be fine. If they're not the same size, then we'd throw an error (and the user would have to use transform_to to specify what they want to happen).

[drasmuss]

I think that is exactly what is being proposed here, that by default there won't be any transform, and everything will have to use the same vocabulary. If the user doesn't want that, they have to be explicit about it (using the syntax discussed above). Unless I'm misunderstanding things.

[tcstewar]

@hunse what you're describing there is an issue to do with write-only subvocabularies, which is related to this but not the thing being addressed here.

I believe that what would happen with read-only subvocabs is that subvocabularies would be treated as the same vocab, and your example would work fine without a transform_to (i.e. you could just do b = a * THING. THING would get created in the master vocab, but if either b or a were read-only it wouldn't be a problem.

What this proposal is about is the ability to map from one vocab to another completely separate vocab. That's the only situation a transform_to would be needed in. And notice that those separate vocabs might have the same dimensionality, so we can't just rely on the number of dimensions as an indicator (and that's also why we can't just have one master default vocab for every dimensionality).

[drasmuss]

@hunse's comment made me think, when you do e.g. a = b.transform_to(a), what will be the behaviour if a and b have the same dimensionality? Should it create a vector THING in a and then compute the transform matrix from b.THING to a.THING (like it would if they were different dimensionalities)? Or should it just set a.THING = b.THING? I think the latter would be nicer, in that it avoids adding unnecessary transforms, but it does mean that there will be some coupling between a and b.

Edit: wrote the example wrong

[s72sue]

Is there something special about cleanup memory regarding vocabs? My
understanding was that when doing b = a * THING, if a particular sp
(THING) didn't
exist in a's vocab, then a new sp should be automatically created for it.
Did you get an error or did it silently create a new sp in a's vocab giving
you unexpected behaviour for your model?

Also, with the new changes being proposed, the plan is to allow the simple
examples (i.e., models having everything with the same dimensionality) to
work without forcing the user to use transform_to.

One reason for allowing different vocabs might be to allow the usage of
components having different dimensions in the model. Right now, if you do
not explicitly create a vocabulary for the model, it uses the same vocab
for all the components as long as they all have the same dimensions.
However, if you create a vocab explicitly, then you can pass in that vocab
to the other components so that they will use the same vocab.

PS: I have my vote for transform_to. However, we could even just use change_to
since its shorter.
'transform' is definitely a better suited term to be used here though.

On Wed, Jul 15, 2015 at 1:24 PM, Eric Hunsberger notifications@github.com
wrote:

I'm also curious to know when this will be used. The reason I ran into
problems is that I had a cleanup memory a with a limited vocabulary, and
I tried to do b = a * THING. The default was to look up THING in a's
vocab, and it wasn't there, causing an error. However, I'm not sure it
would be any better to look up THING in b's vocab, because what if b was
a cleanup instead and you wanted to do b = a * ~THING. It would be nice
for these examples just to work, without having to use transform_to
(assuming everything is the same dimensionality).

It's not clear to me what is the point of Buffer's having vocabs different
from the default vocabs in the model. It seems to me that if everything ran
through the default vocabs, then as long as a and b are the same size in
the above examples, we can just look up THING in the default vocab, and
things would be fine. If they're not the same size, then we'd throw an
error (and the user would have to use transform_to to specify what they
want to happen).

—
Reply to this email directly or view it on GitHub
#789 (comment).

[xchoo]

@drasmuss In your example (a = THING.transform_to(b)), it will give an error unless the vocab of a and the vocab of b are the same. The transform_to syntax is to convert a THING into the (destintation's) vocabulary.

[drasmuss]

Yeah I wrote the example wrong at first, it should have been a = b.transform_to(a) (I edited it real quick but github makes that hard to notice).

[xchoo]

@s72sue The issue with cleanup memories is that when you create it, you pass in a vocabulary to use. And if the transform_to operator adds things into the vocabulary, when you go and plot the outputs, you will find that the vocabulary is not longer the one you used for the cleanup memory (because it has a whole bunch of extra terms). The solution was to create read-only vocabularies (see #699). However, this created new problems when the transform_to operator wanted to add things to a read-only vocabulary.

[xchoo]

@drasmuss Um. Right now, the transform_to function treats a and b as things with different vocabularies. so THING.a != THING.b.

[tcstewar]

@hunse's comment made me think, when you do e.g. a = b.transform_to(a), what will be the behaviour if a and b have the same dimensionality? Should it create a vector THING in a and then compute the transform matrix from b.THING to a.THING (like it would if they were different dimensionalities)? Or should it just set a.THING = b.THING? I think the latter would be nicer, in that it avoids adding unnecessary transforms, but it does mean that there will be some coupling between a and b.

I very strongly disagree with this. If I have two separate vocabularies a and b and I happen to make them have the same dimensionality, I really don't want that to suddenly change the interactions between the vocabularies. If I have defined two separate vocabs then they are two separate vocabs regardless of whether or not they happen to be exactly the same dimensions.

[drasmuss]

I very strongly disagree with this

😵 I don't feel that strongly either way, so happy to be overruled

[jgosmann]

The SPA compiler (now merged into new-spa) adds more expliciteness about the conversion between vocabularies. Does this resolve this issue or was there something else hidden in this long discussion?

[jgosmann]

I skimmed the discussion again and it was mostly about the syntax and naming of stuff I already implemented in the SPA compiler. Of course I forgot about the decisions made here. So this is the naming and syntax I ended up with:

state_with_vocab_b = translate(state_with_vocab_a)
state_with_vocab_b = translate(state_with_vocab_a, vocab_b)
state_with_vocab_b = translate(state_with_vocab_a, vocab=vocab_b)
state_with_vocab_b = reinterpret(state_with_vocab_a)
state_with_vocab_b = reinterpret(state_with_vocab_a, vocab_b)
state_with_vocab_b = reinterpret(state_with_vocab_a, vocab=vocab_b)

If someone dislikes that, I'm happy to discuss other names (@xchoo seemed to be an favour of morph instead of translate). The function style syntax could also be changed to

state._with_vocab_b = state_with_vocab_a.translate()
state._with_vocab_b = state_with_vocab_a.translate(vocab_b)

with a little bit effort.

[tcstewar]

I like the translate(x) and reinterpret(x) syntax and names. :) And just to make sure I understand, if you leave out the vocab= part, it'll just convert it to whatever the obvious thing is?

[jgosmann]

It converts it to the inferred "type"/vocabulary. I hope it's sort of obvious in most cases ...