Start of refactoring of openpsi framework

[amebel]

Main functionality added is the ability to instantiate actions using groundings from context.

This break OpenPsiUTest. Need to resolve #2898 before fixing the unit test failure, or might not be needed as refactoring continues.

[linas]

I assume that you are doing this for performance reasons. However, there are still multiple issues related to demands and modulators that were never fully resolved, and it might not be a good idea to convert those into C++ as that would make the long-term fix harder.

[ngeiswei]

What would be nice is to have openpsi inherits from a more abstract "action selection goal driven" component, that doesn't contains the notions of urges and demands, merely goals and actions.

We'll probably find that such a component will be re-used by more and more by other parts as time goes.

[linas]

Nil, yes, I absolutely agree. I wanted to go one step farther, and define a way of saying "in this situation, try to compose these actions to achieve this goal, but in this other situation, pick from this different set of actions (to achieve the same goal, or maybe a different goal?)" The "in this situation" would be the demand.

This is easy to say, and its fairly easy to imagine. But when I tried to refactor the openpsi code to do this, it was hard, I tripped over multiple issues. I still think that having a working urge/demand subsystem is generically a good and useful thing do do, but we just don't have that yet.

[amebel]

@linas
Modulators are part of (opencog openpsi dyanmics) have no intention of touching that for now.

This changes are being made not only for performance but also for using the variable groundings for instantiating the actions. Why not use the PutLink? b/c it assumes a one to one mapping which is not necessarily the case when it comes to ghost rules.

Yes, in terms of representation having two type of ImplicationLink, where one is used to define the relation between action and goals, and the other between context and action seems more flexible. And would be helpful for experimenting with reinforcement learning. If you guys agree on that representation, then making the changes is simple.

@ngeiswei
Openpsi is being that component with the additional component called demand, which are merely tags for psi-rules, as it is being used presently. (opencog openpsi dynamics) is the component that is dealing with emotion modeling part.

[ngeiswei]

@amebel regarding the 2 types of implication links you mean something like

1. Context & do(Action) -> Goal
2. Context & want(Goal) -> should-do(Action)

right? No problem as long as you introduce predicates want, should-do, etc, to make it clear what it means. If these predicates are axiomatized, then PLN should be able to turn one form into another.

[linas]

Nil, Yes, the old opensi wrote all rules in the form 2. Yes, the new openpsi should consider 1. and 2. to be equivalent. In general, we will want to chain these rules together in a sequence e.g. to climb stairs, take multiple steps, one after another. But also in parallel: e.g. to make someone laugh, make a funny face and fall down. I do not yet understand how this chaining will work.

Naively, it would be

context=standing-on-ground & climb-one-step -> context=standing on first step
context=standing-on-first-step & climb-one-step -> context=standing-on-second-step

and somehow each of these gets closer to the goal of "stand-on-top-of-staircase" But I've never seen any detailed writeup of what the atomese looks like for this, and how PLN or the chainers solve this.

[amebel]

@ngeiswei what I am thinking is more like

1. context -> can_do(action)
2. did(action) -> goal

This way we will be able to define actions that do not achieve a goal(aka high-level goal), but only change the state of the world(aka context). So how to relate the action sequences to a goal? I think that is what URE is good at(aka planning), and is where the opportunity for experimenting on reinforcement learning is at. I think it also intersects with what you are doing with inference control, but I might be wrong.

[linas]

I like this suggestion. It makes chaining clear.

[ngeiswei]

But the probability of achieving a certain goal by doing a certain action depends on a context, perhaps you want to say

Context
  C
  did(action) -> goal

Or perhaps you consider that the truth of C has already been evaluated and so you can express P(G|A) only.

What PLN needs to reason about planning is taking into account temporal knowledge, so that it can make the difference between a context that is true now, or that will be true in the future, etc. For that it needs PredictiveImplication and such, if it has, and has axioms about them, it should be able to construct plans, possibly even efficiently by providing control rules to the URE.

[amebel]

Yes that was what I was thinking, did(action) should be true to imply the goal. But, I see your point as well. If I am not mistaken, the interpretation you have is from the perspective of planning, and I was thinking from the action perspective.

If I understood you correctly then, what you are saying is that if we have

C1 -> can_do(A)
C2 -> can_do(A)
did(A) -> goal

then we have,

Context<tv1>
  C1
  did(A) -> goal

Context<tv2>
  C2
  did(A) -> goal

That assumes you know your goal. A goal is just a preferred context to be in. But what if you don't know what the goal is(perception is weak), or you don't know the action to the goal, or it takes multiple actions with no clear sub-goals to reach it, how should we represent it?

The present representation in openpsi is

Implication
  SequentialAnd
      Context
      Action
  Goal

which I think is lacking, of course I could be missing something.

openpsi-dynamics uses PredictiveImplication. I understand that ImplicationLink declare direction of relationships, but don't imply temporal order. I wonder whether it would be a good idea to delegate that information to the implicant and implicand such that the same set of pln-rules could be used over any set on which a comparator is defined.

[ngeiswei]

I think it all comes down to using temporal knowledge properly, otherwise you'll have incorrect assertions, for instance if right now C has 90% to be true and C & A -> G is generally 50% true, then you can infer that A -> G has 40% to be true, right now. To say that you need to wrap that in a AtTimeLink <25%> Implication A G. Once you have that for various actions, A1, ... An, then you can hand these probabilities (or generally truth values) to your action selection code to make the decision.

As far as I see things, the goal should always known, the uber goal at least. For the sub-goals it's up to the system to discover them, and they're not different from contexts that you may reach along the way.

[ngeiswei]

@amebel said:
The present representation in openpsi is

Implication
  SequentialAnd
      Context
      Action
  Goal

The sequential and I suppose is only used to not shuffle the order and know where is the action and where is the context, right?

[amebel]

The SequentialAnd is a hack to imply PredictiveImplication Context Action. The action inherits from (Concept "OpenPsi: action"), so it is easily identifiable.