mdp learning using AALpy

[roiDaniela]

AALpy/Examples.py

Line 171 in 84b2835

def random_mdp_example(num_states, input_len, num_outputs, n_c=20, n_resample=1000, min_rounds=10, max_rounds=1000):

Hello:)

I'm trying to to do the same for an example I created this mdp and tried to learn it same way:
https://github.com/roiDaniela/AALpy/blob/examples_and_tests/myExample.py

but the prediction of probabilities where different than expected

maybe my configuration is not propriate?

https://github.com/roiDaniela/AALpy/blob/examples_and_tests/graphs/learned.pdf

https://github.com/roiDaniela/AALpy/blob/examples_and_tests/graphs/original.pdf

thanks

[emuskardin]

Hi,

you defined a deterministic automaton, therefore learning returned deterministic automaton.

    s1.transitions['a'].append((s2, 0.2))
    s1.transitions['b'].append((s1, 0.35))
    s1.transitions['c'].append((s3, 0.35))
    s1.transitions['d'].append((s1, 0.1))

In this snipped taken from your example, you can see that 'a' will lead to s2 100% of the time, as the alternative transition for input 'a' from s1 is note defined. This is done for all state/transition pairs, thus this is a deterministic automaton.

In MDP, sum of transitions for each element of input alphabet should be 100%.

As stated, in your example, upon executing 'a' from s1, only possible output is s2.
You would need something like this for it to be an MDP:

    s1.transitions['a'].append((s2, 0.2))
    s1.transitions['a'].append((s3, 0.8))
    s1.transitions['b'].append((s1, 0.35))
    s1.transitions['b'].append((s4, 0.65))
    s1.transitions['c'].append((s3, 0.35))
    s1.transitions['c'].append((s1, 0.65))
    s1.transitions['d'].append((s1, 0.1))
    s1.transitions['d'].append((s5, 0.9))

All best,
Edi

[roiDaniela]

@emuskardin thanks!

Is it possible to add probability to the transactions in dfa, so the 100% will divided to the alphabet?

[roiDaniela]

I think this will be good workaround for my need

`s1.transitions['a'].append((s2, 0.2))
s1.transitions['a'].append((s1 0.8))

s1.transitions['b'].append((s1, 1))

s1.transitions['c'].append((s3, 0.35))
s1.transitions['c'].append((s1, 0.65))

s1.transitions['d'].append((s1, 1))`