Add feature detectors for MineRL

[jadeoneill]

Adds a simple & efficient computer vision system for Minecraft, and features for compass angle and reward. There is documentation in rocca/envs/wrappers/utils.py

[ntoxeg]

Do not track state as a global variable in a module - it would probably be best if you create a derived class from OpencogAgent (or modify NavigateAgent) such that the compass is tracked via an object field. Similarly, this function probably would fit better as a method of such class instead of a free-floating utility one.

[jadeoneill]

Hi, because convert_percept is called from GymWrapper I created a subclass called MineRLWrapper that contains the function and the variable. I couldn't put it in NavigateAgent because GymWrapper objects don't have access to the OpencogAgent object.

…

On Wed, 14 Jul 2021 at 08:34, Adrian Borucki ***@***.***> wrote: ***@***.**** requested changes on this pull request. ------------------------------ In rocca/envs/wrappers/utils.py <#22 (comment)>: > +last_compassAngle = None +def convert_percept(predicate, *args): + """Firstly, the MineRL environment gives us a different floating point + reward number every step. This function converts it into +1 or -1 so that + the pattern miner can find frequent patterns involving reward. + + Secondly, MineRL gives us a 2D image of the agent's view within Minecraft, + but this function gives the agent the average location of each approximate + color on screen. The colors are put in bins of 20 pixel brightness values, + and then we record the average location of the color bin on screen. Note + that this function doesn't divide the screen into blobs of one color; it + may find the average of multiple blobs of one color. Note that the pattern + miner will still have difficulty with this feature so it's a work in + progress. + + Thirdly, MineRL gives us the angle between the agent and the goal (compassAngle). + This function creates a boolean predicate for whether the angle has got closer + or not.""" + if predicate == "pov": + #print (args, type(args)) + #args = ["some image"] + from collections import defaultdict + colors = defaultdict(list) + + for y in range(0, 64): + for x in range(0, 64): + color = args[y][x] + rounded_color = tuple([subpixel // 25 * 25 for subpixel in color]) + colors[rounded_color].append((x, y)) + + #print(f"{len(colors.keys())} colors in this frame") + #args = ["some image"] + links = [] + for (color, locations) in colors.items(): + total_x = total_y = 0 + for (x, y) in locations: + total_x += x + total_y += y + links.append(AtLocationLink(mk_node("color:"+str(color)), mk_node("viewLocation:"+str((total_x//len(locations), total_y//len(locations)))))) + #print(links) + return links + + elif predicate == "Reward": + if float(args[0]) > 0: + args = [1] + elif float(args[0]) < 0: + args = [-1] + + elif predicate == "compassAngle": + global last_compassAngle + lca = last_compassAngle + current = float(args[0]) + links = [] + + if not lca is None: + if abs(0 - current) < abs(0 - lca): + links = [mk_evaluation("compassAngleCloser")] + print(links) + + last_compassAngle = current + return links + + return [mk_evaluation(predicate, *args)] Do not track state as a global variable in a module - it would probably be best if you create a derived class from OpencogAgent (or modify NavigateAgent) such that the compass is tracked via an object field. Similarly, this function probably would fit better as a method of such class instead of a free-floating utility one. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#22 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPWU2VKWILLVDCPCWIDIMDTXSPNVANCNFSM5AIIP6DA> .

[ntoxeg]

Hi, because convert_percept is called from GymWrapper I created a subclass called MineRLWrapper that contains the function and the variable. I couldn't put it in NavigateAgent because GymWrapper objects don't have access to the OpencogAgent object.
…

Right, makes sense.

[ntoxeg]

For 02_minerl_navigate_agent.ipynb (and working with notebooks in general) please remember to run nbstripout to get rid of unnecessary metadata changes. You can setup a git hook to do this automatically before committing.

[jadeoneill]

I installed nbstripout. I ran "nbstripout 02_minerl_navigate_agent.ipyn" but it didn't change anything. "git diff" is empty.

…

On Sat, 17 Jul 2021 at 01:34, Adrian Borucki ***@***.***> wrote: ***@***.**** requested changes on this pull request. For 02_minerl_navigate_agent.ipynb (and working with notebooks in general) please remember to run nbstripout <https://github.com/kynan/nbstripout> to get rid of unnecessary metadata changes. You can setup a git hook to do this automatically before committing. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#22 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPWU2WVEASQYP5K7QQMOITTYAYPPANCNFSM5AIIP6DA> .

[ntoxeg]

Ok, it looks like something is not right but we will fix this later on if needed. Otherwise, it looks good @jadeoneill

[ngeiswei]

Thanks @jadeoneill