Import local libs as extensions #228

Fr6jDJF · 2020-11-15T14:24:33Z

Basic loader for local libraries with optional looping tasks.

Custom libraries placed in the local-lib folder(configuration.default.ini) can be installed as in example "local_lib_pipinstal.sh" with:
venv/bin/python -m pip install -e local-lib/testlib
As it will be installed as *.egg-link in venv/../../site-packages, updating is a matter of updating the files in local-lib folder.
list with libs to be loaded in configuration.default.ini
lib has it's own libconfiguration.ini
lib has it's own lang translation files *example lib holds help message(!test command only) that extend the main help string.
No lib reload(just restart bot)
No auto dependency install(TestLib performs one basic check on it's own for feedparser example), need to figure out if/how to have those installed by the setup files

TerryGeng · 2020-11-15T14:35:45Z

Hello. Thanks for contributing to our project.

I have noticed you made a lot of changes. Can you provide details about the functionalities you'd like to add? I guess what you are aiming for is adding additional commands and hooks in the main loop of the bot by adding "libraries". Did I get it right?

TerryGeng · 2020-11-15T14:38:00Z

local_lib_pipinstal.sh

+
+# 202001108
+
+venv/bin/python -m pip install -e local-lib/test


We don't use pip to manage our project, since @azlux has his own auto-update engine. I think it would be messy if we choose to use pip to manage the lib install...

I was unaware of this update method (I have auto update disabled since I manually inject code to obtain functionalities / one of my reasons for this request).

TerryGeng · 2020-11-15T14:40:01Z

mumbleBot.py

@@ -33,13 +34,15 @@


 class MumbleBot:
-    version = 'git'


Please keep this line. Azlux has the routine for replacing this line with the correct version when doing a release. The 'git' stands for the development version and will be dealt with

botamusique/mumbleBot.py

Line 209 in 0140863

def get_version(self):

.

TerryGeng · 2020-11-15T14:43:44Z

local-lib/testlib/testlib/typical.py

+    return inner_decorator
+
+
+class TestLib():


I think this Lib does something like reading RSS feed. Can you elaborate more? And if this Lib does do some thing, then we should give it a meaningful name like RSSLib.

This lib was pure for example/testing, I could bring it down as suggested being the example RSSLib

TerryGeng · 2020-11-15T14:44:29Z

local-lib/testlib/testlib/typical.py

+        return rssfeeds
+
+
+def loadmod(bot):


TerryGeng · 2020-11-15T14:50:33Z

mumbleBot.py

@@ -459,6 +505,23 @@ def _download(self, item):
    def loop(self):
        while not self.exit and self.mumble.is_alive():

+            # It's posible for libs to start tasks, they are managed/monitored over here.
+            for task in self.task_handle:
+                name = self.task_handle['method']['task_name']


I happen to be in the middle of considering optimizing the logic of the main loop to make it lighter. And now we have these dict lookups in the middle...

I think if you create a specific class like

from enum import Enum class TaskState(Enum): added = 0 started = 1 class TaskHandle: def __init__(self, name, state, handle): self.name = name self.state = TaskState.added self.handle = handle ...

However, I'm just wondering why you put task initialization in the main loop?

Maybe It's better to just release the loop of then already started thread tasks in loop once, this would also give better control of stopping/pause the task.
I currently placed it in the loop the get them running as late as possible, as they live quite on their own.

TerryGeng · 2020-11-15T14:51:02Z

mumbleBot.py

@@ -862,6 +924,8 @@ def start_web_interface(addr, port):
    #  Create bot instance
    # ======================
    var.bot = MumbleBot(args)
+    # load local libraries from /local-lib
+    libimport.import_all_loclibs(var.bot)


import_all_local_libs

TerryGeng

Sorry that I don't have time to do a full review. I just briefly glanced through your changes and posted some comments.

I think we still need to do a lot of clean up and functionalities tests before I can merge it.

Actually, I'd appreciate if you could start an issue and discuss your plan with us before you get started...

@azlux and I are always on IRC channel #mumble at freenode.net and feel free to drop by and have some small discussion with us :)

Fr6jDJF · 2020-11-16T16:55:35Z

Hello. Thanks for contributing to our project.

I have noticed you made a lot of changes. Can you provide details about the functionalities you'd like to add? I guess what you are aiming for is adding additional commands and hooks in the main loop of the bot by adding "libraries". Did I get it right?

Correct,
For me, it's non-general functions that interact with running services(for now I have some broadcast info to chat), but it might just as well extend to more general libs supporting the discussions on Mumble, like IMDB look ups, game server status and such.

So a.t.m I manually inject my own code to interact with my running services, what means auto update must be disabled, aging the instance. functionality like this would be the solution for that.

Started issue: #229

TerryGeng · 2020-11-19T00:36:01Z

mumbleBot.py

@@ -206,22 +203,80 @@ def ctrl_caught(self, signal, frame):

        self.exit = True

-    def get_version(self):


Can you explain this deletion?

TerryGeng · 2020-11-19T00:39:03Z

mumbleBot.py

        self.log.info("bot: music stopped. playlist trashed.")

    def stop(self):
        self.interrupt()
        self.is_pause = True
-        if len(var.playlist) > 0:


Can you explain this part?

Logics related to the loop and its state are messy and modifications, if not thoroughly tested, if prone to errors. Are you sure this will work?

As you can see, this part was introduced in fceb50e, to fix a bug. I think you need to keep the part unchanged.

- catching up

TerryGeng · 2020-11-24T07:21:59Z

Great! Thanks for the improvement.

I'd suggest you check out the ABC of python, and write a template ABC class for all Libs inside libimport.py (or I think you can rename libs into plugins and rename libimport.py into plugins.py to avoid confusion).

Another thing: I think you don't have to use force push in this PR. Using force push makes it hard for me to find out what is your improvement across each commit. You can just commit as usual and we will squash this PR before merging. What's your idea?

TerryGeng · 2020-11-24T07:23:12Z

mumbleBot.py

@@ -459,6 +516,20 @@ def _download(self, item):
    def loop(self):
        while not self.exit and self.mumble.is_alive():

+            # Release started tasks if any


What do you think of moving this part to the place between L516 and 517, outside the while loop?

That would be better as there is no active monitoring by the bot.

TerryGeng · 2020-11-24T07:23:36Z

mumbleBot.py

+                for task in self.task_handle:
+                    self.log.debug(f'bot.loop: about to release task: {task}')
+                    try:
+                        if self.task_handle[task].state[0] == util.TaskState.started:


I'm kind of confused about "started", "released" and "stopped". Can you explain a little bit?

added: init
started: thread is running
released: task may start/resume it's job
paused: task waiting for released
stopped: exit/break task loop

Any renaming suggestions, or would you rather see threads just being started to do their stuff without further "control"?

As for the interaction with the threads maybe queue is something to consider here, and feed the q object some addressed TaskHandle ,
For now I will leave it up to further discussion.

Fr6jDJF · 2020-11-24T17:59:38Z

Great! Thanks for the improvement.

I'd suggest you check out the ABC of python, and write a template ABC class for all Libs inside libimport.py

I'm not known with that, I will check that out, thank you.

(or I think you can rename libs into plugins and rename libimport.py into plugins.py to avoid confusion).

Ok, that would be:

                    shown with log.debug
                             v

/local-lib/rssplugin/rssplugin/typ_rssplugin.Rss

        ^_________^
   would translate(shown by pip freeze) in to "local-rssplugin" by pip setup.cfg

Another thing: I think you don't have to use force push in this PR. Using force push makes it hard for me to find out what is your improvement across each commit. You can just commit as usual and we will squash this PR before merging. What's your idea?

I know sorry, I'm not used to push PR, to me -f seemed the only way to get this PR updated again.

TerryGeng · 2020-11-25T13:54:38Z

I know sorry, I'm not used to push PR, to me -f seemed the only way to get this PR updated again.

Actually you can just keep pushing commits into your 'importlib' branch, and new commits will just show up here :).

-queued entries, to you can have them all on interval/single request -a way to skip big entry list "spam" on first run !getrss (set to latest)

- removed TaskState

- Removed TaskState - Renamed function register_lib => register_plugin

Fr6jDJF

updating

- Forgot to remove variable for warning *line5 "import command" is shadowed by "command" in register_command()

@command

# In order to get rid of another bot as being the argument at commands, a suggestion is to migrate old commands. # pack at mumbleBot.py.message_received / command_exc # self.cmd_handle[command_exc]['handle'](self, user, text, command_exc, parameter) # v # self.cmd_handle[command_exc]['handle'](bot=self, user=user, text=text, command_exc=command_exc, parameter=parameter) # - Requires modification typ_rssplugin.py commands for *args (to ignore it's bot object, since __init__ self.bot) *Example below @command('kwargs') # - Requires modification of all existing command.py commands for dealing with *args (to still obtain that bot object)

@command

# In order to get rid of another bot as being the argument at commands, a suggestion is to migrate old commands. # pack at mumbleBot.py.message_received / command_exc # self.cmd_handle[command_exc]['handle'](self, user, text, command_exc, parameter) # v # self.cmd_handle[command_exc]['handle'](bot=self, user=user, text=text, command_exc=command_exc, parameter=parameter) # - Requires modification typ_rssplugin.py commands for *args (to ignore it's bot object, since __init__ self.bot) *Example below @command('kwargs') # - Requires modification of all existing command.py commands for dealing with *args (to still obtain that bot object)

TerryGeng · 2021-01-21T11:18:41Z

Hello! Any update on this PR?

luca0N · 2023-08-02T23:59:21Z

Hello! Any update on this PR?

Looks like this user hasn't been active on GitHub since January 3, 2021... I would love to contribute to this PR to get it merged on botamusique, though.

@TerryGeng, could we get a checklist going to see what's left for this to get merged?

I will look into this & review the changes as soon as I have some time.

TerryGeng reviewed Nov 15, 2020

View reviewed changes

local-lib/testlib/testlib/typical.py Outdated

return rssfeeds

def loadmod(bot):

Copy link

Collaborator

TerryGeng Nov 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

load_mod.

TerryGeng reviewed Nov 15, 2020

View reviewed changes

TerryGeng suggested changes Nov 15, 2020

View reviewed changes

Fr6jDJF mentioned this pull request Nov 16, 2020

Load custom commands(plugins) #229

Closed

Fr6jDJF force-pushed the importlib branch from 687ab7b to 7e21324 Compare November 18, 2020 19:34

TerryGeng reviewed Nov 19, 2020

View reviewed changes

Import local libs as extensions

9bcf2c9

- catching up

Fr6jDJF force-pushed the importlib branch from 7e21324 to 9bcf2c9 Compare November 23, 2020 19:00

TerryGeng reviewed Nov 24, 2020

View reviewed changes

Fr6jDJF requested a review from TerryGeng November 24, 2020 18:07

wupd

5ac6f0a

-queued entries, to you can have them all on interval/single request -a way to skip big entry list "spam" on first run !getrss (set to latest)

TerryGeng force-pushed the master branch from 9d47549 to 2546e3b Compare November 29, 2020 17:15

Fr6jDJF added 4 commits November 30, 2020 20:42

Update typ_rsslib.py

3e97538

Update util.py

b5c1d11

- removed TaskState

Update mumbleBot.py

a65bccf

- Removed TaskState - Renamed function register_lib => register_plugin

Update interface.py

81d0a6b

Fr6jDJF commented Nov 30, 2020

View reviewed changes

Fr6jDJF added 2 commits November 30, 2020 20:56

Update libimport.py

75067dc

Update mumbleBot.py

8239e45

- Forgot to remove variable for warning *line5 "import command" is shadowed by "command" in register_command()

Fr6jDJF added 2 commits November 30, 2020 23:24

TerryGeng mentioned this pull request Feb 12, 2021

Metadata retrieval? #251

Closed

luca0N mentioned this pull request Aug 3, 2023

Integration with a Large Language Model like LLama #372

Closed

azlux closed this Jun 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import local libs as extensions #228

Import local libs as extensions #228

Fr6jDJF commented Nov 15, 2020 •

edited

Loading

TerryGeng commented Nov 15, 2020

TerryGeng Nov 15, 2020

Fr6jDJF Nov 16, 2020 •

edited

Loading

TerryGeng Nov 15, 2020

TerryGeng Nov 15, 2020

Fr6jDJF Nov 16, 2020

TerryGeng Nov 15, 2020

TerryGeng Nov 15, 2020

Fr6jDJF Nov 16, 2020 •

edited

Loading

TerryGeng Nov 15, 2020

TerryGeng left a comment •

edited

Loading

Fr6jDJF commented Nov 16, 2020 •

edited

Loading

TerryGeng Nov 19, 2020

TerryGeng Nov 19, 2020

TerryGeng Nov 19, 2020

TerryGeng commented Nov 24, 2020

TerryGeng Nov 24, 2020

Fr6jDJF Nov 24, 2020

TerryGeng Nov 24, 2020

Fr6jDJF Nov 24, 2020 •

edited

Loading

Fr6jDJF commented Nov 24, 2020

TerryGeng commented Nov 25, 2020

Fr6jDJF left a comment

TerryGeng commented Jan 21, 2021

luca0N commented Aug 2, 2023 •

edited

Loading

		@@ -206,22 +203,80 @@ def ctrl_caught(self, signal, frame):

		self.exit = True

		def get_version(self):

Import local libs as extensions #228

Import local libs as extensions #228

Conversation

Fr6jDJF commented Nov 15, 2020 • edited Loading

TerryGeng commented Nov 15, 2020

Choose a reason for hiding this comment

Fr6jDJF Nov 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fr6jDJF Nov 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TerryGeng left a comment • edited Loading

Choose a reason for hiding this comment

Fr6jDJF commented Nov 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TerryGeng commented Nov 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fr6jDJF Nov 24, 2020 • edited Loading

Choose a reason for hiding this comment

Fr6jDJF commented Nov 24, 2020

TerryGeng commented Nov 25, 2020

Fr6jDJF left a comment

Choose a reason for hiding this comment

TerryGeng commented Jan 21, 2021

luca0N commented Aug 2, 2023 • edited Loading

Fr6jDJF commented Nov 15, 2020 •

edited

Loading

Fr6jDJF Nov 16, 2020 •

edited

Loading

Fr6jDJF Nov 16, 2020 •

edited

Loading

TerryGeng left a comment •

edited

Loading

Fr6jDJF commented Nov 16, 2020 •

edited

Loading

Fr6jDJF Nov 24, 2020 •

edited

Loading

luca0N commented Aug 2, 2023 •

edited

Loading