Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regex parser : No parsed group detected when using two skills with regex_matcher #872

Closed
iobreaker opened this issue Mar 9, 2019 · 5 comments · Fixed by #873
Closed
Labels

Comments

@iobreaker
Copy link
Contributor

Hi

Description

When activating more than one skill using regex (example : hello skill and welcome skill or loudnoise skill) if the second skill use group name in regex parser, no group are detected and passed into message.regex.group

Steps to Reproduce

1- clone the last version of opsdroid
2- activate regex parser in config file
3- activate hello skill in config file
4- activate welcome skill in config file (parsing sentence like : welcome iobreaker to paris)
5- Activate telegram connector (I tested using telegram)
6- send : welcome iobreaker to paris
7- error will appear in opsdroid logs

Expected Functionality

  • Groups should be present in message.regex.groups()

Experienced Functionality

You can use this skill (welcome skill) i created to facilitate the debug

from opsdroid.matchers import match_regex

import logging

_LOGGER = logging.getLogger(__name__)

@match_regex(r'(welcome)(\s*)(?P<name>\w+)(\s*)(to|at)(\s*)(?P<location>\w+)', case_sensitive=False)
async def welcome(opsdroid, config, message):
	_LOGGER.debug("=========> triggering welcome skill")
	_LOGGER.debug("=========> group('name')     = {}".format(message.regex.group("name")))
	_LOGGER.debug("=========> group('location') = {}".format(message.regex.group("location")))

When sending the message welcome iobreaker to paris without activating hello skill all works fine

INFO opsdroid.web: Started web server on http://127.0.0.1:8080
DEBUG opsdroid.connector.telegram: {'update_id': 246644582, 'message': {'message_id': 286, 'from': {'id': 111222555, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 111222555, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552162058, 'text': 'welcome hicham to paris'}}
DEBUG opsdroid.core: Parsing input: welcome hicham to paris
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.welcome: =========> triggering welcome skill
DEBUG opsdroid-modules.skill.welcome: =========> group('name')     = hicham
DEBUG opsdroid-modules.skill.welcome: =========> group('location') = paris

When activating the hello skill and the welcome skill the error happened : IndexError: no such group

INFO opsdroid.web: Started web server on http://127.0.0.1:8080
DEBUG opsdroid.parsers.crontab: Running crontab skills
DEBUG opsdroid.connector.telegram: {'update_id': 246644579, 'message': {'message_id': 280, 'from': {'id': 111222555, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 111222555, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552161920, 'text': 'welcome hicham to paris'}}
DEBUG opsdroid.core: Parsing input: welcome hicham to paris
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.welcome: =========> triggering welcome skill
DEBUG opsdroid.connector.telegram: Responding with: Whoops there has been an error
DEBUG opsdroid.connector.telegram: Successfully responded
DEBUG opsdroid.connector.telegram: Responding with: Check the log for details
DEBUG opsdroid.connector.telegram: Successfully responded
ERROR opsdroid.core: Exception when running skill 'welcome'
Traceback (most recent call last):
  File "/Users/hicham/Developments/Bots/opsdroid-iobreaker/opsdroid/opsdroid/core.py", line 336, in run_skill
    await skill(self, config, message)
  File "/Users/hicham/Library/Application Support/opsdroid/opsdroid-modules/skill/welcome/__init__.py", line 12, in welcome
    _LOGGER.debug("=========> group('name') = {}".format(message.regex.group("name")))
IndexError: no such group

Perhaps I am missing something !! but it's a strange as a behaviour/isue

Versions

  • Opsdroid version: v0.14.1+28.gfed4488.dirty
  • Python version: 3.7.2

Configuration File

## Parsers
parsers:
#  ## Regex (http://opsdroid.readthedocs.io/en/stable/matchers/regex)
  - name: regex
    enabled: true
## Skill modules
skills:
  - name: welcome
    path: /Users/hicham/Developments/Bots/opsdroid-data/skills/skill-welcome
    debug: true
    no-cache: true
  
  ## Dance (https://github.com/opsdroid/skill-dance)
#  - name: dance

  ## Hello (https://github.com/opsdroid/skill-hello)
  - name: hello

Regards

@iobreaker
Copy link
Contributor Author

Ok i thnink i found why this issue is present;

We can understand the problem trough the debug log bellow

DEBUG opsdroid.connector.telegram: {'update_id': 246644589, 'message': {'message_id': 304, 'from': {'id': 112233445, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 112233445, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552228481, 'text': 'welcome hicham to paris'}}
DEBUG opsdroid.core: Parsing input: welcome hicham to paris
DEBUG opsdroid.parsers.regex: =========> Working on Skill function : <function welcome at 0x10e904ea0>
DEBUG opsdroid.parsers.regex: ==================> Working on matcher : {'regex': {'expression': '(welcome)(\\s*)(?P<name>\\w+)(\\s*)(to|at)(\\s*)(?P<location>\\w+)', 'case_sensitive': False, 'score_factor': 0.6}}
DEBUG opsdroid.parsers.regex: ===========================> Test regex against message is : <re.Match object; span=(0, 23), match='welcome hicham to paris'>
DEBUG opsdroid.parsers.regex: ===========================> Test regex groups are : {'name': 'hicham', 'location': 'paris'}
DEBUG opsdroid.parsers.regex: =========> Working on Skill function : <function hello at 0x10e950840>
DEBUG opsdroid.parsers.regex: ==================> Working on matcher : {'regex': {'expression': 'hi|hello|hey|hallo', 'case_sensitive': True, 'score_factor': 0.6}}
DEBUG opsdroid.parsers.regex: ===========================> Test regex against message is : <re.Match object; span=(8, 10), match='hi'>
DEBUG opsdroid.parsers.regex: ===========================> Test regex groups are : {}
DEBUG opsdroid.parsers.regex: =========> Working on Skill function : <function goodbye at 0x10e9509d8>
DEBUG opsdroid.parsers.regex: ==================> Working on matcher : {'regex': {'expression': "bye( bye)?|see y(a|ou)|au revoir|gtg|I(\\')?m off", 'case_sensitive': True, 'score_factor': 0.6}}
DEBUG opsdroid.parsers.regex: ===========================> Test regex against message is : None
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.welcome: =========> triggering welcome skill
DEBUG opsdroid.connector.telegram: Responding with: Whoops there has been an error
DEBUG opsdroid.connector.telegram: Successfully responded
DEBUG opsdroid.connector.telegram: Responding with: Check the log for details
DEBUG opsdroid.connector.telegram: Successfully responded
ERROR opsdroid.core: Exception when running skill 'welcome'
Traceback (most recent call last):
  File "/Users/hicham/Developments/Bots/opsdroid-iobreaker/opsdroid/opsdroid/core.py", line 336, in run_skill
    await skill(self, config, message)
  File "/Users/hicham/Library/Application Support/opsdroid/opsdroid-modules/skill/welcome/__init__.py", line 12, in welcome
    _LOGGER.debug("=========> group('name')     = {}".format(message.regex.group("name")))
IndexError: no such group

First problem :

The function async def parse_regex(opsdroid, skills, message): in regex.py parser file look for any function with a regex macher and then parse the message against it.

the problem is that this function alter message object each time here :

                if regex:
                    message.regex = regex  #<----------
                    matched_skills.append({
                        "score": await calculate_score(
                            opts["expression"], opts["score_factor"]),
                        "skill": skill,
                        "config": skill.config,
                        "message": message
                    })

Second problem :

The function async def parse(self, message): in core.py always use the original message object instead of the one sent by parse_regex function

                if ranked_skills:
                    tasks.append(
                        self.eventloop.create_task(
                            self.run_skill(ranked_skills[0]["skill"],
                                           ranked_skills[0]["config"],
                                           message)))  #<----------

I fixed those two problems like this :

rexex.py file (making a shallow copy of the message object before playing with it)

...
...
import copy

async def parse_regex(opsdroid, skills, message):
...
...
                if regex:
                    currentMessage = copy.copy(message)
                    data.regex = regex
                    matched_skills.append({
                        "score": await calculate_score(
                            opts["expression"], opts["score_factor"]),
                        "skill": skill,
                        "config": skill.config,
                        "message": currentMessage
                    })
    return matched_skills

core.py file ( passing ranked_skills[0]["message"] instead of message)

    async def parse(self, message):
        """Parse a string against all skills."""
        self.stats["messages_parsed"] = self.stats["messages_parsed"] + 1
        tasks = []
        if message is not None:
            if str(message.text).strip():
                _LOGGER.debug(_("Parsing input: %s"), message.text)

                tasks.append(
                    self.eventloop.create_task(parse_always(self, message)))

                unconstrained_skills = await self._constrain_skills(
                    self.skills, message)
                ranked_skills = await self.get_ranked_skills(
                    unconstrained_skills, message)
                if ranked_skills:
                    tasks.append(
                        self.eventloop.create_task(
                            self.run_skill(ranked_skills[0]["skill"],
                                           ranked_skills[0]["config"],
                                           ranked_skills[0]["message"])))

        return tasks

Now all is working fine

INFO opsdroid.web: Started web server on http://127.0.0.1:8080
DEBUG opsdroid.connector.telegram: {'update_id': 246644673, 'message': {'message_id': 485, 'from': {'id': 112233445, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 112233445, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552237760, 'text': 'welcome ali to paris'}}
DEBUG opsdroid.core: Parsing input: welcome ali to paris
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.welcome: =========> triggering welcome skill
DEBUG opsdroid-modules.skill.welcome: =========> message groups     = {'name': 'ali', 'location': 'paris'}


DEBUG opsdroid.connector.telegram: {'update_id': 246644674, 'message': {'message_id': 486, 'from': {'id': 112233445, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 112233445, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552237768, 'text': 'welcome hicham to paris'}}
DEBUG opsdroid.core: Parsing input: welcome hicham to paris
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.welcome: =========> triggering welcome skill
DEBUG opsdroid-modules.skill.welcome: =========> message groups     = {'name': 'hicham', 'location': 'paris'}


DEBUG opsdroid.connector.telegram: {'update_id': 246644675, 'message': {'message_id': 487, 'from': {'id': 112233445, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 112233445, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552237782, 'text': 'hi'}}
DEBUG opsdroid.core: Parsing input: hi
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.hello: =====> Executing hello function
DEBUG opsdroid.connector.telegram: Responding with: Hi IOBreaker
DEBUG opsdroid.connector.telegram: Successfully responded


DEBUG opsdroid.connector.telegram: {'update_id': 246644676, 'message': {'message_id': 489, 'from': {'id': 112233445, 'is_bot': False, 'first_name': 'IOBreaker', 'language_code': 'en'}, 'chat': {'id': 112233445, 'first_name': 'IOBreaker', 'type': 'private'}, 'date': 1552237785, 'text': 'goodbye'}}
DEBUG opsdroid.core: Parsing input: goodbye
DEBUG opsdroid.core: Processing parsers...
DEBUG opsdroid-modules.skill.hello: =====> Executing goodbye function
DEBUG opsdroid.connector.telegram: Responding with: Au revoir IOBreaker
DEBUG opsdroid.connector.telegram: Successfully responded

I would like to have your feedback before doing a PR with the fix

Regards

@jacobtomlinson
Copy link
Member

Thanks for this! That sounds like a reasonable solution.

I would be keen for you to check the other parsers to see if they suffer form the same problem when raising your PR.

@iobreaker
Copy link
Contributor Author

I checked some parsers, those seems ok.
I will create a PR with the modifications to correct this issue.

Proposition : It will be a good thing to allow user to choose between a full match (re.fullmatch) or a simple match (re.match) This will prevent a confusion between two skills the first parsing hi and the second parsing hihi (it's just an example :-) )
It will add an other layer of security and give the used teh ability to execute the skill only if the message match exactly the patten.

@jacobtomlinson
Copy link
Member

Sounds good. There is already a kwarg to set whether the regex is case sensitive or not. You could add this as another one.

@iobreaker
Copy link
Contributor Author

ok, to keep subjects separated, i will work on the fix first and after on adding the choices between a full or a simple match

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants