Empty blacklist entries block script

[Allard-Chris]

Hello,

I'm working on a script to add/remove blacklist automatically.
To optimize my API calls, I get the list of current blacklist.
But if the blacklist entries is empty, my script is stuck.

How to reproduce

The blacklist entries need to be empty.

query = BlacklistQuery("TEST-CLUSTER")
for bl in query.fetch_as_element():
    print(bl)

You will be stuck at the statement query.fetch_as_element()

Where is the problem

If you call query.fetch_as_element(), you will call the method inside smc_monitoring/monitors/blacklist.py

Extract:

for list_of_results in clone.fetch_raw(**kw):
    for entry in list_of_results:
        data = entry.get('bldata')
        data.update(**entry.get('blid'))
        yield BlacklistEntry(**data)

The problem in inside the method fetch_raw from smc_monitoring/models/query.py

Extract:

with SMCSocketProtocol(self, **self.sockopt) as protocol:
            for result in protocol.receive():
                if 'records' in result and result['records'].get('added'):
                    yield result['records']['added']
                    iteration += 1

if records is not in result, you will never reach the 'yield' statement and block here.

You will need to change the function with something like :

with SMCSocketProtocol(self, **self.sockopt) as protocol:
    for result in protocol.receive():
        if 'records' in result and result['records'].get('added'):
            yield result['records']['added']
            iteration += 1
        else:
            return

Or maybe there is something I don't understant an how works the API.
Thanks in advance.

Regards, Chris.

[Allard-Chris]

Erratum, my fix is not good.
It's more something like that :

iteration = 0
with SMCSocketProtocol(self, **self.sockopt) as protocol:
    for result in protocol.receive():
        if 'records' in result:
            if result['records']['added'] != []:
                yield result['records']['added']
                iteration += 1
        else:
            yield from ()
            iteration += 1
        if iteration == max_recv:
            protocol.abort()

[Allard-Chris]

Ok, the problem was how i used the function.
When we call "query.fetch_as_element()" for BlacklistQuery, we must send the argument "max_recv=1" for example.
With that, we don't hang endlessly.

[thomasdevulder]

You can use parameters to customize websocket results:
1 - max_recv : the number of record you want to retrieve, fetch_as_element will return after
2 - query_timeout: the inactivity period in second after the method will return

In this example, the method will return after 10 seconds of inactivity during record reception (or if blacklist entries is empty)

    print("Retrieve all BlacklistEntry elements using smc_monitoring")
    query = BlacklistQuery(ENGINENAME)
    for element in query.fetch_as_element(max_recv=5, query_timeout=10):
        print("{} {} {} {} {} {}".format(element.engine,
                                         element.href,
                                         element.source,
                                         element.destination,
                                         element.protocol,
                                         element.duration))

Tell me if that could solve your problem.
I am going to try your fix suggestion

Thomas

[Allard-Chris]

Hello Thomas,
Thanks for your reply.
Yes, it's solve my problem. I'm pretty noob about functions and websocket using yields statement. So i was not prepared about the blacklist function that hang if we received nothing.

Thank you for your help.

Regards, Chris.