Skip to content

Commit

Permalink
fix: Slack log service (#2537)
Browse files Browse the repository at this point in the history
### Description

<!--Add a description of your PR here-->

This updates the Slack ``--log-service``  feature added in #170.
The ``Slacker`` package is no longer maintained, and legacy slack tokens
are no longer generatable.
I've replace it with the officially supported [Slack
SDK](https://github.com/slackapi/python-slack-sdk) (see ``setup.cfg``),
preserving the original functionality with minimal adjustments.

This has been minimally tested for functionality. However, the original
implementation has a couple issues (which this PR **does not** address)
-

1. Lack of `docs/` on how to set up the ``--log-service slack`` feature
via Slack
2. Logic is not robust - e.g. when `msg["done"] == msg["total"]`, this
does not mean the workflow is complete (checkpoints)
3. The implementation of ``--log-service slack`` is synchronous, meaning
that Snakemake workflows will wait for the implemented `log_handler`
function to finish. For normal print statements, this isn't an issue,
but repeated calls of Slack WebAPI's `postMessage` can significantly
slow things down. Depending on the workflow (Snakefile), this can become
noticeable.
4. The implementation does not comply with Slack WebAPI's rate limiting.

The current implementation will look like this - messages on slack are
sent by the user, using a user OAuth token.

![image](https://github.com/snakemake/snakemake/assets/43220811/69e3b335-13fc-4af1-8538-8d65920f9494)

I've already addressed points 2, 3, and 4 in a private repository using
``--log-handler`` that should be integrable into this current
implementation, but it has not been robustly tested (integration can
happen at a later PR).

![image](https://github.com/snakemake/snakemake/assets/43220811/97576f52-1c46-4237-806a-e823bd76f9f7)

Thanks!

### QC
<!-- Make sure that you can tick the boxes below. -->

* [ ] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [ ] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).

---------

Co-authored-by: Johannes Köster <johannes.koester@tu-dortmund.de>
  • Loading branch information
vincentczhou and johanneskoester committed Dec 13, 2023
1 parent 76f79f5 commit 26eb4ba
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 12 deletions.
2 changes: 1 addition & 1 deletion setup.cfg
Expand Up @@ -63,7 +63,7 @@ install_requires =
conda-inject >=1.3.1,<2.0

[options.extras_require]
messaging = slacker
messaging = slack_sdk
pep =
eido
peppy
Expand Down
23 changes: 12 additions & 11 deletions snakemake/logging.py
Expand Up @@ -86,38 +86,39 @@ def decorate(self, record):

class SlackLogger:
def __init__(self):
from slacker import Slacker
from slack_sdk import WebClient

self.token = os.getenv("SLACK_TOKEN")
if not self.token:
print(
"The use of slack logging requires the user to set a user specific slack legacy token to the SLACK_TOKEN environment variable. Set this variable by 'export SLACK_TOKEN=your_token'. To generate your token please visit https://api.slack.com/custom-integrations/legacy-tokens."
"The use of slack logging requires the user to set a user specific slack User OAuth token to the SLACK_TOKEN environment variable. Set this variable by 'export SLACK_TOKEN=your_token'. To generate your token please visit https://api.slack.com/authentication/token-types#user."
)
exit(-1)
self.slack = Slacker(self.token)
self.slack = WebClient(self.token)
# Check for success
try:
auth = self.slack.auth.test().body
auth = self.slack.auth_test().data
except Exception:
print(
"Slack connection failed. Please compare your provided slack token exported in the SLACK_TOKEN environment variable with your online token at https://api.slack.com/custom-integrations/legacy-tokens. A different token can be set up by 'export SLACK_TOKEN=your_token'."
"Slack connection failed. Please compare your provided slack token exported in the SLACK_TOKEN environment variable with your online token (app). This token can be tested at https://api.slack.com/methods/auth.test/test. A different token can be set up by 'export SLACK_TOKEN=your_token'."
)
exit(-1)
self.own_id = auth["user_id"]
self.error_occured = False
self.slack.chat_postMessage(
channel=self.own_id, text="Snakemake has connected."
)

def log_handler(self, msg):
if msg["level"] == "error" and not self.error_occured:
self.slack.chat.post_message(
self.own_id, text="At least one error occured.", username="snakemake"
if "error" in msg["level"] and not self.error_occured:
self.slack.chat_postMessage(
channel=self.own_id, text="At least one error occured."
)
self.error_occured = True

if msg["level"] == "progress" and msg["done"] == msg["total"]:
# workflow finished
self.slack.chat.post_message(
self.own_id, text="Workflow complete.", username="snakemake"
)
self.slack.chat_postMessage(channel=self.own_id, text="Workflow complete.")


class WMSLogger:
Expand Down

0 comments on commit 26eb4ba

Please sign in to comment.