Request: Redirect STDERR/STDOUT to DMs

[noirscape]

Not everything I have to test is ran on commands, some of it runs on events instead.

It would be useful for jishaku to have a command to temporarily enable sending the contents of anything send to stderr and stdout to the owners DMs.

A similar feature already exists in Discord-Selfbot which could be used as a base for this.

[ioistired]

Just use _ctx.author.send in your event

[noirscape]

Uhh that's not what I mean.

Basically, let's say I have something in a cog that runs on the on_message() event.

If that prints to stdout or stderr, I want to receive the output in DMs.

_ctx.author.send is for jsk py as far as I am aware.

[ioistired]

Oh anything, including outside of jsk py? Any reason you can't check the logs on your VPS for that?

[noirscape]

The way I manage my processes unfortunately splits off stdout and stderr, meaning it's not really feasible for me to check VPS logs and sometimes I don't notice an exception because it silently appears because I don't watch my VPS logs all the time.

[Gorialis]

Jishaku intentionally doesn't redirect stdout or stderr automatically because it is not async safe - these interfaces were designed "as is" close to 40 years ago with no inherent consideration of the complexity that we get out of modern systems. To have any hope of regulating it "safely" - I would have to take over the entire control flow of your script.

Since that is unreasonable within the design specifications of Jishaku, here are a number of reasons why I will not/can not capture stdout or stderr in good conscience:

Unix streams have no concept of 'context'

When text is written to stdout, the only information that comes with it is the text itself. There are no categories, labels, or denominators under which a program can definitively determine what a given text buffer was for.

This therefore makes it impossible for a capturer to distinguish "text intended for capture" and "unrelated other stdout content". Consider for example that during your REPL session, your logger pushes a critical error related to your bot's functionality to stderr. Not only will this wash up as unintended noise in your REPL session - but as redirection is non-forwarding, it will prevent this error from turning up in your actual logs - giving you no record that it happened.

There is only one stdout/stderr

Due to the simplicity of basic Unix streams as described above - there can be only one stdout in a program, and one handler of it.

This primarily has significance in non-forwarding capture due to its potential to make a capture meaningless - even if I redirect_stdout, if something else does it immediately after me, then just like how I am intercepting text from reaching the real stdout, the new acquirer will stop any text from reaching my capturer, and I will receive nothing.

Asynchronous environments do not guarantee reverse disengagement of context managers

Global-bound synchronous context managers are often built on the assumption that context managers will be exited in the reverse order that they are entered, due to the limitations of the with syntax and therefore the assumed flow in a fully synchronous context.

Let's first consider the simplest situation:

with A() as a:
    ...

In this code, the context manager flow looks like this:

>> A  (enter)
<< A  (exit)

This flow is very simple, and there's very little that can go wrong here.

Now, let's consider a more complicated structure:

with A() as a:
    with B() as b:
        ...

    with C() as c:
        with D() as d:
            ...

For this code, the context manager flow looks like this:

>> A  (enter)
    >> B  (enter)
    << B  (exit)
    >> C  (enter)
        >> D  (enter)
        << D  (exit)
    << C  (exit)
<< A  (exit)

From this, we can begin to notice a pattern. Whenever any context manager is exited, it is always the last-entered non-exited context manager, or more broadly:

In a fully synchronous environment, for any given context manager, that context manager cannot exit until all contained context managers have exited first.

Now, let's discuss the behavior of redirect_stdout. In plain English, redirect_stdout:

• Stores the current value of sys.stdout for later restoration
• Overwrites sys.stdout with a string buffer to serve as the fake stdout for the duration of the context manager.
• Overwrites sys.stdout once the context manager exits with the version stored at the beginning.

We can see how this system preserves a sensible stack, even in a nested case:

with redirect_stdout(...) as A:
    with redirect_stdout(...) as B:
        ...
    ...

Even if we were to create a complicated nest of context managers as before, due to context manager exit always occuring in the reverse order of enter, we will always end like we started.

That is, if we are in a synchronous context.

When we bring asyncio into the equation, things get complicated. Coroutines can suspend and resume during execution, and other coroutines can operate in the interim. While context manager order is guaranteed on a local scale, it is no longer guaranteed on a global scale.

Let's now consider a pair of coroutines that look like this:

# These coroutines are functionally the same, but have different symbol names for illustrative purposes.
async def foo():
    with redirect_stdout(...) as A:
        await asyncio.sleep(5)
        ...

async def bar():
    with redirect_stdout(...) as B:
        await asyncio.sleep(5)
        ...

Thanks to asyncio, we can run both of these coroutines at the same time. Let's give that a try.

>> A  (enter)
    >> B  (enter)
    << A  (exit)
<< B  (exit)

As we can see, when we break the exit guarantee, things stop working as intended. In this case, even though both coroutines have now returned, sys.stdout is left as a dead text buffer. From now on, not only will nothing make it to the regular stdout, but the remaining text buffer will sit there as a memory sink, slowly increasing memory usage every time anything writes to stdout.

While this seems like a manufactured scenario, such a thing could easily happen by accident if one runs multiple REPLs at the same time, as suspension is allowed every time a coroutine awaits.

As the maintainer of a package I wish to preserve as "production-safe", it's pretty clear why I can't let things like this happen automatically and silently just through basic use.

If you really want to redirect stdout, if such a specific case where it is necessary arises, implicit imports allow you to do so in one block:

with io!.StringIO() as f, contextlib!.redirect_stdout(f):
    print("hi")

    await _author.send(f.getvalue())  # send stdout contents to author