KqueueSelector and PollSelector behaves differently with namedpipes

[ahnjaeshin]

Bug report

I think the selectors.KqueueSelector, and selectors.PollSelector behave differently in Mac OS X.
I tested the following code in Python 3.10.4, Mac OS X (12.5) Apple Sillicon.

Below is a simple code that creates a named pipe, opens two descriptors for read (non-block) and write(non-block).
I register a EVENT_READ event on the read file descriptor, and write some message bytes to the pipe.
When the message is small (<8192 b), both Kqueue and Poll invokes a selector event.
However, for large messages, Poll works, but hangs with Kqueue.

import os
import selectors

def test_should_return(selector: selectors.BaseSelector, msg: bytes):
    pipe = "pipe.fifo"

    os.mkfifo(pipe)
    f1 = os.open(pipe, os.O_RDONLY | os.O_NONBLOCK)
    f2 = os.open(pipe, os.O_WRONLY | os.O_NONBLOCK)

    try:
        selector.register(f1, selectors.EVENT_READ)

        os.write(f2, msg)
        assert selector.select()  # should return as pipe can be read now

        print("Passed!")

    finally:
        os.close(f1)
        os.close(f2)
        os.remove(pipe)


test_should_return(selectors.PollSelector(), b"m")                       # Passes
test_should_return(selectors.KqueueSelector(), b"m")                 # Passes
test_should_return(selectors.PollSelector(), b"m" * 10000)          # Passes
test_should_return(selectors.KqueueSelector(), b"m" * 10000)    # Hangs

I expected that for all four cases, the function should return normally (without the selector.select function blocking). However, for Kqueue selector and full-buffer writes to the named pipe leads to an infinite block.

Your environment

Python 3.10.4, Mac OS X (12.5) Apple Sillicon.

[ronaldoussoren]

I get the same on macOS 13.0.1. Interestingly enough the hang happens when the buffer size is larger than the capacity of a pipe: "m"*8192 works fine, "m"*8193 hangs.

This appears to be a bug in macOS, the C program below also hangs and works fine if I change the define for SIZE to 1 (or some other value less than 8193). I have not yet filed an issue with Apple about this, will do so tomorrow.

The best we can do is to add a warning about this problem to the documentation (both for selector and select).

repro.c

#include <sys/types.h>
#include <sys/event.h>
#include <sys/time.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>

#define PIPE "pipe.fifo"

#define SIZE 8193

int main(void)
{
    int r;
    ssize_t n;

    r = mkfifo(PIPE, 0600);
    if (r == -1) {
        perror("mkfifo");
        }

    int kq = kqueue();
    if (kq == -1) {
        perror("kqueue");
        goto error;
    }

    int fd1 = open(PIPE, O_RDONLY|O_NONBLOCK);
    if (fd1 == -1) {
        perror("open 1");
        goto error;
    }

    int fd2 = open(PIPE, O_WRONLY|O_NONBLOCK);
    if (fd2 == -1) {
        perror("open 1");
        goto error;
    }

    struct kevent ev;
    EV_SET(&ev, fd1, EVFILT_READ, EV_ADD|EV_ENABLE, 0, 0, 0);

    r = kevent(kq, &ev, 1, NULL, 0, NULL);
    if (r == -1) {
        perror("first kevent");
        goto error;
    }

    struct kevent ch;

    char buf[SIZE];
    memset(buf, 'm', SIZE);

    n = write(fd2, buf, SIZE);
    if (n == -1) {
        perror("write");
        goto error;
    }
    printf("wrote %ld\n", n);

    struct timespec timeoutspec = { 5, 0 };

    r = kevent(kq, NULL, 0,  &ch, 1, &timeoutspec);
    if (r == -1) {
       perror("kevent");
       goto error;
    }
    printf("done %d %d\n", r, SIZE);
    if (r == 1) {
        n = read(fd1, buf, n);
        printf("read %d\n", (int)n);
    } else {
        printf("nothing ready to read\n");
    }

error:
    unlink(PIPE);
    return 1;
}

[ronaldoussoren]

Apple issue: FB11816962

[ronaldoussoren]

This issue is fixed on the first beta of macOS 14 (23A5257q), both for the original report and for my C-based reproducer.

[sorcio]

@ronaldoussoren I don't know if Apple ever gave feedback on the ticket; but it seems they acknowledged and fixed an issue which looks exactly like this one, according to the latest XNU source release. The error can't be reproduced on FreeBSD (it has a different implementation for fifos). Should this be closed?

[ronaldoussoren]

I haver got a response to my ticket, but that's par for course. As this is an OS bug that's fixed in the current release we might as well close the issue.

I've also looked at our documentation for selectors and select and those don't mention other (older) issues with these APIs on macOS, it is IMHO not worthwhile to document this issue in the stdlib documentation.