Linux: Crash when creating window if more than 1024 file descriptors are open

[fvallee-bnx]

Subject of the issue

My (linux) application using SFML is opening many files (not using SFML at all) then create a SFML Window. Everything works fine if I open less than FD_SETSIZE (1024) files.
If I open more than 1024 files (I'm using ulimit -n 4096) the application crashes in SFML when creating the window.

Your environment

• Linux (Ubuntu 20.04)
• SFML 2.5.1 (ubuntu package or compiled from 2.5.1 tag)
• gcc 9.3
• ulimit -n 4096

Steps to reproduce

Using SFML Release build, any app opening files then creating a window:

#include <SFML/Graphics.hpp>

int main()
{
    /* do something to open more than 1024 files so file descriptor values are > 1024 */
....

    sf::RenderWindow window(sf::VideoMode(1280, 720), "Minimal, complete and verifiable example");
    window.setFramerateLimit(60);

    while (window.isOpen())
    {
        sf::Event event;
        while (window.pollEvent(event))
        {
            if (event.type == sf::Event::Closed)
                window.close();
        }

        window.clear();
        window.display();
    }
}

Expected behavior

No crash and SFML window created

Actual behavior

Crash with "*** buffer overflow detected ***: terminated" message.

Crash comes from hasMonitorEvent() in src/SFML/Window/Unix/JoystickImpl.cpp

     int monitorFd = udev_monitor_get_fd(udevMonitor);
     fd_set descriptorSet;
     FD_ZERO(&descriptorSet);
     FD_SET(monitorFd, &descriptorSet);
     timeval timeout = {0, 0};
      return (select(monitorFd + 1, &descriptorSet, NULL, NULL, &timeout) > 0) &&
               FD_ISSET(monitorFd, &descriptorSet);

In my case monitorFd is > FD_SETSIZE and FD_SET crashes (FD_SET / select crash when the file descriptor is > FD_SET_SIZE is a well know linux issue)

For some reason I can't explain I don't reproduce the crash with a debug SFML build (i.e. CMAKE_BUILD_TYPE=Debug) even if the monitorFd has the exact same value.

[fvallee-bnx]

FYI as I don't care about joysticks I've added a simple

if (monitorFd > FD_SETSIZE) return false;

check to workaround the issue and it seems good enough

[eXpl0it3r]

The mentioned code snippet:

SFML/src/SFML/Window/Unix/JoystickImpl.cpp

Lines 251 to 263 in 87e84bc

	bool hasMonitorEvent()
	{
	// This will not fail since we make sure udevMonitor is valid
	int monitorFd = udev_monitor_get_fd(udevMonitor);
	fd_set descriptorSet;
	FD_ZERO(&descriptorSet);
	FD_SET(monitorFd, &descriptorSet);
	timeval timeout = {0, 0};
	return (select(monitorFd + 1, &descriptorSet, nullptr, nullptr, &timeout) > 0) &&
	FD_ISSET(monitorFd, &descriptorSet);
	}

[eXpl0it3r]

Fixed with #1941 on the 2.6.x branch and will later be merged down to master