allow large buffer to list more entries per directory

[harshavardhana]

Description

allow large buffer to list more entries per directory

Motivation and Context

Default blockSize used as per Go's readdirnames()
is not quite performant and Linux needs you to pass
a large buffer to fill in lots of entries.

How to test this PR?

You need to create a directory with 1million files

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Documentation needed
• Unit tests needed
• Functional tests needed (If yes, add mint PR # here: )

[kannappanr]

LGTM

[harshavardhana]

       #define _GNU_SOURCE
       #include <dirent.h>     /* Defines DT_* constants */
       #include <fcntl.h>
       #include <stdio.h>
       #include <unistd.h>
       #include <stdlib.h>
       #include <sys/stat.h>
       #include <sys/syscall.h>

       #define handle_error(msg) \
               do { perror(msg); exit(EXIT_FAILURE); } while (0)

       struct linux_dirent {
           long           d_ino;
           off_t          d_off;
           unsigned short d_reclen;
           char           d_name[];
       };

       #define BUF_SIZE 1024

       int
       main(int argc, char *argv[])
       {
           int fd, nread;
           char buf[BUF_SIZE];
           struct linux_dirent *d;
           int bpos;
           char d_type;

           fd = open(argc > 1 ? argv[1] : ".", O_RDONLY | O_DIRECTORY);
           if (fd == -1)
               handle_error("open");

           for ( ; ; ) {
               nread = syscall(SYS_getdents, fd, buf, BUF_SIZE);
               if (nread == -1)
                   handle_error("getdents");

               if (nread == 0)
                   break;

               printf("--------------- nread=%d ---------------\n", nread);
               printf("inode#    file type  d_reclen  d_off   d_name\n");
               for (bpos = 0; bpos < nread;) {
                   d = (struct linux_dirent *) (buf + bpos);
                   printf("%8ld  ", d->d_ino);
                   d_type = *(buf + bpos + d->d_reclen - 1);
                   printf("%-10s ", (d_type == DT_REG) ?  "regular" :
                                    (d_type == DT_DIR) ?  "directory" :
                                    (d_type == DT_FIFO) ? "FIFO" :
                                    (d_type == DT_SOCK) ? "socket" :
                                    (d_type == DT_LNK) ?  "symlink" :
                                    (d_type == DT_BLK) ?  "block dev" :
                                    (d_type == DT_CHR) ?  "char dev" : "???");
                   printf("%4d %10lld  %s\n", d->d_reclen,
                           (long long) d->d_off, d->d_name);
                   bpos += d->d_reclen;
               }
           }

           exit(EXIT_SUCCESS);
       }

getdents behavior borrowed from man getdents

[klauspost]

Let's give it a try. LGTM 👍

[minio-trusted]

Mint Automation

Test	Result
mint-xl.sh	✔️
mint-large-bucket.sh	✔️
mint-fs.sh	✔️
mint-dist-xl.sh	✔️
mint-gateway-s3.sh	✔️
mint-gateway-azure.sh	✔️
mint-gateway-nas.sh	✔️
Deleting image on docker hub
Deleting image locally