Rename a folder in cloud storage of google cloud (volume mount) [PHP] .

[firloming]

Why are you starting this discussion?

Question

What GitHub Actions topic or product is this about?

Workflow Deployment

Discussion Details

Hi,

I have :
Warning: rename(/path/to/A,/path/to/B): Too many open files in /var/www/html/path/to/file.php on line 22

It work fine in my local machine, how can i fix it in cloud storage (volume mount) of CLOUD RUN app ?
I know I can create a new folder and move all the files into it, can you tell me about another way, please ?

Doker file :

# Use the official PHP image.
# https://hub.docker.com/_/php

FROM php:8.4-apache

RUN a2enmod rewrite
RUN docker-php-ext-install mysqli pdo pdo_mysql

RUN apt-get update && apt-get install -y \
    zlib1g-dev \
    libzip-dev \
    unzip
RUN docker-php-ext-install zip

# Install system dependencies for GD with WebP support
RUN apt-get update && apt-get install -y \
    libjpeg-dev \
    libpng-dev \
    libwebp-dev \
    && rm -rf /var/lib/apt/lists/*
# Configure and install the GD extension with WebP support
RUN docker-php-ext-configure gd --with-jpeg --with-webp
RUN docker-php-ext-install -j$(nproc) gd

# Configure PHP for Cloud Run.
# Precompile PHP code with opcache.
RUN docker-php-ext-install -j "$(nproc)" opcache
RUN set -ex; \
  { \
    echo "; Cloud Run enforces memory & timeouts"; \
    echo "memory_limit = -1"; \
    echo "max_execution_time = 0"; \
    echo "; File upload at Cloud Run network limit"; \
    echo "upload_max_filesize = 32M"; \
    echo "post_max_size = 32M"; \
    echo "; Configure Opcache for Containers"; \
    echo "opcache.enable = On"; \
    echo "opcache.validate_timestamps = Off"; \
    echo "; Configure Opcache Memory (Application-specific)"; \
    echo "opcache.memory_consumption = 32"; \
  } > "$PHP_INI_DIR/conf.d/cloud-run.ini"
# Copy in custom code from the host machine.
WORKDIR /var/www/html
COPY . .
#COPY . /var/www/html/

# Ensure the webserver has permissions to execute index.php
RUN chown -R www-data:www-data /var/www/html/

# Use the PORT environment variable in Apache configuration files.
# https://cloud.google.com/run/docs/reference/container-contract#port
RUN sed -i 's/80/${PORT}/g' /etc/apache2/sites-available/000-default.conf /etc/apache2/ports.conf

# Configure PHP for development.
# Switch to the production php.ini for production operations.
# RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"
# https://github.com/docker-library/docs/blob/master/php/README.md#configuration
RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini"

[26nikhilkumar]

rename() on a Cloud Storage FUSE mount (Cloud Run volume) won’t reliably rename a directory. It often explodes with “Too many open files” because FUSE has to touch a ton of objects to simulate a folder rename. Cloud Storage isn’t a POSIX FS; “folders” are just prefixes. ([Google Cloud]1)

Here are solid ways to fix it (pick one):

1) If the directory is small: let FUSE do it (set a limit)

Cloud Run now lets you pass FUSE mount options. Set a safe cap so FUSE will only attempt directory renames when the folder has ≤ N descendants:

gcloud beta run services update YOUR_SERVICE \
  --execution-environment gen2 \
  --add-volume name=GCS,type=cloud-storage,bucket=YOUR_BUCKET,mount-options="rename-dir-limit=500" \
  --add-volume-mount volume=GCS,mount-path=/mnt/gcs

rename-dir-limit = max children allowed for a dir rename; default is 0 (disabled). Use this only for small trees; for big ones you’ll still hit descriptor/latency issues. ([Google Cloud]1)

2) The robust way today (works on any bucket): move-by-prefix in PHP

Do a server-side copy → delete of all objects under A/ to B/ with the Storage PHP client:

use Google\Cloud\Storage\StorageClient;

$bucketName = 'YOUR_BUCKET';
$srcPrefix  = 'A/';     // trailing slash
$dstPrefix  = 'B/';

$storage = new StorageClient(); // ADC on Cloud Run
$bucket  = $storage->bucket($bucketName);

// Stream all objects under A/
foreach ($bucket->objects(['prefix' => $srcPrefix]) as $obj) {
    $src = $obj->name(); // e.g. A/foo/bar.txt
    $dst = $dstPrefix . substr($src, strlen($srcPrefix)); // B/foo/bar.txt

    // Server-side copy, then delete source
    $bucket->object($src)->rewrite($bucket->object($dst));
    $bucket->object($src)->delete();
}
// optional: remove the placeholder "A/" object if present
$placeholder = $bucket->object($srcPrefix);
if ($placeholder->exists()) $placeholder->delete();

This mirrors what gsutil -m mv does and avoids the FUSE rename pitfalls. ([Google Cloud]2)

3) The best UX going forward: use an HNS bucket (true folders)

If you create your bucket with Hierarchical Namespace (HNS) enabled, you can rename/move folders as a single atomic metadata operation via the folders.rename API or gcloud storage mv. No mass file touching, no descriptor blowups. (HNS is enabled at bucket creation time.) ([Google Cloud]3)

Example (HNS bucket):

# rename folder inside the same bucket
gcloud storage mv gs://YOUR_BUCKET/A gs://YOUR_BUCKET/B

Or call the JSON API folders.rename from PHP/HTTP. ([Google Cloud]4)

---

Why your rename() worked locally but not on Cloud Run

Local disk does an atomic directory rename. On a GCS FUSE mount, a directory “rename” is emulated and can open many files; doing this on non-trivial trees often leads to “Too many open files.” If you must stay with FUSE, cap it with rename-dir-limit or switch to prefix moves via the API. ([Google Cloud]1)

If you want, tell me roughly how many files live under A/ and whether you can use an HNS bucket; I’ll tailor the exact command or a more parallel PHP mover for your case.

[firloming]

Thank you .
I'm very happy.

[dee480217-cloud]

Someone hack someone fuck off my life