How to ensure mirror syncing has been done?

[peter279k]

As title, I've set up a Composer v2 mirror syncing on my own server with supervisor process yesterday.

When I check my mirror storage size today, it seems that the size has about 83 MB.

It seems that this mirror syncing work is very slow and I think it depends on networking issues.

Is it possible to check the mirror syncing has been done or other approaches about checking the mirror syncing progress?

Thanks :).

[Seldaek]

You can run it manually once with php mirror.php --resync -v It will then output lots of stuff so you can see progress somehow, and when it exits it's done and you can then run it as usual with only --v2

[peter279k]

Great. When I start running php mirror.php --resync -v at once, it present many messages.

And the mirror storage seems to be increased.

I think this command should be added on README and describe this command usage.

[Seldaek]

It's weird though because normally if it did not complete the initial sync it should trigger a resync automatically when you run --v2.

[peter279k]

I got this exception message when I run php mirror.php --resync -v command:

job failed at 2020-11-11 10:23:10
[ErrorException] inflate_add(): data error
PHP Fatal error:  Uncaught ErrorException: inflate_add(): data error in /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseTrait.php:349
Stack trace:
#0 [internal function]: {closure}(2, 'inflate_add(): ...', '/var/www/repov2...', 349, Array)
#1 /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseTrait.php(349): inflate_add(Resource id #4386829, '\x83\x80tK;\xF1B\xA3\xB3\x1ADcu\xE0Z...')
#2 [internal function]: Symfony\Component\HttpClient\Response\CurlResponse::stream(Array, NULL)
#3 /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseStream.php(42): Generator->next()
#4 /var/www/repov2.packagist.tw/mirror.php(278): Symfony\Component\HttpClient\Response\ResponseStream->next()
#5 /var/www/repov2.packagist.tw/mirror.php(204): Mirror->downloadV2Files(Array)
#6 /var/www/repov2.packagist.tw/mirror.php(700): Mirror->resync(16050891035460)
#7 {main}
  thrown in /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseTrait.php on line 349

Fatal error: Uncaught ErrorException: inflate_add(): data error in /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseTrait.php:349
Stack trace:
#0 [internal function]: {closure}(2, 'inflate_add(): ...', '/var/www/repov2...', 349, Array)
#1 /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseTrait.php(349): inflate_add(Resource id #4386829, '\x83\x80tK;\xF1B\xA3\xB3\x1ADcu\xE0Z...')
#2 [internal function]: Symfony\Component\HttpClient\Response\CurlResponse::stream(Array, NULL)
#3 /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseStream.php(42): Generator->next()
#4 /var/www/repov2.packagist.tw/mirror.php(278): Symfony\Component\HttpClient\Response\ResponseStream->next()
#5 /var/www/repov2.packagist.tw/mirror.php(204): Mirror->downloadV2Files(Array)
#6 /var/www/repov2.packagist.tw/mirror.php(700): Mirror->resync(16050891035460)
#7 {main}
  thrown in /var/www/repov2.packagist.tw/vendor/symfony/http-client/Response/ResponseTrait.php on line 349
* Connection #1625 to host repo.packagist.org left intact
* Connection #1624 to host repo.packagist.org left intact

What's that problem?

[Seldaek]

I have never seen that.. Seems like an issue with gzipped responses and http2 perhaps. I guess most likely caused by network failure somehow.

Can you retry see if it goes through this time? It keeps the data locally between runs so it should not redownload everything.

[peter279k]

After running php mirror.php --resync -v command again, it seems that the mirror syncing has been done:

.....
RRRRRRRRRRRRRRRRRRRRRRRRRRRR??RR?RR???????????????????????????????R?RR?????????????????????M?????
Downloaded 566737 files

I think the error I mention is the temporary error.