Issue with setup:di:compile

[stu177]

When magento_dir is set to '.' in my deploy.php, if I run dep build I get the following error:

The command "cd . && (/usr/bin/php .//bin/magento setup:di:compile)" failed.                                                    
                                                                                                                                  
Exit Code: 1(General error)                                                                                                     
                                                                                                                                  
Working directory: /home/user/www/site.uk                                                                               
                                                                                                                                  
Output:                                                                                                                         
================                                                                                                                
Warning: include(/home/user/www/site.uk/vendor/composer/../../generated/code/Magento/Framework/App/ResourceConnection/  
Proxy.php): failed to open stream: No such file or directory in /home/user/www/site.uk/vendor/composer/ClassLoader.php on line 444                                                                                                                    
                                                                                                                                  
                                                                                                                                  
Error Output:                                                                                                                   
================  

If I set the magento_dir to an absolute path, it works for the local build but when pushing to production, this path is incorrect and breaks deployment. If I then change it back to '.' it works for production.

It seems like it should just '.' for local but it throws the ClassLoader.php error. Any ideas?

[jalogut]

Hi @stu177

Could you execute the command in verbose mode -vvv and paste here the output? Are you in the real directory or are you inside a symlink to the real project directory?

You can also try adding the following in your deploy.php file:

• set('magento_bin', 'bin/magento');

If after that, you have the same issue. Please also copy the verbose output here, so we can compare it.

[stu177]

Hi @jalogut ,

Thanks for the response and help.

I've just ran it with -vvv and what's odd is it runs fine with that verbose flag, but doesn't work without it.

Here is my output from the -vvv run:

➤ Executing task build
➤ Executing task deploy:vendors
[localhost] > if hash unzip 2>/dev/null; then echo 'true'; fi
[localhost] < true
[localhost] > if hash composer 2>/dev/null; then echo 'true'; fi
[localhost] < true
[localhost] > command -v 'composer' || which 'composer' || type -p 'composer'
[localhost] < /usr/local/bin/composer
[localhost] > command -v 'php' || which 'php' || type -p 'php'
[localhost] < /usr/bin/php
[localhost] > cd . && /usr/bin/php /usr/local/bin/composer install --verbose --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader
[localhost] < Loading composer repositories with package information
[localhost] < Installing dependencies from lock file
[localhost] < Dependency resolution completed in 0.013 seconds
[localhost] < Analyzed 714 packages to resolve dependencies
[localhost] < Analyzed 5006 rules to resolve dependencies
[localhost] < Nothing to install or update
[localhost] < Generating optimized autoload files
• done on [localhost]
✔ Ok [4s 302ms]
➤ Executing task config:remove-dev-modules
• done on [localhost]
✔ Ok [0ms]
➤ Executing task files:compile
[localhost] > cd . && (/usr/bin/php ./bin/magento setup:di:compile)
[localhost] < Compilation was started.
[localhost] < %message% 0/7 [>---------------------------]   0% < 1 sec 74.5 MiB%message% 0/7 [>---------------------------]   0% < 1 sec 74.5 MiB
[localhost] < Proxies code generation... 0/7 [>------------------------]   0% < 1 sec 74.5 MiB
[localhost] < Proxies code generation... 1/7 [===>---------------------]  14% 1 sec 80.5 MiB
[localhost] < Repositories code generation... 1/7 [===>------------------]  14% 1 sec 80.5 MiB
[localhost] < Repositories code generation... 2/7 [=====>--------------]  28% 6 secs 203.0 MiB
[localhost] < Service data attributes generation... 2/7 [====>---------]  28% 6 secs 203.0 MiB
[localhost] < Service data attributes generation... 3/7 [======>-------]  42% 6 secs 205.0 MiB
[localhost] < Application code generator... 3/7 [======>-------]  42% 6 secs 205.0 MiB
[localhost] < Application code generator... 4/7 [========>-----]  57% 14 secs 227.0 MiB
[localhost] < Interceptors generation... 4/7 [========>-----]  57% 14 secs 227.0 MiB
[localhost] < Interceptors generation... 5/7 [==========>---]  71% 29 secs 253.0 MiB
[localhost] < Area configuration aggregation... 5/7 [==========>---]  71% 29 secs 253.0 MiB
[localhost] < Area configuration aggregation... 6/7 [============>-]  85% 45 secs 341.0 MiB
[localhost] < Interception cache generation... 6/7 [============>-]  85% 45 secs 341.0 MiB
[localhost] < Interception cache generation... 7/7 [==============] 100% 1 min 341.0 MiB
[localhost] < Generated code and dependency injection configuration successfully.
• done on [localhost]
✔ Ok [72s 632ms]
➤ Executing task files:static_assets
[localhost] > cd . && (/usr/bin/php ./bin/magento setup:static-content:deploy en_GB -f --exclude-theme=Magento/blank)
[localhost] < Deploy using quick strategy
[localhost] < frontend/Magento/blank/en_GB            2266/2266           ============================ 100% %  10 secs             
[localhost] < adminhtml/Magento/backend/en_GB         2359/2359           ============================ 100% %  5 secs              
[localhost] < frontend/Magento/luma/en_GB             2282/2282           ============================ 100% %  8 secs
[localhost] < Execution time: 34.920444965363
• done on [localhost]
✔ Ok [39s 466ms]
➤ Executing task files:permissions
[localhost] > cd . && (cd . && chmod -R g+w var vendor pub/static pub/media app/etc && chmod u+x bin/magento)
• done on [localhost]
✔ Ok [10s 819ms]
➤ Executing task artifact:package
[localhost] > cd . && (if [ -d artifacts ]; then echo 'true'; fi)
[localhost] < true
[localhost] > cd . && (tar --exclude-from=/home/site/www/site.uk/vendor/jalogut/magento2-deployer-plus/recipe/magento_2_2/../../config/artifact.excludes -czf artifacts/artifact.tar.gz .)
• done on [localhost]
✔ Ok [46s 588ms]
✔ Ok [46s 588ms]

On a slightly different note, do you know what's the best practice with this deployment script to handle sensitive config values on the production server?

I've read through the Magento Docs on it but it's not very clear on how to actually implement this.

Update:

I've just had it happen again with -vvv and then tried it again without changing anything and it ran so it seems to be intermittent which is very strange.

Update 2:

So it's not intermitted but fails on the first run then succeeds on the second run. This seems to do the same every time, fails once, then runs.

Update 3:

The command it fails on:

cd . && (/usr/bin/php .//bin/magento setup:di:compile)

If I run that by itself, it runs everytime without a problem.

[stu177]

Ok so after looking into this, I think I've figured out what is going on but my knowledge of composer and autoloaders isn't great so I might be way off.

It seems it has something to do with composer messing with the generated/code folder.

In the composer.json there is the PSR-0 section:

"psr-0": {
    "": [
        "app/code/",
        "generated/code/"
    ]
},

If I remove generated/code/ from the composer.json, the issue goes away.

I think what is happening is it's including the generated/code folder in the autloader, however when setup:di:compile is run, it deletes the generated/code folder, then attemps to run which fails because files required by the autoloader are now missing?

I can replicate this by just running these two commands from the build process one after another:

cd . && /usr/bin/php /usr/local/bin/composer install --verbose --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader

cd . && (/usr/bin/php bin/magento setup:di:compile -vvv)

Is there an issue with removing generated/code/ from the composer.json or is there another solution to this? It's also possible this is entirely incorrect and I'm way off.

[erfanimani]

Ha, I had exactly the same issue @stu177 . It would fail every other deployment.

I worked around it by running rm -rf generated/* before running dep build

It's only happening on https://buddy.works/ for me, because they reuse your container for successive deployments unless explicitly told not to. On projects running Bitbucket Pipelines I don't have this issue since they always create a new container and just hydrate Composer cache.

I think deleting generated directory is a good workaround because you're not speeding DI compilation up anyway by having compiled DI before. It just overwrites it every single time.

[stu177]

@erfanimani Ah yeh, should I be doing this instead of removing generated/code from the autoloader? Or will they both achieve the same thing?

@jalogut Any thoughts on this? Should this be changed in repo?

[erfanimani]

@stu177 to be honest, I actually have no idea what removing generated/code from the autoloader does — my thought processes was to rather not tinker with that and just get rid of the directory... I'll keep a keen eye on this though to see if there's a better way and to understand the problem better.

[jalogut]

Hi @stu177 and @erfanimani

Glad to see that you are looking further into this issue. I investigated a little a bit about the reason of this psr-0 and I found that the change was added on version 2.2.6.

• Commit: magento/magento2@c91a25a#diff-b5d0ee8c97c7abd7e3fa29b9a27d1780
• PR: Add generated code to the psr-0 autoloader section so when optimizing… magento/magento2#16435

I honestly never faced this issue because we do not have generated/code in our composer.json files. I was not aware of this change, so we never added it. I would say, there is no problem removing it. As the PR says, this brings just a small performance improvement.

On the other hand, it would be great if you continue looking on the issue and find a better solution. Unfortunately I do not have much time now for that.

I hope these insights help you a little bit ;)

[jalogut]

Hi again guys,

After reading again your comments, I think @erfanimani is completely right here:

On projects running Bitbucket Pipelines I don't have this issue since they always create a new container and just hydrate Composer cache.

I think deleting generated directory is a good workaround because you're not speeding DI compilation up anyway by having compiled DI before. It just overwrites it every single time.

Could any of you maybe create a PR that does a rm -rf generated/* before executing setup:di:compile. I think that would be the best solution. If you cannot create the PR I would try to add it soon.

[erfanimani]

Thanks for coming back to this issue @jalogut  — I can create a PR Monday at work.

I'm not entirely sure what the best approach would be though. Are you OK with me adding a new invocation after this line? https://github.com/jalogut/magento2-deployer-plus/blob/master/recipe/magento_2_2.php#L28

Otherwise I can add it directly to the files:generate task, but I would have to define the location of the generated directory first as it would run on every Magento version (and before 2.2 generated is stored in the var dir).

[jalogut]

Hi,

Yes, let’s add a new invocation there for now as the issue seems to happen only when using the build task. If we notice more issues in the future, we can add it into the files:generated later on. So for now I would go for your first idea. Thx

[scottsb]

FYI, found the root issue here and reported it to Magento (see reference link above).

[jalogut]

Hi @scottsb thanks for the info and for taking the time to report it to magento ;)