Can we work on a better way of keeping a full trellis build under git?

[partounian]

Submit a feature request or bug report

• I've read the guidelines for Contributing to Roots Projects
• This is a feature request
• This is a bug report
• This request isn't a duplicate of an existing issue
• I've read the docs and followed them (if applicable)
• This is not a personal support request that should be posted on the Roots Discourse forums

Replace any X with your information.

---

Of course there is the roots discourse thread from December, but it has issues in different ways, either overwrites local changes or creates merge conflicts for every single change. I imagine other people have issues also, I think we should come up with a solution to his issue with the subtrees.

[partounian]

@tangrufus You have been doing some great work recently, any ideas? Or how did you setup your project(s)?

[tangrufus]

> How did you setup your projects?

More less the same as the subtrees method, but:

• trellis and bedrock are 2 different git repos
• keep .git despite the doc tells you to delete it
• i also have a branch (tangrufus/trellis@project-template) to keep all my common customization to start a project
• 3 git remotes for every project:
  • roots/trellis
  • tangrufus/trellis
  • actual origin of the project

$ git clone -o roots https://github.com/roots/trellis.git www-example-com-trellis
$ cd www-example-com-trellis
$ git remote add tangrufus https://github.com/tangrufus/trellis.git
$ hub create -p
$ git remote -v
origin	https://github.com/TangRufus/www-example-com-trellis.git (fetch)
origin	https://github.com/TangRufus/www-example-com-trellis.git (push)
roots	https://github.com/roots/trellis.git (fetch)
roots	https://github.com/roots/trellis.git (push)
tangrufus	https://github.com/tangrufus/trellis.git (fetch)
tangrufus	https://github.com/tangrufus/trellis.git (push)

## Merge my project temaplate
$ git fetch --all
$ git merge tangrufus/project-template

## Do your job...
## Commit and push to `origin` as usual

## When roots/trellis has been updated...
## Important: Reading change logs, commit messages and `git diff` are necessary
## Tips: Make use of git branches
$ git fetch --all
$ git merge roots/master

Do I recommend everyone to use this method?

No. Use it only if you are comfortable with git. Otherwise, git gives you more headache than benefits.

Should you always keep your forks update with roots/trellis?

Yes! Always!
This is actually missing from the doc: the doc tells you to delete .git but gives no instructions about updating.

However, updating Trellis is challenging, I don't think of a good and easy way that suitable for everyone.

Edit: By forks, I mean both origin and tangrufus remotes in the above example.

Avoiding merge conflicts

Since I use git a bit differently than the subtrees method, I got less merge conflicts.

Tips:

• Whenever possible, don't change existing files in roles/*
• When changing j2 templates, make use of variables
• When changing variable values, define them in group_vars/<env>/xxx.yml
• When a j2 template doesn't allow customizing with variables, submit pull requests, example: Allow overriding php-fpm params by ansible variables #856
• When you need to add more tasks:
  • Bad: change existing files in roles/*
  • Good: make a new role and add it into dev.yml / server.yml
  • Best: extract it to galaxy.ansible.com
  • see also: add provisioning-hooks directory to operate like deploy-hooks #830, Add Provisioning Hooks that Allow Custom Tasks  #882
• Whenever possible, don't use Nginx child templates because you lose all the benefits Trellis update gives you

Let say you want to change something in sites-available/example.com.conf:

• Best: define variables in group_vars/<env>/xxx.yml
• Good: template out to {{ nginx_path }}/includes.d/{{ item.key }}/xxx.conf but beware Trellis might delete your files during this task, see how I do it
• Bad: use Nginx child templates is the last resort

⚠️

• Reading change logs, commit messages and git diff are necessary
• No merge conflict doesn't mean it won't break
• Same commit doesn't mean same server, see: Normalize apt tasks #881 (comment)

The ultimate way to avoid merge conflict

Get it merged into roots/trellis

[partounian]

And just to confirm you create a second repo just for bedrock and the actual WP related stuff?

[tangrufus]

Correct.

For every project, I have 2 git repo:

• example-project-com-trellis
• example-project-com-bedrock

You need to change group_vars/<env>/wordpress_sites.yml accordingly

trellis/group_vars/production/wordpress_sites.yml

Lines 12 to 13 in db89d49

	repo: git@github.com:example/example.com.git # replace with your Git repo URL
	repo_subtree_path: site # relative path to your Bedrock/WP directory in your repo

[swalkinshaw]

Of course there is the roots discourse thread from December, but it has issues in different ways, either overwrites local changes or creates merge conflicts for every single change

If you make changes to Trellis as @tangrufus outlined, merge conflicts and overwritten local changes shouldn't happen too much.

One thing which I don't know has been mentioned on on Discourse is specifying merge strategies for specific sub paths.

So if you know that roles/ is free of local changes, you can run git merge -X theirs subtree=trellis/roles --squash trellis/master (or something like that).

-X theirs means git will automatically pick every change from trellis/master without prompting for conflicts. ours does the reverse which can be useful for certain files/dirs.

[swalkinshaw]

We definitely need some guides/docs about:

• how to properly customize/change Trellis
• how to update Trellis

[partounian]

I believe it is `-X ours` not `yours`, I'll definitely follow Tang's plan in the future, it'd probably be a headache for my pre-existing ones.

[swalkinshaw]

Correct, updated it

[partounian]

@tangrufus Hey Tang, I've almost wrapped up a setup according to your specs, except I'm stuck on a nginx-includes situation. Basically I want to add location blocks to remote servers, in my case one file to do browser caching and another to disable logging for the favicon.

I feel like browser caching should be added into Trellis and enabled when we enable cache. Thoughts? @swalkinshaw

[swalkinshaw]

We can't really do that automatically. Just because you enable fast-cgi caching doesn't mean you have a proper setup with asset digests.

[partounian]

Super sorry @swalkinshaw I was just reading other the tickets about browser caching, forgot that you have some concerns with browser caching.

[strarsis]

Couldn't we make a "child config" from the base roots config?
It works so well with themes 😄

[partounian]

@tangrufus I've been using your method for a current project and it works great. Two notes though.

It can be slightly simplified by your project's trellis having two remotes not three, one being origin and the second being your user's trellis. You would just keep project-template up-to-date and rebase off of that instead of roots remote.

And the second one is a question. Any idea how to keep bedrock up-to-date without having to deal with merge conflicts for composer.lock?

[tangrufus]

deal with merge conflicts for composer.lock?

You don't. First resolve conflicts in composer.json, then $ composer update. Let composer generates a new composer.lock for you.

[partounian]

Yup that's what I do. I know it's best practice to keep composer.lock in vc, but I wonder if it'll work fine without it being in vc. Especially since it's wp packages and not anything with other Composer dependencies.

On Sun, Dec 3, 2017, 7:01 PM Tang Rufus ***@***.***> wrote: deal with merge conflicts for composer.lock? You don't. First resolve conflicts in composer.json, then $ composer update. Let composer generate a new composer.lock for you. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#883 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACbIG0vR9isVNOYp62U5IAn_8HuayeBmks5s82CMgaJpZM4PSJjF> .

-- Thank you, Patrick Artounian

[tangrufus]

It works without composer.lock. However, the deployment will be a little bit different. Trellis may silently upgrade your wp core / plugins / themes during deploy.

Reason

Trellis runs $ composer install during deploy.
$ composer install with .lock always install the exact versions listed.
$ composer install without .lock works as $ composer update.

Workaround

If you really want to exclude .lock from git, always specify the exact package versions in composer.json. Otherwise, trellis will silently upgrade your wp core / plugins / themes during deploy.

Recommendation

Commiting your composer.lock is the best practice.
Use the workaround only if you have a very strong reason.

See:

• https://github.com/roots/trellis/blob/master/roles/deploy/hooks/build-after.yml
• https://getcomposer.org/doc/01-basic-usage.md#installing-without-composer-lock
• https://getcomposer.org/doc/articles/versions.md#exact-version-constraint

[partounian]

Yup, that is what I thought. I actually always specify the exact version, as not all minor versions can be without breaking changes.

[robrecord]

deal with merge conflicts for composer.lock?

You don't. First resolve conflicts in composer.json, then $ composer update. Let composer generates a new composer.lock for you.

To avoid unintentionally upgrading packages, use composer update --lock instead. This will only update the lock file.

[robrecord]

@tangrufus ,

Thank you for laying out your personal method of updating trellis.

Can I ask what is the benefit you see in having bedrock and trellis in two different repositories? For updating trellis it’s a simpler git operation, but beyond that?

Plus, unless I have it set up wrong, the virtualenv isn’t operational inside the bedrock directory when I set it up in two repos.

I love the idea of keeping a set of customisations in a branch of a fork, and keeping it up to date. I will be doing this with bedrock also.

Are you still doing it all this way? Any changes/improvements you could mention?

Thank you.

[strarsis]

@robrecord: My two cents here:
Each Bedrock copy is a site. Trellis is a server that primarily hosts bedrock sites.
A Bedrock site can also be hosted on non-Trellis servers and on multiple, different Trellis servers.

Bedrock-sites are made to be like 12-factor apps, with configuration, data, etc cleanly separated.
So a Bedrock-site shouldn't need to care about the server it is hosted on (besides the credentials and some PHP requirements of course).

[robrecord]

@strarsis - thank you for adding these considerations. Absolutely, I agree with this. It does seem like a there could be a natural possibility for separation here.

The thing that is bringing me back is that my trellis instances only ever reference one site, and as such are configured only for that site. They would not be useful for any other. So while the bedrock instance can exist without the trellis instance, it is not the case the other way round. Plus trellis is used to generate the .env file (though not strictly necessary).

Thus, while the separation is possible, the two also seem to belong together in one repo and it feels unnatural to maintain a separation.

If my resistance ends up being purely about semantics then maybe I need to give it up and try the two separate repos again, but I'd like to be fully aware of all the practical benefits.

[tangrufus]

Can I ask what is the benefit you see in having bedrock and trellis in two different repositories? For updating trellis it’s a simpler git operation, but beyond that?

1. Simpler git operations
2. No need to give away server side stuffs to 3rd party developers
3. Bedrock site can also be hosted on non-Trellis servers

Simpler git operations

With the roots/trellis and my trellis fork as the git remotes, updating with roots or my common customizations is just 1 $ git merge away. And, these git operations is extactly what you do when doing open source PRs. Thus, the whole team is familiar with the process.

However, if your team knows git well enough, using the git subtree way could also achive the same goal.

No need to give away server side stuffs to 3rd party developers

There are times we hire freelancers to do one-off tasks. In those cases, we don't want them to see the trellis repo because they have no need to change server side stuffs nor to know what's in vault.yml.

Besides, when clients leaving us, we could just handover the bedrock repo to the new devlopers. 1 less document to write because:

Bedrock-sites are made to be like 12-factor apps, with configuration, data, etc cleanly separated.
So a Bedrock-site shouldn't need to care about the server it is hosted on

-- #883 (comment)

Bedrock sites can also be hosted on non-Trellis servers

A Bedrock site can also be hosted on non-Trellis servers

-- #883 (comment)

I stand with the idea of "unless your team has experts in all of DevOps, database, linux, security, nginx, redis/memcached, and every other tools WordPress servers might use, don't host clients' websites yourself."

As I am using Kinsta, only the deploy part of Trellis is in use. If a better deployment workflow come along, I might just ditch Trellis.

[robrecord]

Thanks for the detailed reply, @tangrufus !

The second point, I hadn't considered.