Skip to content
This repository has been archived by the owner on Oct 8, 2020. It is now read-only.

Improve buildkit installation docs #224

Merged
merged 7 commits into from
Jul 26, 2017
Merged

Improve buildkit installation docs #224

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
76c8105
Buildkit - relocate post-install steps
seancolsen Jul 24, 2017
eb8e22c
Buildkit - relocate "Applying a patch" content
seancolsen Jul 24, 2017
a698d51
Buildkit - Improve buildkit page
seancolsen Jul 24, 2017
bb27c6e
Buildkit - Mention nginx when mentioning Apache
seancolsen Jul 24, 2017
3485d72
Fix spelling error
seancolsen Jul 24, 2017
0015e66
Simplify amp config steps
seancolsen Jul 24, 2017
36b10fb
Recommend that people use Apache instead of nginx
seancolsen Jul 25, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 91 additions & 40 deletions docs/tools/buildkit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,22 @@ This is the same collection of tools which manages the test/demo/release infrast

### Ubuntu

If you have a new installation of Ubuntu 12.04 or 14.04, then you can download
If you have a new installation of Ubuntu 12.04 or later, then you can download
everything -- buildkit and the system requirements -- with one command. This
command will install buildkit to `~/buildkit`:

```bash
curl -Ls https://civicrm.org/get-buildkit.sh | bash -s -- --full --dir ~/buildkit
```

Note:
!!! note

* When executing the above command, you should *not* run as `root`. However, you *should*
have `sudo` permissions.
* The `--full` option is *very opinionated*; it specifically installs `php`, `apache`, and `mysql` (rather than `hhvm`, `nginx`, `lighttpd`, or `percona`). If you try to mix `--full` with alternative systems, then expect conflicts.
* If you use the Ubuntu feature for "encrypted home directories", then don't put buildkit in `~/buildkit`. Consider `/opt/buildkit`, `/srv/buildkit`, or some other location that remains available during reboot.
* When executing the above command, you should *not* run as `root`. However, you *should*
have `sudo` permissions.
* The `--full` option is *very opinionated*; it specifically installs `php`, `apache`, and `mysql` (rather than `hhvm`, `nginx`, `lighttpd`, or `percona`). If you try to mix `--full` with alternative systems, then expect conflicts.
* If you use the Ubuntu feature for "encrypted home directories", then don't put buildkit in `~/buildkit`. Consider `/opt/buildkit`, `/srv/buildkit`, or some other location that remains available during reboot.

After running the above command, then proceed to the [post-installation configuration](#config).

### Vagrant

Expand Down Expand Up @@ -59,44 +61,15 @@ true for MAMP, XAMPP, and other downloaded packages.
Once the pre-requisites are met, download buildkit to `~/buildkit`:

```bash
git clone https://github.com/civicrm/civicrm-buildkit.git ~/buildkit
cd ~/buildkit
./bin/civi-download-tools
```

### Troubleshooting

* Nodejs version too old or npm update does not work

Download the latest version from nodejs.org and follow to their instructions

* Nodejs problems

It might be handy to run

```bash
npm update
npm install fs-extra
$ git clone https://github.com/civicrm/civicrm-buildkit.git ~/buildkit
$ cd ~/buildkit
$ ./bin/civi-download-tools
```

## Applying a patch

Using buildkit, you can create a CiviCRM environment with the PR applied.

For example:
## Post-install configuration {:#config}

```bash
civibuild create dmaster \
--url http://localhost:8001 \
--patch https://github.com/civicrm/civicrm-core/pull/8494 \
--admin-pass s3cr3t
```

This will create a test environment with the Drupal, CiviCRM master branch
and the patch in PR 8494. More detailed information is in the
[Civibuild documentation](https://github.com/civicrm/civicrm-buildkit/blob/master/doc/civibuild.md)

## Configuring buildkit after installation {:#configuring}
### Configuring your path {:#path}

!!! note "Not needed for Vagrant/Docker installations"
If you set up buildkit using Vagrant or Docker, then you don't need to perform the configuration steps listed here.
Expand Down Expand Up @@ -135,6 +108,84 @@ If you want to ensure that the buildkit CLI tools are always available, then:
Each time you open a new terminal while working on Civi development, you would need to re-run the `export` command.


### Configuring `amp` {:#amp-config}

Buildkit provides a tool called `amp` which [civibuild](/tools/civibuild.md) uses when it needs to set up a new site. Before you can use `civibuild`, need to configure `amp` by telling it a bit about your system (e.g. what webserver you're using).

#### Interactive config

```
$ amp config
```

!!! tip "tips"
* Run this as a non-`root` user who has `sudo` permission. This will ensure that new files are owned by a regular user, and (if necessary) it enables `civibuild` to restart your webserver and edit `/etc/hosts`.
* Pay close attention to any instructions given in the output of this command. They may involve adding a line to your Apache or nginx configuration file.
* To check which version of apache you have, run `apachectl -v`

#### Testing amp's configuration {:#amp-test}

Test that `amp` is correctly configured.

```
$ amp test
```

!!! failure "Troubleshooting errors from `amp test`"

* Try manually adding settings to your sebserver, as described below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

webserver

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tschuettler. Fixed in 3485d72

* Re-run `amp config`.

#### Manually adding settings to your webserver {:#amp-webserver}

During one of the steps in the `amp config` process, `amp` will *attempt to* alter the system-wide settings for your local webserver (apache or nginx). On some platforms, `amp` is not able to perform the necessary configuration and you must do it manually by following these steps:
Copy link
Member

@totten totten Jul 24, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a misunderstanding -- amp makes no attempt to modify the system-wide httpd configuration. Rather, it tells the user to do it (and tries to give some pointers in the right direction). Of course, half the users ignore those instructions, so it's a recurring pain-point.

Random:

  • Guess: there's confusion because the vagrant and docker builds do that part automatically.
  • But as far as amp config and manual setup are concerned, these aren't automated. (And it can't be totally automated because there are too many edge-cases around different builds -- one might not appreciate that from a Debian/Ubuntu perspective, but from a MAMP/XAMPP/Bitnami/brew perspective it's more salient.)
  • It would be patch-welcome for amp config to put up a better prompt.
  • It would be patch-welcome for amp config to try to apply the needful interactively.
  • It would be patch-welcome for civi-download-tools --full to try to apply the needful.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that it doesn't alter apache or nginx it does modify the hostfile which is http related, I suppose the most sensible thing i can think amp doing would be is to write out the include as a new file in the right directory. But as Chris pointed out finding that correct directory maybe tricky

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. Currently the amp config wizard just displays a handful of possibilities and moves on to the next step -- https://github.com/amp-cli/amp/blob/master/src/Amp/Command/ConfigCommand.php#L139

In an ideal world, that script would stop and ask the user what to do. Depending on what config files/directories are around, it would present a menu like:

[1] Include amp in /etc/apache2/httpd.conf
[2] Include amp in /etc/apache2/conf.d/amp.conf
[3] Include amp in /opt/xampp/etc/httpd.conf
[4] Skip this step. I'll configure it manually.

If they choose 1-3, it would call sudo and try to update the selected file.

This way, it's harder for a user to ignore the step, but we still rely on their judgment for where/how to make the change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@totten indeed I was misunderstanding the expected behavior of amp config. I have made changes in 0015e66 to clarify that, indeed, adding the settings to the webserver is mandatory.


1. Identify the location of your `amp` installation. It is probably a `.amp` folder within your home directory. Make sure to *use the full path* to this directory in the settings below. We will use `<amp-installation>` henceforth to refer to the full path of this directory.

1. Identify your webserver. (If using Apache, use `apachectl -v` to see which version you have.)

* For Apache 2.2:

Create a new file `/etc/apache2/conf.d/buildkit.conf` with the following contents:

```
Include <amp-installation>/apache.d/*conf
```

* For Apache 2.4:

Create a new file `/etc/apache2/conf.d/buildkit.conf` with the following contents:

```
IncludeOptional <amp-installation>/apache.d/*conf
```

* For nginx:

Create a new file `/etc/nginx/conf.d/buildkit.conf` with the following contents:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did I get this path correct @xurizaemon ? /etc/nginx/conf.d/buildkit.conf

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"It depends" ... For Debian, the path above is correct assuming they're using the Debian NginX packages yes. But maybe not all ... eg Debian and CentOS differ on Apache (/etc/apache2 vs /etc/httpd), for my OSX with homebrew it's /usr/local/etc/nginx/servers/buildkit.conf, and that's before you get into the madness of things like CPanel and their ilk, or people installing to /usr/local/nginx...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Do you think it's okay to merge this as-is and assume that people will figure it out? Or do you have something better to propose here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, you should only document amp with apache. nginx doesn't read .htaccess, and amp doesn't have enough plumbing to handle bespoke nginx config files. Basically, civibuild/amp needs a way to pass through per-application templates for the nginx config.

Without that, it's easier to (a) just tell people to use apache or (b) turn off amp's httpd integration and do that stuff manually.


```
include <amp-installation>/nginx.d/*.conf;
```

1. Restart your webserver.


## Troubleshooting {:#troubleshooting}

Nodejs version too old or npm update does not work

: Download the latest version from nodejs.org and follow to their instructions

Nodejs problems

: It might be handy to run

```bash
npm update
npm install fs-extra
```


## Upgrading buildkit {:#upgrading}

Expand Down
74 changes: 28 additions & 46 deletions docs/tools/civibuild.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,41 +21,6 @@ these. Unfortunately, such tools generally require extra work for a Civi
developer environment. Civibuild works with these tools and and fills
in missing parts.

## Your First Build {:#start}

!!! tip
Login as a non-`root` user who has `sudo` permission. This will ensure that new files are owned by a regular user, and (if necessary) it enables `civibuild` to restart Apache or edit `/etc/hosts`.

The first build requires only a few commands. However, these are also the
hardest commands -- you need to provide detailed information about the
Apache/MySQL/PHP systems, and you may need to try them a few times.

Configure `amp` with details of your Apache/MySQL environment. Pay close
attention to the instructions. They may involve adding a line to your
Apache configuration file.

```
$ amp config
```

Test that `amp` has full and correct information about Apache/MySQL.

```
$ amp test
```

!!! note
You may need to alternately restart httpd, re-run `amp config`, and/or re-run `amp test` a few times.

Create a new build using Drupal and the CiviCRM `master` branch.
The command will print out URLs and credentials for accessing the website.

```
$ civibuild create dmaster --url http://dmaster.localhost --admin-pass s3cr3t
```

Once you have a working build of `dmaster`, you can continue working with `civibuild` to create different builds as described below.

## Build Types

`civibuild` includes a small library of build scripts for different
Expand Down Expand Up @@ -309,21 +274,26 @@ vi app/civibuild.conf

The template includes documentation and examples.

## Development/Testing of `civibuild` {:#development}

The tests for `civibuild` are stored in `tests/phpunit`. These are
integration tests which create and destroy real builds on the local system.
To run them:
## Examples

* Configure `amp` (as above)
* Ensure that a test site is configured (`civibuild create civibild-test --type empty`)
* Run `phpunit4` or `env DEBUG=1 OFFLINE=1 phpunit4`
* Note that the tests accept some optional environment variables:
* `DEBUG=1` - Display command output as it runs
* `OFFLINE=1` - Try to avoid unnecessary network traffic
### Applying a patch

Using buildkit, you can create a CiviCRM environment with the PR applied.

## Experimental: Multiple demo/training sites {:#demo-training}
For example:

```bash
civibuild create dmaster \
--url http://localhost:8001 \
--patch https://github.com/civicrm/civicrm-core/pull/8494 \
--admin-pass s3cr3t
```

This will create a test environment with the Drupal, CiviCRM master branch
and the patch in PR 8494.

### Experimental: Multiple demo/training sites {:#demo-training}

When creating a batch of identical sites for training or demonstrations,
one may want to create a single source-code-build with several
Expand Down Expand Up @@ -352,6 +322,18 @@ $ for num in $(seq -w 1 20) ; do
done
```

## Development/Testing of `civibuild` {:#development}

The tests for `civibuild` are stored in `tests/phpunit`. These are
integration tests which create and destroy real builds on the local system.
To run them:

* Configure `amp` (as above)
* Ensure that a test site is configured (`civibuild create civibild-test --type empty`)
* Run `phpunit4` or `env DEBUG=1 OFFLINE=1 phpunit4`
* Note that the tests accept some optional environment variables:
* `DEBUG=1` - Display command output as it runs
* `OFFLINE=1` - Try to avoid unnecessary network traffic


## Credits
Expand Down