Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
paliarush Merge pull request #86 from magento/running-tests
Support running tests in cluster from PhpStorm #77
Latest commit c1c66c3 Jun 28, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs Support running tests in cluster from PhpStorm #77 Jun 26, 2019
etc Support running tests in cluster from PhpStorm #77 Jun 26, 2019
lib Eliminated explicit dependency on PHP for Windows hosts Jan 21, 2016
log/email Added email logging support (#9) Jun 2, 2016
scripts Support running tests in cluster from PhpStorm #77 Jun 27, 2019
tests Support running tests in cluster from PhpStorm #77 Jun 27, 2019
.editorconfig start .editorconfig Nov 8, 2016
.gitignore Rplaced "Vagrant" with "Devbox" across project May 1, 2019
.gitmodules Moved ShUnit to submodule version to 2.1.7 May 1, 2019
.travis.yml Support running tests in cluster from PhpStorm #77 Jun 25, 2019
CHANGELOG.md Rplaced "Vagrant" with "Devbox" across project May 1, 2019
CONTRIBUTING.md Configured basic and extended testsuites for PR and nightly builds re… May 2, 2019
LICENSE.txt Updated license Jan 18, 2019
LICENSE_AFL.txt Updated license Jan 18, 2019
README.md Support running tests in cluster from PhpStorm #77 Jun 25, 2019
init_project.sh Support multiple Magento instances simultaneously #52 Jun 20, 2019
k-get-context Support multiple Magento instances simultaneously #52 Jun 20, 2019
k-open-rabbitmq Rplaced "Vagrant" with "Devbox" across project May 1, 2019
k-restart Magento Commerce (EE) linking test May 9, 2019
k-set-context Support multiple Magento instances simultaneously #52 Jun 17, 2019
k-ssh-elasticsearch - Support for linking additional repos May 10, 2019
k-ssh-magento2 - Support for linking additional repos May 10, 2019
k-ssh-mysql - Support for linking additional repos May 10, 2019
k-ssh-nginx - Support for linking additional repos May 10, 2019
k-ssh-redis - Support for linking additional repos May 10, 2019
k-ssh-varnish Varnish support May 15, 2019
k-start Magento Commerce (EE) linking test May 9, 2019
k-status Rplaced "Vagrant" with "Devbox" across project May 1, 2019
k-upgrade-environment Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-bin-magento Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-clear-cache Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-composer Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-open Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-reinstall Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-search-engine Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-switch-to-ce Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-switch-to-ee Support multiple Magento instances simultaneously #52 Jun 17, 2019
m-varnish Support multiple Magento instances simultaneously #52 Jun 17, 2019
plugin.yaml replace sh checking with native --wait argument, add helm plugin to r… Feb 3, 2019

README.md

Magento 2 Kubernetes DevBox

Build Status

What You get

⚠️ This project is under development and may become official Magento DevBox in the future. There is also a DevBox for Magento Cloud.

It's expected that the Magento 2 project source code will be located and managed on the host to allow quick indexing of project files by IDE. All other infrastructure is deployed in kubernetes cluster on Minikube.

Current DevBox aims to support multi-service multi-instance deployment in one click. Multiple Magento projects should be installed in a single Kubernetes cluster and share resoruces. Each of the Magento projects may be deployed as a monolith or a set of services. The DevBox is optimized for development scenarios using local environment.

The environment also suitable for for Magento Commerce and Magento B2B development.

The project initialization script configures a complete development environment:

  1. Configures all software necessary for Magento 2: Nginx, PHP 7.x, MySQL 5.6, Git, Composer, XDebug, Redis, Rabbit MQ, Varnish
  2. Installs Magento 2 from Git repositories or Composer packages (can be configured via checkout_source_from option in etc/instance/config.yaml)

How to install

If you never used Kubernetes before, read the Kubernetes Docs first.

Requirements

The software listed below should be available in PATH (except for PHP Storm).

  • Docker
  • Minikube
  • Helm
  • VirtualBox
  • Git - Ensure that SSH keys are generated and associated with your Github account. See how to check and how to configure, if not configured.
    ℹ️ To obtain the codebase without cloning, just use the Magento 2 codebase instead of devbox-magento/magento2ce. Either method will produce a successful installation.
  • PHP Storm, optional but recommended. To get Helm support in PhpStorm make sure to get v2018.3+
  • NFS server must be installed and running on *nix and OSX hosts; usually available, follow installation steps first

Installation steps

ℹ️ In case of any issues during installation, please read FAQ section

  1. Open terminal and change your directory to the one you want to contain Magento project.

  2. Download or clone the project with DevBox configuration:

    ⚠️ Do not open it in PhpStorm until init_project.sh has completed PhpStorm configuration in the initialize project step below.

    git clone --recursive git@github.com:magento/magento2-kubernetes-devbox.git magento2-devbox

    Optionally, if you use private repositories on GitHub or download packages from the Magento Marketplace using Composer.

    1. Copy etc/composer/auth.json.dist to etc/composer/auth.json.
    2. Specify your GitHub token by adding "github.com": "your-github-token" to the github-oauth section for GitHub authorization.
    3. Add the Magento Marketplace keys for Marketplace authorization to the repo.magento.com section.
    4. Copy (optional) etc/instance/config.yaml.dist as etc/instance/<instance_name>.yaml and make the necessary customizations. Instance name is Magento instance identifier that can only include letters and numbers.
    5. Copy (optional) etc/env/config.yaml.dist as etc/env/config.yaml and make the necessary customizations.
  3. Initialize the project (this will configure the environment, install Magento):

    cd magento2-devbox
    
    # NFS configuration is needed just once for each project, it will prompt for your password to make changes on the host
    
    bash scripts/host/configure_nfs_exports.sh
    
    bash init_project.sh
  4. Use the magento2-devbox directory as the project root in PHP Storm (not magento2-devbox/magento). This is important, because in this case PHP Storm will be configured automatically by init_project.sh.

  5. Configure the remote PHP interpreter in PHP Storm. Go to Preferences, then Languages and Frameworks. Click PHP and add a new remote interpreter. Select Deployment configuration as a source for connection details.

Default credentials and settings

Some of default settings are available for override. These settings can be found in the etc/instance/config.yaml.dist and etc/env/config.yaml.dist.

To override settings create a copy of etc/env/config.yaml.dist under the name 'config.yaml' and add your custom settings.

You can create multiple copies of etc/instance/config.yaml.dist, each of those copies will be responsible for a separate Magento instance deployed in the DevBox. Config file name must only include alpha-numeric characters and will be used to isolate instances (for instance domain name generation, DB name etc).

Upon a successful installation, you'll see the location and URL of the newly-installed Magento 2 application in console.

Web access:

  • Access storefront at http://magento.<instance_name> (can be found in etc/instance/<instance_name>.yaml)
  • Access admin panel at http://magento.<instance_name>/admin/
  • Magento admin user/password: admin/123123q
  • Rabbit MQ control panel: run bash k-open-rabbitmq, credentials admin/123123q

ℹ️ Your admin URL, storefront URL, and admin user and password are located in etc/instance/<instance_name>.yaml.

Codebase and DB access:

  • Path to your Magento installation in the container is the same as on the host
  • MySQL DB host:
    • inside the container: localhost
    • remotely: run minikube ip to get the IP and use port 30306
  • MySQL DB name: magento_<instance_name>, magento_<instance_name>_integration_tests
  • MySQL DB user/password: root:123123q

Codebase on host

  • CE codebase: magento2-devbox/<instance_name>
  • Magento Commerce codebase will be available if path to commerce repository is specified in etc/instance/<instance_name>.yaml: magento2-devbox/<instance_name>/magento2ee

Getting updates and fixes

Current devbox project follows semantic versioning so feel free to pull the latest features and fixes, they will not break your project. For example your current branch is 2.0, then it will be safe to pull any changes from origin/2.0. However branch 3.0 will contain changes backward incompatible with 2.0. Note, that semantic versioning is only used for x.0 branches (not for develop or master).

ℹ️ To apply changes run bash k-upgrade-environment.

Day-to-day development scenarios

Access Magento

Use the following command to open current instance:

./m-open

Hostname can also be found in maghento/host_name section of etc/instance/<instance_name>.yaml.

Reinstall Magento

Use commands described in Switch between CE and EE section with -f flag. Before doing actual re-installation, these commands update linking of EE codebase, clear cache, update composer dependencies.

If no composer update and relinking of EE codebase is necessary, use the following command. It will clear Magento DB, Magento caches and reinstall Magento instance.

Go to the root of the project in command line and execute:

./m-reinstall

Clear Magento cache

Go to the root of the project in command line and execute:

./m-clear-cache

Switch between CE and EE

Assume, that EE codebase is available in magento2-devbox/magento/magento2ee. The following commands will link/unlink EE codebase, clear cache, update composer dependencies and reinstall Magento. Go to 'magento2-devbox' created earlier and run in command line:

./m-switch-to-ce
# OR
./m-switch-to-ee

Force switch can be done using -f flag even if already switched to the target edition. May be helpful to relink EE modules after switching between branches.

Upgrade can be performed instead of re-installation using -u flag.

Sample data installation

Make sure that ce_sample_data and ee_sample_data are defined in etc/instance/<instance_name>.yaml and point CE and optionally EE sample data repositories. During initial project setup or during bash init_project.sh -fc (with -fc project will be re-created from scratch), sample data repositories willl be checked out to magento2-devbox/magento/magento2ce-sample-data and magento2-devbox/magento/magento2ee-sample-data.

To install Magento with sample data specify/uncomment sample data repository link at repository_url_additional_repositories in etc/instance/<instance_name>.yaml and run ./m-switch-to-ce -f or ./m-switch-to-ee -f, depending on the edition to be installed. To disable sample data, comment out additional repositories and force-switch to necessary edition (using the same commands).

Basic data generation

Several entities are generated for testing purposes by default using REST API after Magento installation:

  • Customer with address (credentials customer@example.com:123123qQ)
  • Category
  • Couple simple products
  • Configurable product

To disable this feature, set magento/generate_basic_data in etc/instance/<instance_name>.yaml to 0 and run ./m-switch-to-ce -f or ./m-switch-to-ee -f, depending on the edition to be installed.

Use Magento CLI (bin/magento)

Go to 'magento2-devbox' created earlier and run in command line:

./m-bin-magento <command_name>
# e.g.
./m-bin-magento list

Debugging with XDebug

XDebug is already configured to connect to the host machine automatically. So just:

  1. Set XDEBUG_SESSION=1 cookie (e.g. using 'easy Xdebug' extension for Firefox). See XDebug documentation for more details
  2. Start listening for PHP Debug connections in PhpStorm on default 9000 port. See how to integrate XDebug with PhpStorm
  3. Set beakpoint or set option in PhpStorm menu 'Run -> Break at first line in PHP scripts'

Connecting to MySQL DB

Go to 'magento2-devbox' created earlier and run in command line:

bash k-ssh-mysql

After successful login to the container run the following command and enter 123123q when prompted for a password:

mysql -uroot -p

To connect remotely run minikube ip to get the IP and use port 30306

View emails sent by Magento

Not available yet.

Accessing PHP and other config files

The following configuration files are used by default:

Upgrading Magento

Sometimes it is necessary to test upgrade flow. This can be easily done as follows (assuming that you have installed instance):

  • For git-based installation - check out codebase corresponding to the target Magento version. Or modify your composer.json in case of composer-based installation
  • Use commands described in Switch between CE and EE section with -u flag

Multiple Magento instances

Not available yet.

Update Composer dependencies

Go to 'magento2-devbox' created earlier and run in command line:

./m-composer install
# OR
./m-composer update

Running Magento tests

See how to run Magento tests from PhpStorm using remote PHP in Kubernetes cluster

Environment configuration

Switch between PHP versions

Not available yet.

Activating Varnish

Use the following commands to enable/disable varnish : m-varnish disable or m-varnish enable.

You can also set use_varnish: 1 in etc/env/config.yaml to use varnish. Changes will be applied on init_project.sh -f.

The VCL content can be found in configmap.yaml.

Activating ElasticSearch

Set search_engine: "elasticsearch" in etc/env/config.yaml to use ElasticSearch as current search engine or search_engine: "mysql" to use MySQL. Changes will be applied on m-reinstall.

Use the following commands to switch between search engines without reinstalling Magento: m-search-engine elasticsearch or m-search-engine mysql.

Redis for caching

Redis is configured as cache backend by default.

Reset environment

It is possible to reset project environment to default state, which you usually get just after project initialization. The following command will re-initialize Kubernetes cluster. Magento 2 code base (magento directory) and etc/instance/<instance_name>.yaml and PhpStorm settings will stay untouched.

Go to 'magento2-devbox' created earlier and run in command line:

./init_project.sh -f

It is possible to reset Magento 2 code base at the same time. Magento 2 code base will be deleted and then cloned from the repositories specified in etc/config.yaml

./init_project.sh -fc

To reset PhpStorm project configuration, in addition to -f specify -p option:

./init_project.sh -fp

Ultimate project reset can be achieved by combining all available flags:

./init_project.sh -fcp

Switch NodeJS Versions

NodeJS not available yet.

DevBox tests

The tests are executed on every PR on Travis CI. It is possible to configure the same tests to run on the forked repository. In order to run composer-based Magento tests for the fork, repo.magento.com credentials must be set to COMPOSER_AUTH environment variable on Travis CI, the variable value should be:

'{"http-basic": {"repo.magento.com": {"username": "<public_key>","password": "<secret_key>"}}}'

An extended testsuite by default is executed against the master branch only. It is possible to execute an extended testsuite on every build by commenting out if: branch = master in the .travis.yaml

The same tests can be run on local using the following command. ⚠️ only one devbox can be running on the same host at the same time. The tests will destroy existing devbox installation.

cd tests
bash ./<test-name>.sh

FAQ

  1. To debug any CLI script in current Devbox project, set debug:devbox_project option in etc/env/config.yaml to 1
  2. Make sure that you used magento2-devbox directory as project root in PHP Storm (not magento2-devbox/magento)
  3. If project opened in PhpStorm looks broken, close PhpStorm and remove magento2-devbox/.idea. Run ./magento2-devbox/scripts/host/configure_php_storm.sh. After opening project in PhpStorm again everything should look good
  4. Please make sure that currently installed software, specified in requirements section, meets minimum version requirement
  5. Be careful if your OS is case-insensitive, NFS might break the symlinks if you cd into the wrong casing and you power the devbox up. Just be sure to cd in to the casing the directory was originally created as.
  6. Cannot run unit tests from PHPStorm on Magento 2.2, see possible solution here
  7. Permission denied (publickey)
  8. If you get minikube time out error restarting cluster while initializing project, run minikube stop && minikube delete && ./init_project.sh.
  9. To modify the docker image used for php-fpm container:
    • Make changes in etc/docker/monolith/Dockerfile
    • Run ./k-upgrade-environment
    • Run ./k-status to open kubernetes dashboard and delete Replica Set named magento2-monolith-*. The container should be restarted and its Age should reset
You can’t perform that action at this time.