NEXT-33729 - Add staging mode

.wordlist.txt

@@ -47,6 +47,7 @@ amqp
 AndRule
 antipattern
 Analytics
+anonymization
 api
 ApiAware
 ApiController
@@ -1633,3 +1634,6 @@ YYYY
 zlib
 zsh
 ZSH
+anonymize
+anonymized
+Cloudflare

guides/hosting/configurations/shopware/staging.md

@@ -0,0 +1,216 @@
+---
+nav:
+  title: Staging
+  position: 10
+
+---
+
+# Staging
+
+Since Shopware 6.6.1.0, Shopware has an integrated staging mode. This mode prepares the shop to be used in a staging environment. This means the shop is prepared to be used in a test environment, where changes can be made without affecting the live shop.
+
+## The workflow
+
+The staging mode is designed to modify data only inside the Shopware instance. This means the staging mode does not duplicate the current installation, copy the database, or copy the files. It only changes the data inside the Shopware instance.
+
+So, the real-world use case would be something like this:
+
+### Creating the second Shopware instance
+
+The recommended way to create a second Shopware instance would be to deploy from your Git repository to the new environment. This way, you ensure the codebase is equal to the live environment.
+
+An alternative way would be to copy the files from the live environment to the staging environment.
+
+### Copying the database
+::: info
+Ensure that the `mysqldump` and `mysql` binary are from the same major version and vendor. If you use `mysqldump` from MariaDB, you should also use `mysql` from MariaDB. The same applies to MySQL.
+:::
+
+To have the staging environment similar to the live environment, it's recommended that the database be duplicated. You can use the `mysqldump` command to export the database and import it into the staging environment. 
+
+::: info
+`shopware-cli` is a separate Go command line application that contains a lot of useful commands for Shopware. [Checkout the docs](https://sw-cli.fos.gg/install/) to learn how to install it.
+:::
+
+We recommend using `shopware-cli project dump` to create a dump of the database and import it with the regular mysql command. Shopware cli also has a flag to anonymize the data, so you can be sure that no personal data is in the staging environment.
+
+```bash
+# creating a regular dump, the clean parameter will not dump the data of cart table
+shopware-cli project dump --clean --host localhost --username db_user --password db_pass --output shop.sql shopware
+
+# create a dump with anonymize data
+shopware-cli project dump --clean --anonymize --host localhost --username db_user --password db_pass --output shop.sql shopware
+```
+
+You can configure the dump command with a `.shopware-project.yml`. This file allows you to specify tables that should be skipped, define additional fields for anonymization, and more. Check out the [CLI](https://sw-cli.fos.gg/commands/project/#shopware-cli-project-dump-database) for more information.
+
+### Configuration
+::: info
+It is not recommended to share resources like MySQL, Redis, ElasticSearch/OpenSearch between the live and staging environments. This could lead to data corruption when the configuration is not done correctly. Also, the performance of the live environment could be affected by the staging environment.
+:::
+
+After importing the database, you should modify the `.env` to use the staging database. If you use ElasticSearch/OpenSearch, you should set a `SHOPWARE_ES_INDEX_PREFIX` to avoid conflicts with the live environment.
+
+### Activate the staging mode
+
+After the database is imported and the configuration is done, you can activate the staging mode. This can be done using:
+
+```bash
+./bin/console system:setup:staging
+```
+
+This command will modify the database to be used in a staging environment. You can pass `--no-interaction` to the command to avoid the interactive questions.
+
+### Protecting the staging environment
+
+The staging environment should be protected from unauthorized access. It is advisable to employ protective measures like password protection, IP restriction, or OAuth authentication.
+
+The simplest way to protect the staging environment is utilizing `.htaccess` for  Apache or `auth_basic` for Nginx. You can also use a firewall to restrict access to the staging environment based on IP addresses.
+
+Example configuration for Apache:
+
+```apache
+# <project-root>/public/.htaccess
+SetEnvIf Request_URI /api noauth=1
+<RequireAny>
+Require env noauth
+Require env REDIRECT_noauth
+Require valid-user
+</RequireAny>
+```
+
+An alternative way could be to use an Application Proxy before the staging environment like:
+
+- [Cloudflare Access](https://www.cloudflare.com/teams/access/)
+- [Azure Application Gateway](https://azure.microsoft.com/en-us/services/application-gateway/)
+- [Generic oauth2 proxy](https://oauth2-proxy.github.io/oauth2-proxy/)
+
+
+## Staging mode
+
+The staging mode is designed to be used in a test environment. This means the shop is prepared to be used in a test environment, where changes can be made without affecting the live shop.
+
+### What staging mode does?
+
+- Deletes all apps that have an active connection to an external service and the integrations in Shopware.
+- Resets the instance ID used for registration of apps.
+- It turns off the sending of emails.
+- Rewrites the URLs to the staging domain (if configured).
+- Checks that the ElasticSearch/OpenSearch indices do not exist yet.
+- Shows a banner in the administration and storefront to indicate that the shop is in staging mode.
+
+### What staging mode does not?
+
+- Doesn't duplicate the current installation.
+- Doesn't copy database or files.
+- Doesn't modify the live environment.
+
+### Configuration
+
+The staging mode is fully configurable with `config/packages/staging.yaml`. You can configure the following options:
+
+```yaml
+# <shopware-root>/config/packages/staging.yaml
+shopware:
+    staging:
+        mailing:
+            # Disables the sending of mails (default: true)
+            disable_delivery: true
+        storefront:
+            # Shows a banner in the storefront when staging mode is active (default: true)
+            show_banner: true
+        administration:
+            # Shows a banner in the administration when staging mode is active (default: true)
+            show_banner: true
+        sales_channel:
+            domain_rewrite:
+                # See below for more information
+        elasticsearch:
+            # Checks that no indices are existing yet (default: true)
+            check_for_existence: true
+```
+
+One of the most important options is the `domain_rewrite`. This option allows you to rewrite the URLs to the staging domain. This allows multiple ways to rewrite the URLs:
+
+- Using direct match (`equal`)
+
+
+```yaml
+# <shopware-root>/config/packages/staging.yaml
+shopware:
+    staging:
+        sales_channel:
+            domain_rewrite:
+                - type: equal
+                  match: https://my-live-store.com
+                  replace: https://my-staging-store.com
+                - # ... second rule
+```
+
+This compares the Sales Channel URLs. When it's equal to `https://my-live-store.com`, it will be replaced with `https://my-staging-store.com`.
+
+- Replace using prefix (`prefix`)
+
+```yaml
+# <shopware-root>/config/packages/staging.yaml
+shopware:
+    staging:
+        sales_channel:
+            domain_rewrite:
+                - type: prefix
+                  match: https://my-live-store.com
+                  replace: https://my-staging-store.com
+                - # ... second rule
+```
+
+The difference here to the `equal` type is that it will only replace the URL when it starts with `https://my-live-store.com`, so all paths to that beginning will be replaced. For example, `https://my-live-store.com/en` will be replaced with `https://my-staging-store.com/en`
+
+- Replace using regex (`regex`)
+
+```yaml
+# <shopware-root>/config/packages/staging.yaml
+shopware:
+    staging:
+        sales_channel:
+            domain_rewrite:
+                - type: regex
+                  match: '/https?:\/\/(\w+)\.(\w+)$/m'
+                  replace: 'http://$1-$2.local'
+                - # ... second rule
+```
+
+This will use the regex to replace the URL. The match and replace are regular expressions. In this example, `https://my-live-store.com` will be replaced with `http://my-live-store.local`.
+
+### Usage of apps
+
+The staging command will delete all apps that have an active connection to an external service. This will be done to avoid data corruption or leaks in the live environment, as the staging environment is a copy of the live environment, so they keep a connection. After executing the command, you can install the app again, creating a new instance ID, so the app will think it's an entirely different shop. In this way, the app installation is completely isolated from the live environment.
+
+## Integration into plugins
+
+The `system:setup:staging` is dispatching an Event which all plugins can subscribe to `Shopware\Core\Maintenance\Staging\Event\SetupStagingEvent` and modify the database for them to be in staging mode.
+
+Example of a subscriber for a payment provider to turn on the test mode:
+
+```php
+<?php
+
+namespace Swag\PaymentProvider\Subscriber;
+
+use Shopware\Core\Maintenance\Staging\Event\SetupStagingEvent;
+
+class StagingSubscriber implements EventSubscriberInterface
+{
+    public static function getSubscribedEvents(): array
+    {
+        return [
+            SetupStagingEvent::class => 'onSetupStaging'
+        ];
+    }
+
+    public function onSetupStaging(SetupStagingEvent $event): void
+    {
+        // modify the database to turn on the test mode
+    }
+}
+```
+

guides/hosting/performance/performance-tweaks.md

@@ -7,7 +7,7 @@ nav:
 
 # Performance Tweaks
 
-Shopware is a platform for many different projects. It needs to handle a broad range of load characteristics and environments. That means that the default configuration is optimized for the best out-of-the-box experience. But there are many opportunities to increase the performance by fitting the configuration to your needs.
+Shopware is a platform for many different projects. It needs to handle a broad range of load characteristics and environments. It means that the default configuration is optimized for the best out-of-the-box experience. However, there are many opportunities to increase the performance by fitting the configuration to your needs.
 
 ## HTTP cache
 
@@ -35,7 +35,7 @@ shopware:
 
 ### Delayed invalidation
 
-A delay for the cache invalidation can be activated for systems with a high update frequency for the inventory (products, categories). Once the instruction to delete the cache entries for a specific product or category occurs, they are not deleted instantly but processed by a background task afterwards. Thus, if two processes invalidate the cache in quick succession, the timer for the invalidation of this cache entry will only reset.
+A delay for cache invalidation can be activated for systems with a high update frequency for the inventory (products, categories). Once the instruction to delete the cache entries for a specific product or category occurs, they are not deleted instantly but processed by a background task later. Thus, if two processes invalidate the cache in quick succession, the timer for the invalidation of this cache entry will only reset.
 
 ```yaml
 # config/packages/prod/shopware.yaml
@@ -64,7 +64,7 @@ and then you can set `SQL_SET_DEFAULT_SESSION_VARIABLES=0` to your `.env` file
 
 ## SQL is faster than DAL
 
-We designed the DAL (Data Abstraction Layer) to provide developers a flexible and extensible data management. However, features in such a system come at the cost of performance. Therefore, using DBAL (plain SQL) is much faster than using the DAL in many scenarios, especially when it comes to internal processes, where often only one ID of an entity is needed.
+DAL(Data Abstraction Layer) has been designed suitably to provide developers with a flexible and extensible data management. However, features in such a system come at the cost of performance. Therefore, using DBAL (plain SQL) is much faster than using the DAL in many scenarios, especially when it comes to internal processes, where often only one ID of an entity is needed.
 
 Refer to this article to know more on [when to use plain SQL and DAL](../../../resources/references/adr/2021-05-14-when-to-use-plain-sql-or-dal).
 
@@ -93,7 +93,7 @@ shopware:
         update_mail_variables_on_send: false
 ```
 
-If you wonder, why it is in `prod`, have a look into the [Symfony configuration about configuration environments](https://symfony.com/doc/current/configuration.html#configuration-environments).
+If you ever wonder why it is in `prod`, take a look into the [Symfony configuration environments](https://symfony.com/doc/current/configuration.html#configuration-environments).
 
 ## Increment storage
 
@@ -115,7 +115,7 @@ shopware:
             url: 'redis://host:port/dbindex'
 ```
 
-If you don't need such functionality, it is highly recommended to disable this behavior by using `array` as a type.
+If you don't need such functionality, it is highly recommended that you disable this behavior by using `array` as a type.
 
 ## Lock storage
 
@@ -135,7 +135,7 @@ The generation of the number ranges is an **atomic** operation, which guarantees
 
 By default, the number range states are stored in the database.
 In scenarios where high throughput is required (e.g., thousands of orders per minute), the database can become a performance bottleneck because of the requirement for atomicity.
-Redis offers better support for atomic increments than the database. Therefore the number ranges should be stored in Redis in such scenarios.
+Redis offers better support for atomic increments than the database. Therefore, the number ranges should be stored in Redis in such scenarios.
 
 ```yaml
 # config/packages/prod/shopware.yaml
@@ -163,14 +163,14 @@ framework:
 zend.assertions=-1
 
 # cache file_exists,is_file
-# WARNING: this will lead to thrown errors after clearing cache, while it tries to access cached Shopware_Core_KernelProdDebugContainer.php
+# WARNING: this will lead to thrown errors after clearing cache while it tries to access cached Shopware_Core_KernelProdDebugContainer.php
 opcache.enable_file_override=1
 
 # increase opcache string buffer as shopware has many files
 opcache.interned_strings_buffer=20
 
 # disables opcache validation for timestamp for reinvalidation of the cache
-# WARNING: you need to clear on deployments the opcache by reloadding php-fpm or cachetool (https://github.com/gordalina/cachetool)
+# WARNING: you need to clear on deployments the opcache by reloading php-fpm or cachetool (https://github.com/gordalina/cachetool)
 opcache.validate_timestamps=0
 
 # disable check for BOM
@@ -190,7 +190,7 @@ For an additional 2-5% performance improvement, it is possible to provide a prel
 
 - Each cache clear requires a PHP-FPM restart
 - Each file change requires a PHP-FPM restart
-- Extension Manager does not work
+- The Extension Manager does not work
 
 The PHP configuration would look like:
 
@@ -201,7 +201,7 @@ opcache.preload_user=nginx
 
 ## Cache ID
 
-The Shopware cache has a global cache id to clear the cache faster and work in a cluster setup. This cache id is saved in the database and will only be changed when the cache is cleared. This ensures that the new cache is used and the message queue can clean the old folder. If this functionality is not used, this cache id can also be hardcoded `SHOPWARE_CACHE_ID=foo` in the `.env` to save one SQL query on each request.
+The Shopware cache has a global cache ID to clear the cache faster and work in a cluster setup. This cache ID is saved in the database and will only be changed when the cache is cleared. This ensures that the new cache is used and the message queue can clean the old folder. If this functionality is not used, this cache ID can also be hardcoded `SHOPWARE_CACHE_ID=foo` in the `.env` to save one SQL query on each request.
 
 ## .env.local.php
 
@@ -234,7 +234,7 @@ The `business_event_handler_buffer` handler logs flow. Setting it to `error` wil
 
 ## Disable App URL external check
 
-On any Administration load Shopware tries to request himself to test that the configured `APP_URL` inside `.env` is correct.
+On any Administration load, Shopware tries to request itself to test that the configured `APP_URL` inside `.env` is correct.
 If your `APP_URL` is correct, you can disable this behavior with an environment variable `APP_URL_CHECK_DISABLED=1`.
 
 ## Disable fine-grained caching