Scripts that simplify project update

Before running scripts please configure them by creating config.ini file using config.dist.ini as a template. All the configurations are described in the config.dist.ini file.

composer_outdated_packages_analyzer.php

Run this script to see outdated packages. The tool creates report file named outdated_packages_comparing_to_%desired_version%_version.txt in the outputs folder.

Script output description:

• Total counts of outdated pakages.
• Major outdated packages - a list of packages that have new major versions required in the demoshop version you want to update to based on composer.lock files analyze.
• Minor and Patch outdated packages - a list of packages that have new minor or patch versions required in the demoshop version you want to update to based on composer.lock files analyze.
• Feature outdated packages - a list of feature packages that are outdated compared to the demoshop version you want to update to.

semver_update_order_analyzer.php

Run this script to get a recommended instruction on updating a project to a desired version specified in the configuration file. After script execution, the result is in the outputs folder named update_instruction_to_%desired_version%_version.csv.

The output file contains several blocks that must be executed one by one, the order of blocks is important. It allows you to update project packages in small parts and avoid -w and -W options in most cases. Groups of packages inside each section are independent and might be split among multiple developers.

The instruction contains composer commands you should execute to perform package updates, it also contains package names that will get minor/major updates for each group to make it easier to check project-level customization and fix conflicts after the update.

For extremely outdated projects the instruction might contain long chains of packages in a single composer command because they depend on each other and cannot be updated separately. Such a chain might cause composer exceptions even if all the required packages are listed in the command. For this case, you should use -w or -W option for the composer command. Unfortunately, in this case, the composer will update more packages than actually required. Hence, you need to see the list of updated packages in the composer output to fix conflicts for just updated packages on the project level. Actual composer versions have the --minimal-changes (-m) option available, which allows you to perform only necessary updates safely.

Note: the tool does not work with private packages, so if they block the update process, the tool will recommend removing them.

upgrade_estimator.php

Run this script to estimate project update efforts. It determines which Spryker packages are outdated compared to the demoshop specified in the configuration, e.g. b2b-demo-shop, version 202307.0. Then it scans project level codebase to find new and extended PHP and Twig files to estimate how hard it would be to fix compatibility issues with a new module version. After script execution, the result is in the outputs folder named upgrade_to_%desired_version%_estimation_report.csv.

The script has 3 options:

• -o: for optimistic estimation (every 4th file requires compatibility fix). By default, the script uses pessimistic estimation (each file requires compatibility fix);
• -d: to add compatibility fix estimation details to the report;
• -t: process type (example: php upgrade_estimator.php -tpartial).

Allowed process types are:

• regular: estimates full update efforts to desired Spryker version (default, not need to be specified);
• php: estimates efforts to update project to PHP version specified in the configuration file, uses PHP upgrade instruction report. Before using this option please run the composer_php8_compatibility_analyzer.php script.
• partial: estimates efforts to update modules to versions specified in the configuration file, uses Partial upgrade instruction report. Before using this option please run the semver_partial_update_analyzer.php script.

For optimistic estimation we assume that we will not fix all the project level files but about 50% of them.

Compatibility fix details add to the report all the outdated modules with information about how deeply each module is customized.

extended_files_analyzer.php

Run this script to see which PHP classes have been extended on the project level from vendor. Having the list of extended classes it will be easier to check the compatibility of the updated package and project customizations.

The tool creates report file named extended_project_level_files_list.txt in the outputs folder.

glossary_analyzer.php

Run this script to see glossary keys that are used in the vendor but are missed in the project glossary.csv import file.

The script saves the result in the outputs folder in file named missing_glossary_keys.txt.

missing_plugins_finder.php

Run this script to see plugins that are not injected into the project in comparison with demoshop.

The script saves the result in the outputs folder in file named missing_plugins_comparing_to_%desired_version%_version.txt.

UpgradeDocumentationScraper.php

1. put your github token to github_token in config.ini (it will be needed further to search package names through the github api)
2. manually download migration guides to UpgradeFiles folder from https://github.com/spryker/spryker-docs/tree/master/_includes/pbc/all/upgrade-modules
3. Run this script to get an outputs/effort_estimations.txt e.g "spryker/manual-order-entry-gui" => [ '0.7.0' => 0, '0.8.0' => 0, '0.9.0' => 0, ], "spryker/merchant" => [ 2 => 2, 3 => 2, ]

key it is version key , value it is effort (in hours) needed to migrate to this version

4. copy the result and insert it to \SprykerPackages::$packages , in order to provide proper data for UpgradeEstimator.php

5. configure flags in order to run with needed option e.g $scraper->scrape('UpgradeFiles', false, false, true);

   • $folderPath put upgrade guides there,
   • $includeUnknownVersions set to true if you need to have all version , otherwise the version that were not found will not be included in the result
   • excludeManuallyReviewedModules set to true if you previously reviewed and added some packages with some effort to \SprykerPackages::$packages , and you dont want to have them overwritten
   • $includeGithubPackageSearch set to true if you want to use full power of the scraper in terms of searching of packages names , that you see in output file , otherwise some packages will have default names equls to upgrade guides filenames

Details of implementation :

As all upgrade guides templates varies , script introduces several regular expressions that are used to find :

• package version e.g ## Upgrading from version 3.* to version 4.*
• package name e.g spryker/touch
• estimated migration time e.g Estimated migration time: 5 min

! For the package name Github API used only if regular expressions didnt handle this properly ; see \UpgradeDocumentationScraper::extractPackageName

composer_php_compatibility_analyzer.php

This script calculates which packages must be updated and to which versions on a project to make it compatible with the PHP version specified in the configuration. It is better to specify the latest available patch for the PHP minor version you want to upgrade to, because some packages might have patch version specified in requirements, e.g. you want to update to PHP 8.0, but a package might have requirement 8.0.2 which is higher than 8.0. In this case better add to the config version 8.0.30.

As an output, it creates a CVS file with composer commands you should execute, and also it contains a report with all packages that are supposed to be updated with their current and new versions.

By default, the output contains multiple independent blocks of composer commands that might be split among multiple developers. But for extremely outdated projects the tool might generate too long commands because too many packages must be updated and they depend on each other. It leads to composer errors and the only way to execute such commands is by using the -W composer option which leads to unnecessary updates. (In latest composer version the --minimal-changes (-m) option is available which allows you to perform only necessary updates safely) An alternative way is to use the -b option for this tool which locks all the packages to proper versions so you can update everything together. Alternative output contains following sections:

• Failed to process packages: exists if for some reason the script was not able to resolve conflicts between packages. Must be an exceptional case.
• Incompatible root requirements: exists if composer.json contains packages that have no compatible versions with the PHP version specified in the configuration. It happens when root requirements are not compatible with required PHP version or their inner dependencies have no compatible versions.
• Replaced root requirements: exists if composer.json contains packages that are inner dependencies of other packages and when those packages from composer.json are no longer required and were replaced with alternatives.
• Upgrade instruction: contains composer commands you need to execute in order to make project compatible with desired PHP version. It consists of several steps:
  • lock all packages to strict versions the script found;
  • lock all dev packages to strict versions the script found;
  • perform update for all the packages;
  • remove redundant packages from coomposer.json;
  • unlock packages back to initial requirements existed in composer.json if possible or change version requirement to higher if required;
  • unlock dev packages back to initial requirements existed in composer.json if possible or change version requirement to higher if required;
  • execute update for single packages to refresh composer.lock file.

Note: the tool does not work with private packages, so if they block the update process, the tool will recommend removing them. Note: the tool does not process the "replace" section of composer.json, this might lead to incorrect results. Rare case, needs additional development to cover this scenario.

semver_partial_update_analyzer.php

Run this script to get a recommended instruction on updating specific modules to desired versions specified in the configuration file. After script execution, the result is in the outputs folder named partial_update_instruction_for_%project_name%.csv.

The tool works based on the composer_php_compatibility_analyzer.php, so it has the same behavior and output structure as the composer_php_compatibility_analyzer.php script.

RestorationCommandsValidators

Two plugins are supposed to be added to the Console module in the Zed layer and injected into the ConsoleDependencyProvider.

PublishTriggerEventsCapabilityValidationConsole

Validates if data is restorable using the publish:trigger-events command. Displays a list of entities for which a publisher plugin is not injected or does not exist in the codebase. In order to run this command: APPLICATION_ENV=development vendor/bin/console publish:trigger-events:capability:validation.

SyncDataCapabilityValidationConsole

Validates is data is restorable using the sync:data command. Displays a list of tables for which a synchronization plugin is not injected or does not exist in the codebase. In order to run this command: APPLICATION_ENV=development vendor/bin/console sync:data:capability:validation.