Upgrading from 2.1.16 to 3.0.1 or 3.0.5 fails

[neoloc]

sysPass Version
2.1 (2.1.16.18061901)

Describe the bug
Upgrading from 2.1.16 to 3.0.1 or 3.0.5 fails to complete. This upgrade is being done in a separate testing environment as we previously ran into bugs after upgrading in the past.

Previous upgrade from 2.1.16 to 3.0.0 was successful but we ran into an issue (unrelated to upgrade) where we couldnt have more users added. We rolled back to the pre-upgrade database and website files successfully and continued to use syspass 2.1.16 until now. The previous upgrade November 2018.

Testing environment is same servers but different URL,Database and DB Credentials. I am not sure if this is going to create a problem while trying to upgrade.

To Reproduce
Steps to reproduce the behavior:

1. Create testing environment.
2. Import an export of the database to different DB (syspass_dev)
3. Download 3.0.1 or 3.0.5 and extract into webserver directory for test environment
4. Copy files from /config directory as described in upgrade process.
5. Fix permissions as described in upgrade process.
6. Update dbname, dbuser and dbpass in config.xml file.
7. Install composer and all addons.
8. Visit testing URL.
9. Initiate Upgrade process.

Expected behavior
Expect upgrade to complete successfully.

Event log
Eventlog here is from 3.0.1 upgrade attempt. I have renamed part of the path as it includes our domain.

[root@webapp1 public_html]# cat app/config/syspass.log
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [EXCEPTION] [N/A] File not found (/var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/cache/actions.cache)
#0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/File/FileCacheBase.php(70): SP\Storage\File\FileHandler->checkFileExists()
#1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Core/Acl/Actions.php(89): SP\Storage\File\FileCacheBase->isExpired(86400)
#2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Core/Acl/Actions.php(189): SP\Core\Acl\Actions->loadCache()
#3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/UpgradeController.php(56): SP\Core\Acl\Actions->reset()
#4 [internal function]: SP\Modules\Web\Controllers\UpgradeController->indexAction()
#5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array)
#6 [internal function]: SP\Bootstrap->SP{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#7 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#8 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array)
#9 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request))
#10 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container))
#11 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...')
#12 {main}
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::mapAndSave] ACTION CACHE MISS
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::saveCache] Saved actions cache
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked
2019-08-12 17:15:41 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache
2019-08-12 17:15:41 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache
2019-08-12 17:15:41 [EXCEPTION] [N/A] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'syspass_dev.Plugin' doesn't exist
#0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(236): PDO->query('SELECT id, nam...') #1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(174): SP\Storage\Database\Database->prepareQueryData(Object(SP\Storage\Database\QueryData)) #2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(135): SP\Storage\Database\Database->doQuery(Object(SP\Storage\Database\QueryData)) #3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Repositories/Plugin/PluginRepository.php(445): SP\Storage\Database\Database->doSelect(Object(SP\Storage\Database\QueryData)) #4 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Plugin/PluginService.php(290): SP\Repositories\Plugin\PluginRepository->getEnabled() #5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Plugin/PluginManager.php(360): SP\Services\Plugin\PluginService->getEnabled() #6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(114): SP\Plugin\PluginManager->getEnabledPlugins() #7 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(67): SP\Modules\Web\Controllers\BootstrapController->getPlugins() #8 [internal function]: SP\Modules\Web\Controllers\BootstrapController->getEnvironmentAction() #9 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array) #10 [internal function]: SP\Bootstrap->SP\{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array) #11 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array) #12 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array) #13 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request)) #14 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container)) #15 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...') #16 {main} 2019-08-12 17:15:41 [EXCEPTION] [N/A] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'syspass_dev.Plugin' doesn't exist #0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(174): SP\Storage\Database\Database->prepareQueryData(Object(SP\Storage\Database\QueryData)) #1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(135): SP\Storage\Database\Database->doQuery(Object(SP\Storage\Database\QueryData)) #2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Repositories/Plugin/PluginRepository.php(445): SP\Storage\Database\Database->doSelect(Object(SP\Storage\Database\QueryData)) #3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Plugin/PluginService.php(290): SP\Repositories\Plugin\PluginRepository->getEnabled() #4 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Plugin/PluginManager.php(360): SP\Services\Plugin\PluginService->getEnabled() #5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(114): SP\Plugin\PluginManager->getEnabledPlugins() #6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(67): SP\Modules\Web\Controllers\BootstrapController->getPlugins() #7 [internal function]: SP\Modules\Web\Controllers\BootstrapController->getEnvironmentAction() #8 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array) #9 [internal function]: SP\Bootstrap->SP\{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array) #10 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array) #11 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array) #12 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request)) #13 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container)) #14 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...') #15 {main} 2019-08-12 17:15:41 [EXCEPTION] [N/A] (P) SQLSTATE[42S02]: Base table or view not found: 1146 Table 'syspass_dev.Plugin' doesn't exist #0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(236): PDO->query('SELECT id, nam...')
#1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(174): SP\Storage\Database\Database->prepareQueryData(Object(SP\Storage\Database\QueryData))
#2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(135): SP\Storage\Database\Database->doQuery(Object(SP\Storage\Database\QueryData))
#3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Repositories/Plugin/PluginRepository.php(445): SP\Storage\Database\Database->doSelect(Object(SP\Storage\Database\QueryData))
#4 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Plugin/PluginService.php(290): SP\Repositories\Plugin\PluginRepository->getEnabled()
#5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Plugin/PluginManager.php(360): SP\Services\Plugin\PluginService->getEnabled()
#6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(114): SP\Plugin\PluginManager->getEnabledPlugins()
#7 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(67): SP\Modules\Web\Controllers\BootstrapController->getPlugins()
#8 [internal function]: SP\Modules\Web\Controllers\BootstrapController->getEnvironmentAction()
#9 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array)
#10 [internal function]: SP\Bootstrap->SP{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#11 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#12 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array)
#13 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request))
#14 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container))
#15 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...')
#16 {main}
2019-08-12 17:15:41 [EXCEPTION] [N/A] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'syspass_dev.Plugin' doesn't exist
#0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(174): SP\Storage\Database\Database->prepareQueryData(Object(SP\Storage\Database\QueryData))
#1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(135): SP\Storage\Database\Database->doQuery(Object(SP\Storage\Database\QueryData))
#2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Repositories/Plugin/PluginRepository.php(445): SP\Storage\Database\Database->doSelect(Object(SP\Storage\Database\QueryData))
#3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Plugin/PluginService.php(290): SP\Repositories\Plugin\PluginRepository->getEnabled()
#4 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Plugin/PluginManager.php(360): SP\Services\Plugin\PluginService->getEnabled()
#5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(114): SP\Plugin\PluginManager->getEnabledPlugins()
#6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(67): SP\Modules\Web\Controllers\BootstrapController->getPlugins()
#7 [internal function]: SP\Modules\Web\Controllers\BootstrapController->getEnvironmentAction()
#8 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array)
#9 [internal function]: SP\Bootstrap->SP{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#10 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#11 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array)
#12 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request))
#13 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container))
#14 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...')
#15 {main}
2019-08-12 17:15:41 [EXCEPTION] [N/A] (P) SQLSTATE[42S02]: Base table or view not found: 1146 Table 'syspass_dev.Plugin' doesn't exist
#0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(236): PDO->query('SELECT id, nam...') #1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(174): SP\Storage\Database\Database->prepareQueryData(Object(SP\Storage\Database\QueryData)) #2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Storage/Database/Database.php(135): SP\Storage\Database\Database->doQuery(Object(SP\Storage\Database\QueryData)) #3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Repositories/Plugin/PluginRepository.php(445): SP\Storage\Database\Database->doSelect(Object(SP\Storage\Database\QueryData)) #4 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Plugin/PluginService.php(290): SP\Repositories\Plugin\PluginRepository->getEnabled() #5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Plugin/PluginManager.php(360): SP\Services\Plugin\PluginService->getEnabled() #6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(114): SP\Plugin\PluginManager->getEnabledPlugins() #7 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/BootstrapController.php(67): SP\Modules\Web\Controllers\BootstrapController->getPlugins() #8 [internal function]: SP\Modules\Web\Controllers\BootstrapController->getEnvironmentAction() #9 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array) #10 [internal function]: SP\Bootstrap->SP\{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array) #11 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array) #12 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array) #13 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request)) #14 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container)) #15 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...') #16 {main} 2019-08-12 17:16:01 [INFO] [SP\Core\PhpExtensionChecker::checkMandatory] Extensions checked 2019-08-12 17:16:01 [INFO] [SP\Core\UI\Theme::saveIcons] Saved icons cache 2019-08-12 17:16:01 [INFO] [SP\Core\Acl\Actions::loadCache] Loaded actions cache 2019-08-12 17:16:01 [EXCEPTION] [N/A] SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'CustomFieldType' already exists #0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Upgrade/UpgradeDatabaseService.php(172): PDO->exec('CREATE TABLE C...')
#1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Upgrade/UpgradeDatabaseService.php(107): SP\Services\Upgrade\UpgradeDatabaseService->applyUpgrade('300.18010101')
#2 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/UpgradeController.php(81): SP\Services\Upgrade\UpgradeDatabaseService->upgrade('0.0', Object(SP\Config\ConfigData))
#3 [internal function]: SP\Modules\Web\Controllers\UpgradeController->upgradeAction()
#4 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array)
#5 [internal function]: SP\Bootstrap->SP{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#7 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array)
#8 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request))
#9 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container))
#10 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...')
#11 {main}
2019-08-12 17:16:01 [EXCEPTION] [N/A] Error while updating the database
#0 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Services/Upgrade/UpgradeDatabaseService.php(107): SP\Services\Upgrade\UpgradeDatabaseService->applyUpgrade('300.18010101')
#1 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/app/modules/web/Controllers/UpgradeController.php(81): SP\Services\Upgrade\UpgradeDatabaseService->upgrade('0.0', Object(SP\Config\ConfigData))
#2 [internal function]: SP\Modules\Web\Controllers\UpgradeController->upgradeAction()
#3 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(234): call_user_func_array(Array, Array)
#4 [internal function]: SP\Bootstrap->SP{closure}(Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#5 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(886): call_user_func(Object(Closure), Object(Klein\Request), Object(Klein\Response), Object(Klein\ServiceProvider), Object(Klein\App), Object(Klein\Klein), Object(Klein\DataCollection\RouteCollection), Array)
#6 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/vendor/klein/klein/src/Klein/Klein.php(588): Klein\Klein->handleRouteCallback(Object(Klein\Route), Object(Klein\DataCollection\RouteCollection), Array)
#7 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/SP/Bootstrap.php(456): Klein\Klein->dispatch(Object(Klein\Request))
#8 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/lib/Base.php(75): SP\Bootstrap::run(Object(DI\Container))
#9 /var/www/webapps/dev-sp.example.domain/sysPass-3.0.1.19012002/index.php(28): require('/var/www/webapp...')
#10 {main}

Platform (please complete the following information):

• OS: Linux
• OS: CentOS 7
• Browser: Chromium

Additional context
I have seen other issues/bugs posted where people have have the same issue. We cannot revert our database back to before the original backup as too much data has been entered since.

Is there are way to 'un-upgrade' the database if that is what the issue is? If no, is there another way to migrate? e.g. export accounts, build new environment and import accounts

[nuxsmin]

@neoloc Hello, It seems that you're trying to perform an upgrade over a failed one, but unfortunately it won't work, since some database changes are related to DDL (structure) and they can't be rolled back.

What about the database backup?. Are you using custom fields?

Regards

[neoloc]

We take backups with automysqlbackup. One custom field is in use in the Accounts module.

The system was rolled back to before the successful upgrade (database and files on web server). Would we be able to use syspass if the upgrade failed?

[nuxsmin]

Sorry, I missed this part: "before the successful upgrade". Did you finish the upgrading process?.

sysPass won't work if not upgraded, because there are major database changes that need to be performed before using the new release. Every release is versioned using SEMVER which stands for a well-known nomenclature for each version number, so changes made to the first number are major and breaking ones, so jumping from 2.1 to 3.0 is a major upgrade.

Asking about custom fields was for trying to perform a manual upgrade by applying SQL scripts, but if there are any of them, it won't work...

Regards

[neoloc]

We completed the upgrade to 3.0.0 successfully but rolled back as we hit an issue we thought was caused from the upgrade. The issue that I am referring to was due to the size of the varbinary field for plugin_data in the plugins table (sysPass/plugin-Authenticator#5). We found the same issue in 2.1.16 after rolling back, so it was just a coincidence. One new user after upgrading, and the same user created again after rolling back.

To resolve this, the field was increased in size to 32k.

MariaDB [syspass]> describe plugins;
+----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+------------------+------+-----+---------+----------------+
| plugin_id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| plugin_name | varchar(100) | NO | UNI | NULL | |
| plugin_data | varbinary(32768) | YES | | NULL | |
| plugin_enabled | bit(1) | NO | | b'0' | |
+----------------+------------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)

It has been half a year since the previous upgrade and rollback, syspass has been working fantastic (thank you so much!) so can I assume that the database shouldn't think its in a partially upgraded state?

We have one custom field, but its not extensively used and we can live without it. If we remove the custom field, can the SQL scripts be applied?

[nuxsmin]

Regarding the field size issue it has been fixed in 3.1 (already in RC stage) by moving the plugin's data to another table, increasing the field size and using encryption.

Because you're using the authenticator plugin, some extra tasks need to be performed by the app code, so a script-only upgrade is not possible...

Regards

[neoloc]

Hi nuxsmin,

Just a followup to this in case someone else has a similar issue.

We were able to successfully migrate to syspass 3.1 using the export/import features. This actually helped us clean up our categories too since the account history wasnt migrated.