Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors with fresh install to Mysql on Heroku #1750

Closed
hamuz opened this issue Sep 30, 2018 · 17 comments
Closed

Errors with fresh install to Mysql on Heroku #1750

hamuz opened this issue Sep 30, 2018 · 17 comments
Labels
stale No replies or comments. Will be auto-closed in 14 days.

Comments

@hamuz
Copy link
Contributor

hamuz commented Sep 30, 2018

Bug description
I am running Firefly III version 4.7.7@develop

Errors with fresh install to MySQL on Heroku

Steps to reproduce
What do you need to do to trigger this bug?

  1. https://heroku.com/deploy?template=https://github.com/firefly-iii/firefly-iii/tree/develop
  2. Go to Manage app > Resources > PostgreSQL attachment > Delete
  3. Go to Settings > Reveal config vars > Add the following
    Note: DB_CONNECTION needs to be "mysql" (no quotes)
    image
  4. More > Run console > type "bash"
    ( Follow instructions from docs: https://firefly-iii.readthedocs.io/en/latest/installation/server.html#initialize-the-database )
  5. Run "php artisan migrate:refresh --seed"
  6. See errors bellow.

Expected behavior
What do you expect to see after those steps?

No errors.

Extra info
Please add extra info here, such as OS, browser, and the output from the /debug page of your Firefly III installation (click the version at the bottom).

Firefly III: 4.7.7@develop
Heroku
Mysql: 5.7.23

Bonus points
Earn bonus points by:

  • Post a stacktrace from your log files
  • Add a screenshot

~ $ php artisan migrate:refresh --seed
Migration table not found.
Migration table created successfully.
Migrating: 2016_06_16_000000_create_support_tables
Migrated: 2016_06_16_000000_create_support_tables
Migrating: 2016_06_16_000001_create_users_table
Migrated: 2016_06_16_000001_create_users_table
Migrating: 2016_06_16_000002_create_main_tables
Migrated: 2016_06_16_000002_create_main_tables
Migrating: 2016_08_25_091522_changes_for_3101
Migrated: 2016_08_25_091522_changes_for_3101
Migrating: 2016_09_12_121359_fix_nullables
Migrated: 2016_09_12_121359_fix_nullables
Migrating: 2016_10_09_150037_expand_transactions_table
Migrating: 2016_10_22_075804_changes_for_v410
Migrated: 2016_10_22_075804_changes_for_v410
Migrating: 2016_11_24_210552_changes_for_v420
Migrated: 2016_11_24_210552_changes_for_v420
Migrating: 2016_12_22_150431_changes_for_v430
Migrated: 2016_12_22_150431_changes_for_v430
Migrating: 2016_12_28_203205_changes_for_v431
Migrated: 2016_12_28_203205_changes_for_v431
Migrating: 2017_04_13_163623_changes_for_v440
Migrated: 2017_04_13_163623_changes_for_v440
Migrating: 2017_06_02_105232_changes_for_v450
Migrated: 2017_06_02_105232_changes_for_v450
Migrating: 2017_08_20_062014_changes_for_v470
[2018-09-30 14:07:49] heroku.ERROR: SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 1000 bytes (SQL: alter table link_types add unique link_types_name_outward_inward_unique(name, outward, inward)) {"exception":"[object] (Illuminate\Database\QueryException(code: 42000): SQLSTATE[42000]: Syntax error or access violatio
n: 1071 Specified key was too long; max key length is 1000 bytes (SQL: alter table link_types add unique link_types_name_outward_inward_unique(name, outward, inward)) at /app/vendor/
laravel/framework/src/Illuminate/Database/Connection.php:664, Doctrine\DBAL\Driver\PDOException(code: 42000): SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too l
ong; max key length is 1000 bytes at /app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:144, PDOException(code: 42000): SQLSTATE[42000]: Syntax error or access violation: 1071
Specified key was too long; max key length is 1000 bytes at /app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:142)
[stacktrace]
#0 /app/vendor/laravel/framework/src/Illuminate/Database/Connection.php(624): Illuminate\Database\Connection->runQueryCallback('alter table li...', Array, Object(Closure)) #1 /app/vendor/laravel/framework/src/Illuminate/Database/Connection.php(459): Illuminate\\Database\\Connection->run('alter table li...', Array, Object(Closure))
#2 /app/vendor/laravel/framework/src/Illuminate/Database/Schema/Blueprint.php(88): Illuminate\Database\Connection->statement('alter table `li...')
#3 /app/vendor/laravel/framework/src/Illuminate/Database/Schema/Builder.php(264): Illuminate\Database\Schema\Blueprint->build(Object(Illuminate\Database\MySqlConnection), Object(Illuminat
e\Database\Schema\Grammars\MySqlGrammar))
#4 /app/vendor/laravel/framework/src/Illuminate/Database/Schema/Builder.php(165): Illuminate\Database\Schema\Builder->build(Object(Illuminate\Database\Schema\Blueprint))
#5 /app/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(223): Illuminate\Database\Schema\Builder->create('link_types', Object(Closure))
#6 /app/database/migrations/2017_08_20_062014_changes_for_v470.php(61): Illuminate\Support\Facades\Facade::__callStatic('create', Array)
#7 /app/vendor/laravel/framework/src/Illuminate/Database/Migrations/Migrator.php(360): ChangesForV470->up()
#8 /app/vendor/laravel/framework/src/Illuminate/Database/Migrations/Migrator.php(367): Illuminate\Database\Migrations\Migrator->Illuminate\Database\Migrations\{closure}()
#9 /app/vendor/laravel/framework/src/Illuminate/Database/Migrations/Migrator.php(178): Illuminate\Database\Migrations\Migrator->runMigration(Object(ChangesForV470), 'up')
#10 /app/vendor/laravel/framework/src/Illuminate/Database/Migrations/Migrator.php(147): Illuminate\Database\Migrations\Migrator->runUp('/app/database/m...', 1, false)
#11 /app/vendor/laravel/framework/src/Illuminate/Database/Migrations/Migrator.php(96): Illuminate\Database\Migrations\Migrator->runPending(Array, Array)
#12 /app/vendor/laravel/framework/src/Illuminate/Database/Console/Migrations/MigrateCommand.php(71): Illuminate\Database\Migrations\Migrator->run(Array, Array)
#13 [internal function]: Illuminate\Database\Console\Migrations\MigrateCommand->handle()
#14 /app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(29): call_user_func_array(Array, Array)
#15 /app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(87): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#16 /app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(31): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(C
losure))
#17 /app/vendor/laravel/framework/src/Illuminate/Container/Container.php(564): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#18 /app/vendor/laravel/framework/src/Illuminate/Console/Command.php(183): Illuminate\Container\Container->call(Array)
#19 /app/vendor/symfony/console/Command/Command.php(251): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArrayInput), Object(Illuminate\Console\OutputStyle)
)
#20 /app/vendor/laravel/framework/src/Illuminate/Console/Command.php(170): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArrayInput), Object(Ill
uminate\Console\OutputStyle))
#21 /app/vendor/laravel/framework/src/Illuminate/Console/Command.php(198): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArrayInput), Object(Illuminate\Console
\OutputStyle))
#22 /app/vendor/laravel/framework/src/Illuminate/Database/Console/Migrations/RefreshCommand.php(65): Illuminate\Console\Command->call('migrate', Array)
#23 [internal function]: Illuminate\Database\Console\Migrations\RefreshCommand->handle()
#24 /app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(29): call_user_func_array(Array, Array)
#25 /app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(87): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#26 /app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(31): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(C
losure))
#27 /app/vendor/laravel/framework/src/Illuminate/Container/Container.php(564): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#28 /app/vendor/laravel/framework/src/Illuminate/Console/Command.php(183): Illuminate\Container\Container->call(Array)
#29 /app/vendor/symfony/console/Command/Command.php(251): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#30 /app/vendor/laravel/framework/src/Illuminate/Console/Command.php(170): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illu
minate\Console\OutputStyle))
#31 /app/vendor/symfony/console/Application.php(886): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\Conso
leOutput))
#32 /app/vendor/symfony/console/Application.php(262): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Database\Console\Migrations\RefreshCommand), Object(Symfony\
Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#33 /app/vendor/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console
\Output\ConsoleOutput))
#34 /app/vendor/laravel/framework/src/Illuminate/Console/Application.php(89): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfon
y\Component\Console\Output\ConsoleOutput))
#35 /app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(122): Illuminate\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony
\Component\Console\Output\ConsoleOutput))
#36 /app/artisan(37): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#37 {main}
"}
In Connection.php line 664:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was t
oo long; max key length is 1000 bytes (SQL: alter table link_types add un
ique link_types_name_outward_inward_unique(name, outward, inward))

In PDOStatement.php line 144:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was t
oo long; max key length is 1000 bytes

In PDOStatement.php line 142:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was t
oo long; max key length is 1000 bytes

@hamuz
Copy link
Contributor Author

hamuz commented Sep 30, 2018

When you rerun the seed command, it appear to succeed:

~ $ php artisan migrate:refresh --seed
Rolling back: 2017_06_02_105232_changes_for_v450
Rolled back: 2017_06_02_105232_changes_for_v450
Rolling back: 2017_04_13_163623_changes_for_v440
Rolled back: 2017_04_13_163623_changes_for_v440
Rolling back: 2016_12_28_203205_changes_for_v431
Rolled back: 2016_12_28_203205_changes_for_v431
Rolling back: 2016_12_22_150431_changes_for_v430
Rolled back: 2016_12_22_150431_changes_for_v430
Rolling back: 2016_11_24_210552_changes_for_v420
Rolled back: 2016_11_24_210552_changes_for_v420
Rolling back: 2016_10_22_075804_changes_for_v410
Rolled back: 2016_10_22_075804_changes_for_v410
Rolling back: 2016_10_09_150037_expand_transactions_table
Rolled back: 2016_10_09_150037_expand_transactions_table
Rolling back: 2016_09_12_121359_fix_nullables
Rolled back: 2016_09_12_121359_fix_nullables
Rolling back: 2016_08_25_091522_changes_for_3101
Rolled back: 2016_08_25_091522_changes_for_3101
Rolling back: 2016_06_16_000002_create_main_tables
Rolled back: 2016_06_16_000002_create_main_tables
Rolling back: 2016_06_16_000001_create_users_table
Rolled back: 2016_06_16_000001_create_users_table
Rolling back: 2016_06_16_000000_create_support_tables
Rolled back: 2016_06_16_000000_create_support_tables
Migrating: 2016_06_16_000000_create_support_tables
Migrated: 2016_06_16_000000_create_support_tables
Migrating: 2016_06_16_000001_create_users_table
Migrated: 2016_06_16_000001_create_users_table
Migrating: 2016_06_16_000002_create_main_tables
Migrated: 2016_06_16_000002_create_main_tables
Migrating: 2016_08_25_091522_changes_for_3101
Migrated: 2016_08_25_091522_changes_for_3101
Migrating: 2016_09_12_121359_fix_nullables
Migrated: 2016_09_12_121359_fix_nullables
Migrating: 2016_10_09_150037_expand_transactions_table
Migrated: 2016_10_09_150037_expand_transactions_table
Migrating: 2016_10_22_075804_changes_for_v410
Migrated: 2016_10_22_075804_changes_for_v410
Migrating: 2016_11_24_210552_changes_for_v420
Migrated: 2016_11_24_210552_changes_for_v420
Migrating: 2016_12_22_150431_changes_for_v430
Migrated: 2016_12_22_150431_changes_for_v430
Migrating: 2016_12_28_203205_changes_for_v431
Migrated: 2016_12_28_203205_changes_for_v431
Migrating: 2017_04_13_163623_changes_for_v440
Migrated: 2017_04_13_163623_changes_for_v440
Migrating: 2017_06_02_105232_changes_for_v450
Migrated: 2017_06_02_105232_changes_for_v450
Migrating: 2017_08_20_062014_changes_for_v470
Migrated: 2017_08_20_062014_changes_for_v470
Migrating: 2017_11_04_170844_changes_for_v470a
Migrated: 2017_11_04_170844_changes_for_v470a
Migrating: 2018_01_01_000001_create_oauth_auth_codes_table
Migrated: 2018_01_01_000001_create_oauth_auth_codes_table
Migrating: 2018_01_01_000002_create_oauth_access_tokens_table
Migrated: 2018_01_01_000002_create_oauth_access_tokens_table
Migrating: 2018_01_01_000003_create_oauth_refresh_tokens_table
Migrated: 2018_01_01_000003_create_oauth_refresh_tokens_table
Migrating: 2018_01_01_000004_create_oauth_clients_table
Migrated: 2018_01_01_000004_create_oauth_clients_table
Migrating: 2018_01_01_000005_create_oauth_personal_access_clients_table
Migrated: 2018_01_01_000005_create_oauth_personal_access_clients_table
Migrating: 2018_03_19_141348_changes_for_v472
Migrated: 2018_03_19_141348_changes_for_v472
Migrating: 2018_04_07_210913_changes_for_v473
Migrated: 2018_04_07_210913_changes_for_v473
Migrating: 2018_04_29_174524_changes_for_v474
Migrated: 2018_04_29_174524_changes_for_v474
Migrating: 2018_06_08_200526_changes_for_v475
Migrated: 2018_06_08_200526_changes_for_v475
Migrating: 2018_09_05_195147_changes_for_v477
Migrated: 2018_09_05_195147_changes_for_v477
Seeding: AccountTypeSeeder
Seeding: TransactionCurrencySeeder
Seeding: TransactionTypeSeeder
Seeding: PermissionSeeder
Seeding: LinkTypeSeeder
Seeding: ConfigSeeder
[2018-09-30 14:21:06] heroku.WARNING: No database version entry is present. Database is assumed to be OLD (version 1).

@hamuz
Copy link
Contributor Author

hamuz commented Sep 30, 2018

This is a consistent pattern. 3rd time errors. 4th time no errors. and it goes on, as you re-run seeding.
No double link types thou.

@hamuz
Copy link
Contributor Author

hamuz commented Sep 30, 2018

Note: this is not a bad write up on how to install firefly with mysql on heroku :) should update docs with this.

@JC5
Copy link
Member

JC5 commented Sep 30, 2018

Which database encoding and collation does Heroku add by default? Can you tell?

@hamuz
Copy link
Contributor Author

hamuz commented Sep 30, 2018

image

This mysql database is not hosted on Heroku.

@JC5
Copy link
Member

JC5 commented Sep 30, 2018

Ah, unicode. I use utf8mb4_general_ci myself. I'll have to Google the difference.

@stale
Copy link

stale bot commented Oct 14, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale No replies or comments. Will be auto-closed in 14 days. label Oct 14, 2018
@stale stale bot closed this as completed Oct 21, 2018
@hamuz
Copy link
Contributor Author

hamuz commented Oct 21, 2018

@JC5 ?

@JC5
Copy link
Member

JC5 commented Oct 21, 2018

Oh completely forgot about this. The problem is the MyISAM storage engine that MySQL defaults to. Future versions of Firefly III will force this to be InnoDB. That should resolve the issue.

@hamuz
Copy link
Contributor Author

hamuz commented Oct 21, 2018

Can you briefly explain the design choice?

@JC5
Copy link
Member

JC5 commented Oct 21, 2018

Mainly the advantages of InnoDB over the advantages of MyISAM.

MyISAM is better when reading lots of data (which Firefly III does) but the write functionality of InnoDB seems to blow MyISAM out of the water. This is also relevant for the error you reported: long indexes for text data aren't supported in MyISAM.

https://www.percona.com/blog/2009/01/12/should-you-move-from-myisam-to-innodb/
https://stackoverflow.com/questions/20148/myisam-versus-innodb

@Bubka
Copy link

Bubka commented Jan 2, 2019

@JC5 I saw in the changelog that since v4.7.8 InnoDB is forced to prevent current issue and #1748.

Does this change is supposed to fix an existing install when upgrading to 4.7.8 (and higher) or does it apply to fresh install only ?

Because I encounter the #1748 issue (each link type is duplicated 6 times) even after my today's upgrade to 4.7.9. My MySQL db is still set with MyISAM - utf8mb4_unicode_ci

Can I just hard delete duplicates link_types records in MySQL (while they have no associated records) to clean up my install?

@JC5
Copy link
Member

JC5 commented Jan 2, 2019

It applies to fresh installations only, because it is only used for new tables. Could you check which indexes are on that table? Those should prevent extra rows from being created.

@JC5
Copy link
Member

JC5 commented Jan 2, 2019

And yes, you can safely delete them.

@Bubka
Copy link

Bubka commented Jan 3, 2019

Could you check which indexes are on that table? Those should prevent extra rows from being created.

I have only the primary one :
image

Compare to a fresh local install i made this morning the link_types_name_outward_inward_unique is missing :
image

@Bubka
Copy link

Bubka commented Jan 3, 2019

Obviously index creation failed because of Specified key was too long; max key length is 1000 bytes

@JC5
Copy link
Member

JC5 commented Jan 3, 2019

Because it's a MyISAM table. Figures 😅

@lock lock bot locked as resolved and limited conversation to collaborators Jan 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
stale No replies or comments. Will be auto-closed in 14 days.
Projects
None yet
Development

No branches or pull requests

3 participants