Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All database tables doesn't have a primary key column #12971

Closed
swtch1 opened this issue Aug 17, 2018 · 7 comments · Fixed by #22255
Closed

All database tables doesn't have a primary key column #12971

swtch1 opened this issue Aug 17, 2018 · 7 comments · Fixed by #22255

Comments

@swtch1
Copy link
Contributor

swtch1 commented Aug 17, 2018

When using Grafana with MariaDB with Galera cluster the database migration/creation fails because of a the setting innodb_force_primary_key=1, which is required for Galera cluster to work properly. You don't need to set up a Galera cluster to reproduce this issue, you can do it by setting that property on a normal MariaDB instance.

I have corrected the issue in PR 12970.

What Grafana version are you using?

5.2.2

What OS are you running grafana on?

RHEL 7.3

What did you do?

bin/grafana-server web

What was the expected result?

Successful service start

What happened instead?

EROR[08-17|10:29:06] Executing migration failed               logger=migrator id="Create annotation_tag table v2" error="Error 1173: This table type requires a primary key"
EROR[08-17|10:29:06] Exec failed                              logger=migrator error="Error 1173: This table type requires a primary key" sql="CREATE TABLE IF NOT EXISTS `annotation_tag` (\n`annotation_id` BIGINT(20) NOT NULL\n, `tag_id` BIGINT(20) NOT NULL\n) ENGINE=InnoDB DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci;"
EROR[08-17|10:29:06] Server shutdown                          logger=server reason="Service init failed: Migration failed err: Error 1173: This table type requires a primary key"
@bergquist
Copy link
Contributor

#12970 (comment)

@azhiltsov
Copy link
Contributor

I faced the same issue running MySQL > 8.0.13 with sql_require_primary_key=ON
I tried to resolve it by adding new column 'id' with primary index to the tables without primaries.
However this break the tests and I couldn't figure why.
Would really appreciate if someone can help me.

azhiltsov added a commit to azhiltsov/grafana that referenced this issue Mar 29, 2020
The annotation_tag and alert_rule_tag tables did not have
PRIMARY KEY defined what cause problems with migration to
MariaDB with Galera setup with
 innodb_force_primary_key=1
Or MySQL > 8.0.13 with
 sql_require_primary_key=ON

Which can manifest as follows:

MariaDB
Error 1173: This table type requires a primary key

MySQL
ERROR 3750 (HY000): Unable to create or change a table
without a primary key, when the system variable 'sql_require_primary_key' is set.

Extra reading for curious:
https://jfg-mysql.blogspot.com/2017/08/danger-no-pk-with-RBR-and-mariadb-protection.html

Fixes grafana#12971
@marefr marefr changed the title migration on MariaDB: table type requires primary key All database tables doesn't have a primary key column Apr 22, 2020
azhiltsov added a commit to azhiltsov/grafana that referenced this issue Jun 9, 2020
The annotation_tag and alert_rule_tag tables did not have
PRIMARY KEY defined what cause problems with migration to
MariaDB with Galera setup with
 innodb_force_primary_key=1
Or MySQL > 8.0.13 with
 sql_require_primary_key=ON

Which can manifest as follows:

MariaDB
Error 1173: This table type requires a primary key

MySQL
ERROR 3750 (HY000): Unable to create or change a table
without a primary key, when the system variable 'sql_require_primary_key' is set.

Extra reading for curious:
https://jfg-mysql.blogspot.com/2017/08/danger-no-pk-with-RBR-and-mariadb-protection.html

Fixes grafana#12971
@hongkongkiwi
Copy link

I'm facing the same issue, DigitalOcean Managed MySQL databases require a primary key.

So I can't use grafana with this because the tables have no primary keys defined.

Can this be fixed? Any reason for specifically not adding primary keys?

@hongkongkiwi
Copy link

I had to dump the sql schema and then add some primary keys on the missing tables then import. It would be great if this was just part of the original schema.

marefr added a commit that referenced this issue Nov 17, 2020
The annotation_tag and alert_rule_tag tables did not have
PRIMARY KEY defined what cause problems with migration to
MariaDB with Galera setup with
 innodb_force_primary_key=1
Or MySQL > 8.0.13 with
 sql_require_primary_key=ON

Which can manifest as follows:

MariaDB
Error 1173: This table type requires a primary key

MySQL
ERROR 3750 (HY000): Unable to create or change a table
without a primary key, when the system variable 'sql_require_primary_key' is set.

Extra reading for curious:
https://jfg-mysql.blogspot.com/2017/08/danger-no-pk-with-RBR-and-mariadb-protection.html

Fixes #12971

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
@marefr marefr added this to the 7.4.0 milestone Nov 17, 2020
ryantxu pushed a commit that referenced this issue Nov 18, 2020
The annotation_tag and alert_rule_tag tables did not have
PRIMARY KEY defined what cause problems with migration to
MariaDB with Galera setup with
 innodb_force_primary_key=1
Or MySQL > 8.0.13 with
 sql_require_primary_key=ON

Which can manifest as follows:

MariaDB
Error 1173: This table type requires a primary key

MySQL
ERROR 3750 (HY000): Unable to create or change a table
without a primary key, when the system variable 'sql_require_primary_key' is set.

Extra reading for curious:
https://jfg-mysql.blogspot.com/2017/08/danger-no-pk-with-RBR-and-mariadb-protection.html

Fixes #12971

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
ryantxu pushed a commit that referenced this issue Nov 18, 2020
The annotation_tag and alert_rule_tag tables did not have
PRIMARY KEY defined what cause problems with migration to
MariaDB with Galera setup with
 innodb_force_primary_key=1
Or MySQL > 8.0.13 with
 sql_require_primary_key=ON

Which can manifest as follows:

MariaDB
Error 1173: This table type requires a primary key

MySQL
ERROR 3750 (HY000): Unable to create or change a table
without a primary key, when the system variable 'sql_require_primary_key' is set.

Extra reading for curious:
https://jfg-mysql.blogspot.com/2017/08/danger-no-pk-with-RBR-and-mariadb-protection.html

Fixes #12971

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
ryantxu pushed a commit that referenced this issue Nov 18, 2020
The annotation_tag and alert_rule_tag tables did not have
PRIMARY KEY defined what cause problems with migration to
MariaDB with Galera setup with
 innodb_force_primary_key=1
Or MySQL > 8.0.13 with
 sql_require_primary_key=ON

Which can manifest as follows:

MariaDB
Error 1173: This table type requires a primary key

MySQL
ERROR 3750 (HY000): Unable to create or change a table
without a primary key, when the system variable 'sql_require_primary_key' is set.

Extra reading for curious:
https://jfg-mysql.blogspot.com/2017/08/danger-no-pk-with-RBR-and-mariadb-protection.html

Fixes #12971

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
@mjseaman mjseaman added this to the 7.4.0 milestone Jan 8, 2021
@Silvenga
Copy link

Silvenga commented Feb 5, 2021

@marefr I'm not sure this is fixed, with an empty database:

t=2021-02-05T04:44:38+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2"
t=2021-02-05T04:44:38+0000 lvl=eror msg="Executing migration failed" logger=migrator id="copy alert_rule_tag v1 to v2" error="Error 3098: The table does not comply with the requirements by an external plugin."
t=2021-02-05T04:44:38+0000 lvl=eror msg="Exec failed" logger=migrator error="Error 3098: The table does not comply with the requirements by an external plugin." sql="INSERT INTO `alert_rule_tag` (`tag_id`\n, `alert_id`) SELECT `tag_id`\n, `alert_id` FROM `alert_rule_tag_v1`"
service init failed: migration failed: Error 3098: The table does not comply with the requirements by an external plugin.

If I'm reading the code right, the temporary table used for migration doesn't have a primary key, so the migration fails.

@Silvenga
Copy link

Silvenga commented Feb 5, 2021

Anyone who comes across this from Google:

  • You can start Grafana targeting a temp database server.
  • Dump the schema (generated by the migrations).
  • Import it into your MySql cluster that requires primary keys.
  • Profit!

@dansimone
Copy link

This issue was closed as of #22255, but one thing not addressed by that PR is creating all tables initially with primary keys. What Grafana does now is:

  1. Create alert_rule_tag and annotation_tag tables without primary keys
  2. Migrate them to new schemas that contain primary keys

However, if you are running using MySQL with sql_require_primary_key=ON, then step 1) fails out of the gate.

Is there any reason to not change things so that the initial schema created (for a fresh Grafana instance) for alert_rule_tag and annotation_tag contain primary keys.

The workaround provided by @Silvenga and others technically work, but seem like unnecessary hoops to have to jump through.

double-yaya added a commit to double-yaya/grafana that referenced this issue Mar 3, 2023
fix mysql PrimaryKey  grafana#12971
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment