Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration doesn't like rows with null characters #40835

Closed
bagafoot opened this issue Apr 1, 2024 · 5 comments
Closed

Migration doesn't like rows with null characters #40835

bagafoot opened this issue Apr 1, 2024 · 5 comments
Assignees
Labels
Database/Postgres .Escalation Operation/Database Migrations Issues with application DB migrations when launching Metabase Priority:P1 Security holes w/o exploit, crashing, setup/upgrade, login, broken common features, correctness .Team/BackendComponents also known as BEC Type:Bug Product defects
Milestone

Comments

@bagafoot
Copy link

bagafoot commented Apr 1, 2024

Describe the bug

Hello,
I'm running metabase service on ubuntu and postgresql 14 as database.
psql (14.9 (Ubuntu 14.9-0ubuntu0.22.04.1))
Trying to update from 48.8 to 49.2 but metabase service doent work properly,
when check journalctl there are some errors as

Caused by: liquibase.exception.MigrationFailedException: Migration failed for changeset migrations/001_update_migrations.yaml::v49.2024-01-22T11:52:00::qnkhuat:
Reason: clojure.lang.ExceptionInfo: ERROR: unsupported Unicode escape sequence

another error:

ERROR jdbcjobstore.JobStoreTX :: MisfireHandler: Error handling misfires: Couldn't store trigger 'DEFAULT.metabase.task.truncate-audit-log.trigger' for 'DEFAULT.metabase.task.truncate-audit-log.job' job:Cou>
 org.quartz.JobPersistenceException: Couldn't store trigger 'DEFAULT.metabase.task.truncate-audit-log.trigger' for 'DEFAULT.metabase.task.truncate-audit-log.job' job:Couldn't retrieve job because a required class was not found: met>

full log:
https://hackmd.io/@OdFNQ07XQeCmq8QL3wcWzw/BJluLgOJ0

To Reproduce

follow update instructions.
Change newer merabase.jar and start metabase service

Expected behavior

No response

Logs

No response

Information about your Metabase installation

Chrome Version 121.0.6167.140 (Official Build) (64-bit)
Ubuntu 22.04.1
Postgresql 14.9
Metabase 0.48.8
Jar File on Ubuntu
Postgresql 14.9

Severity

FATAL

Additional context

No response

@bagafoot bagafoot added .Needs Triage Type:Bug Product defects labels Apr 1, 2024
@qnkhuat qnkhuat added Priority:P3 Cosmetic bugs, minor bugs with a clear workaround Database/Postgres Operation/Database Migrations Issues with application DB migrations when launching Metabase .Team/BackendComponents also known as BEC and removed .Needs Triage labels Apr 1, 2024
@qnkhuat
Copy link
Contributor

qnkhuat commented Apr 1, 2024

the query being executed is

UPDATE revision
               SET object = jsonb_set(
                  object::jsonb, '{type}',
                  to_jsonb(CASE
                              WHEN (object::jsonb->>'dataset')::boolean THEN 'model'
                              ELSE 'question'
                           END)::jsonb, true)
               WHERE model = 'Card' AND (object::jsonb->>'dataset') IS NOT NULL;

it's probably something weird in your revision.object that makes this query fail.

@paoliniluis
Copy link
Contributor

Apr 01 11:34:37 METABASE06S01 metabase[1162568]:   Detail: \u0000 cannot be converted to text.
Apr 01 11:34:37 METABASE06S01 metabase[1162568]:   Where: JSON data, line 1: ....\\..\\..\\..\\..\\..\\..\\windows\\win.ini\u0000... {:toucan2/context-trace [["execute SQL with class com.mchange.v2.c3p0.impl.NewProxyConnection" {:toucan2.jdbc.query/sql-args ["UPDATE revision\n >

this is interesting, what's that win.ini thing in the database?

@bagafoot
Copy link
Author

bagafoot commented Apr 1, 2024

@qnkhuat @paoliniluis
this is the problematic row

{
  "description": "Description",
  "archived": false,
  "collection_position": null,
  "table_id": 180,
  "database_id": 2,
  "enable_embedding": false,
  "collection_id": null,
  "query_type": "query",
  "name": "Beptr FactTumProjeler",
  "embedding_params": null,
  "cache_ttl": null,
  "dataset_query": {
    "database": 2,
    "type": "query",
    "query": {
      "source-table": 180
    }
  },
  "parameter_mappings": [],
  "display": "table",
  "collection_preview": true,
  "visualization_settings": {
    "table.pivot_column": "ProjectState_Id",
    "table.cell_column": "..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\windows\\win.ini\u0000Project_Id"
  },
  "parameters": [],
  "dataset": false
}

@paoliniluis paoliniluis changed the title Migration failed for changeset migrations/001_update_migrations.yaml::v49. Migration doesn't like rows with slashes Apr 1, 2024
@paoliniluis
Copy link
Contributor

So the fix is simple, wipe that stuff from the database. I just changed the name of the issue to reflect the actual problem

@darksciencebase darksciencebase removed the .Team/BackendComponents also known as BEC label Apr 2, 2024
@cbalusek cbalusek added Priority:P1 Security holes w/o exploit, crashing, setup/upgrade, login, broken common features, correctness .Escalation labels Apr 18, 2024
@darksciencebase darksciencebase removed the Priority:P3 Cosmetic bugs, minor bugs with a clear workaround label Apr 18, 2024
@calherries calherries self-assigned this Apr 18, 2024
@calherries calherries added the .Team/BackendComponents also known as BEC label Apr 18, 2024
@calherries calherries changed the title Migration doesn't like rows with slashes Migration doesn't like rows with null characters Apr 18, 2024
@calherries
Copy link
Contributor

resolved by #41584, the issue was that for postgres, converting text to JSONB doesn't work if the text contains any null characters.

@calherries calherries added this to the 0.49.7 milestone Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Database/Postgres .Escalation Operation/Database Migrations Issues with application DB migrations when launching Metabase Priority:P1 Security holes w/o exploit, crashing, setup/upgrade, login, broken common features, correctness .Team/BackendComponents also known as BEC Type:Bug Product defects
Projects
None yet
Development

No branches or pull requests

6 participants