Serialization: Cannot load into empty/blank target

[223kazuki]

Describe the bug
When I try to dump and load data in metabase v1.39.3, it throws an exception in Inserting User step. So I can't restore data.

Logs

2021-06-16 10:59:37,443 INFO serialization.cmd :: BEGIN LOAD from dump with context {:mode :update, :on-error :abort}
2021-06-16 10:59:37,464 INFO serialization.upsert :: Inserting User
2021-06-16 10:59:37,485 ERROR serialization.cmd :: ERROR LOAD from dump: Assert failed: ((every-pred string? (complement str/blank?)) password)
java.lang.AssertionError: Assert failed: ((every-pred string? (complement str/blank?)) password)
        at metabase.models.user$pre_insert.invokeStatic(user.clj:28)
        at metabase.models.user$pre_insert.invoke(user.clj:25)
        at toucan.models$fn__24435$fn__24469$G__24442__24474.invoke(models.clj:178)
        at toucan.models$do_pre_insert.invokeStatic(models.clj:329)
        at toucan.models$do_pre_insert.invoke(models.clj:324)
        at toucan.db$insert_BANG_.invokeStatic(db.clj:610)
        at toucan.db$insert_BANG_.invoke(db.clj:597)
        at metabase_enterprise.serialization.upsert$insert_many_individually_BANG_$iter__57434__57438$fn__57439$fn__57440.invoke(upsert.clj:82)
        at metabase_enterprise.serialization.upsert$insert_many_individually_BANG_$iter__57434__57438$fn__57439.invoke(upsert.clj:80)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:51)
        at clojure.lang.RT.seq(RT.java:535)
        at clojure.core$seq__5402.invokeStatic(core.clj:137)
        at clojure.core$map$fn__5870.invoke(core.clj:2757)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:58)
        at clojure.lang.RT.seq(RT.java:535)
        at clojure.core$seq__5402.invokeStatic(core.clj:137)
        at clojure.core$concat$cat__5495$fn__5496.invoke(core.clj:734)
        at clojure.lang.LazySeq.sval(LazySeq.java:42)
        at clojure.lang.LazySeq.seq(LazySeq.java:51)
        at clojure.lang.RT.seq(RT.java:535)
        at clojure.core$seq__5402.invokeStatic(core.clj:137)
        at clojure.core$sort.invokeStatic(core.clj:3113)
        at clojure.core$sort_by.invokeStatic(core.clj:3119)
        at clojure.core$sort_by.invokeStatic(core.clj:3119)
        at clojure.core$sort_by.invoke(core.clj:3119)
        at metabase_enterprise.serialization.upsert$maybe_upsert_many_BANG_.invokeStatic(upsert.clj:144)
        at metabase_enterprise.serialization.upsert$maybe_upsert_many_BANG_.invoke(upsert.clj:119)
        at metabase_enterprise.serialization.load$fn__58123.invokeStatic(load.clj:641)
        at metabase_enterprise.serialization.load$fn__58123.invoke(load.clj:638)
        at clojure.lang.MultiFn.invoke(MultiFn.java:234)
        at metabase_enterprise.serialization.cmd$fn__58635$load__58640$fn__58641.invoke(cmd.clj:52)
        at metabase_enterprise.serialization.cmd$fn__58635$load__58640.invoke(cmd.clj:39)
        at clojure.lang.Var.invoke(Var.java:388)
        at metabase.cmd$load.invokeStatic(cmd.clj:150)
        at metabase.cmd$load.doInvoke(cmd.clj:141)
        at clojure.lang.RestFn.applyTo(RestFn.java:139)
        at clojure.core$apply.invokeStatic(core.clj:665)
        at clojure.core$apply.invoke(core.clj:660)
        at metabase.cmd$run_cmd$fn__82234.invoke(cmd.clj:191)
        at metabase.cmd$run_cmd.invokeStatic(cmd.clj:191)
        at metabase.cmd$run_cmd.invoke(cmd.clj:187)
        at clojure.lang.Var.invoke(Var.java:388)
        at metabase.core$run_cmd.invokeStatic(core.clj:149)
        at metabase.core$run_cmd.invoke(core.clj:147)
        at metabase.core$_main.invokeStatic(core.clj:171)
        at metabase.core$_main.doInvoke(core.clj:166)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at metabase.core.main(Unknown Source)

To Reproduce

1. Run metabase by java -jar metabase-v1.39.3.jar.
2. Create a user according to Web UI.
3. Stop metabase and dump data by java -jar metabase-v1.39.3.jar dump ./dump --user [my mail address]
4. Remove existing h2 database by rm -rf metabase.db.*
5. Load dump data by java -jar metabase-v1.39.3.jar load ./dump --mode update --on-error abort
6. See the error above

Expected behavior
Should restore all dump data.

Information about your Metabase Installation:

{
  "browser-info": {
    "language": "en-US",
    "platform": "MacIntel",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36",
    "vendor": "Google Inc."
  },
  "system-info": {
    "file.encoding": "UTF-8",
    "java.runtime.name": "OpenJDK Runtime Environment",
    "java.runtime.version": "11.0.9.1+1",
    "java.vendor": "AdoptOpenJDK",
    "java.vendor.url": "https://adoptopenjdk.net/",
    "java.version": "11.0.9.1",
    "java.vm.name": "OpenJDK 64-Bit Server VM",
    "java.vm.version": "11.0.9.1+1",
    "os.name": "Mac OS X",
    "os.version": "10.16",
    "user.language": "en",
    "user.timezone": "America/Los_Angeles"
  },
  "metabase-info": {
    "databases": [
      "h2"
    ],
    "hosting-env": "unknown",
    "application-database": "h2",
    "application-database-details": {
      "database": {
        "name": "H2",
        "version": "1.4.197 (2018-03-18)"
      },
      "jdbc-driver": {
        "name": "H2 JDBC Driver",
        "version": "1.4.197 (2018-03-18)"
      }
    },
    "run-mode": "prod",
    "version": {
      "date": "2021-05-27",
      "tag": "v1.39.3",
      "branch": "release-x.39.x",
      "hash": "598a112"
    },
    "settings": {
      "report-timezone": null
    }
  }
}

Severity
Can workaround by using a previous version.

Additional context
It doesn't happen in v1.39.2

[flamber]

Hi @223kazuki

That was unfortunate. I tested all the fixes in #15858 so many times, but never on an empty target, since the main use-case for Serialization is existing targets. I'm curious, do you normally always load to an empty target?

Workaround would be to do the setup again before step 5, so the load-process can see the user. It could be done via API.

[223kazuki]

@flamber Hi, thanks for the reply. Yes, we always load to an empty target in our local development workflow. I think your workaround works and I can also use the previous version which doesn't have this issue. And I understand restoring from empty is not the main use-case. But do you have a plan to fix this or not? Thanks.

[flamber]

@223kazuki We will address this, since it's highly recommended to use 1.39.3+ for Serialization, which fixed more than 15 problems and that version provides much better logging too. I just don't have a timeline for a fix right now.

[223kazuki]

@flamber Thanks. Yeah, I also found some other serialization issues. Hopefully, 1.39.3+ solves them. So I will adopt your workaround for now.

[223kazuki]

@flamber I tried your workaround. But to create initial user by setup endpoint, we need setup-token. I found it in html. But I think it's not a clean way to use it. Do we have a clean way to create an initial user vi API?

[223kazuki]

I found MB_SETUP_TOKEN. But even if I can get a setup token, calling REST API to create user makes our development workflow difficult. So we need to solve the original issue in order to upgrade our metabase to 1.39.3.

[flamber]

@223kazuki An alternative is to copy a target database in, which just has one user and nothing else. Then you don't need to use the API. We are prioritizing this issue internally.

[223kazuki]

@flamber Thanks. That's one idea. But since we also need to use postgres in dev, we'd like to wait for a fixed version. (Not easy to copy db)

And I also realized there's a new version, 1.39.4. But this issue still exists.

[jeff303]

Should be fixed by commit 2 under #17388

The behavior now will be:

• If a new user is inserted by the load process (i.e. not matching an email for any existing one), they will be assigned a randomly generated password (which is not printed anywhere), and a password reset email will be generated for that new user, and sent to them, the same way that would happen in the webapp (assuming SMTP is set up in the target instance)

[223kazuki]

@jeff303 Thanks so much!