New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Ecto 3.0 #728

Merged
merged 14 commits into from Oct 17, 2018

Conversation

Projects
None yet
3 participants
@wojtekmach
Member

wojtekmach commented Oct 6, 2018

TODO:

  • Telemetry-related error?

    $ mix test
    18:04:46.201 [error] an exception was raised logging %DBConnection.LogEntry{call: :commit, connection_time: 422000, decode_time: nil, params: nil, pool_time: nil, query: :commit, result: {:ok, %Postgrex.Result{columns: nil, command: :commit, connection_id: 11103, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
        (stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query])
        lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1
        lib/telemetry.ex:76: Telemetry.execute/4
        lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4
        (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
        (db_connection) lib/db_connection.ex:1363: DBConnection.run_transaction/4
        lib/ecto/adapters/sql.ex:686: Ecto.Adapters.SQL.lock_for_migrations/4
        lib/ecto/migrator.ex:285: Ecto.Migrator.lock_for_migrations/3
        lib/ecto/adapters/sql/sandbox.ex:472: Ecto.Adapters.SQL.Sandbox.unboxed_run/2
        lib/mix/tasks/ecto.migrate.ex:92: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
        (elixir) lib/enum.ex:768: Enum."-each/2-lists^foreach/1-0-"/2
        (elixir) lib/enum.ex:768: Enum.each/2
        (mix) lib/mix/task.ex:319: Mix.Task.run_task/3
        (mix) lib/mix/task.ex:353: Mix.Task.run_alias/3
        (mix) lib/mix/task.ex:280: Mix.Task.run/2
        (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
        (elixir) src/elixir_compiler.erl:79: :elixir_compiler.dispatch/4
        (elixir) src/elixir_compiler.erl:63: :elixir_compiler.compile/3
        (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
        (elixir) src/elixir_compiler.erl:23: :elixir_compiler.quoted/2
    
  • per Ecto changelog, config :ecto, json_library: Jason should emit a warning but it doesn't

  • remaining calendar types-related errors

cc @josevalim

@josevalim

This comment has been minimized.

Member

josevalim commented Oct 6, 2018

Telemetry-related error?

It seems the Telemetry app is not being started? Try removing _build. If it works, this can be an Elixir bug, so it would be nice if you can reproduce it consistently.

per Ecto changelog, config :ecto, json_library: Jason should emit a warning but it doesn't

Fixed in EctoSQL master.

@wojtekmach

This comment has been minimized.

Member

wojtekmach commented Oct 6, 2018

@josevalim Removing _build didn't help. I'm getting the error when running migrations, stack trace below. Starting app with just e.g. iex -S mix is fine.

``` ~/src/hexpm/hexpm[wm-ecto-3]% mix ecto.migrate [error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 383000, decode_time: 2335000, params: [], pool_time: 112201000, query: %Postgrex.Query{columns: nil, name: "", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: ["CREATE TABLE ", "IF NOT EXISTS ", [34, "schema_migrations", 34], 32, 40, [[[], [[34, "version", 34], 32, "bigint", [[], []]], ", "], [34, "inserted_at", 34], 32, ["timestamp", "(0)"], [[], []]], [", ", "PRIMARY KEY (", [[], 34, "version", 34], ")"], 41, []], types: nil}, result: {:ok, %Postgrex.Query{columns: nil, name: "", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.3362657775.3597664257.207272>, result_formats: [], result_oids: nil, result_types: nil, statement: ["CREATE TABLE ", "IF NOT EXISTS ", [34, "schema_migrations", 34], 32, 40, [[[], [[34, "version", 34], 32, "bigint", [[], []]], ", "], [34, "inserted_at", 34], 32, ["timestamp", "(0)"], [[], []]], [", ", "PRIMARY KEY (", [[], 34, "version", 34], ")"], 41, []], types: {Postgrex.DefaultTypes, #Reference<0.3362657775.3597795329.207228>}}, %Postgrex.Result{columns: nil, command: :create_table, connection_id: 13585, messages: [%{code: "42P07", file: "parse_utilcmd.c", line: "209", message: "relation \"schema_migrations\" already exists, skipping", routine: "transformCreateStmt", severity: "NOTICE", unknown: "NOTICE"}], num_rows: 0, rows: nil}}}: ** (ArgumentError) argument error (stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query]) lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1 lib/telemetry.ex:76: Telemetry.execute/4 lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4 (db_connection) lib/db_connection.ex:1303: DBConnection.log/5 (postgrex) lib/postgrex.ex:146: Postgrex.query/4 lib/ecto/adapters/sql.ex:288: Ecto.Adapters.SQL.query!/4 (elixir) lib/enum.ex:1326: Enum."-map/2-lists^map/1-0-"/2 lib/ecto/adapters/sql.ex:671: Ecto.Adapters.SQL.execute_ddl/4 lib/ecto/migrator.ex:453: Ecto.Migrator.verbose_schema_migration/3 lib/ecto/migrator.ex:237: Ecto.Migrator.run/4 lib/mix/tasks/ecto.migrate.ex:94: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2 (elixir) lib/enum.ex:768: Enum."-each/2-lists^foreach/1-0-"/2 (elixir) lib/enum.ex:768: Enum.each/2 (mix) lib/mix/task.ex:319: Mix.Task.run_task/3 (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2 (elixir) src/elixir_compiler.erl:79: :elixir_compiler.dispatch/4 (elixir) src/elixir_compiler.erl:63: :elixir_compiler.compile/3 (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2 (elixir) src/elixir_compiler.erl:23: :elixir_compiler.quoted/2

[error] an exception was raised logging %DBConnection.LogEntry{call: :begin, connection_time: 413000, decode_time: nil, params: nil, pool_time: 29000, query: :begin, result: {:ok, %DBConnection{conn_mode: nil, conn_ref: #Reference<0.3362657775.3597664257.207392>, pool_ref: {:pool_ref, #PID<0.267.0>, #Reference<0.3362657775.3597795329.207073>, nil, #Reference<0.3362657775.3597795329.207270>}}, %Postgrex.Result{columns: nil, command: :commit, connection_id: 13584, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
(stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query])
lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1
lib/telemetry.ex:76: Telemetry.execute/4
lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4
(db_connection) lib/db_connection.ex:1303: DBConnection.log/5
(db_connection) lib/db_connection.ex:760: DBConnection.transaction/3
lib/ecto/adapters/sql.ex:686: Ecto.Adapters.SQL.lock_for_migrations/4
lib/ecto/migrator.ex:285: Ecto.Migrator.lock_for_migrations/3
lib/mix/tasks/ecto.migrate.ex:94: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
(elixir) lib/enum.ex:768: Enum."-each/2-lists^foreach/1-0-"/2
(elixir) lib/enum.ex:768: Enum.each/2
(mix) lib/mix/task.ex:319: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
(elixir) src/elixir_compiler.erl:79: :elixir_compiler.dispatch/4
(elixir) src/elixir_compiler.erl:63: :elixir_compiler.compile/3
(elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
(elixir) src/elixir_compiler.erl:23: :elixir_compiler.quoted/2
(elixir) lib/code.ex:767: Code.require_file/2
(elixir) lib/kernel/cli.ex:503: Kernel.CLI.wrapper/1
(elixir) lib/enum.ex:1326: Enum."-map/2-lists^map/1-0-"/2

[error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 940000, decode_time: 16000, params: [], pool_time: nil, query: %Postgrex.Query{columns: nil, name: "ecto_1634", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: "SELECT s0."version"::bigint FROM "schema_migrations" AS s0 FOR UPDATE", types: nil}, result: {:ok, %Postgrex.Query{columns: ["version"], name: "ecto_1634", param_formats: [], param_oids: [], param_types: [], ref: #Reference<0.3362657775.3597664257.207439>, result_formats: [:binary], result_oids: [20], result_types: [Postgrex.Extensions.Int8], statement: "SELECT s0."version"::bigint FROM "schema_migrations" AS s0 FOR UPDATE", types: {Postgrex.DefaultTypes, #Reference<0.3362657775.3597795329.207228>}}, %Postgrex.Result{columns: ["version"], command: :select, connection_id: 13584, messages: [], num_rows: 0, rows: []}}}: ** (ArgumentError) argument error
(stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query])
lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1
lib/telemetry.ex:76: Telemetry.execute/4
lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4
(db_connection) lib/db_connection.ex:1303: DBConnection.log/5
lib/ecto/adapters/sql.ex:525: Ecto.Adapters.SQL.execute!/4
lib/ecto/adapters/sql.ex:517: Ecto.Adapters.SQL.execute/5
(ecto) lib/ecto/repo/queryable.ex:147: Ecto.Repo.Queryable.execute/4
(ecto) lib/ecto/repo/queryable.ex:18: Ecto.Repo.Queryable.all/3
lib/ecto/migrator.ex:285: anonymous fn/3 in Ecto.Migrator.lock_for_migrations/3
lib/ecto/adapters/sql.ex:774: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection) lib/db_connection.ex:1341: DBConnection.run_transaction/4
lib/ecto/adapters/sql.ex:686: Ecto.Adapters.SQL.lock_for_migrations/4
lib/ecto/migrator.ex:285: Ecto.Migrator.lock_for_migrations/3
lib/mix/tasks/ecto.migrate.ex:94: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
(elixir) lib/enum.ex:768: Enum."-each/2-lists^foreach/1-0-"/2
(elixir) lib/enum.ex:768: Enum.each/2
(mix) lib/mix/task.ex:319: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
(elixir) src/elixir_compiler.erl:79: :elixir_compiler.dispatch/4

[error] an exception was raised logging %DBConnection.LogEntry{call: :begin, connection_time: 289000, decode_time: nil, params: nil, pool_time: 43000, query: :begin, result: {:ok, %DBConnection{conn_mode: nil, conn_ref: #Reference<0.3362657775.3597664258.204220>, pool_ref: {:pool_ref, #PID<0.267.0>, #Reference<0.3362657775.3597795329.207073>, nil, #Reference<0.3362657775.3597795329.207267>}}, %Postgrex.Result{columns: nil, command: :commit, connection_id: 13585, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
(stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query])
lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1
lib/telemetry.ex:76: Telemetry.execute/4
lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4
(db_connection) lib/db_connection.ex:1303: DBConnection.log/5
(db_connection) lib/db_connection.ex:760: DBConnection.transaction/3
lib/ecto/migrator.ex:183: Ecto.Migrator.do_run_maybe_in_transaction/3
(elixir) lib/task/supervised.ex:88: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
(stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[error] an exception was raised logging %DBConnection.LogEntry{call: :rollback, connection_time: 327000, decode_time: nil, params: nil, pool_time: nil, query: :rollback, result: {:ok, %Postgrex.Result{columns: nil, command: :rollback, connection_id: 13585, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
(stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query])
lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1
lib/telemetry.ex:76: Telemetry.execute/4
lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4
(db_connection) lib/db_connection.ex:1303: DBConnection.log/5
(db_connection) lib/db_connection.ex:1359: DBConnection.run_transaction/4
lib/ecto/migrator.ex:183: Ecto.Migrator.do_run_maybe_in_transaction/3
(elixir) lib/task/supervised.ex:88: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
(stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[error] an exception was raised logging %DBConnection.LogEntry{call: :commit, connection_time: 386000, decode_time: nil, params: nil, pool_time: nil, query: :commit, result: {:ok, %Postgrex.Result{columns: nil, command: :commit, connection_id: 13584, messages: [], num_rows: nil, rows: nil}}}: ** (ArgumentError) argument error
(stdlib) :ets.lookup(Telemetry.Impl.Ets, [:hexpm, :repo_base, :query])
lib/telemetry/impl/ets.ex:49: Telemetry.Impl.Ets.list_handlers_for_event/1
lib/telemetry.ex:76: Telemetry.execute/4
lib/ecto/adapters/sql.ex:749: Ecto.Adapters.SQL.log/4
(db_connection) lib/db_connection.ex:1303: DBConnection.log/5
(db_connection) lib/db_connection.ex:1363: DBConnection.run_transaction/4
lib/ecto/adapters/sql.ex:686: Ecto.Adapters.SQL.lock_for_migrations/4
lib/ecto/migrator.ex:285: Ecto.Migrator.lock_for_migrations/3
lib/mix/tasks/ecto.migrate.ex:94: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
(elixir) lib/enum.ex:768: Enum."-each/2-lists^foreach/1-0-"/2
(elixir) lib/enum.ex:768: Enum.each/2
(mix) lib/mix/task.ex:319: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
(elixir) src/elixir_compiler.erl:79: :elixir_compiler.dispatch/4
(elixir) src/elixir_compiler.erl:63: :elixir_compiler.compile/3
(elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
(elixir) src/elixir_compiler.erl:23: :elixir_compiler.quoted/2
(elixir) lib/code.ex:767: Code.require_file/2
(elixir) lib/kernel/cli.ex:503: Kernel.CLI.wrapper/1
(elixir) lib/enum.ex:1326: Enum."-map/2-lists^map/1-0-"/2

** (exit) exited in: GenServer.call(Ecto.Migration.Supervisor, {:start_child, [#PID<0.278.0>, Hexpm.RepoBase, :forward, :up, %{level: :info, sql: false}]}, :infinity)
** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
(elixir) lib/gen_server.ex:951: GenServer.call/3
lib/ecto/migration/runner.ex:22: Ecto.Migration.Runner.run/6
lib/ecto/migrator.ex:195: Ecto.Migrator.attempt/6
lib/ecto/migrator.ex:112: anonymous fn/3 in Ecto.Migrator.do_up/4
lib/ecto/adapters/sql.ex:774: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection) lib/db_connection.ex:1341: DBConnection.run_transaction/4
lib/ecto/migrator.ex:183: Ecto.Migrator.do_run_maybe_in_transaction/3
(elixir) lib/task/supervised.ex:88: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
(stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

</details>
@josevalim

This comment has been minimized.

Member

josevalim commented Oct 6, 2018

Ohhhhhhhhhh, fixed in ecto_sql master.

@wojtekmach

This comment has been minimized.

Member

wojtekmach commented Oct 6, 2018

@josevalim perfect, I still have a couple failing tests but nothing major, will let you know if I'm blocked. thanks!

@wojtekmach wojtekmach changed the title from WIP: Update to Ecto 3.0 to Update to Ecto 3.0 Oct 10, 2018

mix.exs Outdated
{:phoenix_ecto, "~> 3.1"},
{:phoenix_html, "~> 2.3"},
{:phoenix_live_reload, "~> 1.0", only: :dev},
{:phoenix_pubsub, "~> 1.0"},
{:phoenix, "~> 1.3"},
{:plug_attack, "~> 0.3"},
{:plug, "~> 1.2"},
{:postgrex, ">= 0.0.0"},
{:rollbax, "~> 0.5"},
{:sbroker, "~> 1.0"},

This comment has been minimized.

@josevalim

josevalim Oct 10, 2018

Member

Should you remove sbroker dep?

mix.exs Outdated
@@ -46,14 +46,16 @@ defmodule Hexpm.MixProject do
{:hackney, "~> 1.7"},
{:libcluster, "~> 3.0"},
{:mox, "~> 0.3.1", only: :test},
{:ecto, github: "elixir-ecto/ecto", override: true},

This comment has been minimized.

@josevalim

josevalim Oct 10, 2018

Member

I think you don't need this dep.

This comment has been minimized.

@wojtekmach

wojtekmach Oct 11, 2018

Member

I need to overwrite the dependency for now because phoenix_ecto requires ~> 2.1

This comment has been minimized.

@josevalim

josevalim Oct 11, 2018

Member

Ah, ok. It is temporary then until a new phoenix_ecto version is out.

@wojtekmach

I think this is ready, just waiting for the RC!

def change do
alter table(:emails) do
modify(:verification_expiry, :utc_datetime_usec)

This comment has been minimized.

@wojtekmach

wojtekmach Oct 11, 2018

Member

@ericmj do you have a strong opinion whether this should have or don't have usec? What happened was we previously had it as non-usec and were filling it in with DateTime.utc_now() (which has usec), and with Ecto 3.0 we're stricter with regards to precision. Alternatively, we could truncate it before saving into the DB, but because we use usec everywhere else I think it's better to be consistent.

This comment has been minimized.

@ericmj

ericmj Oct 11, 2018

Member

I would rather have higher precision than lower. I guess we store usec in all other fields?

This comment has been minimized.

@wojtekmach

wojtekmach Oct 11, 2018

Member

Correct, it's usec across the board now.

Show resolved Hide resolved config/config.exs Outdated
@@ -206,12 +206,12 @@ defmodule Hexpm.Repository.Releases do
|> validate_reserved_version(reserved_packages)
|> Repo.insert_or_update()

This comment has been minimized.

@ericmj

ericmj Oct 17, 2018

Member

Lowercase repo here.

Show resolved Hide resolved priv/repo/migrations/20181011082425_update_timestamp_fields.exs
@ericmj

ericmj approved these changes Oct 17, 2018

@wojtekmach wojtekmach merged commit b9fa546 into master Oct 17, 2018

3 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
ebert Ebert did not find any new or fixed issues.
Details

@wojtekmach wojtekmach deleted the wm-ecto-3 branch Oct 17, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment