Inserting data?

[chiroptical]

I'll be honest. I am still very confused how I use this tool after making the migration. I am trying to remove the not-null constraint I added in the previous example,

reshape_example> SET search_path TO migration_6_make_name_null_again;
SET
Time: 0.001s
reshape_example> insert into users (name) values (null);
null value in column "name" of relation "users" violates not-null constraint
DETAIL:  Failing row contains (10, null, null).

Time: 0.005s

And the migration,

[[actions]]
type = "alter_column"
table = "users"
column = "name"

  [actions.changes]
  nullable = true

Is this expected to work?

[chiroptical]

Ohh, I think it is because the migrations for the id column aren't carried to the new schema!

reshape_example> SET search_path TO migration_6_make_name_null_again;
reshape_example> \d users
+--------+---------+-----------+
| Column | Type    | Modifiers |
|--------+---------+-----------|
| id     | integer |           |
| name   | text    |           |
+--------+---------+-----------+
Time: 0.009s

[chiroptical]

I need to check quickly if this is because my first set of migrations were using the old version. One second!

[chiroptical]

I started from scratch and this behavior persists. The id column needs to carry over the ALWAYS AS IDENTITY and PRIMARY KEY constraints I guess?

[chiroptical]

If I always go from nullable to non-nullable it is fine.

[fabianlindfors]

Hi! Do you have a migration which alters the id column? Altering columns that are part of primary keys isn't supported at the moment (I should add that to the docs!)

[chiroptical]

I am not modifying the id column in the migration.

• migration: add table with id serial primary key and not null name
• migration: alter table, make name nullable
• set the search_path to the new migration
• try insert into users (name) values (null); and get the failure

[fabianlindfors]

Then it should work. Could you attach copies of your migration files so I can run them myself? Thanks!

[chiroptical]

Yeah, I'll drop them tomorrow. I had to switch machines.

[chiroptical]

migrations/1_create_users_table.toml

[[actions]]
type = "create_table"
name = "users"
primary_key = ["id"]

	[[actions.columns]]
	name = "id"
	type = "INTEGER"
	generated = "ALWAYS AS IDENTITY"

	[[actions.columns]]
	name = "name"
	type = "TEXT"
        nullable = false

migrations/2_make_name_null.toml

[[actions]]
type = "alter_column"
table = "users"
column = "name"

  [actions.changes]
  nullable = true

[fabianlindfors]

I see the issue now! You haven't provided a down function for your second migration meaning Reshape doesn't know how to convert the null value into a value that is safe for the old schema. You could do something like this to for example fill in a default value in the old schema:

[[actions]]
type = "alter_column"
table = "users"
column = "name"

# Use 'N/A' for the old schema
down = "COALESCE(name, 'N/A')"

  [actions.changes]
  nullable = true

I've been wanting to add some explicit validation for these cases but haven't gotten around to it yet :)

[chiroptical]

This makes so much more sense now. Not sure if there is a better name here but I am starting to understand how this works

[fabianlindfors]

Glad to hear that! I could definitely do a better job making these things clearer. I've been working on some proper documentation but unfortunately I'm not a particularly good writer!

[chiroptical]

I can definitely help with that when I get further along. One of my plans is to write some blog posts or make videos on how to use this in a real project with a Haskell project.

[fabianlindfors]

That'd be brilliant! Let me know if you publish anything and I'd be happy to link to it from the repo.