New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skeema pull deletes dir for ignored schema #55

Closed
mugli opened this Issue Jan 7, 2019 · 6 comments

Comments

Projects
None yet
2 participants
@mugli
Copy link

mugli commented Jan 7, 2019

I have a mydb_test schema for test/dev environments. The production database obviously doesn't have the schema, so I've added it to ignore-schema option for production environment in config.
Like this:

[production]
ignore-schema='tmp|mydb_test'

But every time I run skeema pull -p production, skeema deletes the directory with this message:
Deleted directory .../schemas/mydb_test -- schema mydb_test no longer exists

Is this the intended behaviour? How can I prevent skeema from deleting it?

@mugli mugli changed the title skeema pull deltes ignored schema skeema pull deletes ignored schema Jan 7, 2019

@mugli

This comment has been minimized.

Copy link

mugli commented Jan 7, 2019

Also I haven't found a recommended workflow for test environments: https://github.com/skeema/skeema/blob/master/doc/examples.md#advanced-configuration

In traditional db-migration tools with up-down migrations, we'd use a different schema name (like mydb_test) to apply the same migrations. The test schema usually exists on the same local database server alongside the development schema (mydb). A configuration file would handle which schema to point to in which environment and run the same migrations on them.

What's the skeema way of doing so? How do I keep mydb and mydb_test in sync? Or do I need to run two database servers on my local machine for two different environments (develop and test), both of which will contain a schema with the same name (like mydb)? Or should I just symlink two directories?

@mugli

This comment has been minimized.

Copy link

mugli commented Jan 7, 2019

Forgot to mention, I'm using skeema version 1.1.2-dev (snapshot build from source)

@evanelias

This comment has been minimized.

Copy link
Member

evanelias commented Jan 7, 2019

Thank you for the report! I can confirm the first part is a bug in skeema pull; I should have a fix released soon.

(I'm also tweaking the issue subject here just to clarify the directory is being deleted, not your actual database 😉)

Regarding your second question, the way to handle this in Skeema is to have a single directory, no need for symlinks. You just need to manually edit the .skeema file there to map the schema names correctly:

[production]
schema=mydb

[test]
schema=mydb_test

One question though -- do you just have one production schema, or several? If you only have one, I would recommend "flattening" the dir structure, so that there's no host-level parent dir, and the host/port/socket is defined in the same .skeema file as the schema name. Skeema will default to this automatically if you start with skeema init ... --schema mydb. With that setup, skeema pull will also work properly without needing to use ignore-schema.

If you have multiple production schemas, the correct layout is a bit harder to achieve, but still doable. If so, let me know more about your db layout, and I'm happy to help figure out the right config to match.

@evanelias evanelias changed the title skeema pull deletes ignored schema skeema pull deletes dir for ignored schema Jan 7, 2019

@evanelias evanelias added the bug label Jan 7, 2019

@mugli

This comment has been minimized.

Copy link

mugli commented Jan 9, 2019

There are multiple production schemas.

Different people in the team have other temporary schemas in their local db. The production has some of them too that are not being used anymore. Those all are declared in ignore-schema config.

This is my current layout:

.
├── README.md
└── schemas
    ├── .skeema
    ├── mydb         
    │   ├── .skeema
    │   ├── table_name.sql
    ├── another_db
    │   ├── .skeema
    │   ├── table_name.sql
    └── mydb_test      <----- This is/was for test environment only
        ├── .skeema
        ├── table_name.sql

So , if I understand it correctly now, I actually don't need a separate mydb_test directory at all. That's awesome, because I guess that'll solve my primary issue too (there will be no "extra" directory there that can be deleted during pulling from production) 👍

evanelias added a commit that referenced this issue Jan 9, 2019

Extend ignore-schema to pull, lint, push/diff; other lint improvement
Previously, the ignore-schema option only prevented creation of matching
schema dir names in `skeema init` and `skeema pull --new-schemas`. However,
if a dir already existed, it would still be used by other commands, preventing
conditional use of ignore-schema based on environment in .skeema files.

This commit extends ignore-schema logic to all relevant commands. See #55 for
more background.

In the process of fixing that issue, this commit also makes a minor
improvement to `skeema lint` when using workspace=docker: as long as a flavor
is correctly set, `skeema lint` no longer needs to communicate with other DBs
besides the Dockerized instance. Previously, `skeema lint` would connect to
live databases unnecessarily, not actually using the connection for anything
if the flavor was already supplied in .skeema.
@evanelias

This comment has been minimized.

Copy link
Member

evanelias commented Jan 9, 2019

Yes, that's correct. If mydb and mydb_test represent two different environments of the same conceptual schema, then they should share a directory. You just have to manually configure those environments in the .skeema file in that dir. I think that should fully solve the issue in your case, but please let me know if you hit any roadblocks in that config.

Separately, regarding the issue of ignore-schema not properly affecting some commands, I've just committed 212720e which fixes that bug. Although this probably won't matter in your case (since combining the dirs should avoid the problem anyway, as you mentioned), there may be other situations where it could be useful for ignore-schema to affect these commands. Thanks again for reporting! I'll likely build a new binary release later this week.

@evanelias

This comment has been minimized.

Copy link
Member

evanelias commented Jan 15, 2019

Just released Skeema v1.1.3 which includes the ignore-schema fix from 212720e. Closing this out, but please feel free to reopen if you encounter any related questions/issues.

@evanelias evanelias closed this Jan 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment