yeet_dba - find missing foreign key constraints
yeet_dba scans your rails tables for missing foreign key constraints. If there are no dangling records, it will create a migration to add the foreign key constraints on all the table it is safe.
If you have dangling migrations, check the generator logs to see where you have invalid orphaned rows. Orphaned row meaning a row with an id that doesn't exist in the associated table.
But why should I use foreign keys?
You can save yourself an N+1 call by checking if the id has a value instead of loading up the object.
user.company.id # bad - N+1 user.company_id # good
But this doesn't work if you don't nullify the
company_id when the company is deleted. Foreign key constraints prevent you from deleting a record without cleaning out the associated tables.
lol_dba will only add indexes for RoR models. yeet_dba looks at every table (including join tables) to add foreign key constraints.
Add this line to your application's Gemfile:
And then execute:
1. Find invalid rows
If a row has an id, but there doesn't exist an id the expected associated table, then the row has bad data and should either be fixed by nulling the orphaned row or assigning it to an existing row.
This rake task will scan every column for orphaned rows.
$ RAILS_ENV=production rake yeet_dba:find_invalid_columns
---RESULTS--- 🚨Houston, we have a problem 🚨. We found 1 invalid column. -> notifications.primary_image_id Invalid rows: 83 Foreign table: active_storage_attachments This query should return no results: SELECT "notifications".* FROM "notifications" left join active_storage_attachments as association_table on association_table.id = notifications.primary_image_id WHERE "notifications"."primary_image_id" IS NOT NULL AND (association_table.id is null)
2. Fix invalid rows
You can either manually repair your data via rails console or direct SQL queries, or you can run a rake task to resolve failures.
If your rails association requires a value (e.g.
belongs_to :user, required: true), then we try to delete the row.
If your rails association says a value is optional, then we try to nullify the value if the schema alls that column to be null.
If your schema does not allow you to nullify a column, we print a warning.
$ RAILS_ENV=production rake yeet_dba:nullify_or_destroy_invalid_rows
3a. Add foreign keys via migration
Now that the database is in a valid state, we can add the foreign keys in a migration.
$ RAILS_ENV=production rails g yeet_dba:foreign_key_migration
This will create a new migration with for every foreign_key that can safely be added without running into orphaned data errors. We also warn you if active_record models that are missing association declarations (
WARNING - cannot find an association for alternative_housings . supplier_id | suppliers
We also warn if we have tables that don't have existing models attached to them. This can be safe to ignore because join tables on many to many relations don't need models, but ideally, everything should have an AR model backing it.
WARNING - cannot find a model for alternative_housings . supplier_id | suppliers
Finally, if there is a table that we think should have a foreign key constraint, but there are dangling values we warn you against that too.
WARNING - orphaned rows alternative_housings . supplier_id | suppliers
3b. Add missing foreign keys as a rake task
You might want to add foreign keys outside of your regular deployment flow in case there are failures and deployment would be blocked by bad data. This would be especially obnoxious for MySql users since you can't rollback migrations on failure.
$ RAILS_ENV=production rake yeet_dba:add_foreign_keys
ERROR - users . profile_id failed to add key
This rake task is idempotent (safe to run as many times as you need).
- Rails 5.2 (but it may work with 5.0+)
- Ruby 2.4+
Road map to v1
- rspec tests
- add rake task identify all dangling records
- add rake task to automatically nullify or destroy dangling records
- run adding foreign keys as rake task instead of generating a migration
- Use rails associations to find columns that should be "not null" to improve performance
After checking out the repo, run
bin/setup to install dependencies. Then, run
rake spec to run the tests. You can also run
bin/console for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run
bundle exec rake install. To release a new version, update the version number in
version.rb, and then run
bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the
.gem file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/kevincolemaninc/yeet_dba. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
The gem is available as open source under the terms of the MIT License.
Code of Conduct
Everyone interacting in the YeetDb project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.
Logo design attribute
Foreign Key by Ary Prasetyo from the Noun Project
Kevin Coleman, https://kcoleman.me/