New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQL Storage Support #37
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like what you've done in this PR. I especially like that you've improved the storage layer as a whole in addition to adding a new backend!
Thanks so much for the effort!
docs/configuration.md
Outdated
@@ -42,6 +42,9 @@ disableMetadata: false | |||
# Optional, defaults to 8000 | |||
port: 8000 | |||
storage: | |||
# What type of storage do you want? inmemory (default), directory, or postgresql | |||
type: "" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking we make a breaking change here (we're not 1.0.0 yet).
I'd like to use URI schemes so that it's really easy to add different storage backends in the future.
file://
for filesystempostgresql://user:pass@localhost:5432/database
for postgres
I think we should update the config so that it's just storage: <uri>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can then probably refactor the name PostgresStorage
to be SQLStorage
and implement it to support all SQL backends that gorm supports (pg, mysql and sqlite3 at least).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried hard not to break anything :)
https://gorm.io/docs/connecting_to_the_database.html#Sqlite3
Could just drop json disk and use sqlite in memory, sqlite on disk, mysql, and postgres
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oooh, i can use my new yaml unmarshaling learnings so it could be done directly in storage and not in main? is that dumb?
add in sqlite, postgres, mysql Tests don't pass cause they don't have a db to talk to Signed-off-by: Gavin Mogan <git@gavinmogan.com>
Okay, lemme know what you think. I don't like the db.Open inside the new function cause it makes it impossible to test. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm happy with this PR. There are a few minor modifications/refactors I might apply but you've nailed the parts the matter I think.
if you're happy, i'd like to merge this, do a few small changes, update the change log and then release an RC so we can test a little.
Directory string `yaml:"directory"` | ||
} `yaml:"storage"` | ||
Port int `yaml:"port"` | ||
Storage storage.StorageWrapper `yaml:"storage"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's simpler to just use a string here. This is a bit too fancy I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I liked the wrapper solution cause it meant that the logic was in a single place. I'm open to other solution ideas though
Sure merge and fix it or let me know |
It's been a couple weeks now. Any idea what your plan is? Do you have changes you want to make? Or are you going to merge and make them? |
Hey sorry. I'll get this done now! |
@halkeye again, sorry I ghosted for the past 17 days. give me a kick up the butt sooner next time :P This is now published as 0.2.0-rc6 for docker and helm. |
I usually nudge at two weeks. Everyone gets busy or distracted and open source should be fun and voluntary |
Mostly need help how to non breakingly do the configs
Added a new config.storage.type to set it to postgres
refactored storage interface so it takes in owner/name, or the device only, and only do key logic for disk/in memory