Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL Storage Support #37

Merged
merged 5 commits into from May 10, 2020
Merged

SQL Storage Support #37

merged 5 commits into from May 10, 2020

Conversation

halkeye
Copy link
Contributor

@halkeye halkeye commented Apr 17, 2020

Mostly need help how to non breakingly do the configs

Added a new config.storage.type to set it to postgres
refactored storage interface so it takes in owner/name, or the device only, and only do key logic for disk/in memory

@halkeye
Copy link
Contributor Author

halkeye commented Apr 17, 2020

image

Copy link
Owner

@Place1 Place1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like what you've done in this PR. I especially like that you've improved the storage layer as a whole in addition to adding a new backend!

Thanks so much for the effort!

@@ -42,6 +42,9 @@ disableMetadata: false
# Optional, defaults to 8000
port: 8000
storage:
# What type of storage do you want? inmemory (default), directory, or postgresql
type: ""
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking we make a breaking change here (we're not 1.0.0 yet).

I'd like to use URI schemes so that it's really easy to add different storage backends in the future.

  • file:// for filesystem
  • postgresql://user:pass@localhost:5432/database for postgres

I think we should update the config so that it's just storage: <uri>

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can then probably refactor the name PostgresStorage to be SQLStorage and implement it to support all SQL backends that gorm supports (pg, mysql and sqlite3 at least).

https://gorm.io/docs/connecting_to_the_database.html

Copy link
Contributor Author

@halkeye halkeye Apr 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried hard not to break anything :)

https://gorm.io/docs/connecting_to_the_database.html#Sqlite3

Could just drop json disk and use sqlite in memory, sqlite on disk, mysql, and postgres

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooh, i can use my new yaml unmarshaling learnings so it could be done directly in storage and not in main? is that dumb?

internal/storage/postgresql.go Outdated Show resolved Hide resolved
internal/storage/postgresql.go Outdated Show resolved Hide resolved
internal/storage/postgresql.go Outdated Show resolved Hide resolved
add in sqlite, postgres, mysql

Tests don't pass cause they don't have a db to talk to

Signed-off-by: Gavin Mogan <git@gavinmogan.com>
@halkeye halkeye changed the title Postgresql (could easily be more generic) support SQL Storage Support Apr 18, 2020
@halkeye
Copy link
Contributor Author

halkeye commented Apr 18, 2020

Okay, lemme know what you think.

I don't like the db.Open inside the new function cause it makes it impossible to test.

Copy link
Owner

@Place1 Place1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with this PR. There are a few minor modifications/refactors I might apply but you've nailed the parts the matter I think.

if you're happy, i'd like to merge this, do a few small changes, update the change log and then release an RC so we can test a little.

Directory string `yaml:"directory"`
} `yaml:"storage"`
Port int `yaml:"port"`
Storage storage.StorageWrapper `yaml:"storage"`
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's simpler to just use a string here. This is a bit too fancy I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I liked the wrapper solution cause it meant that the logic was in a single place. I'm open to other solution ideas though

@halkeye
Copy link
Contributor Author

halkeye commented Apr 22, 2020

Sure merge and fix it or let me know

@halkeye
Copy link
Contributor Author

halkeye commented May 9, 2020

It's been a couple weeks now. Any idea what your plan is? Do you have changes you want to make? Or are you going to merge and make them?

@Place1
Copy link
Owner

Place1 commented May 10, 2020

Hey sorry. I'll get this done now!

@Place1 Place1 merged commit bda474c into Place1:master May 10, 2020
@Place1
Copy link
Owner

Place1 commented May 10, 2020

@halkeye again, sorry I ghosted for the past 17 days. give me a kick up the butt sooner next time :P

This is now published as 0.2.0-rc6 for docker and helm.

@halkeye
Copy link
Contributor Author

halkeye commented May 10, 2020

I usually nudge at two weeks. Everyone gets busy or distracted and open source should be fun and voluntary

DasSkelett referenced this pull request in DasSkelett/wg-access-server Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants