This repository has been archived by the owner. It is now read-only.
The new location for this repository is https://github.com/gobuffalo/pop.
Pull request Compare This branch is 8 commits ahead, 290 commits behind gobuffalo:master.
Latest commit a82eaaf Jul 28, 2018
Permalink
Failed to load latest commit information.
associations add SkippedAssociation to make explicit an association should be skipped Feb 6, 2018
columns add many_to_many tag association and allow model to use TableName to … Jan 30, 2018
fizz fixed mattn/anko dep Apr 11, 2018
grifts added the --rm-dist flag for goreleaser Jan 2, 2018
migrations skip associations where ID is null Feb 6, 2018
nulls Fix for issue satori/go.uuid#66 Jan 3, 2018
slices Fix for issue satori/go.uuid#66 Jan 3, 2018
soda version bump Feb 25, 2018
.gitignore Adjust FK code for cockroach according to DB Jan 29, 2018
.goreleaser.yml small tweak to releaser Jan 2, 2018
.travis.yml fingers crossed this works on the first try! Jan 4, 2018
Dockerfile added back sqlite Jul 19, 2016
Gopkg.lock update github.com/markbates/inflect package Feb 8, 2018
Gopkg.toml make sure uuid is locked to a specific commit Jan 8, 2018
LICENSE.txt added a whole bunch of documentation Feb 2, 2016
README.md updated README about move Feb 26, 2018
belongs_to.go use alias instead of model association name Jan 19, 2018
belongs_to_test.go use alias instead of model association name Jan 19, 2018
bench.sh Initial Commit Oct 6, 2015
benchmarks_test.go Update import path for nulls Jan 15, 2017
callbacks.go AfterFind now works on slices and the members of that slice Oct 1, 2017
callbacks_test.go AfterFind now works on slices and the members of that slice Oct 1, 2017
clause.go changed a lot of the clause to be private and not public Feb 2, 2016
cockroach.go *fixed typo, missing database restriction and added a test for Trunca… Feb 20, 2018
commands.go Fix some golint issues Oct 18, 2017
config.go Handle remaining golint issues Oct 19, 2017
config_test.go *seems to be mostly working minus migrations with changes to columns … Jan 9, 2018
connection.go *seems to be mostly working minus migrations with changes to columns … Jan 9, 2018
connection_details.go *seems to be mostly working minus migrations with changes to columns … Jan 9, 2018
connection_details_test.go fixed mysql url parser using underlying driver's parser. Aug 30, 2017
connection_test.go fixed issue with sqlite loosing it's forward slash during parse Jul 27, 2016
database.yml Fix CockroachDB AddForeignKey, DropForeignKey during test Jan 18, 2018
db.go Export Tx, which enables users to define their own store as long as i… Dec 28, 2017
dialect.go Fix for issue satori/go.uuid#66 Jan 3, 2018
doc.go Update doc.go Jul 28, 2018
docker-compose.yml *narrowed down initial working version of cockroach and added it to R… Jan 9, 2018
executors.go Merge branch 'master' of github.com:markbates/pop into pop Oct 20, 2017
executors_test.go *fixed typo, missing database restriction and added a test for Trunca… Feb 20, 2018
file_migrator.go Fix file migrations run from code Jan 6, 2018
finders.go add SkippedAssociation to make explicit an association should be skipped Feb 6, 2018
finders_test.go add SkippedAssociation to make explicit an association should be skipped Feb 6, 2018
group.go Handle remaining golint issues Oct 19, 2017
having.go Handle remaining golint issues Oct 19, 2017
join.go Join On condition as a single string Apr 19, 2017
migration.go Handle remaining golint issues Oct 19, 2017
migration_box.go refactored the entire migration system to make it easier to add new Jun 30, 2017
migration_info.go Handle remaining golint issues Oct 19, 2017
migrator.go deprecated MapTableName infavor of the TableNameAble interface Jan 11, 2018
model.go Fix TableName issue for arrays or slices Jan 25, 2018
model_test.go Fix TableName issue for arrays or slices Jan 25, 2018
mysql.go Merge branch 'handle-remaining-golint-issues' of https://github.com/s… Oct 20, 2017
pagination_test.go Update import path for nulls Jan 15, 2017
paginator.go Handle remaining golint issues Oct 19, 2017
pop.go Handle remaining golint issues Oct 19, 2017
pop_test.go skip associations where ID is null Feb 6, 2018
postgresql.go implemented a better truncate function for postgres Oct 22, 2017
query.go return error when a field is passed over eager and it does not exist. Jan 29, 2018
query_groups.go Handle remaining golint issues Oct 19, 2017
query_having.go Handle remaining golint issues Oct 19, 2017
query_joins.go Handle remaining golint issues Oct 19, 2017
query_test.go support for query.toSQL to support As fromclause Jul 27, 2017
schema_migrations.go refactored the entire migration system to make it easier to add new Jun 30, 2017
schema_migrations_appengine.go refactored the entire migration system to make it easier to add new Jun 30, 2017
scopes.go Handle remaining golint issues Oct 19, 2017
scopes_test.go added a whole bunch of documentation Feb 2, 2016
slices_test.go wrapped specific dialects into suites Aug 15, 2016
sql_builder.go Handle remaining golint issues Oct 19, 2017
sqlite.go Handle remaining golint issues Oct 19, 2017
sqlite_shim.go wip gae Jun 30, 2017
store.go Export Tx, which enables users to define their own store as long as i… Dec 28, 2017
test.sh fixed issue with test runner Feb 7, 2018
tx.go Export Tx, which enables users to define their own store as long as i… Dec 28, 2017
validations.go added a BeforeValidations callback Sep 20, 2017

README.md

POP GoDoc Build Status

!!NOTICE: THIS REPO HAS MOVED!!

The new location for this repository is https://github.com/gobuffalo/pop.

We recommend you migrate your applications over to this new repository as soon as you can.

This repository is now in READ-ONLY mode, and will no longer be updated.

See https://github.com/markbates/pop/issues/164 for more details.

A Tasty Treat For All Your Database Needs

So what does Pop do exactly? Well, it wraps the absolutely amazing https://github.com/jmoiron/sqlx library. It cleans up some of the common patterns and workflows usually associated with dealing with databases in Go.

Pop makes it easy to do CRUD operations, run migrations, and build/execute queries. Is Pop an ORM? I'll leave that up to you, the reader, to decide.

Pop, by default, follows conventions that were defined by the ActiveRecord Ruby gem, http://www.rubyonrails.org. What does this mean?

  • Tables must have an "id" column and a corresponding "ID" field on the struct being used.
  • If there is a timestamp column named "created_at", "CreatedAt" on the struct, it will be set with the current time when the record is created.
  • If there is a timestamp column named "updated_at", "UpdatedAt" on the struct, it will be set with the current time when the record is updated.
  • Default database table names are lowercase, plural, and underscored versions of the struct name. Examples: User{} is "users", FooBar{} is "foo_bars", etc...

Supported Databases

  • PostgreSQL (>= 9.3)
  • MySQL (>= 5.7)
  • SQLite (>= 3.x)
  • CockroachDB (>= 1.1.1)

Connecting to Databases

Pop is easily configured using a YAML file. The configuration file should be stored in config/database.yml or database.yml.

Example Configuration File

development:
  dialect: "postgres"
  database: "your_db_development"
  host: "localhost"
  port: "5432"
  user: "postgres"
  password: "postgres"

test:
  dialect: "mysql"
  database: "your_db_test"
  host: "localhost"
  port: "3306"
  user: "root"
  password: "root"

staging:
  dialect: "sqlite3"
  database: "./staging.sqlite"

production:
  dialect: "postgres"
  url: {{ env "DATABASE_URL" }}

Note that the database.yml file is also a Go template, so you can use Go template syntax. There are two special functions that are included, env and envOr.

  • env - This function will look for the named environment variable and insert it into your file. This is useful for configuring production databases without having to store secret information in your repository. {{ env "DATABASE_URL" }}
  • envOr - This function will look for the named environment variable and use it. If the variable can not be found a default value will be used. {{ envOr "MYSQL_HOST" "localhost" }}

You can generate a default configuration file using the init command:

$ soda g config

The default will generate a database.yml file in the current directory for a PostgreSQL database. You can override the type of database using the -t flag and passing in any of the supported database types: postgres, cockroach, mysql, or sqlite3.

CockroachDB currently works best if you DO NOT use a url and instead define each key item. Because CockroachDB more or less uses the same driver as postgres you have the same configuration options for both. In production you will also want to make sure you are using a secure cluster and have set all the needed connection parameters for said secure connection. If you do not set the sslmode or set it to disable this will put dump and load commands into --insecure mode.

In your code

Once you have a configuration file defined you can easily connect to one of these connections in your application.

db, err := pop.Connect("development")
if err != nil {
  log.Panic(err)
}

Now that you have your connection to the database you can start executing queries against it.

CLI Support

Pop features CLI support via the soda command for the following operations:

  • creating databases
  • dropping databases
  • migrating databases

Installing CLI Support

$ go get github.com/markbates/pop/...
$ go install github.com/markbates/pop/soda

Creating Databases

Assuming you defined a configuration file like that described in the above section you can automatically create those databases using the soda command:

Create All Databases

$ soda create -a

Create a Specific Database

$ soda create -e development

Dropping Databases

Assuming you defined a configuration file like that described in the above section you can automatically drop those databases using the soda command:

Drop All Databases

$ soda drop -a

Drop a Specific Database

$ soda drop -e development

Models

The soda command supports the generation of models.

A full list of commands available for model generation can be found by asking for help:

$ soda generate help

Generate Models

The soda command will generate Go models and, optionally, the associated migrations for you.

$ soda generate model user name:text email:text

Running this command with generate the following files:

models/user.go
models/user_test.go
migrations/20170115024143_create_users.up.fizz
migrations/20170115024143_create_users.down.fizz

The models/user.go file contains a structure named User with fields ID, CreatedAt, UpdatedAt, Name, and Email. The first three correspond to the columns commonly found in ActiveRecord models as mentioned before, and the last two correspond to the additional fields specified on the command line. The known types are:

  • text (string in Go)
  • time or timestamp (time.Time)
  • nulls.Text (nulls.String) which corresponds to a nullifyable string, which can be distinguished from an empty string
  • uuid (uuid.UUID)
  • Other types are passed thru and are used as Fizz types.

The models/user_test.go contains tests for the User model and they must be implemented by you.

The other two files correspond to the migrations as explained below.

Migrations

The soda command supports the creation and running of migrations.

A full list of commands available for migration can be found by asking for help:

$ soda migrate --help

Create Migrations

The soda command will generate SQL migrations (both the up and down) files for you.

$ soda generate fizz name_of_migration

Running this command with generate the following files:

./migrations/20160815134952_name_of_migration.up.fizz
./migrations/20160815134952_name_of_migration.down.fizz

The generated files are fizz files. Fizz lets you use a common DSL for generating migrations. This means the same .fizz file can be run against any of the supported dialects of Pop! Find out more about Fizz

If you want to generate old fashion .sql files you can use the -t flag for that:

$ soda generate sql name_of_migration

Running this command with generate the following files:

./migrations/20160815134952_name_of_migration.up.sql
./migrations/20160815134952_name_of_migration.down.sql

The soda migrate command supports both .fizz and .sql files, so you can mix and match them to suit your needs.

Running Migrations

The soda command will run the migrations using the following command:

$ soda migrate up

Migrations will be run in sequential order. The previously run migrations will be kept track of in a table named schema_migrations in the database.

Migrations can also be run in reverse to rollback the schema.

$ soda migrate down

Find

user := models.User{}
err := tx.Find(&user, id)

Query

tx := models.DB
query := tx.Where("id = 1").Where("name = 'Mark'")
users := []models.User{}
err := query.All(&users)

err = tx.Where("id in (?)", 1, 2, 3).All(&users)
Join Query
// page: page number
// perpage: limit
roles := []models.UserRole{}
query := models.DB.LeftJoin("roles", "roles.id=user_roles.role_id").
  LeftJoin("users u", "u.id=user_roles.user_id").
  Where(`roles.name like ?`, name).Paginate(page, perpage)

count, _ := query.Count(models.UserRole{})
count, _ := query.CountByField(models.UserRole{}, "*")
sql, args := query.ToSQL(&pop.Model{Value: models.UserRole{}}, "user_roles.*",
  "roles.name as role_name", "u.first_name", "u.last_name")
//log.Printf("sql: %s, args: %v", sql, args)
err := models.DB.RawQuery(sql, args...).All(&roles)

Eager Loading

pop allows you to perform an eager loading for associations defined in a model. By using pop.Connection.Eager() function plus some fields tags predefined in your model you can extract associated data from a model.

type User struct {
  ID           uuid.UUID
  Email        string
  Password     string
  Books        Books     `has_many:"books" order_by:"title asc"`
  FavoriteSong Song      `has_one:"song" fk_id:"u_id"`
  Houses       Addresses `many_to_many:"users_addresses"`
}
type Book struct {
  ID      uuid.UUID
  Title   string
  Isbn    string
  User    User        `belongs_to:"user"`
  UserID  uuid.UUID
}
type Song struct {
  ID      uuid.UUID
  Title   string
  UserID  uuid.UUID   `db:"u_id"`
}
type Address struct {
  ID           uuid.UUID
  Street       string
  HouseNumber  int
}

type Addresses []Address

has_many: will load all records from the books table that have a column named user_id or the column specified with fk_id that matches the User.ID value.

belongs_to: will load a record from users table that have a column named id that matches with Book.UserID value.

has_one: will load a record from the songs table that have a column named user_id or the column specified with fk_id that matches the User.ID value.

many_to_many: will load all records from the addresses table through the table users_addresses. Table users_addresses MUST define address_id and user_id columns to match User.ID and Address.ID values. You can also define a fk_id tag that will be used in the target association i.e addresses table.

fk_id: defines the column name in the target association that matches model ID. In the example above Song has a column named u_id that represents id of users table. When loading FavoriteSong, u_id will be used instead of user_id.

 order_by: used in has_many and many_to_many to indicate the order for the association when loading. The format to use is order_by:"<column_name> <asc | desc>"

u := Users{}
err := tx.Eager().Where("name = 'Mark'").All(&u)  // preload all associations for user with name 'Mark', i.e Books, Houses and FavoriteSong
err  = tx.Eager("Books").Where("name = 'Mark'").All(&u) // preload only Books association for user with name 'Mark'.

Callbacks

Pop provides a means to execute code before and after database operations. This is done by defining specific methods on your models. For example, to hash a user password you may want to define the following method:

type User struct {
	ID       uuid.UUID
	Email    string
	Password string
}

func (u *User) BeforeSave(tx *pop.Connection) error {
	hash, err := bcrypt.GenerateFromPassword([]byte(u.Password), bcrypt.DefaultCost)
	if err != nil {
		return errors.WithStack(err)
	}

	u.Password = string(hash)

	return nil
}

The available callbacks include:

  • BeforeSave
  • BeforeCreate
  • BeforeUpdate
  • BeforeDestroy
  • AfterSave
  • AfterCreate
  • AfterUpdate
  • AfterDestroy
  • AfterFind

Further reading

The Unofficial pop Book: a gentle introduction to new users.