Skip to content

badu/reGorm

 
 

Repository files navigation

Overview

  • Full-Featured ORM (almost)
  • Associations (Has One, Has Many, Belongs To, Many To Many, Polymorphism)
  • Callbacks (Before/After Create/Save/Update/Delete/Find)
  • Preloading (eager loading)
  • Transactions
  • Composite Primary Key
  • SQL Builder
  • Auto Migrations
  • Logger
  • Extendable, write Plugins based on GORM callbacks
  • Every feature comes with tests
  • Developer Friendly

Getting Started

Comments and thoughts

  • As a general idea on golang projects : "fail fast" type of logic is the best approach
  • When a function is called everytime, best idea is to allow golang to inline it
  • regexp.MustCompile is slow inside functions (10 times slower)

Last merge

  • 05.12.2016 - "Add gorm:association:source for association operations for plugins to extend GORM"
  • Note : what's not included before that merge, it's because I don't think it's good to do so

Breaking changes

  • DB struct - renamed to DBCon, since that is what it represents. However, you can do the following, to use the old gorm.DB: dbcon, err := gorm.Open("mysql", dbstr+"?parseTime=true") db = &gorm.DB{*dbcon}
  • Removed MSSQL support - out of my concerns with this project

Changes log (29.10.2016-present)

Todo

  • Documentation for tests and build examples
  • Generated SQL let's the SQL engine cast : SELECT * FROM aTable WHERE id = '1' (id being int). I think it's a bad practice and it should be fixed
  • convertInterfaceToMap in utils it's terrible : simplify

28.01.2017

  • update IsZero as in last original gorm commit

21.12.2016

  • replace fields access (where possible) via modelstruct instead of scope fields
  • Scope rType property holds reflect.Type of the Value (on clone)

20.12.2016

  • SetZero, IsZero in utils
  • Scope has getColumnAsArray instead of generic getColumnAsArray in utils
  • added field not found error
  • removed errors.New() - replaced with fmt.Errorf

19.12.2016

  • removed Scope NewScope - inlined into utils
  • Scope holds rValue (prepare removal of IndirectValue calls)
  • unified DBCon NewScope with newScope
  • replace IndirectValue calls
  • removed GetType from utils, renamed GetTType to GetType

18.12.2016

  • Quote fields are now cached and DBCon is holding them
  • Removed Search struct TableName() method
  • Replaced, where possible, GetSliceSetting with GetForeignFieldNames, GetAssociationForeignFieldNames, GetForeignDBNames, GetAssociationDBNames
  • Fix for Association.Count when there are no where conditions
  • Fixed some tests (duplicate entries errors or misspellings)
  • removed Scope's attrs interface{} - since we can pass it as argument to postUpdate()
  • Scope TableName fix : if implements tabler or dbtabler should override the search table name
  • Fix for Update after save, e.g. TestDB.Save(&animal).Update("name", "Francis") - scope table name is empty, do not execute query
  • getScannerValue removed from utils
  • toQueryMarks optimisation (concat instead of string joins)

17.12.2016

  • When error occurs, print the SQL that was executed (via AddErr of the Scope, we're passing SQL and SQLVars)
  • DBCon has modelsStructMap property which keeps the map that was a "global" variable (separate of concerns and encapsulation)
  • convertInterfaceToMap in utils - created a scope without connection (should be forbidden)
  • SetJoinTableHandler in DBCon - created a scope without connection (should be forbidden)
  • safeModelStructsMap made private
  • DBCon exposes modelsStructMap via KnownModelStructs()
  • DBCon has namesMap property which keeps the map that was a "global" variable (separate of concerns and encapsulation) same as above
  • DBCon KnownNames(name string) for checking against namesMap
  • had to modify ModelStruct FieldByName signature to FieldByName(column string, con *DBCon) to give access to namesMap

16.12.2016

  • Replaced - where possible - conditions for relation checking (readability)
  • fieldsMap struct - all methods are private
  • moved errors from utils_relations and performed some cleanups
  • Search moved flags in types and renamed to be private
  • extracted strings from dialects

15.12.2016

  • Bug fix for ManyToManyWithCustomizedForeignKeys2
  • Fix for calling AutoMigrate with JoinTableHandlerInterface
  • refactored createJoinTable in utils_migrations
  • Fix DoJoinTable test : SetJoinTableHandler should be used only if AutoMigrate creates the tables first
TestDB.AutoMigrate(&Person{},&PersonAddress{})
TestDB.SetJoinTableHandler(&Person{}, "Addresses", &PersonAddress{})

instead of

TestDB.AutoMigrate(&Person{})
TestDB.SetJoinTableHandler(&Person{}, "Addresses", &PersonAddress{})

14.12.2016

  • Bug fix in StructField ParseFieldStructForDialect : slice of bytes is wrong (probably all kind of slices are)
  • Fix for ManyToManyWithMultiPrimaryKeys test : toQueryMarks optimisation in utils was breaking it
  • Cleanup in StructField after using COLUMN tag setting
  • GetHandlerStruct of JoinTableHandler and interface - for debugging

13.12.2016

  • removed ORDER_BY_PK_SETTING and logic from dbCon First and Last (order it's kept in search)
  • removed QUERY_DEST_SETTING and logic : Scope's postQuery accepts a parameter which is destination for dbCon Scan
  • created QueryOption test for "gorm:query_option"
  • Association Count IS failing in mysql tests, because for some reason we get field and/or scope nil
  • removed IGNORE_PROTEC_SETTING : set in dbCon Updates, but never used

11.12.2016

  • CallMethod - extract string constants
  • reduced Scope callbacks
  • more string concat instead of string slices

08.12.2016

  • removed STARTED_TX_SETTING : Scope's Begin returns also a bool, CommitOrRollback accepts a bool param
  • removed BLANK_COLS_DEFAULT_SETTING : Scope's beforeCreateCallback returns also a string, forceReloadAfterCreateCallback accepts that string
  • string concat instead of string slices where possible (cheaper)
  • DBCon SetLogMode(mode int) and LOG_OFF, LOG_VERBOSE, LOG_DEBUG constants
  • removed UPDATE_ATTRS_SETTING : instead, Scope has now updateMaps map[string]interface{} which holds that data
  • got rid of InstanceSet and InstanceGet from Scope, instanceID (uint64) property was removed
  • callCallbacks gets called only we have registered functions, so we won't call function for nothing

07.12.2016

  • "Add gorm:association:source for association operations for plugins to extend GORM" from original commit
  • removed SelectWithArrayInput test
  • changed dbcon signature for Select - it doesn't accept slices of strings anymore
  • Skip order sql when quering with distinct commit
  • Search Select is using only first select clause - now it's overriding existing select clauses
  • Scope createCallback was adding BLANK_COLS_DEFAULT_SETTING inside a for loop (since it was a map, was not malfunctioning, but was bad logic)
  • BLANK_COLS_DEFAULT_SETTING is now storing a string, not a StrSlice
  • Add gorm:association:source for association operations for plugins to extend GORM commit

05.12.2016

  • removed getTableOptions from Scope : was used in creation operations, so we don't need it there
  • after processing relations, we cleanup tag setting's ASSOCIATIONFOREIGNKEY and FOREIGNKEY for allocation sake
  • added sort for test results
  • autoIndex in tables_operations calls directly addIndex (instead of going through dbcon)

04.12.2016

  • StrSlices for association and foreign keys gets created on parseTagSettings of StructField
  • tag settings map changed from map[uint8]string to map[uint8]interface{}
  • Relationship : PolymorphicType, PolymorphicDBName, PolymorphicValue were removed
  • Relationship : Moved relationship kind to tag settings
  • Relationship : removed JoinTableHandler (holded by tag settings now)
  • Relationship removed.
  • seems JoinTableHandler did nothing with sources parameter of Delete method : in association Replace method was passing relationship

03.12.2016

  • Warning for relationship
  • Unset field flag, so we can use HasRelations() instead of checking for relationship == nil (HAS_RELATIONS)
  • test/types.go
  • checking field.HasRelations() - instead of checking for relationship == nil

02.12.2016

  • tag settings are kept only if they have values - flags are set in parent StructField
  • tag settings is concurrent map
  • StructField Set method invalid logic : after setting it to zero, we don't verify is blank, we just set the flag
  • Rearranged tests, with a form of benchmarking
  • Removed SetJoinTableFK from StructField (unused)

30.11.2016

  • Simplify reflections
  • Removed the way internal callbacks were called - gorm can't be broken by un-registering internal callbacks
  • removed Struct property of StructField - added StructName property
  • removed handleBelongsToPreload from Search

29.11.2016

  • Stringer for Relationship, ModelStruct and StructField

28.11.2016

  • Benchmark Quote with regexp, runes, runes conversion and byte shifting
  • Optimized Quote in utils : uses "prepared" regexp
  • changed the Dialect interface : Quote(key string) string is now GetQuoter() string - which returns the quote string(rune)

27.11.2016

  • created Search struct methods for Query, QueryRow and Exec
  • created Search struct RowsAffected field (to replace DBCon's one)
  • argsToInterface in utils
  • updatedAttrsWithValues in utils
  • getValueFromFields in utils
  • initialize moved from Scope to Search
  • implement Warnings (like logs, but always)
  • All create, migrate and alter functions should be moved from the scope inside a separate file (since we're automigrating just at startup)
  • put back PrimaryKey() of Scope shadow of PKName()
  • put back PrimaryField() of Scope shadow of PK()
  • put back PrimaryFields() of Scope shadow of PKs()

26.11.2016

  • polished Scope methods
  • removed inlineConditions from Search
  • rearranged Search combinedConditionSql to have fewer calls
  • rearranged Search prepareQuerySQL to have fewer calls
  • Scope's instanceID is now uint64 and holds the pointer address of that Scope
  • replace current method of keeping data Set / SetInstance / Get
  • switch back to some inline function : getColumnAsArray, generatePreloadDBWithConditions, toQueryValues, toQueryMarks, toQueryCondition, QuoteIfPossible, Quote
  • moved Value from DBCon struct to Search struct (temporary)

25.11.2016

  • utils, removed toSearchableMap
  • utils, convertInterfaceToMap moved to Scope
  • DBConFunc func(*DBCon) *DBCon
  • benchmarks organized
  • DBCon, Scope have Warn
  • Scope related() fail fast logic

24.11.2016

  • moved SQL related functions from Scope to Search struct
  • Search struct replaced scope.AddToVars with addToVars , Scope AddToVars() method removed
  • slimmer Search struct : group, limit, offset gone

23.11.2016

  • file for expr struct - removed, replaced with SqlPair
  • removed Scope SelectAttrs() method and Scope selectAttrs property
  • removed Scope OmitAttrs() []string
  • because above removals, Scope changeableField() method simplified
  • Search struct - added more flags instead of using length for conditions
  • moved SQL and SQLVars from Scope to Search
  • removed Search "con" property : DBCon struct has now clear unscoped methods (stores in search property)
  • removed Search getInterfaceAsSQL since it was used by Group, which takes string parameter

22.11.2016

  • slimmer Search struct - preload gone
  • slimmer Search struct - selects gone
  • slimmer Search struct - order gone
  • Search true clone
  • Search conditions renamed to Conditions, sqlConditions struct renamed to SqlConditions, so search_test.go could be moved in tests

21.11.2016

  • slimmer Search struct - notConditions are gone
  • slimmer Search struct - flags instead of booleans
  • slimmer Search struct - initAttrs gone + method GetInitAttr()
  • slimmer Search struct - assignAttrs gone + method GetAssignAttr()

16.11.2016

  • slimmer search struct - whereConditions, orConditions, havingConditions, joinConditions are gone

15.11.2016

  • minor modification on ModelStruct Create() : moved HasRelations flag setters into StructField
  • moved getValueFromFields from utils.go to string_slice.go (even if it don't belong there)

14.11.2016

  • StructField - optimized creation
  • StructField - optimized makeSlice()
  • StructField - method PtrToValue() called in Scope (scan)
  • integrate Omit duplicates and zero-value ids in preload queries. Resolves #854 and #1054.

13.11.2016

  • ModelStruct - removed properties PrimaryFields and StructFields - they are kept in fieldsMap struct
  • ModelStruct - fieldsMap struct has method PrimaryFields() which are cached into cachedPrimaryFields
  • extracted string "id" into a const (Scope and ModelStruct)
  • ModelStruct - method for number of primary fields
  • Added flag for IS_AUTOINCREMENT (and logic for it)
  • renamed PrimaryFields to PKs, PrimaryField to PK
  • polished relationship.go methods
  • added errors on relationship.go when fields not found, but they break the tests (TODO : investigate why)
  • got rid of checkInterfaces() method of ModelStruct (simplification)
  • make StructField be able to provide a value's interface (Interface() method)
  • make ModelStruct be able to provide a value's interface (Interface() method)
  • cleanup reflect.New(blah... blah) - replaced with Interface() call (WIP)

12.11.2016

  • switched bitflag from uint64 to uint16 (we really don't need more than 16 at the time)
  • make ModelStruct map it's fields : fieldsMap struct
  • make ModelStruct map it's fields : logic modification in fieldByName() - if mapped not found, looking into NamesMap
  • make ModelStruct map it's fields : ModelStruct has addField(field) method
  • make ModelStruct map it's fields : ModelStruct has addPK(field) method (primary keys)
  • make ModelStruct map it's fields : ModelStruct has HasColumn(name) method
  • make ModelStruct map it's fields : removed Scope method HasColumn(name)
  • refactored Scope Fields() method - calls a cloneStructFields method of ModelStruct
  • simplified further the GetModelStruct() of Scope to cleanup the fields
  • renamed DB() of Scope to Con()
  • renamed NewDB() of Scope to NewCon()
  • make Relationship have some methods so we can move code from ModelStruct (part 1 - ModelStruct sliceRelationships() removed)
  • make Relationship have some methods so we can move code from ModelStruct (part 2 - ModelStruct structRelationships() removed)

11.11.2016

  • instead of having this bunch of flags in StructField - bitflag
  • removed joinTableHandlers property from DBCon (was probably leftover of work in progress)
  • simplified Setup(relationship *Relationship, source reflect.Type, destination reflect.Type) of JoinTableHandlerInterface
  • added SetTable(name string) to JoinTableHandlerInterface
  • renamed property "db" of DBCon to "sqli"
  • renamed interface sqlCommon to sqlInterf
  • renamed property "db" of Scope to "con"
  • renamed property "db" of search struct to "con"
  • search struct has collectAttrs() method which loads the cached selectAttrs of the Scope

10.11.2016

  • StructField has field UnderlyingType (should keep reflect.Type so we won't use reflection everywhere)
  • finally got rid of defer inside loop of Scope's GetModelStruct method (ModelStruct's processRelations method has a loop in which calls relationship processors)
  • introduced a HasRelations and IsTime in StructField

09.11.2016

  • Collector - a helper to avoid multiple calls on fmt.Sprintf : stores values and string
  • replaced some statements with switch
  • GetModelStruct refactoring
  • GromErrors change and fix (from original gorm commits)

08.11.2016

  • adopted skip association tag from https://github.com/slockij/gorm (gorm:"save_associations:false")
  • adopted db.Raw().First() makes wrong sql fix #1214 #1243
  • registerGORMDefaultCallbacks() calls reorder at the end of registration
  • Scope toQueryCondition() from utils.go
  • Moved callbacks into Scope (needs closure functions)
  • Removed some postgres specific functions from utils.go

07.11.2016

  • have NOT integrate original-gorm pull request #1252 (prevent delete/update if conditions are not met, thus preventing delete-all, update-all) tests fail
  • have looked upon all original-gorm pull requests - and decided to skip them
  • have NOT integrate original-gorm pull request #1251 - can be done with Scanner and Valuer
  • have NOT integrate original-gorm pull request #1242 - can be done simplier
  • ParseFieldStructForDialect() moved to struct_field.go from utils.go
  • makeSlice() moved to struct_field.go from utils.go
  • indirect() from utils.go, swallowed where needed (shows logic better when dereferencing pointer)
  • file for expr struct (will add more functionality)
  • cloneWithValue() method in db.go

06.11.2016

  • got rid of parseTagSetting method from utils.go
  • moved ToDBName into safe_map.go - renamed smap to NamesMap
  • StrSlice is used in Relationship
  • more cleanups
  • more renaming

05.11.2016

  • DefaultCallback removed from types - it's made under open and registers all callbacks there
  • callback.go has now a method named registerDefaults
  • scope's GetModelStruct refactored and fixed a few lint problems

02.11.2016

  • avoid unnecessary calls in CallbackProcessors reorder method (lengths zero)
  • Refactored sortProcessors not to be recursive, but have a method called sortCallbackProcessor inside CallbackProcessor
  • Concurent slice and map in utils (so far, unused)
  • type CallbackProcessors []*CallbackProcessor for readability
  • callback_processors.go file which holds methods for type CallbackProcessors (add, len)
  • moved sortProcessors from utils.go to callback_processors.go as method
  • created type ScopedFunc func(*Scope)
  • created type ScopedFuncs []*ScopedFunc
  • replaced types ScopedFunc and ScopedFuncs to be more readable

01.11.2016

  • TestCloneSearch could not be moved
  • Exposed some methods on Callback for tests to run (GetCreates, GetUpdates, GetQueries, GetDeletes)
  • Moved tests to tests folder (helps IDE)
  • Extracted strings from dialect_common.go
  • Extracted strings from dialect_mysql.go
  • Modified some variable names to comply to linter ("collides with imported package name")
  • Remove explicit variable name on returns
  • Removed method newDialect from utils.go (moved it into Open() method)
  • Removed MSSQL support - out of my concerns with this project
  • Fix (chore) in StructField Set method : if implements Scanner don't attempt to convert, just pass it over
  • Test named TestNot
  • CallbackProcessor kind field changed from string to uint8

30.10.2016 (Others)

  • DB struct - renamed to DBCon, since that is what it represents

30.10.2016 (Operation Field -> StructField)

  • StructFields has it's own file to get rid of append() everywhere in the code
  • TagSettings map[uint8]string in StructField will become a struct by itself, to support has(), get(), set(), clone(), loadFromTags()
  • TagSettings should be private in StructField
  • replace everywhere []*StructField with type StructFields
  • create StructFields type []*StructField for code readability
  • NewStructField method to create StructField from reflect.StructField
  • Field struct, "Field" property renamed to "Value", since it is a reflect.Value
  • StructField should swallow Field model field definition
  • created cloneWithValue(value reflect.Value) on StructField -> calls setIsBlank()
  • moved isBlank(fieldValue) from utils to StructField named setIsBlank()
  • remove getForeignField from utils.go -> ModelStruct has a method called getForeignField(fieldName string)

29.10.2016

  • Moved code around
  • Numbered tests - so I can track what fails
  • Replaced some string constants like "many_to_many" and refactor accordingly
  • StructField is parsing by it's own gorm and sql tags with method ParseTagSettings
  • Replaced string constants for the tags and created a map string-to-uint8
  • Removed field Name from StructField since Struct property of it exposes Name
  • Created method GetName() for StructField to return that name
  • Created method GetTag() for StructField to return Struct property Tag (seems unused)