Initialize/merge policy from CSV to GoRM adapter

[TJM]

Want to prioritize this issue? Try:

---

What's your scenario? What do you want to achieve?

I am trying to use the gorm adapter with casbin. I would like to be able to "initialize" the data in the database using a CSV policy file that would be distributed in the container with my code. I can sorta make it work by using "NewEnforcer" with the file, then setting enforcer.SetAdapter ... then SavePolicy, but that overwrites the policy in the database (not ok).

I would like to load from CSV, then load from the database, then save? (I think)

Your model:

# CASBIN RBAC Configuration
# https://github.com/casbin/casbin/blob/master/examples/rbac_with_resource_roles_model.conf

[request_definition]
r = sub, obj, act

[policy_definition]
p = sub, obj, act

[role_definition]
g = _, _
g2 = _, _

[policy_effect]
e = some(where (p.eft == allow))

[matchers]
m = g(r.sub, p.sub) && g2(r.obj, p.obj) && r.act == p.act

Your policy:

# RBAC Policies

## RBAC policy for role: admin

p, admin, patchRunStuff, read
p, admin, patchRunStuff, write
p, admin, patchRunStuff, delete
p, admin, patchRunStuff, run

p, admin, config, read
p, admin, config, write
p, admin, config, delete

## RBAC Policy for role: user

p, user, patchRunStuff, read
p, user, patchRunStuff, run
p, user, config, read

# p, admin, water, drink
# p, admin, water, fill

## Role (group) Assignments - these should be in the database
### g, tommy@company.com, admin


## Group (Resource) Assignments
g2, patchRun, patchRunStuff
g2, application, patchRunStuff
g2, environment, patchRunStuff
g2, component, patchRunStuff
g2, server, patchRunStuff
g2, puppetTaskRun, patchRunStuff
g2, puppetPlanRun, patchRunStuff
g2, jenkinsJobRun, patchRunStuff

g2, puppetServer, config
g2, puppetTask, config
g2, puppetPlan, config
g2, jenkinsServer, config
g2, jenkinsJob, config
g2, chatRoom, config

Your request(s):

tommy@company.com, patchRun, read ---> false (expected: true)

This takes a couple runs to reproduce...

The first time through I either add the role ("g", "tommy@company.com", "admin") through code or insert it directly in the database. When the application starts, it mostly seems to overwrite whatever is in the database with the file...

I have also tried "enforcer.LoadPolicy()" after setting the adapter, but that appears to overwrite anything that was loaded from the file with what is in the database. In the event that I add a new feature and want to add a new "g2" I need to be able to add those to an existing database. (test this by uncommenting the "water" lines in the policy and restart.

• createEnforcer :

// createEnforcer will create and initialize the "enforcer" for CASBIN
func createEnforcer() (enforcer *casbin.Enforcer) {
	var err error
	args := config.GetArgs()

	log.Info("Loading RBAC Configuration...")

	// Generic Text based policies
	enforcer, err = casbin.NewEnforcer(casbinConfig, casbinPolicy)
	if err != nil {
		panic("Error with NewEnforcer: " + err.Error())
	}
	if args.DebugAuth {
		enforcer.EnableLog(true)
	}
	if args.DBType != "sqlite" && args.DBType != "sqlite3" {

		// Initialize casbin GORM (database) adapter
		adapter, err := gormadapter.NewAdapterByDB(models.GetDB())
		if err != nil {
			log.Fatal("failed to initialize casbin gorm adapter: " + err.Error())
		}

		log.Info(" - Change to DB adapter")
		enforcer.SetAdapter(adapter)

		log.Info(" - Load DB Policy")
		err = enforcer.LoadPolicy()
		if err != nil {
			log.Fatal("Error Reading from DB: " + err.Error())
		}

		if !enforcer.HasGroupingPolicy("tommy@company.com", "admin") {
			log.Info(" - Add an admin (just for testing)")
			result, err := enforcer.AddGroupingPolicy("tommy@company.com", "admin")
			if err != nil {
				log.Error("Error Adding Admin: " + err.Error())
			}
			log.WithField("result", result).Info("Result")
		}

		// Save Policies
		log.Info(" - Save Policy...")
		err = enforcer.SavePolicy()
		if err != nil {
			log.Fatal("Error saving policies: " + err.Error())
		}
	} else {
		log.Warn("DUE TO SQLITE ISSUES (id column) - FALLING BACK TO FILE BASED POLICIES")
	}

	log.Info(" - end state")
	fmt.Printf("\n*** Roles: %+v\n\n", enforcer.GetGroupingPolicy())
	fmt.Printf("\n*** Policies: %+v\n\n", enforcer.GetPolicy())

	return enforcer
}

// getEnforcer returns the current enforcer and initializes if needed
func getEnforcer() *casbin.Enforcer {
	// Uses global variable `enforcer`
	if globalEnforcer == nil {
		globalEnforcer = createEnforcer()
	}
	return globalEnforcer
}

[casbin-bot]

@tangyang9464 @closetool @sagilio

[hsluoyz]

@Abingcbc

/cc @tangyang9464 @sagilio @abichinger

[Abingcbc]

@TJM LoadPolicy and SavePolicy are both operations that first delete all and then import all.
FYI:
LoadPolicy

casbin/enforcer.go

Line 285 in 833f1af

newModel.ClearPolicy()

SavePolicy
https://github.com/casbin/gorm-adapter/blob/7b3bc27c2b301e05607450cdf68aad047507943f/adapter.go#L508

So you may better use AddPolicies to insert your policies in files into the db.

[TJM]

So you may better use AddPolicies to insert your policies in files into the db.

We also tried to use this:

		err = enforcer.LoadIncrementalFilteredPolicy(gormadapter.Filter{
			Ptype: []string{"g"},
		})

... but then we couldn't save, because it said we couldn't save a filtered policy :(

I was hoping to use the built in methods for parsing the CSV policies instead of parsing it myself and looping through the Add* operation.

That is why I asked, I had more or less come to the same conclusion, but was hoping I was just missing something.

~tommy

[TJM]

FYI, for anyone else that stumbles here through search, here is what I ended up doing. I wrote some helper functions to test the existence (enforcer.HasXXX) before attempting to add (enforcer.AddXXX) to tame database errors. I could have probably done "variadic" but actually preferred to have named arguments.

package middleware

import (
	"github.com/casbin/casbin/v2"
	log "github.com/sirupsen/logrus"
)

// AddUserToRole will assign a user to a role (userGroup) if they are not already assigned
func AddUserToRole(user, role string) (result bool, err error) {
	return addUserToRole(getEnforcer(), user, role)
}

func addUserToRole(e *casbin.Enforcer, user, role string) (result bool, err error) {
	if !e.HasGroupingPolicy(user, role) {
		log.WithFields(log.Fields{
			"user": user,
			"role": role,
		}).Info("Add user to role.")
		result, err = e.AddGroupingPolicy(user, role)
		if err != nil {
			log.WithFields(log.Fields{
				"user": user,
				"role": role,
			}).Error("Error adding user to role: " + err.Error())
		}
	} else {
		log.WithFields(log.Fields{
			"user": user,
			"role": role,
		}).Debug("User was already assigned the role.")
	}
	return
}

// AddResourceToGroup will add a resource object (path target) to a resource group if it does not exist already.
func AddResourceToGroup(resource, group string) (result bool, err error) {
	return addResourceToGroup(getEnforcer(), resource, group)
}

func addResourceToGroup(e *casbin.Enforcer, resource, group string) (result bool, err error) {
	if !e.HasNamedGroupingPolicy("g2", resource, group) {
		log.WithFields(log.Fields{
			"resource": resource,
			"group":    group,
		}).Infof("Add resource to group")
		result, err = e.AddNamedGroupingPolicy("g2", resource, group)
		if err != nil {
			log.WithFields(log.Fields{
				"resource": resource,
				"group":    group,
			}).Error("Error adding resource to group: " + err.Error())
		}
	} else {
		log.WithFields(log.Fields{
			"resource": resource,
			"group":    group,
		}).Debug("Reource already found in group.")
	}
	return
}

// AddPolicy will add a policy to the enforcer if it does not exist.
func AddPolicy(subject, object, action string) (result bool, err error) {
	return addPolicy(getEnforcer(), subject, object, action)
}

func addPolicy(e *casbin.Enforcer, subject, object, action string) (result bool, err error) {
	if !e.HasPolicy(subject, object, action) {
		log.WithFields(log.Fields{
			"subject": subject,
			"object":  object,
			"action":  action,
		}).Info("Add policy")
		result, err = e.AddPolicy(subject, object, action)
		if err != nil {
			log.WithFields(log.Fields{
				"subject": subject,
				"object":  object,
				"action":  action,
			}).Error("Error adding policy: " + err.Error())
		}
	} else {
		log.WithFields(log.Fields{
			"subject": subject,
			"object":  object,
			"action":  action,
		}).Debug("Policy was found")
	}
	return
}

Then I have an "initEnforcer" function that is called at startup to initialize the database enforcer with the results of the CSV enforcer policy files...

// initEnforcer will initialize the enforcer with default policies, mostly for first run
//   or if an upgrade provides new features or policies. It will also add initial admins.
func initEnforcer(e *casbin.Enforcer) {
	log.Info("Initializing RBAC enforcer...")

	args := config.GetArgs()

	for _, user := range args.InitAdmins {
		_, err := addUserToRole(e, user, "admin")
		if err != nil {
			log.Error("Error addUserToRole: " + err.Error())
		}
	}

	for _, user := range args.InitUsers {
		_, err := addUserToRole(e, user, "user")
		if err != nil {
			log.Error("Error addUserToRole: " + err.Error())
		}
	}

	if args.DBType != "sqlite" && args.DBType != "sqlite3" {
		log.Warn("DUE TO SQLITE ISSUES (id column) - SKIPPING INIT - as it would be pointless.")

		// Read Policy from CSV
		csvEnforcer, err := casbin.NewEnforcer(casbinConfig, casbinPolicy)
		if err != nil {
			log.Fatal("Failed to load initial policies from CSV")
		}

		// Process "p" objects (policies)
		for _, p := range csvEnforcer.GetPolicy() {
			_, err := addPolicy(e, p[0], p[1], p[2])
			if err != nil {
				log.Error("Error addPolicy: " + err.Error())
			}
		}

		// Process "g" objects (groups/roles)
		for _, g := range csvEnforcer.GetGroupingPolicy() {
			_, err := addResourceToGroup(e, g[0], g[1])
			if err != nil {
				log.Error("Error addResourceToGroup: " + err.Error())
			}
		}

		// Process "g2" objects (resource groups)
		for _, g2 := range csvEnforcer.GetNamedGroupingPolicy("g2") {
			_, err := addResourceToGroup(e, g2[0], g2[1])
			if err != nil {
				log.Error("Error addResourceToGroup: " + err.Error())
			}
		}
	}

	if args.DebugAuth {
		fmt.Printf("\n\n*** initEnforcer Results\n")
		fmt.Printf("*** - Roles (User Groups): %+v\n", e.GetGroupingPolicy())
		fmt.Printf("*** - Resource Groups: %+v\n", e.GetNamedGroupingPolicy("g2"))
		fmt.Printf("*** - Policies: %+v\n", e.GetPolicy())
		fmt.Printf("*** END initEnforcer Results\n\n")
		log.Info("initEnforer: Done!")
	}
}

NOTE: I preferred to keep the CSV file, which I have to use when using sqlite as a database (another issue). I could have just used an array of strings or whatever to initialize these, but we already had the CSV from developing.