Skip to content

Commit

Permalink
add the project-quota asset manager
Browse files Browse the repository at this point in the history
  • Loading branch information
majewsky committed Jul 15, 2019
1 parent 35b8250 commit e04121a
Show file tree
Hide file tree
Showing 32 changed files with 2,284 additions and 13 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ Castellum understands access rules in the [`oslo.policy` JSON format][os-pol]. A
All project-level policy rules can use the object attribute `%(project_id)s`.

When policy rule names reference the asset type, only the part of the asset type up until the first colon is used. For
example, access to project resources with asset type `quota:compute:instances` would be gated by the rules
`project:show:quota` and `project:edit:quota`.
example, access to project resources with asset type `project-quota:compute:instances` would be gated by the rules
`project:show:project-quota` and `project:edit:project-quota`.

See also: [List of available API attributes](https://github.com/sapcc/go-bits/blob/53eeb20fde03c3d0a35e76cf9c9a06b63a415e6b/gopherpolicy/pkg.go#L151-L164)

Expand Down
9 changes: 9 additions & 0 deletions docs/asset-managers/nfs-shares.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,12 @@ metrics emitted by the [netapp-api-exporter](https://github.com/sapcc/netapp-api
| Variable | Default | Explanation |
| -------- | ------- | ----------- |
| `CASTELLUM_NFS_PROMETHEUS_URL` | *(required)* | The URL of the Prometheus instance providing usage metrics to this asset manager, e.g. `https://prometheus.example.org:9090`. |

## Required permissions

The Castellum service user must be able to list, extend and shrink Manila shares in all projects.

## Policy considerations

- `project:show:nfs-shares` can usually be given to everyone who can interact with Manila shares.
- `project:edit:nfs-shares` should only be given to users who can extend and shrink Manila shares.
26 changes: 26 additions & 0 deletions docs/asset-managers/project-quota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Asset manager: `project-quota`

The asset manager `project-quota` provides one asset type for each quota in [Limes](https://github.com/sapcc/limes), e.g.,

```
project-quota:compute:cores
project-quota:compute:instances
project-quota:compute:ram
project-quota:network:floating_ips
etc.
```

Each such project resource contains exactly one asset, the project itself. The asset UUID is the project ID.

## Required permissions

The Castellum service user must be able to:

- retrieve information about any project from Keystone (to find the domain ID for a project), and
- get/set any project quotas in Limes.

## Policy considerations

- `project:show:project-quota` can be given to everyone who has read access to Limes quotas in the project.
- `project:edit:project-quota` should only be given to users who can set project quota, i.e. usually only to domain
admins or cluster admins.
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
github.com/prometheus/common v0.0.0-20181126121408-4724e9255275
github.com/rs/cors v1.6.0
github.com/sapcc/go-bits v0.0.0-20190522121402-dabab492e20b
github.com/sapcc/gophercloud-limes v0.0.0-20190319123021-2ed8e1224541
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5 // indirect
golang.org/x/net v0.0.0-20190607181551-461777fb6f67 // indirect
golang.org/x/sys v0.0.0-20190610200419-93c9922d18ae // indirect
Expand Down
15 changes: 5 additions & 10 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ github.com/golang-migrate/migrate v3.5.4+incompatible h1:R7OzwvCJTCgwapPCiX6DyBi
github.com/golang-migrate/migrate v3.5.4+incompatible/go.mod h1:IsVUlFN5puWOmXrqjgGUfIRIbU7mr8oNBE2tyERd9Wk=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/gophercloud/gophercloud v0.0.0-20190212181753-892256c46858 h1:QBK4YvYPJGsghVtVJcEZcQ70UNGFnWUCjtv2NTcEFHA=
github.com/gophercloud/gophercloud v0.0.0-20190212181753-892256c46858/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gophercloud/gophercloud v0.0.0-20190318015731-ff9851476e98/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gophercloud/gophercloud v0.1.0 h1:P/nh25+rzXouhytV2pUHBb65fnds26Ghl8/391+sT5o=
github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gophercloud/utils v0.0.0-20190313033024-0bcc8e728cb5 h1:8USoe8m65WcTOYy+MUu+EtLJJysSODnoNDNCEWhDMso=
Expand All @@ -20,11 +20,8 @@ github.com/gorilla/mux v1.7.1/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2z
github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/lib/pq v1.1.0 h1:/5u4a+KGJptBRqGzPvYQL9p0d/tPR4S31+Tnzj9lEO4=
github.com/lib/pq v1.1.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/majewsky/sqlproxy v0.0.0-20170413172413-66c34829b29d h1:ZaEhslK/60Wc1yCUuasPqR1tNj4T7YjU1KoHrEV5or8=
github.com/majewsky/sqlproxy v0.0.0-20170413172413-66c34829b29d/go.mod h1:2wtXqbyVn/SrnklFtle3Kuncd4ytDddW7539rYI+caA=
github.com/majewsky/sqlproxy v0.0.0-20190524080451-ba724456c4af h1:o+sSGflMaFTg/r5iUYYX6PxQ2+nJf6dkjY6CaUgDxg4=
github.com/majewsky/sqlproxy v0.0.0-20190524080451-ba724456c4af/go.mod h1:2wtXqbyVn/SrnklFtle3Kuncd4ytDddW7539rYI+caA=
github.com/mattn/go-sqlite3 v1.10.0 h1:jbhqpg7tQe4SupckyijYiy0mJJ/pRyHvXf7JdWK860o=
github.com/mattn/go-sqlite3 v1.10.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
Expand All @@ -41,14 +38,12 @@ github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a h1:9a8MnZMP0X2nL
github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/rs/cors v1.6.0 h1:G9tHG9lebljV9mfp9SNPDL36nCDxmo3zTlAf1YgvzmI=
github.com/rs/cors v1.6.0/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU=
github.com/sapcc/go-bits v0.0.0-20190111093410-620b317f22bc h1:sIOSzr29YZS0LSl8Ocku6RCRNjxUgbdb6faHjDaA1Fs=
github.com/sapcc/go-bits v0.0.0-20190111093410-620b317f22bc/go.mod h1:KeP5OOqQIgrauiDb/8shKWOq1YoplBALYNvpV4jizPo=
github.com/sapcc/go-bits v0.0.0-20190425131010-53eeb20fde03 h1:2fJaWdbNys9gCBfTvQ7sM4cxsHtc4y8MB1urtyyfmgo=
github.com/sapcc/go-bits v0.0.0-20190425131010-53eeb20fde03/go.mod h1:KeP5OOqQIgrauiDb/8shKWOq1YoplBALYNvpV4jizPo=
github.com/sapcc/go-bits v0.0.0-20190515124317-ff6faf17ad02 h1:TdKE6fGWVELxpDf7SQbDDBNuugtLe7grokMCRKRTN+g=
github.com/sapcc/go-bits v0.0.0-20190515124317-ff6faf17ad02/go.mod h1:KeP5OOqQIgrauiDb/8shKWOq1YoplBALYNvpV4jizPo=
github.com/sapcc/go-bits v0.0.0-20190522121402-dabab492e20b h1:uNHlkQrn6OfRnZA7C/UBDJsD0QLsThgpLur3BVlRmdQ=
github.com/sapcc/go-bits v0.0.0-20190522121402-dabab492e20b/go.mod h1:KeP5OOqQIgrauiDb/8shKWOq1YoplBALYNvpV4jizPo=
github.com/sapcc/gophercloud-limes v0.0.0-20190319123021-2ed8e1224541 h1:rWKMPSjDiQdSulbB32lYwZUuN6zZLvTt8aQafl1rTjk=
github.com/sapcc/gophercloud-limes v0.0.0-20190319123021-2ed8e1224541/go.mod h1:+CE/Fj6tEVycQJBEKxDTAJdwTpFRAJpX1+yd2+fISUo=
github.com/sapcc/limes v0.0.0-20190311092800-fb212143c5f5 h1:X2CPf4pXZoq83mhAjfnEXp8p0AmyoXn0zPcZgoOGGd8=
github.com/sapcc/limes v0.0.0-20190311092800-fb212143c5f5/go.mod h1:Fq0sDroTbraRqiS+OWjf8GFMREFVgrF8nhvJ76SgAk4=
golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
Expand Down
197 changes: 197 additions & 0 deletions internal/plugins/project-quota.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
/******************************************************************************
*
* Copyright 2019 SAP SE
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
******************************************************************************/

package plugins

import (
"fmt"

"github.com/gophercloud/gophercloud"
"github.com/gophercloud/gophercloud/openstack"
keystone_projects "github.com/gophercloud/gophercloud/openstack/identity/v3/projects"
"github.com/gophercloud/gophercloud/openstack/identity/v3/tokens"
"github.com/sapcc/castellum/internal/core"
"github.com/sapcc/castellum/internal/db"
"github.com/sapcc/go-bits/logg"
"github.com/sapcc/gophercloud-limes/resources"
"github.com/sapcc/gophercloud-limes/resources/v1/projects"
"github.com/sapcc/limes"
)

//This asset manager has one asset type for each resource (i.e. each type of
//quota). For a given type of quota, there is only one quota per project, so
//for each project resource, exactly one asset is reported, the project itself
//(i.e. `asset.UUID == resource.ScopeUUID`).
type assetManagerProjectQuota struct {
KeystoneV3 *gophercloud.ServiceClient
Limes *gophercloud.ServiceClient
KnownResources []limesResourceInfo
DomainIDCache map[string]string //maps project ID -> domain ID
}

type limesResourceInfo struct {
ServiceType string
ResourceName string
Unit limes.Unit
}

func (info limesResourceInfo) AssetType() db.AssetType {
return db.AssetType(fmt.Sprintf("project-quota:%s:%s", info.ServiceType, info.ResourceName))
}

func init() {
core.RegisterAssetManagerFactory("project-quota", func(provider *gophercloud.ProviderClient, eo gophercloud.EndpointOpts) (core.AssetManager, error) {
keystoneV3, err := openstack.NewIdentityV3(provider, eo)
if err != nil {
return nil, err
}
limes, err := resources.NewLimesV1(provider, eo)
if err != nil {
return nil, err
}

//get project ID where we're authenticated (see below for why)
var (
currentProjectID string
currentProjectDomainID string
)
if result, ok := provider.GetAuthResult().(tokens.CreateResult); ok {
project, err := result.ExtractProject()
if err == nil {
currentProjectID = project.ID
currentProjectDomainID = project.Domain.ID
} else {
return nil, err
}
} else {
return nil, fmt.Errorf("cannot extract project ID from %t", provider.GetAuthResult())
}

//list all resources that exist, by looking at the current project
report, err := projects.Get(limes, currentProjectDomainID, currentProjectID, nil).Extract()
var knownResources []limesResourceInfo
for _, srv := range report.Services {
for _, res := range srv.Resources {
knownResources = append(knownResources, limesResourceInfo{
ServiceType: srv.Type,
ResourceName: res.Name,
Unit: res.Unit,
})
}
}

return &assetManagerProjectQuota{
KeystoneV3: keystoneV3,
Limes: limes,
KnownResources: knownResources,
DomainIDCache: make(map[string]string),
}, nil
})
}

//AssetTypes implements the core.AssetManager interface.
func (m *assetManagerProjectQuota) AssetTypes() []core.AssetTypeInfo {
result := make([]core.AssetTypeInfo, len(m.KnownResources))
for idx, info := range m.KnownResources {
result[idx] = core.AssetTypeInfo{
AssetType: info.AssetType(),
ReportsAbsoluteUsage: true,
}
}
return result
}

//ListAssets implements the core.AssetManager interface.
func (m *assetManagerProjectQuota) ListAssets(res db.Resource) ([]string, error) {
//see notes on type declaration above
return []string{res.ScopeUUID}, nil
}

//SetAssetSize implements the core.AssetManager interface.
func (m *assetManagerProjectQuota) SetAssetSize(res db.Resource, projectID string, oldSize, newSize uint64) error {
info, err := m.parseAssetType(res)
if err != nil {
return err
}
domainID, err := m.getDomainIDForProjectID(projectID)
if err != nil {
return err
}

srvQuotaReq := limes.ServiceQuotaRequest{}
srvQuotaReq[info.ResourceName] = limes.ValueWithUnit{Value: newSize, Unit: info.Unit}
quotaReq := limes.QuotaRequest{}
quotaReq[info.ServiceType] = srvQuotaReq

respBytes, err := projects.Update(m.Limes, domainID, projectID, projects.UpdateOpts{Services: quotaReq})
if len(respBytes) > 0 {
logg.Info("encountered non-critical error while setting %s/%s quota on project %s: %q",
info.ServiceType, info.ResourceName, projectID, string(respBytes))
}
return err
}

//GetAssetStatus implements the core.AssetManager interface.
func (m *assetManagerProjectQuota) GetAssetStatus(res db.Resource, projectID string, previousStatus *core.AssetStatus) (core.AssetStatus, error) {
info, err := m.parseAssetType(res)
if err != nil {
return core.AssetStatus{}, err
}
domainID, err := m.getDomainIDForProjectID(projectID)
if err != nil {
return core.AssetStatus{}, err
}

opts := projects.GetOpts{Service: info.ServiceType, Resource: info.ResourceName}
report, err := projects.Get(m.Limes, domainID, projectID, opts).Extract()
if err != nil {
return core.AssetStatus{}, err
}
for _, srv := range report.Services {
for _, res := range srv.Resources {
return core.AssetStatus{
Size: res.Quota,
AbsoluteUsage: p2u64(res.Usage),
UsagePercent: uint32(100 * res.Usage / res.Quota),
}, nil
}
}
return core.AssetStatus{}, fmt.Errorf("Limes does not report %s/%s quota for project %s",
info.ServiceType, info.ResourceName, projectID)
}

func (m *assetManagerProjectQuota) getDomainIDForProjectID(projectID string) (string, error) {
if id, ok := m.DomainIDCache[projectID]; ok {
return id, nil
}
project, err := keystone_projects.Get(m.KeystoneV3, projectID).Extract()
if err != nil {
return "", err
}
m.DomainIDCache[projectID] = project.DomainID
return project.DomainID, nil
}

func (m *assetManagerProjectQuota) parseAssetType(res db.Resource) (limesResourceInfo, error) {
for _, info := range m.KnownResources {
if info.AssetType() == res.AssetType {
return info, nil
}
}
return limesResourceInfo{}, fmt.Errorf("unknown asset type: %s", res.AssetType)
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit e04121a

Please sign in to comment.