Resource Plugin #491

DuodenumL · 2021-10-15T04:21:51Z

关于资源插件化的PR。

设计想法：对于core来说，它不需要知道各种资源的作用&分配方式，只需要把请求的参数转发给资源插件即可。资源插件会进行资源的具体分配工作，并对core返回engine args，用于后续engine层创建workload。

资源插件采用可执行文件的方式，core会加载指定目录下的所有可执行文件作为资源插件。

在创建workload的流程中，eru-cli要把所有关于资源的请求参数转化为map[string][]string的形式，例如{"cpu": ["1.2"]}。由于core不知道哪些参数对应哪个资源插件，所以它会把所有的参数依次转发给所有的资源插件。各个插件自己负责解析自己需要的参数，并做出正确的处理。

由于要实现插件化，所以资源相关的信息并不会再存储在node metadata里了，而是由各个资源插件自己负责存储，存储也不一定用ETCD，处理得当的话MySQL / MongoDB之类的也行（就是部署麻烦点）。

因此在workload metadata中，关于资源的描述换成了map[string]map[string][]string的方式（是不是看着有点绕），例如：

{
  "cpu-plugin": {
    "cpu": ["1.2"],
    "file": ["some-file"]
  },
  "mem-plugin": {
    "mem": ["1PB"],
    "file": ["some-other-file"]
  }
}

要分别记录各个资源插件分配的资源，否则在rollback / remove要释放资源的时候不知道怎么释放。

主要要做的事情：

把当前资源处理的逻辑改成插件的形式。
cluster层和engine层要基于resource args的改动做出变化。

DuodenumL · 2021-10-15T04:25:59Z

针对当前commit的一些说明：

cluster/calcalcium就是“使用资源插件的calcium”，为了防止混乱复制出来一份，不然IDE一顿报错难顶。这里大部分的文件没啥改动，主要就是在create.go及与其关联的其它文件上有变动。
PluginManager目前还没抽象化，但是Plugin做成了Interface。先撸实现再优化吧...

CMGS · 2021-10-15T07:08:07Z

所以我应该看哪些- -

DuodenumL · 2021-10-15T07:16:31Z

所以我应该看哪些- -

现在完成度还没那么高，主要动的是

calcalium/create.go以及这里面调用到的各种东西
resources/plugin.go + manager.go

jschwinger233

整体方向还行, 就是有些实现有点别扭, 现在先不用在意.

jschwinger233 · 2021-10-15T08:42:16Z

resources/manager.go

+			res[node] = &resourcetypes.NodeResourceInfo{
+				NodeName: node,
+				Capacity: utils.Min(info1.Capacity, info2.Capacity),
+				Rate:     info1.Rate + info2.Rate*info2.Weight,


加权平均

这里是加权平均，L134的是当前带加权的总和，最后会除以总数来算平均

jschwinger233 · 2021-10-15T08:50:32Z

resources/plugin.go

+	UnlockNodes(ctx context.Context, nodes []string) error
+
+	// GetAvailableNodes returns available nodes and total capacity
+	GetAvailableNodes(ctx context.Context, rawRequest RawParams) (map[string]*types.NodeResourceInfo, int, error)


名字不好, 之后换一个.

There are only two hard things in Computer Science...

CMGS · 2021-10-18T03:22:43Z

utils/file.go

+
+// ListAllExecutableFiles returns all the executable files in the given path
+func ListAllExecutableFiles(path string) ([]string, error) {
+	entries, err := os.ReadDir(path)


filepath.Walk 轻松重构

CMGS · 2021-10-18T03:32:10Z

types/resource.go

 }
+
+// ResourcePluginOptions .
+type ResourcePluginOptions map[string][]string


interface map 呢

CMGS · 2021-10-18T03:38:35Z

types/options.go

 type DeployOptions struct {
-	ResourceOpts   ResourceOptions
+	//ResourceOpts   ResourceOptions
+	ResourceOpts   ResourcePluginOptions


为啥不叫 ResourceOptions

DuodenumL · 2021-10-29T09:49:26Z

纪念一下，终于能build了...下一步是把cpu和mem先写成插件，持续debug

DuodenumL · 2021-10-29T09:52:29Z

cluster/calcium/build.go

-	return node, err
+	// node, err := c.scheduler.MaxIdleNode(nodes)
+	// todo: node, err := c.resource.MaxIdleNode(nodes)
+	node := nodes[0]


标记一下，这里还得改改...

计划是让每个plugin都实现max idle node，并且返回一个优先级，选择优先级最大的那个node。

实际操作的时候只有cpu插件返回比较大的优先级。

DuodenumL · 2021-10-29T10:17:41Z

resources/plugin.go

+// Plugin resource plugin
+type Plugin interface {
+	// LockNodes locks the given nodes
+	LockNodes(ctx context.Context, nodes []string) error


这里也是有点纠结，感觉插件这边不一定需要有lock的功能，依赖core里那个ETCD的锁也不是不行？

这个已经讨论过了, 先依然用 core 的锁, 如果有什么问题再来讨论

CMGS · 2021-11-02T03:03:41Z

core.go


 	if config.Profile != "" {
-		http.Handle("/metrics", metrics.Client.ResourceMiddleware(cluster)(promhttp.Handler()))
+		// todo: http.Handle("/metrics", metrics.Client.ResourceMiddleware(cluster)(promhttp.Handler()))


这是做啥

metrics里删了一大堆东西，后续也许还要补上，这里放一个todo...

CMGS · 2021-11-02T03:05:37Z

utils/transaction.go

+// `prepare` should be a pure calculation process without side effects.
+// `commit` writes the calculation result of `prepare` into database.
+// if `commit` returns error, `rollback` will be performed.
+func PCR(ctx context.Context, prepare func() error, commit func() error, rollback func() error) error {


按照之前的风格也应该叫 Pcr

CMGS · 2021-11-02T03:06:13Z

utils/transaction.go

+// `commit` writes the calculation result of `prepare` into database.
+// if `commit` returns error, `rollback` will be performed.
+func PCR(ctx context.Context, prepare func() error, commit func() error, rollback func() error) error {
+	err := prepare()


if err := prepare(); err != nil { bla bla } 紧凑点

CMGS · 2021-11-02T03:06:24Z

utils/transaction.go

+		log.Errorf(ctx, "[PCR] failed to prepare, err: %v", err)
+		return err
+	}
+	err = commit()


同样这里也是，风格保持一致

CMGS · 2021-11-02T03:07:08Z

utils/transaction.go

+		}
+		return err
+	}
+	return nil


而且按照这个注释写的逻辑，我感觉其实用 Txn 包一下就可以了啊

类似于 if prepare then commit else rollback，为什么需要一个新函数

其实是完全可以用Txn来替换的，用这个PCR主要是为了体现Prepare过程是无副作用的，这样感觉逻辑清晰一些...

目前的设计里plugin的大部分API都是纯计算无副作用的，例如Alloc / Realloc / Remap，它们就适用于Prepare过程

CMGS · 2021-11-02T03:13:06Z

types/config.go

 	MaxConcurrency int64         `yaml:"max_concurrency" default:"20"`                  // concurrently call single runtime in the same time
 	Store          string        `yaml:"store" default:"etcd"`                          // store type

+	ResourcePluginDir     string        `yaml:"resource_plugin_dir" default:"/etc/eru/resource_plugins"` // resource plugins path


Plugins，注意是复数

另外折腾了这里的话，你的 yaml sample 也得改

CMGS · 2021-11-02T03:23:59Z

metrics/metrics.go

-		log.Errorf(nil, "[SendNodeInfo] Error occurred while sending cpu used data to statsd: %v", err) //nolint
-	}
-}
+//


因为Node里的ResourceMeta也换成map[string]interface{}了，所以不能直接用之前这个metrics，这里等流程跑通了之后要改...

CMGS · 2021-11-02T03:24:08Z

metrics/handler.go

+//
+//// ResourceMiddleware to make sure update resource correct
+//// todo: remove resource types
+//func (m *Metrics) ResourceMiddleware(cluster cluster.Cluster) func(http.Handler) http.Handler {


CMGS · 2021-11-02T03:25:40Z

lock/mocks/template.go

+
+func FromTemplate() lock.DistributedLock {
+	m := &DistributedLock{}
+	m.On("Lock", mock.Anything).Return(context.Background(), nil)


这是用在哪里的，既不是测试又不是 mock？

这个是store的FromTemplate里调用的...

CMGS · 2021-11-02T03:26:08Z

engine/transform.go

+	if err != nil {
+		return res, err
+	}
+	err = json.Unmarshal(body, &res)


CMGS · 2021-11-02T07:14:47Z

utils/file.go

+	if err != nil {
+		return nil, err
+	}
+	return files, nil


files := []string{} if err := filepath.Walk(basedir, func(path string, info fs.FileInfo, err error) error { if err != nil { return err } if info.IsDir() && path != basedir { return filepath.SkipDir } if isExecutable(info.Mode().Perm()) { files = append(files, path) } return nil }); err!= nil { return nil, err }

可以写紧凑点的

jschwinger233 · 2021-11-10T07:29:36Z

rpc/gen/core.proto

 }

+message StringSlice {
+  repeated string slice = 1;


好像没有必要单独定义这个 message

jschwinger233 · 2021-11-10T07:49:38Z

resources/plugin.go

+	UpdateNodeResourceUsage(ctx context.Context, node string, resourceArgs []coretypes.RawParams, direction bool) error
+
+	// UpdateNodeResourceCapacity updates node resource capacity
+	UpdateNodeResourceCapacity(ctx context.Context, node string, resourceOpts coretypes.RawParams, direction bool) error


node 这个名字不好, 这个仓库里 node 变量都是 *types.Node 类型, string 类型的变量名是 nodename

jschwinger233 · 2021-11-10T08:12:14Z

resources/manager.go

+					rollbackPluginChan <- plugin
+				}
+			})
+			close(rollbackPluginChan)


rollbackPluginChan 听起来像是要回滚的 plugin, 但是 len()==all 的时候又不回滚, 这个逻辑虽然正确但是很奇怪, 想办法调整一下

我终于知道这里哪里最别扭了, 是你把 fan-out fan-in 的逻辑的一部分放到 callPlugins 里, 而这个拆分的结果是, 如你在当前这一层来观察 rollbackPluginChan 这个 chan, 你会发现没有并发场景去使用它, 但是如果改成 []Plugin slice 也不对, 因为并发被 callPlugins 藏起来的, 但是必要的并发同步却又留给了调用方.
我觉得这里可以使用泛型了, callPlugins[T any](xxx, func(Plugin) T) []T, 让 callPlugin 内部的 goroutine 完成所有的同步, 最后返回 slice.

jschwinger233 · 2021-11-10T08:42:39Z

resources/plugin.go

+// Plugin resource plugin
+type Plugin interface {
+	// LockNodes locks the given nodes
+	LockNodes(ctx context.Context, nodes []string) error


这个已经讨论过了, 先依然用 core 的锁, 如果有什么问题再来讨论

jschwinger233 · 2021-11-10T08:46:27Z

resources/plugin.go

+	UnlockNodes(ctx context.Context, nodes []string) error
+
+	// SelectAvailableNodes returns available nodes and total capacity
+	SelectAvailableNodes(ctx context.Context, nodes []string, resourceOpts coretypes.RawParams) (map[string]*types.NodeResourceInfo, int, error)


声明类型的时候, 对那些读上去不知道是啥的最好加上变量名增加可读性, 比如这里的返回值 (map, int, error) 可以写成 (_ map, variableName int, _ error), 可以让人知道这个 int 是啥, 这样注释都可以直接省略

jschwinger233 · 2021-11-10T08:59:34Z

resources/plugin.go

+
+	// GetNodeResourceInfo returns total resource info and available resource info of the node, format: {"cpu": 2}
+	// also returns diffs, format: ["node.VolumeUsed != sum(workload.VolumeRequest"]
+	GetNodeResourceInfo(ctx context.Context, node string, workloads []*coretypes.Workload, fix bool) (coretypes.RawParams, coretypes.RawParams, []string, error)


这个接口拆两个吧, Get 操作的同时在改数据...

大力支持！之前的Get我就觉得有点奇怪...

jschwinger233 · 2021-11-10T09:02:33Z

resources/plugin.go

+	// SetNodeResourceInfo sets both total node resource info and allocated resource info
+	// used for rollback of RemoveNode
+	// notice: here uses absolute values, not delta values
+	SetNodeResourceInfo(ctx context.Context, resourceCapacity coretypes.RawParams, resourceUsage coretypes.RawParams) error


这个接口第三参数是 resourceUsage, 和 UpdateNodeResourceUsage 这个接口是不是语义重复了

还有 UpdateNodeResourceCapacity, 功能重复?

这里的Set是设置绝对值，Update用的是相对值...

jschwinger233 · 2021-11-10T09:07:12Z

resources/plugin.go

+	// Alloc allocates resource, returns engine args for each workload, format: [{"cpus": 1.2}, {"cpus": 1.2}]
+	// also returns resource args for each workload, format: [{"cpus": 1.2}, {"cpus": 1.2}]
+	// pure calculation
+	Alloc(ctx context.Context, node string, deployCount int, resourceOpts coretypes.RawParams) ([]coretypes.RawParams, []coretypes.RawParams, error)


没看懂前两个 return 值是啥, 不如你把变量名加到返回值的类型声明里; RawParam 这个类型是只和资源有关的吗? 从名字来看像是能承载万物

打算把RawParams改成具体的类型，WorkloadResourceOpts / NodeResourceOpts / WorkloadResourceArgs / NodeResourceArgs / EngineArgs

jschwinger233 · 2021-11-10T09:08:56Z

resources/plugin.go

+
+	// AddNode adds a node with requested resource, returns resource capacity and (empty) resource usage
+	// should return error if the node already exists
+	AddNode(ctx context.Context, node string, resourceOpts coretypes.RawParams) (coretypes.RawParams, coretypes.RawParams, error)


第二返回值是啥

resource usage

jschwinger233

感觉还是 callPlugins 太恶心了, 改成泛型就好了

jschwinger233 · 2021-11-10T09:12:53Z

resources/plugin.go

+	Remap(ctx context.Context, node string, workloadMap map[string]*coretypes.Workload) (map[string]coretypes.RawParams, error)
+
+	// UpdateNodeResourceUsage updates node resource usage
+	UpdateNodeResourceUsage(ctx context.Context, node string, resourceArgs []coretypes.RawParams, direction bool) error


direction 是啥?

Incr / Decr，表示usage的改变方向

那把这个类型建立一个 alias 吧, 不要用 bool, 语义模糊

jschwinger233 · 2021-11-10T10:06:04Z

resources/manager.go

+// SelectAvailableNodes returns available nodes which meet all the requirements
+// the caller should require locks
+// pure calculation
+func (pm *PluginManager) SelectAvailableNodes(ctx context.Context, nodes []string, resourceOpts types.RawParams) (map[string]*resourcetypes.NodeResourceInfo, int, error) {


doCalculateCapacity 和 SelectAvailableNodes 这两个函数几乎拥有一样的语义吧, 我觉得前者的名字好一点, 然后似乎不需要 doCalculateCapacity 这个函数了

jschwinger233 · 2021-11-10T10:17:45Z

resources/manager.go

+}
+
+// Alloc .
+func (pm *PluginManager) Alloc(ctx context.Context, node string, deployCount int, resourceOpts types.RawParams) ([]types.RawParams, []map[string]types.RawParams, error) {


pm.Alloc 是有 IO 的而 plugin.Alloc 是纯计算的, 最好统一一下 Alloc 这个函数的语义

不只是Alloc，为了方便回滚 + PCR，plugin里大多数func都是纯计算的...但是在plugin manager里为了方便外部调用，就成了有IO的。

如果plugin的Alloc也有IO，那么plugin manager回滚的时候就不能统一调用UpdateNodeResourceUsage，而是要反向调用Alloc，也就是说Alloc本身要提供rollback的能力，感觉这样就啰嗦了很多。

*当然不可以在plugin manager里将所有resource args取反向然后直接UpdateNodeResourceUsage，因为默认core这边是不知道如何计算“反向”的，而且不是所有resource args都能通过加个负号就变成“反向”的。

所以这里的解决方案...可能是把plugin里的Alloc改成一个看起来无副作用的名字？

jschwinger233

有个问题, 我们对命令行接口好像有点理解不同, 我心想的其实是那种..以文档 spec 来规定接口的命令行, 比如 https://github.com/containernetworking/cni/blob/master/SPEC.md#cni-operations, spec 规定一个正确的实现应该有哪些命令行调用选项, 最后的实现方只要提供一个二进制满足 spec 就好;
但是你的接口好像是让 plugin 去实现一个 golang interface?

jschwinger233 · 2021-11-10T10:33:00Z

cluster/calcium/create.go

 // deploy scheduled workloads on one node
-func (c *Calcium) doDeployWorkloadsOnNode(ctx context.Context, ch chan *types.CreateWorkloadMessage, nodename string, opts *types.DeployOptions, deploy int, plans []resourcetypes.ResourcePlans, seq int) (indices []int, err error) {
-	logger := log.WithField("Calcium", "doDeployWorkloadsOnNode").WithField("nodename", nodename).WithField("opts", opts).WithField("deploy", deploy).WithField("plans", plans).WithField("seq", seq)
+func (c *Calcium) doDeployWorkloadsOnNode(ctx context.Context, ch chan *types.CreateWorkloadMessage, nodename string, opts *types.DeployOptions, deploy int, engineArgs []types.RawParams, resourceArgs []map[string]types.RawParams, seq int) (indices []int, err error) {


这个参数太多了, 手动格式化一下, 一个参数一行

借个楼，最终实现的时候是“提供一个二进制满足spec”的，plugin这个interface一方面是为了mock，一方面是“万一以后用rpc插件呢？”

我对这个 go interface 没意见, 我是看到你没有实现具体的命令行调用方式, 如果按照 spec 来调用, 那么命令行的调用方式都是一样的, 只需要换二进制.

jschwinger233 · 2021-11-10T10:33:21Z

engine/transform.go

+	if err != nil {
+		return res, err
+	}
+	if err = json.Unmarshal(body, &res); err != nil {


jschwinger233 · 2021-11-10T10:37:09Z

resources/plugin.go

+
+	command := exec.CommandContext(ctx, cmd, args...)
+	if err := command.Run(); err != nil {
+		log.Errorf(ctx, "[callBinaryPlugin] failed to run plugin %s, command %s %v, err %s", bp.path, cmd, args, err)


这里 err 的时候输出一下 stderr, 非常有帮助

jschwinger233 · 2021-11-10T10:45:07Z

cluster 层的逻辑整体方向没太大问题, plugin 那里需要再讨论一下, 我可以直接看看你实现的一个具体的 plugin 仓库先.
不过还有很多实现细节要打磨.

DuodenumL · 2021-11-29T10:16:53Z

用泛型重写了一下plugin manager，显著减少了编码过程的痛苦。但是现在各种工具对泛型的支持还不全，包括但并不限于：

Goland识别不出来[T any]里这个any，试了下VSCode也不太行
Goland还有各种奇奇怪怪的报错，比如说会提示func() int不能匹配func() T
mockery在Parse的过程中会报错，看了下现在mockery自己还在用go1.16。
go vet在泛型的地方会报错，导致go test的时候还要手动指定一下-vet=off。
golangci-lint跟mockery有一样的问题。

mockery这个问题我想办法解决了：要pull下来mockery的代码，然后把go版本改成1.17，在go install的时候要这么用：`go install -gcflags=all=-G=3 -tags="typeparams"。但是golangci-lint还没搞...

CMGS · 2022-03-27T17:48:14Z

utils/generics.go

+package utils
+
+// Map like map in Python
+func Map[T1, T2 any](slice []T1, f func(T1) T2) []T2 {


太骚了，测试加起来，这部分一看 ut 的覆盖率急剧下降了

CMGS · 2022-03-27T17:48:58Z

utils/ram.go

+		return ramInBytes, nil
+	}
+
+	flag := int64(1)


CMGS · 2022-03-27T17:49:31Z

utils/transaction.go

+// `prepare` should be a pure calculation process without side effects.
+// `commit` writes the calculation result of `prepare` into database.
+// if `commit` returns error, `rollback` will be performed.
+func Pcr(ctx context.Context, prepare func(ctx context.Context) error, commit func(ctx context.Context) error, rollback func(ctx context.Context) error, ttl time.Duration) error {


缩写用 PCR

CMGS · 2022-03-27T17:51:51Z

types/resource.go

-func (p *VolumePlan) UnmarshalJSON(b []byte) (err error) {
-	if *p == nil {
-		*p = VolumePlan{}
+// StringSlice .


DuodenumL · 2022-03-30T10:40:02Z

迁移工具 & 回滚工具做好啦

jschwinger233

整体结构还行, 兼容性可能要考虑再改改.

rpc/gen/core.proto

jschwinger233 · 2022-04-06T08:09:50Z

cluster/calcium/realloc.go

-				Storage:       resourceMeta.StorageLimit,
+			// note here will change the node resource meta (stored in resource plugin)
+			// todo: add wal here
+			engineArgs, deltaResourceArgs, resourceArgs, err = c.resource.Realloc(ctx, workload.Nodename, workload.ResourceArgs, opts.ResourceOpts)


这里的 txn 有没有可能做得对称一点, 比如这里用 source.SetNodeResourceUsage + store.UpdateWorkload 来回滚 resource.Realloc, 语义是非常隐晦的, 最好是有对称的 API.

其实应该理解为Realloc的回滚是resource.SetNodeResourceUsage，store.UpdateWorkload回滚的是上面的store.UpdateWorkload，算是一一对应的，就是Realloc回滚这里名字不大对称...

manager里把realloc和alloc的rollback包了一下，这样看起来就是对称的了。

DuodenumL · 2022-04-14T07:54:26Z

准备合啦，接下来的工作都要基于插件化来做了。

This reverts commit ac037e7.

jschwinger233 reviewed Oct 15, 2021

View reviewed changes

CMGS force-pushed the master branch from 9fd6774 to fbbb278 Compare October 18, 2021 03:12

CMGS reviewed Oct 18, 2021

View reviewed changes

types/resource.go Outdated

}

// ResourcePluginOptions .

type ResourcePluginOptions map[string][]string

Copy link

Contributor

CMGS Oct 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interface map 呢

CMGS reviewed Oct 18, 2021

View reviewed changes

DuodenumL force-pushed the feature/plugin branch 4 times, most recently from 221192b to b12f3f8 Compare October 21, 2021 04:51

DuodenumL force-pushed the feature/plugin branch 3 times, most recently from cae47ef to 39843ee Compare October 29, 2021 09:47

DuodenumL commented Oct 29, 2021

View reviewed changes

DuodenumL force-pushed the feature/plugin branch from 39843ee to 7e63725 Compare October 29, 2021 10:03

DuodenumL commented Oct 29, 2021

View reviewed changes

CMGS force-pushed the master branch from e2be356 to a5586a8 Compare November 2, 2021 03:00

CMGS requested changes Nov 2, 2021

View reviewed changes

DuodenumL force-pushed the feature/plugin branch from ef0676e to 0f1c0b1 Compare November 2, 2021 04:07

CMGS reviewed Nov 2, 2021

View reviewed changes

jschwinger233 requested changes Nov 10, 2021

View reviewed changes

jschwinger233 reviewed Nov 10, 2021

View reviewed changes

DuodenumL force-pushed the feature/plugin branch from 4fc68d5 to 292d65d Compare November 19, 2021 10:09

DuodenumL force-pushed the feature/plugin branch from 2e1d867 to 65be1b5 Compare November 29, 2021 10:16

CMGS reviewed Mar 27, 2022

View reviewed changes

utils/ram.go

return ramInBytes, nil

}

flag := int64(1)

Copy link

Contributor

CMGS Mar 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

测试 too

CMGS reviewed Mar 27, 2022

View reviewed changes

DuodenumL force-pushed the feature/plugin branch from 2f77b72 to 9e12d3b Compare March 30, 2022 10:39

DuodenumL force-pushed the feature/plugin branch 2 times, most recently from 60efde5 to 38de27a Compare March 31, 2022 07:55

DuodenumL force-pushed the master branch 2 times, most recently from 0c06f3a to 34dacc4 Compare April 4, 2022 09:15

DuodenumL force-pushed the feature/plugin branch from 38de27a to 1f68954 Compare April 5, 2022 03:11

jschwinger233 reviewed Apr 6, 2022

View reviewed changes

jschwinger233 approved these changes Apr 11, 2022

View reviewed changes

DuodenumL added 10 commits April 13, 2022 16:19

resource plugin

5c06893

modify rpc layer to adapt to the resource plugin

fd7b61b

temporarily remove metrics

ef39dcb

modify the engine layer to adapt to the resource plugin

a0efdaf

modify cluster layer to adapt to the resource plugin

c3fa7f6

unit tests

71c9aa2

metrics

df25c91

compatible

2651b95

lint & disable nolintlint temporarily

37896b1

migerate tools & unit tests

15fd09a

DuodenumL force-pushed the feature/plugin branch from d32ff7e to c270f5c Compare April 13, 2022 08:19

review

dfeb5a2

DuodenumL force-pushed the feature/plugin branch from c270f5c to dfeb5a2 Compare April 13, 2022 08:28

DuodenumL merged commit ac037e7 into projecteru2:master Apr 14, 2022

Aceralon added a commit to Aceralon/core that referenced this pull request May 19, 2022

Revert "Resource Plugin (projecteru2#491)"

572559f

This reverts commit ac037e7.

Aceralon added a commit to Aceralon/core that referenced this pull request May 20, 2022

Revert "Resource Plugin (projecteru2#491)"

45fcff7

This reverts commit ac037e7.

Resource Plugin #491

Resource Plugin #491

Uh oh!

Conversation

DuodenumL commented Oct 15, 2021

Uh oh!

DuodenumL commented Oct 15, 2021

Uh oh!

CMGS commented Oct 15, 2021

Uh oh!

DuodenumL commented Oct 15, 2021

Uh oh!

jschwinger233 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DuodenumL commented Oct 29, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

jschwinger233 Nov 10, 2021 •

edited

Loading