Improve permission system that supports dynamic tokens for DEC

[lurenpluto]

The current permission system has two layers: rmeta and object access, with the following characteristics:

• RMeta layer: It can set permissions based on the global-state path and assign permissions to (dec, zone) groups using access-string control. It can also set permissions for specific (dec, zone) pairs through (dec-id, device-id/owner-id).
• Object-level access: This is at the noc layer, where permissions can be set for each object based on (dec, zone) groups using access-string control.

These two levels of permissions can handle many static configurations but are insufficient for some dynamic permission scenarios, such as:

A DEC needs to share a specific resource with others and sets a token (maybe a random password string). As long as others know this token, they can access the resource. The shared token has an expiration time, after which others will no longer have access.

This is a typical token-based sharing mechanism, with tokens being custom-generated and validated by each DEC. Tokens can be a random password or a public key format, etc.

The protocol stack needs to consider how to support this form of dynamic permission. It is reasonable to add it to the rmeta layer, as introduced from the beginning. Since DEC validation is required, the handler system also needs to be involved. The DEC can dynamically handle permission validation requests for each rmeta to determine whether to grant or deny access.

[lurenpluto]

On an existing basis codes with ACL and protocols, consider a version of the design as follows:

1. Add handler permission to rmeta

The upgraded access types are as follows:

#[derive(Clone, Debug, Serialize, Deserialize, Eq, PartialEq)]
pub enum GlobalStatePathGroupAccess {
    Specified(GlobalStatePathSpecifiedGroup),
    Default(u32 /*AccessString*/),
    Handler, // New Type, should be handled by the dec's acl handler
}

2. Reuse the previous handler.acl event

This event once played the role of acl callback in the old ACL mechanism, but after the ACL was restructured into a new logic, this event has been deprecated but still retained. We can reactivate this event, but the Request needs to be redesigned and should include the following fields:

pub struct AclHandlerRequest {
    pub dec: ObjectId,
    pub path: String,
    pub source: RequestSourceInfo,
    pub permissions: AccessPermissions,
}

3. rmeta management

We can use add_access and remove_access to set the handler permission for a global-state path. When a request needs to use rmeta for verification, it matches the corresponding path. If it corresponds to a handler, it will call the ACL event registered by the corresponding dec for dynamic judgment, thus determining whether the request is rejected or accepted.

4. Custom token for requests

Since the ACL handler is a general rmeta processing mechanism, all places that need to use rmeta to handle permissions need to support custom tokens. It is somewhat difficult to recklessly add a field to many different request modes, so after researching, we consider adding it to the composite structure of req-path:

https://github.com/buckyos/CYFS/blob/8f8e88c2089d3a5573c2e71504020642b9ffdcfd/src/component/cyfs-lib/src/prelude/global_state_common.rs#L28

RequestGlobalStatePath

Since req-path itself is a composite structure, you can refer to the design pattern of URLs, providing parameters in the form of ? followed by the parameter, like:

{dec-id}/a/b/c?token=xxx

However, note the following two points:

• In the 'o' link protocol, the req-path itself is passed through the URL query pairs parameters. So, if you need to add a token parameter in the req-path, you must escape the characters ? and & (at least the & symbol) to avoid the token not being recognized by the req-path parameters.

• In the 'r' link protocol, since the req-path itself is a part of the URL path, the token needs to be carried separately in the 'r' protocol. Additionally, since the internal implementation of the 'r' protocol relies on the global-state accessor mechanism, the accessor mechanism also needs to add relevant support for the token field.

[streetycat]

Great!

Is there some built-in authentication strategy? compare password or verify the signature?

By the way, I think we should use uniform names for some special identifiers.

For example: the dec in AclHandlerRequest , I also find dec_id used in other structures (eg: TypelessObjectDesc.dec_id).

and there are req_path and rpath already existed. what is the path in AclHandlerRequest mean?

[lurenpluto]

Great!

Is there some built-in authentication strategy? compare password or verify the signature?

By the way, I think we should use uniform names for some special identifiers.

For example: the dec in AclHandlerRequest , I also find dec_id used in other structures (eg: TypelessObjectDesc.dec_id).

and there are req_path and rpath already existed. what is the path in AclHandlerRequest mean?

The token should be more precisely called user_token, and it should be transparent to the cyfs-stack. The specific verification mechanism depends on the app's own handling(during acl handler callback). It can use a password form or asymmetric key verification such as a signature. Cyfs-base provides some utility functions and classes for these purposes.

The AclHandlerRequest mentioned above is just pseudocode for illustration purposes, where the path refers to the req-path and dec refers to the dec-id. In the actual implementation, this will be unified.

[lurenpluto]

rmeta adds support for dynamic permissions

1. Add Handler type to the permission type of rmeta

If the corresponding permission type is Handler when adding rmeta access, then when rmeta access check, if it matches the rule, it will try to call the corresponding dec registered acl handler callback, and the return value of the callback method will determine whether the permission request is passed or denied.

rmeta related is defined as follows:

CYFS/src/component/cyfs-lib/src/rmeta/def/access.rs

Lines 91 to 96 in afd05eb

	#[derive(Clone, Debug, Serialize, Deserialize, Eq, PartialEq)]
	pub enum GlobalStatePathGroupAccess {
	Specified(GlobalStatePathSpecifiedGroup),
	Default(u32 /*AccessString*/),
	Handler,
	}

Usage as the following code:

CYFS/src/tests/cyfs-stack-test/src/case/acl_handler.rs

Lines 73 to 83 in afd05eb

	// Add rmeta 'Handler' access for specific req path
	let item = GlobalStatePathAccessItem {
	path: TEST_REQ_PATH.to_owned(),
	access: GlobalStatePathGroupAccess::Handler,
	};
	device1
	.root_state_meta_stub(None, None)
	.add_access(item)
	.await
	.unwrap();

2. Refactoring the acl events in the Router Handlers event system

The new acl event is adapted to the rmeta callback request, the corresponding request and response are defined as follows:

• Acl Request
  

  CYFS/src/component/cyfs-lib/src/acl/handler/request.rs

  Lines 5 to 19 in afd05eb

  	#[derive(Debug, Clone)]
  	pub struct AclHandlerRequest {
  	// The request's target dec
  	pub dec_id: ObjectId,
  	// request source
  	pub source: RequestSourceInfo,
  	// full req_path = {req_path}?{query_string}
  	pub req_path: String,
  	pub req_query_string: Option<String>,
  	// The required permissions
  	pub permissions: AccessPermissions,
  	}

• Acl Response
  

  CYFS/src/component/cyfs-lib/src/acl/handler/request.rs

  Lines 35 to 38 in afd05eb

  	#[derive(Debug, Clone)]
  	pub struct AclHandlerResponse {
  	pub action: AclAction,
  	}

dec needs to check the permission request inside the handler based on the following params:

• source
  The source of the request, including key information such as zone and dec
• req_path
  The req_path of the request, which generally corresponds to the path on the root-state, can be filtered by the req_path when the dec service adds the acl handler event handler.
• req_query_string
  The additional parameters of the request, negotiated between the caller and the dec service, are designed according to the URL query string specification, and can carry some custom parameters, such as token

Here is a complete req_path with query_string

req_path = /a/b/c?token=123456

• permissions
  The permissions required for the permission request

The request can be answered by the Acl handler with the following AclAction:

CYFS/src/component/cyfs-lib/src/acl/action.rs

Lines 12 to 16 in afd05eb

	#[derive(Debug, Clone, Eq, PartialEq)]
	pub enum AclAction {
	Accept = 0,
	Reject = 1,
	}

3. How to register acl events for Dec service

Registering acl events is similar to registering post-object events, you can filter the corresponding rmeta access callback events by filter and req_path

The filter supports the following fields and types:

- Request side

FieldName	Type
source.protocol	String
source.dec_id	String
source.zone	String
source.device	String
source.zone_category	String
dec_id	String
req_path	Glob
req_query_string	Glob
permissions	String

- Response side

FieldName	Type
action	String

The sample code for registering an event is as follows

- The acl handler callback:

CYFS/src/tests/cyfs-stack-test/src/case/acl_handler.rs

Lines 20 to 58 in afd05eb

	#[async_trait::async_trait]
	impl EventListenerAsyncRoutine<RouterHandlerAclRequest, RouterHandlerAclResult> for OnDynamicAcl {
	async fn call(&self, param: &RouterHandlerAclRequest) -> BuckyResult<RouterHandlerAclResult> {
	info!("will handle dynamic acl: {}, query={:?}", param.request.req_path, param.request.req_query_string);
	let mut action = AclAction::Accept;
	let querys: Vec<_> = param.request.req_query_string.as_ref().unwrap().split('&').collect();
	for query in querys {
	if let Some((k, v)) = query.split_once('=') {
	if k == "token" {
	if v != USER_ACCESS_TOKEN {
	warn!("invalid user token: {}", v);
	action = AclAction::Reject;
	} else {
	action = AclAction::Accept;
	}
	break;
	}
	} else {
	unreachable!("should not come here in out test case! {}", query);
	}
	}
	// let url = Url::parse(&param.request.req_path).unwrap();
	let resp = AclHandlerResponse {
	action,
	};
	let result = RouterHandlerAclResult {
	action: RouterHandlerAction::Response,
	request: None,
	response: Some(Ok(resp)),
	};
	Ok(result)
	}
	}

- Register acl handler:

CYFS/src/tests/cyfs-stack-test/src/case/acl_handler.rs

Lines 85 to 95 in afd05eb

	// Add acl handler to handle request
	let handler = OnDynamicAcl {};
	device1.router_handlers().add_handler(
	RouterHandlerChain::Acl,
	"test-dynamic-acl-1",
	0,
	Some("*".to_owned()),
	Some(TEST_REQ_PATH.to_owned()),
	RouterHandlerAction::Reject,
	Some(Box::new(handler)),
	).unwrap();

4. How to use the custom req_query_string on the request side

The req_query_string is a custom parameter for the rmeta permissions dynamic Handler, which can be passed with various pre-designed parameters from dec to better verify the permissions of the request.

For all requests that use req_path to rely on rmeta permissions, you can specify additional req_query_string to specify additional parameters for the permission request, such as token, password, etc., including the following scenarios

- NON/NDN/Crypto/Trans etc.

Directly attach a query_string to the req_path of the request: {req_path}? {query_string}

- Global-state based on op-env/accessor's related interface

directly on the path/inner_path parameter of the request, in the following format: {path}? {query_string}

- r-link

You need to include additional query pairs in the query string of the url, but you need to avoid some parameters reserved by the protocol

The current list of reserved parameters is as follows:

• mode
• format
• flags
• action
• page_index
• page_size
• context
• group

For example, here is an example of an r link with token=123456 query string:

cyfs://r/$/{dec-id}/test/dynamic/call?token=123456

- o-link

Since the req-path of the o link is itself provided as a query string parameter, the req_path with user-defined query_string needs to be encoded to escape the ? and & special characters to avoid not being recognized correctly by cyfs-stack internally

For example, the following o link:

cyfs://o/{device-id}/{object-id}?req_path=/a/b%3Ftoken=xxx%26id=xxx&dec_id=xxx

where got the decoded parameters are as follows

• req_path=/a/b?token=xxx&id=xxx
• dec_id=xxx

[lurenpluto]

The dynamic acl base on rmeta handler has been merged into the main, and a simple test case is in

https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/tests/cyfs-stack-test/src/case/acl_handler.rs

The detailed usage documentation is above, hopefully providing some detailed testing help @lizhihongTest

[lizhihongTest]

How can I set cyfs.GlobalStatePathGroupAccess.Handler() by cyfs.AccessString ? If I put_object or publish_file can set access, But the request params type is cyfs.AccessString

[lurenpluto]

The current permission system in cyfs-stack is divided into two layers:

• rmeta level
  Permissions are controlled by root-state+path, and Handler only supports this level of permissions

• object level
  Each object can be associated with an access_string, and the put_object/publish_file etc. you mentioned here, which involve "object" writing operations, use permissions at this level, and Handler type is not supported at this level

[lizhihongTest]

The interface of dynamic tokens for DEC has been test finished, The remaining r/a-link related tests will be manually performed in the browser.

Testing report

http://bdttest.tinyappcloud.com/cyfs_test_package/test_report/2023_05_04_dynamic_token/

Testing code

cyfs-test-lab:test_dynamic_token_scenario.ts

Testing result

Scenario Testing: Dynamic token
Testing register different Acl Handler
√ Normal Case: Share object with dynamic token,Another user get_object_by_path with correct token (702 ms)
√ Normal Case: Share object with dynamic token,Another user get_object_by_path with incorrect token (769 ms)
Testing parameter chain : Just use cyfs.RouterHandlerChain.Acl will take effect
√ Normal Case:set chain cyfs.RouterHandlerChain.Acl (598 ms)
√ Abnormal Case: set chain cyfs.RouterHandlerChain.NDN will not take effect (551 ms)
√ Abnormal Case: set chain cyfs.RouterHandlerChain.Handler will not take effect (559 ms)
√ Abnormal Case: set chain cyfs.RouterHandlerChain.PostCrypto will not take effect (575 ms)
√ Abnormal Case: set chain cyfs.RouterHandlerChain.PostRouter will not take effect (553 ms)
√ Abnormal Case: set chain cyfs.RouterHandlerChain.PreNOC will not take effect (531 ms)
Testing parameter default_action : just run effect when routine is error . developer just can use routine to verify token
√ Abnormal Case: Not use routine acl handler,default_action set Reject (556 ms)
√ Abnormal Case: Not use routine acl handler,default_action set Default Reject (553 ms)
√ Abnormal Case: Not use routine acl handler,default_action set Drop (542 ms)
√ Abnormal Case: Not use routine acl handler,default_action set Response,but empty response.Acl handler must use routine handler (605 ms)
√ Normal Case: Not use routine acl handler,default_action set Pass will use next routine handler will accept (627 ms)
√ Abnormal Case: Not use routine acl handler,default_action set Pass will use next routine handler will reject (572 ms)
Testing parameter req_path : handler will take effect on the req_path and child req_path
√ Normal Case: handler will take effect on the req_path (626 ms)
√ Normal Case: handler will take effect on the child req_path (573 ms)
√ Abmormal Case: handler will not take effect on the brother req_path (535 ms)
√ Abmormal Case: handler will not take effect on the parent req_path (618 ms)
Testing parameter index : The smaller the index value, the higher the priority
√ Normal Case: set index 0 handler accept,set index 1 handler reject. Result access accept (619 ms)
√ Normal Case: set index 1 handler accept,set index 0 handler reject. Result access reject (528 ms)
Testing parameter filter[Unknown the filter rules?] : Only the handlers that meet the filter rules will take effect
√ Normal Case: set filter match success (608 ms)
√ Normal Case: set filter match failed (549 ms)
√ Normal Case: set filter = undefined (616 ms)
Testing parameter id :The id is the unique identifier of the handler
√ Normal Case: Registering a handler with the same id will overwrite it,Set access Accept -> Reject (545 ms)
√ Normal Case: Registering a handler with the same id will overwrite it,Set access Reject -> Accept (623 ms)
Testing parameter routine : the routine will register a handler event ,When call it,it can response accept、reject、BuckyError
√ Normal Case: Registering a handler the routine will response accept (574 ms)
√ Normal Case: Registering a handler the routine will response reject (649 ms)
√ Normal Case: Registering a handler the routine will response BuckyError (651 ms)
√ Normal Case: Registering a handler the routine will return BuckyError (300101 ms)

[lizhihongTest]

Test environment

OOD : Nightly 1.1.0.755
cyfs-runtime : bulid by code c477a17

Random test data code

cyfs-test-lab:test_dynamic_token_r_a_link.ts

Test Data

• CYFS O-Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

• CYFS R-Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

R-link get object frist time use with correct token

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

• get object success
• trigger ACL handler success

R-link get object second time

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

• get object success
• Not trigger ACL handler，get object by local noc

R-link get object second use flags=1 refresh object

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&flags=1

Test Result:

• get object success
• trigger ACL handler success,use flags=1 will refresh object

R-link get object second set ketwords params

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&flags=1&page_index=1

Test Result:

• get object success
• trigger ACL handler success,use flags=1 will refresh object
• page_index and flags is ketwords params,the handler param.request.req_query_string is missing flags and page_index params

R-link get object use error token

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=error-hc1Rf0ndAdddcG1S2iic&flags=1

Test Result:

• get object failed,raccess reject by handler

O-link get object frist time

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

• get object success
• trigger ACL handler success

O-link get object second time

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

• get object success
• Not trigger ACL handler，get object by local noc

O-link get object second use flags=1 refresh object

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&&flags=1

Test Result:

• get object success
• trigger ACL handler success,use flags=1 will refresh object

O-link get object second set ketwords params

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&page_index=1&flags=1

Test Result:

• get object success
• trigger ACL handler success,use flags=1 will refresh object
• page_index and flags is ketwords params,the handler param.request.req_query_string is missing flags and page_index params

O-link get object use error token

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=error-hc1Rf0ndAdddcG1S2iic&&flags=1

Test Result:

• get object failed,raccess reject by handler