Provide a clearer flow for people to create and consume custody-light data assets (ie DatatokenEnterprise)

[mariacarmina]

I have seen some reports from Discord lately regarding how we can consume free assets through ocean.py. And my first thought was that we can improve certain comments that we have in main-flow.md.

For example in this section, we can specify that:
"For a free asset, order_tx_id is retrieved from the dispense_and_order().txid call, not from pay_for_access_service. ", or something similar.

On the same page, I saw that some users want to mint enterprise datatokens to start_order which is impossible, due to the fact that enterprise datatokens are burnt after the startOrder call and maybe we can add this too, if it's needed.

Discord thread where the problem was discussed: here

Temporary fix for the user

import os
from ocean_lib.models.datatoken_enterprise import DatatokenEnterprise
from ocean_lib.web3_internal.utils import connect_to_network
connect_to_network("polygon-test") # mumbai is "polygon-test"s

from ocean_lib.example_config import get_config_dict
from ocean_lib.ocean.ocean import Ocean
config = get_config_dict("polygon-test")
ocean = Ocean(config)

OCEAN = ocean.OCEAN_token

ddo = ocean.assets.resolve("did:op:7b14ebca07410ba3226178255cea467df6c9ce49b3ca17e10f94fdfdbecc29ee")
datatoken = DatatokenEnterprise(ocean.config_dict, "0xA63d7C622547E197321Cbd7A0886a524c4Ba7b06")
print(datatoken)

from brownie import accounts

wallet = accounts.add(os.environ["REMOTE_TEST_PRIVATE_KEY1"])

provider_fees = ocean.retrieve_provider_fees(
    ddo,
    ddo.services[0],
    publisher_wallet=wallet
)

tx = datatoken.dispense_and_order(wallet.address, 0,
                              provider_fees,
                              {"from": wallet})

asset_dir = ocean.assets.download_asset(ddo, wallet, './', tx.txid)

I have also let the community know about this here.

I have also discussed with @AnaLoznianu about this and she agreed on specifying those information in the README, but I wanted to have your input on this as well @trentmc.

[trentmc]

For a free asset, order_tx_id is retrieved from the dispense_and_order().txid call, not from pay_for_access_service

This is not quite correct.

It doesn't matter where you get a datatoken from, whether from minting or a transfer or an exchange or a faucet: you have a datatoken then you can consume it with pay_for_access_service.

That's part of the point of datatokens! Max composability of atomic building blocks.

And that's the point of section 2 plus section 3 of the readme. Section 2 shows four different ways for Bob to get a datatoken, and section 3 has him consume it.

As for calls that combine together steps 2 and 3, such as dispense free tokens then order, or for enterprise template: this is already covered in the appendix, section on datatokens.

Therefore I advise NOT to do the change suggested in the description.

There likely are ways to improve readability however, without compromising the emphasis on composability. Feel free to read the README more throughly, reflect on it, and then propose changes.

[trentmc]

I have marked this as "invalid" for now. We can discuss further if you like.

[calina-c]

@mariacarmina is this fixed now with pricing schema parameters? Does #1306 cover this aspect?

[mariacarmina]

It doesn't matter where you get a datatoken from, whether from minting or a transfer or an exchange or a faucet: you have a datatoken then you can consume it with pay_for_access_service.

I created a snippet code and attached a screenshot that reflects the following scenario:
A. If the datatoken is enterprise(has template 2), you cannot dispense or mint and have the funds directly, because those tokens will get burnt, so you need to store the tx as a "proof" when downloading the asset (that shows the fact that you dispensed some enterprise datatokens in order to consume).

Code snippet expected to fail

import os

from ocean_lib.web3_internal.utils import connect_to_network
connect_to_network("polygon-test") # mumbai is "polygon-test"s

from ocean_lib.example_config import get_config_dict
from ocean_lib.ocean.ocean import Ocean
config = get_config_dict("polygon-test")
ocean = Ocean(config)

OCEAN = ocean.OCEAN_token

ddo = ocean.assets.resolve("did:op:7b14ebca07410ba3226178255cea467df6c9ce49b3ca17e10f94fdfdbecc29ee")
datatoken = ocean.get_datatoken("0xA63d7C622547E197321Cbd7A0886a524c4Ba7b06")
print(datatoken)

from brownie import accounts

wallet = accounts.add(os.environ["REMOTE_TEST_PRIVATE_KEY1"])

provider_fees = ocean.retrieve_provider_fees(
    ddo,
    ddo.services[0],
    publisher_wallet=wallet
)

datatoken.dispense_and_order(wallet.address, 0,
                              provider_fees,
                              {"from": wallet})

order_tx_id = ocean.assets.pay_for_access_service(ddo, {"from": wallet})

asset_dir = ocean.assets.download_asset(ddo, wallet, './', order_tx_id)

Screenshot with the error

B. I have also tested with a standard datatoken (template 1) and retrieved the same error on a free asset (which has dispenser)

Code snippet expected to fail

config = get_config_dict("polygon-test")
ocean = Ocean(config)

OCEAN = ocean.OCEAN_token

from brownie import accounts

seller = accounts.add(os.environ["REMOTE_TEST_PRIVATE_KEY1"])
buyer = accounts.add(os.environ["REMOTE_TEST_PRIVATE_KEY2"])

name = "Branin dataset"
url = "https://raw.githubusercontent.com/trentmc/branin/main/branin.arff"

#create data asset
(data_nft, datatoken, ddo) = ocean.assets.create_url_asset(name, url, {"from": seller}, pricing_schema_args=DispenserArguments(to_wei(1), to_wei(1)))

#print
print("Just published asset:")
print(f"  data_nft: symbol={data_nft.symbol}, address={data_nft.address}")
print(f"  datatoken: symbol={datatoken.symbol}, address={datatoken.address}")
print(f"  did={ddo.did}")





provider_fees = ocean.retrieve_provider_fees(
    ddo,
    ddo.services[0],
    publisher_wallet=buyer
)

datatoken.dispense_and_order(buyer.address, 0,
                              provider_fees,
                              {"from": buyer})

order_tx_id = ocean.assets.pay_for_access_service(ddo, {"from": buyer})

asset_dir = ocean.assets.download_asset(ddo, buyer, './', order_tx_id)

Logs with the error

(venv) maria@ubuntu:~/Desktop/ocean.py$ python3 test.py 
Transaction sent: 0x7eca50b802801ebcbda89353c885a9a2bf4d8309b5012a232992a8a3b2c3bf15
  Gas price: 1.505440017 gwei   Gas limit: 2067769   Nonce: 295
  ERC721Factory.createNftWithErc20WithDispenser confirmed   Block: 31907205   Gas used: 1833234 (88.66%)

INFO:ocean_lib.data_provider.data_encryptor:Asset urls encrypted successfully, encrypted urls str: 0x0472c21e63e836e9df60485233be9846eca0d350083dbc856a9664b7a97654cccc73a44b778e26a3bf94a48c362dda381c1e8afb6b8fd14ce8694abf199b08d738dc96192e45f172f83a349cee2f19ecbee715b66f65449a9c32bd5df409098f7a5a9bee8cdd9901b9535da61ab6be6a3699e7da3a8f9263a4125bb97742e5a4f114f224cfcd96203ad496386cdfc077577990878f82b74c17d5e83111cfd4090d59578b983ab95753016e9d4c10503aa1dac1fdca7e860920ecbaae4ee9ade14e7ff52797414fca126c7001821c2bd8790bb7534008e55e90476f14c115617c3f35cf3e557912b5e9351ab270dcfd8c4bb89e71428319b47296a16a23ee42752ff5856101c813a710e83ad0e51307abcb0b2b8d30f068185dcff6919df2ab11a5685a3f2dc461a07009a21366ec0a7ef5c8b92ff896464bb618d0f47bbd84, encryptedEndpoint https://v4.provider.mumbai.oceanprotocol.com/api/services/encrypt
INFO:ocean_lib.data_provider.data_encryptor:Asset urls encrypted successfully, encrypted urls str: 0x0437bff5148a61a984197bb7c8b3472187766a14fae1a2667bc98dbce8b8558802a4acbcc0e394b7d0c7071fc3a775aaaf59a61f52ddc81326dafddfb0cb8f02733675a29007425581386221dd0997c0176d6bd248b26f2296031d6f7d181de291ca0e16e9101c37651fd519c1ab123fc5121beb843930e45310674844e9666df98ab342231646d159d726e9d25810dfff318382fa425da72c4bfe17df87bec4cc769340499fc971f2dee0e257e6c8833c0cdb60641db53accb49361fee5dbdad7b9054d9f7ce55c20abed1d972b17201a5c6b291dda55c689e6b5d6a3a99b7a649c8cc14f817bf0e90589744d858d91fb8814bddc5e594052a6cbef199ac4f32a20cbc04f4cf974e40aa36bd42c17adc76fe6c0d5da7f797a0c7df2a4b9b09aca617a2f9fff361cd72cf00b530e2e517fb81e83a7c58e3d2ebe69cd83a17684fba5160a26180ee79f9b681a656b05120c604a63704bd5df5453ffd898278fe9406ea52f0082bc163f586ea0eaf1ae672f34530629c0246d9c6d40f5e4edab77fe48d124ae4975ee2c4dcd3e61a925253b075418e8400e79db8614db3b1606c8118ac6437c11c9396358db5df2d9b0939131563b317de68b4da3da89496614f547eecb6f10eef947068f821abce194050bfedcd209785bb416e159ce29011d61a4df3980e479d751cfd966ca8a4b79a78e8adfba2ca415c62aaccc6b8eefc654b34eeeef418a3ba1289dbad7f8d04a8264f49b02f8c3c36481c67910892aea417a382a02183c7bdca3feb9cc9d18b78dab70638846ec2a22c92e28babaa71cdb45dec08d6ac94942ced17ba39116ac23f1b05de4aa6e0b1ca49cf55fb8a7a6e8cdb985684580ce15fc37c7f156a47a544482a0543d0a58cf96b55992f7ab40d6443e632e83d40f117b6f9dafd54cff66c4f4bf081e0d1ebab422eaffc0b36069b703c7005892458ff53436b1e0493219615062d2edd348d67e60fd3c6b510690d6ecdad759a074fa51fb9c7587900985c697b096f20a2f7e9160c8693fa94f7cdfb510bacfd724944cfbb649bff4f15f2e44c1eacf78d41fcdf5e7f1551f855268ce47572c058416d34d74696b403709349da872a8cb25d341dcdd7961f92dd01df415db5b6290af9690379b2ea5d5374b5ef560a0bb676382748c42e132f8b8ffdfcff01bc1570b7000cebb8adc960e2a0158350ca39c528a1a1bf3629c465b6c57f24655d902f9058dcc7eb7a0c138b64af1e9869352434fb11218c9c2beb06b0f7cff00443256ee70a355ccddb8aedcfa4297cb035e727afa23ca2f8c8ddb68b01e8d5d9a7c9447ddf5c7f004c94e0bea92209939fcb33a3e80e48fef5af90468db85ec0dfcc3a843369c8042cafd8d338486a3e1c39642fe4349e4767bc8a0aa0c91921841913e604a827daf83b00aee1a82cb, encryptedEndpoint https://v4.provider.mumbai.oceanprotocol.com/api/services/encrypt
Transaction sent: 0x00d47fa62a6fef8f7e290f9294a546d210742ac06c006435ee28461aebff61e2
  Gas price: 1.505440017 gwei   Gas limit: 235844   Nonce: 296
  ERC721Template.setMetaData confirmed   Block: 31907208   Gas used: 211799 (89.80%)

Just published asset:
  data_nft: symbol=<ContractCall 'symbol()'>, address=0x86d1Bc2aCF51f447Eb159f89DCf03ABcA11185fe
  datatoken: symbol=<ContractCall 'symbol()'>, address=0xF3BB0C87d0D1D5b3b004AFb651A903Ab2e2bFc67
  did=did:op:f427119c4a16fcb1719ef508a3b6e9240a5ac6b2f856bb6680a2fbfb782f98b0
INFO:ocean_lib.data_provider.data_service_provider:Service initialized successfully initializeEndpoint https://v4.provider.mumbai.oceanprotocol.com/api/services/initialize
Transaction sent: 0x86c4b9f602988e492ac24c15e3edbc023fb67b20278ac6604e6b05cb5c3f05c1
  Gas price: 1.505440017 gwei   Gas limit: 142092   Nonce: 42
  Dispenser.dispense confirmed   Block: 31907215   Gas used: 104291 (73.40%)

Transaction sent: 0xc0cc0eb1a92d8de1ee0b12b8a7c18b41897d62c03f86f13e3514392243d27dc4
  Gas price: 1.505440017 gwei   Gas limit: 134383   Nonce: 43
  ERC20Template.startOrder confirmed   Block: 31907217   Gas used: 110947 (82.56%)

Traceback (most recent call last):
  File "test.py", line 46, in <module>
    order_tx_id = ocean.assets.pay_for_access_service(ddo, {"from": buyer})
  File "/home/maria/Desktop/ocean.py/venv/lib/python3.8/site-packages/enforce_typing/decorator.py", line 29, in wrapper
    return func(*args, **kwargs)
  File "/home/maria/Desktop/ocean.py/ocean_lib/ocean/ocean_assets.py", line 702, in pay_for_access_service
    raise InsufficientBalance(
ocean_lib.exceptions.InsufficientBalance: Your token balance 0 DT1 is not sufficient to execute the requested service. This service requires 1 wei.

From the two hypotheses A and B it follows the fact that pay_for_access_service should not be used in the free asset scenario (when the asset has a deployed dispenser) and moreover, to specify which tx_id should be provided for download_asset function.
As an observation, I have followed the main-flow README while reproducing both scenarios and I will come up with two options:

• Option 1
  Add a section in the Appendix Consume Details section, similar to Consume General, something like this:

Consume Free Asset

For consuming a free asset, you need to dispense datatokens and order and retrieve the transaction hash as a proof for having funds. Then, you typically download the asset from the url using the transaction hash provided previously.

tx = datatoken.dispense_and_order(bob.address, 0,
                              provider_fees,
                              {"from": bob})

asset_dir = ocean.assets.download_asset(ddo, bob, './', tx.txid)

• Option 2

Modify pay_for_access_service in order to accept dispenser funds and make it return the transaction hash provided by dispense_and_order() function.

@trentmc, @calina-c Any thoughts?

[mariacarmina]

@mariacarmina is this fixed now with pricing schema parameters? Does #1306 cover this aspect?

Still have the above topic which explains better the context.

[trentmc]

Re option 1: I don't like it, because it reads as if that's the only way that one can consume a free asset. Yet the main part of the readme, in section 3, shows already good you can consume a free asset (when you're holding datatokens).

Re option 2: I don't like this overloading if functionality of pay_for_access_service, it's outside the scope of what it should do.

Below is a new option. But first, I recommend to rename this issue to "provide a clearer flow for people to create and consume custody-light data assets" (ie DatatokenEnterprise).

• I say "custody-light" because that is the USP of DatatokenEnterprise: it allows orgs to buy & consume data without having custody of assets.
• Now that the goal is clearer, we can solve it with more clarity.

---

Option 3: create a new README called eg "custody-light flows".

• It should have two sections. One for each of:
  • free steps: publish for free, then consume
  • priced steps: publish with a price, then buy / consume
• at the top, it should give the USP, like given above
• link to this new README from the main readme, as a fourth bullet in the "Use-case flows" section
• the flow will use DatatokenEnterprise of course. (Note that we will rename that; see #1315. However these issues can be done orthogonally.)

---

I advocate option 3, ie to have a completely separate readme. Why: the flow is really quite different, since the consumer doesn't see datatokens at all. (And datatokens are at the heart of the main flow.)

[mariacarmina]

I agree with option 3, makes totally sense. Thank you for the discussion, that helped to elucidate the mystery :)

[mariacarmina]

From what we discussed previously, I will remove the Invalid status for this issue.

[calina-c]

Please put this on pause until we merge in the bundled pay for service. That will change a lot of the functionality.

[mariacarmina]

My opinion is that #1312 does not represent a blocker for this issue, because pay_for_access_service is not included here. If there are changes that could break this README, we will manage to fix those in a PR, but there won't be such major changes.