Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asset backed offers #1718

Merged
merged 13 commits into from
Aug 2, 2018
Merged

Asset backed offers #1718

merged 13 commits into from
Aug 2, 2018

Conversation

jonjove
Copy link
Contributor

@jonjove jonjove commented Jun 25, 2018

Implements cap-0003, which has been discussed in stellar/stellar-protocol#36. Note that this is a preliminary implementation and does not yet contain all necessary tests, so it should not be merged.

@@ -123,6 +129,16 @@ struct AccountEntry
{
case 0:
void;
case 1:

This comment was marked as resolved.

This comment was marked as resolved.

@@ -261,7 +261,7 @@ compareSizes(std::string const& objType, uint64_t inDatabase,
// in the ledger frames and db layer to make that work.

void
checkDBAgainstBuckets(medida::MetricsRegistry& metrics,
checkDBAgainstBuckets(uint32_t ledgerVersion, medida::MetricsRegistry& metrics,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should split this commit:

  • one for adding ledger version support when loading ledger entries
  • the other for adding liabilities support to account (including SQL schema upgrade)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version support no longer exists after these fixes.

@@ -99,7 +100,8 @@ BucketListIsConsistentWithDatabase::checkOnBucketApply(
default:
abort();
}
auto s = EntryFrame::checkAgainstDatabase(e.liveEntry(), mDb);
auto s = EntryFrame::checkAgainstDatabase(
mLedgerManager.getCurrentLedgerVersion(), e.liveEntry(), mDb);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is logically wrong: mLedgerManager.getCurrentLedgerVersion() has nothing to do with the buckets being applied ; in fact this is only set after buckets are applied (look for ProgressState::APPLIED_BUCKETS)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed version-based loading.

@@ -448,6 +452,7 @@ class ApplyBucketsWorkModifyEntry : public ApplyBucketsWork
, mKey(target.first)
, mFrame(target.second)
, mModified{false}
, mLedgerVersion(app.getLedgerManager().getCurrentLedgerVersion())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to catchup: currentLedgerVersion doesn't seem to be a version that has any good meaning

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed version-based loading.

@@ -273,6 +282,17 @@ AccountFrame::loadAccount(AccountID const& accountID, Database& db)
signers.end());
}

if (ledgerVersion >= 11)
{
assert(buyingLiabilitiesInd == sellingLiabilitiesInd);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we should just be doing this:

    assert(buyingLiabilitiesInd == sellingLiabilitiesInd);
    if (buyingLiabilitiesInd == soci::i_ok)
    {
        account.ext.v(1);
        account.ext.v1().liabilities = liabilities;
    }

the check as you wrote it doesn't actually enforce anything useful.
The liabilities being set transparently to 0 like this is just error prone as we throw away the information (by calling account.ext.v(1)).
The conversion from no liabilities -> with liabilities should be done 100% in the upgrade code - modifying entries on load is strange as, in theory, we're supposed to enforce that entries get written back if we do it like this (or risk having SQL and buckets get out of sync).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this also gets rid of the need to pass around the version number (previous commit)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, modifying on load is strange. Liabilities are now only enabled via addBuyingLiabilities and addSellingLiabilities.

case 6:
mSession << "ALTER TABLE peers ADD flags INT NOT NULL DEFAULT 0";
break;

case 7:
Upgrades::dropAll(*this);

This comment was marked as resolved.

This comment was marked as resolved.

if (shouldCheckAccount(*iter, ledgerVersion))
{
auto const& account = current.data.account();
Liabilities liabilities = (ledgerVersion >= 11)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of using ledgerVersion, you can simply use account.ext.v() == 1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect with the new changes that liabilities should be managed only by AccountFrame and TrustFrame functions get*Liabilities and add*Liabilities. In tests, accounts and trust lines created during the set-up phase (which occurs in the current protocol version) may have liabilities. These would then be incorrectly checked here regardless of the version if this change was made.

{
auto const& trust = current.data.trustLine();
Liabilities liabilities =
(ledgerVersion >= 11) ? trust.ext.v1().liabilities : Liabilities{};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, better to use trust.ext.v()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same response as above.

@@ -0,0 +1,249 @@
// Copyright 2018 Stellar Development Foundation and contributors. Licensed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new/deleted files -> need to update visual studio project file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -88,7 +88,7 @@ getTestConfig(int instanceNumber, Config::TestDbMode mode)
"CacheIsConsistentWithDatabase",
"ConservationOfLumens",
"LedgerEntryIsValid",
"MinimumAccountBalance"};
"LiabilitiesMatchOffers"};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forgot to update the example config (documentation for invariants)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -16,11 +17,13 @@ std::shared_ptr<Invariant>
CacheIsConsistentWithDatabase::registerInvariant(Application& app)
{
return app.getInvariantManager()
.registerInvariant<CacheIsConsistentWithDatabase>(app.getDatabase());
.registerInvariant<CacheIsConsistentWithDatabase>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To think about: is it better to pass just the objects that we need or just give up and pass whole Application everywhere (as it seems that over time more and more objects are required in multiple places)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I think it is better to pass just the objects that are needed. However, the Invariant subsystem was designed so that all invariants appear the same externally (registering and enabling every invariant should look exactly the same). That is why we pass Application into the Invariant-level registerInvariant which then forwards only the relevant subsystems to the InvariantManager-level registerInvariant.

: Invariant(false), mDb{db}
CacheIsConsistentWithDatabase::CacheIsConsistentWithDatabase(Database& db,
LedgerManager& lm)
: Invariant(false), mDb{db}, mLedgerManager(lm)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: false here is cryptic, should be replaced with enum when that makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this is cryptic and should be fixed, but I think it is outside the scope of this PR as it doesn't really have anything specific to do with cap-0003.

bool
AccountFrame::addSellingLiabilities(int64_t delta, LedgerManager& lm)
{
return stellar::addBalance(mAccountEntry.ext.v1().liabilities.selling,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that is valid. getBalanceAboveReserve already uses current value of getSellingLiabilities.

Lets assume current sellingliabilities = 20, balance = 60, minbalance = 20.
If we want to increase sellingliabilities by 20 to 40 (which should be possible), we call this method.

getBalanceAboveReserve then returns balance - minbalance - sellingliabiliies, which is 20. This is passes as maxBalance to the addBalance method, so we are not able to increase sellingliabilities.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent catch, fixed.


auto offers = OfferFrame::loadOffersByAccountAndAsset(
mAllowTrust.trustor, ci, db);
for (auto& offer : offers)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or even better - extract this part to method called deletAllLinkedOffers

mSheepLineA.reset();

int64_t buyingLiabilities = mSellSheepOffer->getBuyingLiabilities();
Asset const& buyingAsset = mSellSheepOffer->getBuying();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part looks very similar to change in account

is there any chance of some function extraction and code de-duplication here or is that not possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This segment has been refactored into a pair of functions releaseLiabilities and acquireLiabilities which facilitate liability management and is used in a few places such as ManageOfferOpFrame, AllowTrustOpFrame, and OfferExchange.

auto accountFrame =
AccountFrame::loadAccount(ld, accountOffers.first, db);
auto& account = accountFrame->getAccount();
assert(accountFrame);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes no sense here, should be moved line up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Previous line is actually deleted after these fixes, so done.

AccountID const& accountID, Asset const& asset, int64_t delta)
{
auto iter =
liabilities.insert(std::make_pair(asset, std::make_unique<int64_t>(0)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are probably executing make_unique too many times here, but as it is code that will be executed only once, it should no affect performance

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it is used many times, but I think trying to use it less might make determining the correctness much more difficult.

}

void
Upgrades::storeUpgradeHistory(LedgerManager& ledgerManager,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could probable be split into two commits - one for storeUpgradeHistory and friends, other for prepareLiabilities

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done.

}

static void
prepareLiabilities(LedgerManager& ledgerManager, LedgerDelta& ld)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I understand this method correctly, please let me know if this is correct:

For each account, remove all offers that are using assets, for which the liabilities are too big. So it does not keep any offers from that set, even if keeping only some of them would make all the liabilities proper?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You do understand this correctly, and I have added a comment to clarify. It works like this for two reasons:

  1. I don't think there is any natural method to determine which offers should remain from a given asset
  2. This method is independent of the order in which the offers are processed across assets


SECTION("liabilities overflow")
{
auto createOfferSmall =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems not to be used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, it was a remnant of an earlier test. Fixed.

for_versions_to(11, *app, [&] {
issuer.pay(a1, usd, 20000);

// ensure we could receive proceeds from the offer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably invalid comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -1039,6 +1170,12 @@ TEST_CASE("create offer", "[tx][offers]")
{b1, {{usd, 150}, {idr, trustLineBalance - 100}}},
{c1, {{usd, trustLineBalance - 225}, {idr, 150}}}});
});

for_versions_from(11, *app, [&] {
// makes "A" only capable of holding 75 "USD"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invalid comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

{
std::vector<OfferFrame::pointer> retOffers;
std::string sql = offerColumnSelector;
sql += " WHERE sellerid = :acc"

This comment was marked as resolved.

LedgerDelta& delta, Database& db,
LedgerManager& ledgerManager)
{
auto loadAccountIfNecessaryAndValidate = [this, &account, &delta, &db]() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to fix this but I think that explicit captures is only useful when doing multi-threaded/async calls.

{
// this would indicate a bug in OfferExchange
throw std::runtime_error("offer claimed over limit");
}

mSourceAccount->storeChange(delta, db);
mSourceAccount->storeChange(tempDelta, db);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sad, thanks for catching this

@@ -352,6 +350,35 @@ TransactionFrame::commonValid(SignatureChecker& signatureChecker,
}

return ValidationType::kFullyValid;

//// if we are in applying mode fee was already deduced from signing account
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs to be removed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was commented out as I was updating this to use getBalanceAboveReserve. Fixed.


## upgradehistory

Defined in [`src/transactions/TransactionFrame.cpp`](/src/herder/Upgrades.cpp)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong link

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -572,6 +590,17 @@ TEST_CASE("create offer", "[tx][offers]")
auto actualPayment = a1IDrs - delta;
checkCrossed(b1, actualPayment, offerAmount);
});

for_versions_from(10, *app, [&]() {
REQUIRE_THROWS_AS(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not what this test is trying to test:
test ensures that we compute reserve as if creating the offer.
I think that, with liabilities, there should be both a check that verifies that creating an offer with exactly actualPayment works, but trying to sell more (by 1 stroop) fails.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


for_versions_from(10, *app, [&]() {
// make it that c1 can only receive 9 USD (for 1 IDR)
REQUIRE_THROWS_AS(c1.changeTrust(usd, 9 * assetMultiplier),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same thing here: the way the setup is done is now impossible, probably clearer to not test this anymore like that and instead rely on changeTrust tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

});
REQUIRE_THROWS_AS(market.updateOffer(a1, offer.key.offerID,
{xlm, usd, oneone, 111},
{xlm, usd, oneone, 110}),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a bit confusing to have an expected state different as it's a failure case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, fixed.

@@ -1292,6 +1431,15 @@ TEST_CASE("create offer", "[tx][offers]")
{xlm, usd, oneone, 110});
});
});
for_versions_from(10, *app, [&] {
auto offer = market.requireChangesWithOffer({}, [&] {
return market.addOffer(a1, {xlm, usd, oneone, 9});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should change the amounts to be the ones around the edge case:
so use "10" instead of "9", and use "11" instead of "111"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

{mm34, {{xlm, minBalance3 - 3 * txfee}, {cur1, 0}, {cur2, 0}, {cur3, 20}, {cur4, 0}}},
{destination, {{xlm, minBalance1 - txfee}, {cur1, 0}, {cur2, 0}, {cur3, 0}, {cur4, 10}}}});
// clang-format on
REQUIRE_THROWS_AS(mm12a.pay(gateway, cur2, 39),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should those edge cases be tested differently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These edge cases aren't possible anymore, since it is not possible to have an offer with excess liabilities. Let's not test them starting in version 10.

VirtualClock clock;
auto cfg = getTestConfig(0);
cfg.LEDGER_PROTOCOL_VERSION = 9;
auto app = Application::create(clock, cfg);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be better to use createTestApplication instead (same for line 1011).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. All uses of Application::create in this file changed to createTestApplication.

int64_t
getBuyingLiabilities(AccountEntry const& acc, LedgerManager const& lm)
{
assert(lm.getCurrentLedgerVersion() >= 10);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're comparing to version 10 is many places, any reason not to have it as a constant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't followed this convention with other version numbers, and I'm not sure it would add much clarity. For example, do you think assert(lm.getCurrentLedgerVersion() >= PROTOCOL_VERSION_10); would be considerably more clear?

Copy link
Contributor

@marta-lokhova marta-lokhova Jul 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reasons I suggested it:

  1. You can easily change it. Previously you had 11, so instead of changing it to 10 in every place, you could just change one variable.
  2. you could name it something like PROTOCOL_VERSION_WITH_LIABILITIES and it would be way less vague than just 10

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not do that in this PR: v10 is broader than liabilities.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could even to as far as if (lm.supportsLiabilities()) { ... }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah like I said, not for this PR; and this would make the code even more complicated to understand as features depend on each other on a per protocol basis.

acc1.manageData(t1, &value);
});
for_versions_from(10, *app, [&] {
REQUIRE_THROWS_AS(acc1.manageData(t1, &value),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, I was thinking that we need to cover this in general

auto const minBal0 = app->getLedgerManager().getMinBalance(0);
auto const minBal3 = app->getLedgerManager().getMinBalance(3);

auto txfee = app->getLedgerManager().getTxFee();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

txfee: this is redefining/shadowing line 43 (can be removed)

// Note: Index from 1 rather than 0 to match the behavior of
// storeTransaction and storeTransactionFee.
Upgrades::storeUpgradeHistory(*this, lupgrade,
upgradeDelta.getChanges(), i + 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i+1 is of type size_t - yet storeUpgradeHistory takes an int, so this triggers a warning


// It is required to rollback the current LedgerHeader in order to satisfy
// the consistency checks enforced by LedgerDelta::commit.
getCurrentLedgerHeader() = initialHeader;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mean previousHeader here as to only rollbck whatever upgrade did?

@@ -23,6 +23,9 @@ namespace stellar
class LedgerManager;
class LedgerRange;

int64_t getBuyingLiabilities(AccountEntry const& acc, LedgerManager const& lm);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like versions of those for AccountEntry and TrustLineEntry could be merged as a template.

};

auto loadTrustIfNecessaryAndValidate = [this, &delta, &db](
TrustFrame::pointer const& trust,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing clang-format?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is only partially rebased right now, so no clang-format on the newest commits yet.

acc = AccountFrame::loadAccount(delta, getSellerID(), db);
assert(acc);
}
assert(acc->getID() == getSellerID());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see need for such asserts here. If we wish to ensure that loadAccount does what it says it does we could either:

  • test it with some unit tests
  • make that assert a postcondition on loadAccount inside that function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assert was actually put here to guarantee that if the parameter account != nullptr then that account matches the account that owns this offer. I agree that it does not need to test the case for AccountFrame::loadAccount. But given that I already wanted to test this condition for one case, I did not see a big disadvantage in checking all cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I have missed that case.

@@ -108,7 +108,7 @@ InflationOpFrame::doApply(Application& app, LedgerDelta& delta,
{
lcl.totalCoins += toDoleThisWinner;
}
if (!winner->addBalance(toDoleThisWinner))
if (!winner->addBalance(toDoleThisWinner, ledgerManager))
{
throw std::runtime_error(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This throw was added here as it was impossible for above condition even to get true.

But is it impossible now? Imagine an issuer that creates asset that requires authorization. It sends INT64_MAX of that asset to one account that has enough XLM to be a winner in inflation. Then that account can make an offer to sell this asset 1:1 for XLM, making its buying liabilities so big, that its addBalance returns 0. Then at the next inflation round all nodes will drop this transaction with txINTERNAL_ERROR, making inflation operations no-op for as long as they want.

I think we need another solution for handling that case now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch @vogel : I think you are correct, toDoleThisWinner should be capped by the amount that the account can receive instead of just being bigDivide(amountToDole, w.mVotes, totalVotes, ROUND_DOWN).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch indeed. Fixed as described above.


int64_t availableBalance =
(sheep.type() == ASSET_TYPE_NATIVE)
? mSourceAccount->getBalanceAboveReserve(ledgerManager)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could change getBalanceAboveReserver name to getAvailableBalance for symmetry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done.

ledgerManager.getCurrentLedgerHeader().baseReserve = newReserve;
if (header.ledgerVersion >= 10 && didReserveIncrease)
{
prepareLiabilities(ledgerManager, ld);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it is possible to execute prepareLiabilities twice during closing one ledger. Maybe applyUpgrade could return bool/enum/set to show that prepareLiabilities needs to be performed, then it would be only called once in closeLedger?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, this implementation is better as it associates upgrade side effects to the appropriate upgrade.

@bartekn bartekn mentioned this pull request Jul 9, 2018
1 task
// Test when no existing offers
root.pay(acc1, xlm, 500 + txfee);
REQUIRE_THROWS_AS(
market.addOffer(acc1, {xlm, usd, Price{1, 1}, 501},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that addOffer API allows you to leave last parameter if it will be the same as second one, as it defaults to OfferState::SAME

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, fixed on many lines.


SECTION("fees with liabilities")
{
auto acc = root.create("acc", lm.getMinBalance(1) + baseFee + 1000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that edge case here is that if you added +1 to that balance, it would pass even for version 1. Maybe it is worth adding such test case for contrast?

// using the initial result of step (1), so it does not matter what order the
// offers are processed.
static void
prepareLiabilities(LedgerManager& ledgerManager, LedgerDelta& ld)
Copy link
Contributor

@bartekn bartekn Jul 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this function also remove offers from accounts with trustline.isAuthorized() == false? AllowTrust operation removes them when revoking trust so it seems that leaving them out is an invalid state.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prepareLiabilities does remove offers that are not authorized, as demonstrated in https://github.com/stellar/stellar-core/pull/1718/files#diff-64c71f9133aa77f79d1d123bc6a59cf0R1086. The implementation uses the following logic:

  1. On lines 426-434, getAvailableBalance returns 0 if not authorized.
  2. On lines 453-461, getAvailableLimit returns 0 if not authorized.
  3. On line 542, erase == true if offerFrame.getSellingLiabilities() == 0 || offerFrame.getBuyingLiabilities() == 0.
  4. On lines 544-550, these checks can only be reached if erase == false after line 542. If so, we check that the liabilities do not exceed the relevant cap (either available balance or available limit, depending on whether it is a buying asset or a selling asset for the offer). If either the buying or selling asset is not authorized, then the cap will be 0. But because erase == false, the liabilities cannot be 0 so we end up with erase == true.

asset, db, &ld);
int64_t deltaSelling =
liab.selling -
trustFrame->getSellingLiabilities(ledgerManager);
Copy link
Contributor

@bartekn bartekn Jul 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function errors when there are orphaned offers:

Exception during upgrade: trustline does not exist

Repro:

  1. Run standalone testnet (testnet passphrase).
  2. Submit:
AAAAAGL8HQvQkbK2HA3WVjRrKmjX00fG8sLI7m0ERwJW/AX3AAABkAAAAAAAAAABAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAADJNEgW9b6gnIKGtYeSq1SiCyMJLujoFBoDtnljGf65oAAAAXSHboAAAAAAEAAAAADJNEgW9b6gnIKGtYeSq1SiCyMJLujoFBoDtnljGf65oAAAAGAAAAAVVTRAAAAAAAYvwdC9CRsrYcDdZWNGsqaNfTR8bywsjubQRHAlb8BfcAAAAAO5rKAAAAAAEAAAAADJNEgW9b6gnIKGtYeSq1SiCyMJLujoFBoDtnljGf65oAAAADAAAAAAAAAAFVU0QAAAAAAGL8HQvQkbK2HA3WVjRrKmjX00fG8sLI7m0ERwJW/AX3AAAAAACYloAAAAABAAAAAQAAAAAAAAAAAAAAAQAAAAAMk0SBb1vqCcgoa1h5KrVKILIwku6OgUGgO2eWMZ/rmgAAAAYAAAABVVNEAAAAAABi/B0L0JGythwN1lY0aypo19NHxvLCyO5tBEcCVvwF9wAAAAAAAAAAAAAAAAAAAAJW/AX3AAAAQK6ngzT5yLjXBKPibogU0NBm9M+W+ROt2JcPiIe1mf9eGNGaWhj0DYyG3GkTzrN4Tzg+LpXhiEO4cu6gylK6bQkxn+uaAAAAQAADEbTeCGHZ3RQVmUOX2pM8J+lFr5cq1PjOQZtoxSy1jYTnrvoRHoc7z9eSj9tv8aU/9ufEYBUYawc/l6hLMAI=
  1. Upgrade to V10.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was fixed in d8f0241.

Copy link
Contributor

@MonsieurNicolas MonsieurNicolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit (that you may decide to ignore). also, you need to rebase and reorder commits in commit order. If you decide to keep some of the commits as is (with fixes from previous commits), try to include the updated test cases with the commit (or have the next commit deal with the test cases).

@@ -505,7 +513,7 @@ TEST_CASE("liabilities", "[ledger][liabilities]")
REQUIRE(tf->getSellingLiabilities(lm) == 0);
if (res)
{
REQUIRE(tf->getTrustLine().ext.v() == 1);
REQUIRE(tf->getTrustLine().ext.v() == (deltaLiabilities != 0));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while this is correct, I find the implicit conversion from true to 1 quite obscure when used like this.

{
return stellar::addBalance(mAccountEntry.balance, delta);
if (delta == 0 && lm.getCurrentLedgerVersion() >= 10)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is currentLedgerVersion checked here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I see now that it doesn't need to be checked. Fixed.

@vogel
Copy link
Contributor

vogel commented Jul 20, 2018

LGTM, I'll merge when acceptance tests are working

@vogel
Copy link
Contributor

vogel commented Aug 1, 2018

r+ 3c9dfc0

latobarita added a commit that referenced this pull request Aug 1, 2018
Asset backed offers

Reviewed-by: vogel
@vogel
Copy link
Contributor

vogel commented Aug 1, 2018

@latobarita: retry

@vogel
Copy link
Contributor

vogel commented Aug 2, 2018

@latobarita: retry

latobarita added a commit that referenced this pull request Aug 2, 2018
Asset backed offers

Reviewed-by: vogel
@latobarita latobarita merged commit 3c9dfc0 into stellar:master Aug 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants