Merge pull request #425 from steemit/develop

v0.14.2

Dockerfile

@@ -46,6 +46,11 @@ RUN \
     ./tests/chain_test && \
     rm -rf /usr/local/src/steemtest
 
+RUN \
+    cd /usr/local/src/steem && \
+    doxygen && \
+    programs/build_helpers/check_reflect.py
+
 RUN \
     cd /usr/local/src/steem && \
     cmake \

Doxyfile

@@ -1558,7 +1558,7 @@ EXTRA_SEARCH_MAPPINGS  =
 # If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
 # The default value is: YES.
 
-GENERATE_LATEX         = YES
+GENERATE_LATEX         = NO
 
 # The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put. If a
 # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of

doc/git-guildelines.md

@@ -1,37 +1,111 @@
-The git guidelines for Steemit are influenced by the [Graphene](https://github.com/cryptonomex/graphene/wiki/How-we-use-version-control) git guidelines as well as [Git Flow](http://nvie.com/posts/a-successful-git-branching-model/) and [this blog post](http://www.draconianoverlord.com/2013/09/07/no-cherry-picking.html).
+The git guidelines for SteemIt are influenced by the
+[Graphene](https://github.com/cryptonomex/graphene/wiki/How-we-use-version-control)
+git guidelines as well as [Git
+Flow](http://nvie.com/posts/a-successful-git-branching-model/) and [this
+blog
+post](http://www.draconianoverlord.com/2013/09/07/no-cherry-picking.html).
 
-### Branches
+## Branches
 
-- `origin/master`  : Contains releases of Steem. `origin/master` and `origin/HEAD` should always be at the most recent release. All witnesses should be running this branch.
-- `origin/staging` : Staging branch for new releases. Release candidates of develop will be merged into staging for review. Witnesses wanting to run the latest builds and test new features and bug fixes should be on this branch.
-- `origin/develop` : The development branch. Develop should always build but may contain bugs. This branch is only for developers.
+- `master`: Points to the current release of Steem.  Witnesses should be
+  running this branch. Each release commit will be tagged
+  `vMajor.Hardfork.Release`. When we get ready to release we will merge
+  feature branches into `develop` and then do a single merge into `master`
+  via a Pull Request. All PRs to `master` must pass automated testing to be
+  merged.
+- `develop`: The active development branch. We will strive to keep `develop`
+  in a working state. All PRs must pass automated tests before being merged.
+  While this is not a guarantee that `develop` is bug-free, it will
+  guarantee that the branch is buildable in our standard build configuration
+  and passes the current suite of tests. That being said, running a node
+  from `develop` has risks.  We recommend that any block producing node
+  build from `master`. If you want to test new features, develop is the
+  correct branch.
 
 ### Patch Branches
 
-All issues should be developed against their own branch. These branches should have the most recent staging branch as a working parent and then merged into develop when they are tested and ready to be deployed.
-If an issue needs another issue as a dependency, branch from `staging`, merge the dependent issue branch into the new branch, and begin development. The naming scheme we use is the issue number, then a dash, followed by a shorthand description of the issue. For example, issue 22 is to allow the removal of an upvote. Branch `22-undo-vote` was used to devlop the patch for this issue.
+All changes should be developed in their own branch. These branches
+should branch from `develop` and then merged back into `develop` when they are
+tested and ready. If an issue needs another issue as a
+dependency, branch from `develop`, merge the dependent issue branch into the
+new branch, and begin development. The naming scheme we use is the issue
+number, then a dash, followed by a shorthand description of the issue. For
+example, issue 22 is to allow the removal of an upvote. Branch
+`22-undo-vote` was used to develop the patch for this issue.
 
 ### Non-Issue Branches
 
-Some changes are so minor as to not require an issue, e.g. changes to logging. If in doubt, create an issue for the change and follow the guidelines above. Issues serve as documentation for many changes. These should still be done against `staging` where possible (or the earliest commit they depend on, where not possible) and merged into `develop`.  In practice you may develop such patches against `master`; then rebase against `develop` before pushing.
+Some changes are so minor as to not require an issue, e.g. changes to
+logging. Because the requirement of automated testing, create an issue for
+them. We err on the side of over-documentation rather than
+under-documentation.  Branch from `develop`, make your fix, and create a pull
+request into `develop`.
 
-##Policies
+Suggested formatting for such branches missing an issue is
+`YYYYMMDD-shortname`, e.g. `20160913-documentation-update`.  (The date in
+the branch is so that we can prune old/defunct ones periodically to keep the
+repo tidy.)
 
-### Force-push policy
+## Pull Requests
+
+All changes to `develop` and `master` are done through GitHub Pull Requests
+(PRs). This is done for several reasons:
+
+- It enforces two factor authentication. GitHub will only allow merging of a
+  pull request through their interface, which requires the dev to be logged
+  in.
+- If enforces testing. All pull requests undergo automated testing before
+  they are allowed to be merged.
+- It enforces best practices. Because of the cost of a pull request,
+  developers are encouraged to do more testing themselves and be certain of
+  the correctness of their solutions.
+- If enforces code review. All pull requests must be reviewed by a developer
+  other than the creator of the request. Pull requests made by external
+  developers should be reviewed by two internal developers. When a developer
+  reviews and approves a pull request they should +1 the request or leave a
+  comment mentioning their approval of the request. Otherwise they should
+  describe what the problem is with the request so the developer can make
+  changes and resubmit the request.
+
+All pull requests should reference the issue(s) they relate to in order to
+create a chain of documentation.
+
+If your pull request has merge conflicts, rebase against `origin/develop`,
+resolve the merge conflicts, force push to your branch, and resubmit the
+pull request.
+
+## Policies
+
+### Force-Push Policy
 
 - `origin/master` should never be force pushed.
-- `origin/staging` should rarely be force pushed. Exceptions to this policy may be made on a case-by-case basis.
-- `origin/develop` should rarely be force pushed. It may be force-pushed to kill patches that are prematurely merged. Force pushing beyond this is likely an idication development should have occured on an issue branch.
-- Individual patch branches may be force-pushed at any time, at the discretion of the developer or team working on the patch.
+- `origin/develop` should never be force pushed. All updates to `develop`
+  are done through PRs so force pushing is not allowed.
+- Individual patch branches may be force-pushed at any time, at the
+  discretion of the developer or team working on the patch.
 
 ### Tagging Policy
 
-- Tags are reserved for releases. The version scheme is `vMajor.Hardfork.Release` (Ex. v0.5.0 is the version for the Hardfork 5 release). Releases should be made only on `master`.
+- Tags are reserved for releases. The version scheme is
+  `vMajor.Hardfork.Release` (Ex. v0.5.0 is the version for the Hardfork 5
+  release). Releases should be made only on `master`.
 
-### Code review policy
+### Code Review Policy / PR Merge Process
 
-- Two developers *must* review *every* consensus-breaking change before it moves into `graphene/master`.
-- Two developers *should* review *every* patch before it moves into `graphene/master`.
+- Two developers *must* review *every* release before merging it into
+  `master`, enforced through pull requests.
+- Two developers *must* review *every* consensus-breaking change before it
+  moves into `develop`, enforced through pull requests.
+- Two developers *should* review *every* patch before it moves into
+  `develop`, enforced through pull requests.
 - One of the reviewers may be the author of the change.
-- This policy is designed to encourage you to take off your "writer hat" and put on your "critic/reviewer hat."  If this was a patch from an unfamiliar community contributor, would you accept it?  Can you understand what the patch does and check its correctness based only on its commit message and diff?  Does it break any existing tests, or need new tests to be written?  Is it stylistically sloppy -- trailing whitespace, multiple unrelated changes in a single patch, mixing bugfixes and features, or overly verbose debug logging?
-- Having multiple people look at a patch reduces the probability it will contain bugs.
+- This policy is designed to encourage you to take off your "writer hat" and
+  put on your "critic/reviewer hat."  If this were a patch from an
+  unfamiliar community contributor, would you accept it?  Can you understand
+  what the patch does and check its correctness based only on its commit
+  message and diff? Does it break any existing tests, or need new tests to
+  be written? Is it stylistically sloppy -- trailing whitespace, multiple
+  unrelated changes in a single patch, mixing bug fixes and features, or
+  overly verbose debug logging?
+- Having multiple people look at a patch reduces the probability it will
+  contain uncaught bugs.

libraries/chain/database.cpp

@@ -2414,9 +2414,9 @@ void database::account_recovery_processing()
 void database::expire_escrow_ratification()
 {
    const auto& escrow_idx = get_index_type< escrow_index >().indices().get< by_ratification_deadline >();
-   auto escrow_itr = escrow_idx.lower_bound( boost::make_tuple( false, false ) );
+   auto escrow_itr = escrow_idx.lower_bound( false );
 
-   while( escrow_itr != escrow_idx.end() && !escrow_itr->to_approved && !escrow_itr->agent_approved && escrow_itr->ratification_deadline <= head_block_time() )
+   while( escrow_itr != escrow_idx.end() && !escrow_itr->is_approved() && escrow_itr->ratification_deadline <= head_block_time() )
    {
       const auto& old_escrow = *escrow_itr;
       ++escrow_itr;
@@ -2439,13 +2439,19 @@ void database::process_decline_voting_rights()
    {
       const auto& account = itr->account(*this);
 
-      adjust_proxied_witness_votes( account, -account.witness_vote_weight() );
+      /// remove all current votes
+      std::array<share_type, STEEMIT_MAX_PROXY_RECURSION_DEPTH+1> delta;
+      delta[0] = -account.vesting_shares.amount;
+      for( int i = 0; i < STEEMIT_MAX_PROXY_RECURSION_DEPTH; ++i )
+         delta[i+1] = -account.proxied_vsf_votes[i];
+      adjust_proxied_witness_votes( account, delta );
+
       clear_witness_votes( account );
 
       modify( itr->account(*this), [&]( account_object& a )
       {
          a.can_vote = false;
-         a.proxy = a.name;
+         a.proxy = STEEMIT_PROXY_TO_SELF_ACCOUNT;
       });
 
       remove( *itr );

libraries/chain/hardfork.d/0_14.hf

@@ -12,6 +12,7 @@
 #define STEEMIT_HARDFORK_0_14__317 (STEEMIT_HARDFORK_0_14)
 #define STEEMIT_HARDFORK_0_14__327 (STEEMIT_HARDFORK_0_14)
 #define STEEMIT_HARDFORK_0_14__240 (STEEMIT_HARDFORK_0_14)
+#define STEEMIT_HARDFORK_0_14__410 (STEEMIT_HARDFORK_0_14)
 
 
 #define STEEMIT_HARDFORK_0_14_TIME  1474383600 // Tue, 20 Sep 2016 15:00:00 GMT

libraries/chain/include/steemit/chain/config.hpp

@@ -3,7 +3,7 @@
  */
 #pragma once
 
-#define STEEMIT_BLOCKCHAIN_VERSION              ( version(0, 14, 1) )
+#define STEEMIT_BLOCKCHAIN_VERSION              ( version(0, 14, 2) )
 #define STEEMIT_BLOCKCHAIN_HARDFORK_VERSION     ( hardfork_version( STEEMIT_BLOCKCHAIN_VERSION ) )
 
 #ifdef IS_TEST_NET

libraries/chain/include/steemit/chain/steem_objects.hpp

@@ -46,6 +46,8 @@ namespace steemit { namespace chain {
          bool           to_approved = false;
          bool           agent_approved = false;
          bool           disputed = false;
+
+         bool           is_approved()const { return to_approved && agent_approved; }
    };
 
    class savings_withdraw_object : public abstract_object<savings_withdraw_object> {
@@ -282,12 +284,11 @@ namespace steemit { namespace chain {
          >,
          ordered_unique< tag< by_ratification_deadline >,
             composite_key< escrow_object,
-               member< escrow_object, bool, &escrow_object::to_approved >,
-               member< escrow_object, bool, &escrow_object::agent_approved >,
+               const_mem_fun< escrow_object, bool, &escrow_object::is_approved >,
                member< escrow_object, time_point_sec, &escrow_object::ratification_deadline >,
                member< object, object_id_type, &object::id >
             >,
-            composite_key_compare< std::less< bool >, std::less< bool >, std::less< time_point_sec >, std::less< object_id_type > >
+            composite_key_compare< std::less< bool >, std::less< time_point_sec >, std::less< object_id_type > >
          >,
          ordered_unique< tag< by_sbd_balance >,
             composite_key< escrow_object,

libraries/chain/protocol/steem_operations.cpp

@@ -309,10 +309,15 @@ namespace steemit { namespace chain {
       FC_ASSERT( steem_amount.amount >= 0, "steem amount cannot be negative" );
       FC_ASSERT( sbd_amount.amount > 0 || steem_amount.amount > 0, "escrow must transfer a non-zero amount" );
       FC_ASSERT( from != agent && to != agent, "agent must be a third party" );
-      FC_ASSERT( fee.symbol != VESTS_SYMBOL, "fee cannot be paid in VESTS" );
+      FC_ASSERT( (fee.symbol == STEEM_SYMBOL) || (fee.symbol == SBD_SYMBOL), "fee must be STEEM or SBD" );
       FC_ASSERT( sbd_amount.symbol == SBD_SYMBOL, "sbd amount must contain SBD" );
       FC_ASSERT( steem_amount.symbol == STEEM_SYMBOL, "steem amount must contain STEEM" );
       FC_ASSERT( ratification_deadline < escrow_expiration, "ratification deadline must be before escrow expiration" );
+      if ( json_meta.size() > 0 )
+      {
+         FC_ASSERT( fc::is_utf8(json_meta), "JSON Metadata not formatted in UTF8" );
+         FC_ASSERT( fc::json::is_valid(json_meta), "JSON Metadata not valid JSON" );
+      }
    }
 
    void escrow_approve_operation::validate()const
@@ -378,12 +383,16 @@ namespace steemit { namespace chain {
       validate_account_name( to );
       FC_ASSERT( amount.amount > 0 );
       FC_ASSERT( amount.symbol == STEEM_SYMBOL || amount.symbol == SBD_SYMBOL );
+      FC_ASSERT( memo.size() < STEEMIT_MAX_MEMO_SIZE, "Memo is too large" );
+      FC_ASSERT( fc::is_utf8( memo ), "Memo is not UTF8" );
    }
    void transfer_from_savings_operation::validate()const {
       validate_account_name( from );
       validate_account_name( to );
       FC_ASSERT( amount.amount > 0 );
       FC_ASSERT( amount.symbol == STEEM_SYMBOL || amount.symbol == SBD_SYMBOL );
+      FC_ASSERT( memo.size() < STEEMIT_MAX_MEMO_SIZE, "Memo is too large" );
+      FC_ASSERT( fc::is_utf8( memo ), "Memo is not UTF8" );
    }
    void cancel_transfer_from_savings_operation::validate()const {
       validate_account_name( from );

libraries/chain/steem_evaluator.cpp

@@ -56,7 +56,25 @@ void witness_update_evaluator::do_apply( const witness_update_operation& o )
 {
    db().get_account( o.owner ); // verify owner exists
 
-   if ( db().has_hardfork( STEEMIT_HARDFORK_0_1 ) ) FC_ASSERT( o.url.size() <= STEEMIT_MAX_WITNESS_URL_LENGTH, "url is too long" );
+   if ( db().has_hardfork( STEEMIT_HARDFORK_0_1 ) )
+   {
+      FC_ASSERT( o.url.size() <= STEEMIT_MAX_WITNESS_URL_LENGTH, "url is too long" );
+   }
+   else if( o.url.size() > STEEMIT_MAX_WITNESS_URL_LENGTH )
+   {
+      // after HF, above check can be moved to validate() if reindex doesn't show this warning
+      wlog( "URL is too long in block ${b}", ("b", db().head_block_num()+1) );
+   }
+
+   if ( db().has_hardfork( STEEMIT_HARDFORK_0_14__410 ) )
+   {
+      FC_ASSERT( o.props.account_creation_fee.symbol == STEEM_SYMBOL );
+   }
+   else if( o.props.account_creation_fee.symbol != STEEM_SYMBOL )
+   {
+      // after HF, above check can be moved to validate() if reindex doesn't show this warning
+      wlog( "Wrong fee symbol in block ${b}", ("b", db().head_block_num()+1) );
+   }
 
    const auto& by_witness_name_idx = db().get_index_type< witness_index >().indices().get< by_name >();
    auto wit_itr = by_witness_name_idx.find( o.owner );
@@ -472,36 +490,30 @@ void escrow_transfer_evaluator::do_apply( const escrow_transfer_operation& o )
       FC_ASSERT( o.ratification_deadline > db().head_block_time(), "ratification deadline must be after head block time" );
       FC_ASSERT( o.escrow_expiration > db().head_block_time(), "escrow expiration must be after head block time" );
 
+      asset steem_spent = o.steem_amount;
+      asset sbd_spent = o.sbd_amount;
       if( o.fee.symbol == STEEM_SYMBOL )
-      {
-         FC_ASSERT( from_account.balance >= o.steem_amount + o.fee, "account cannot cover steem costs of escrow" );
-         FC_ASSERT( from_account.sbd_balance >= o.sbd_amount, "account cannot cover sbd costs of escrow" );
-      }
+         steem_spent += o.fee;
       else
-      {
-         FC_ASSERT( from_account.balance >= o.steem_amount, "account cannot cover steem costs of escrow" );
-         FC_ASSERT( from_account.sbd_balance >= o.sbd_amount + o.fee, "account cannot cover sbd costs of escrow" );
-      }
+         sbd_spent += o.fee;
 
-      if( o.fee.amount > 0 )
-      {
-         db().adjust_balance( from_account, -o.fee );
-      }
+      FC_ASSERT( from_account.balance >= steem_spent, "account cannot cover steem costs of escrow" );
+      FC_ASSERT( from_account.sbd_balance >= sbd_spent, "account cannot cover sbd costs of escrow" );
 
-      db().adjust_balance( from_account, -o.steem_amount );
-      db().adjust_balance( from_account, -o.sbd_amount );
+      db().adjust_balance( from_account, -steem_spent );
+      db().adjust_balance( from_account, -sbd_spent );
 
       db().create<escrow_object>([&]( escrow_object& esc )
       {
          esc.escrow_id              = o.escrow_id;
          esc.from                   = o.from;
          esc.to                     = o.to;
          esc.agent                  = o.agent;
+         esc.ratification_deadline  = o.ratification_deadline;
+         esc.escrow_expiration      = o.escrow_expiration;
          esc.sbd_balance            = o.sbd_amount;
          esc.steem_balance          = o.steem_amount;
          esc.pending_fee            = o.fee;
-         esc.ratification_deadline  = o.ratification_deadline;
-         esc.escrow_expiration      = o.escrow_expiration;
       });
    }
    FC_CAPTURE_AND_RETHROW( (o) )
@@ -1761,7 +1773,7 @@ void cancel_transfer_from_savings_evaluator::do_apply( const cancel_transfer_fro
    const auto& from = db().get_account( op.from );
    db().modify( from, [&]( account_object& a )
    {
-      a.savings_withdraw_requests++;
+      a.savings_withdraw_requests--;
    });
 }
 

libraries/plugins/account_statistics/include/steemit/account_statistics/account_statistics_plugin.hpp

@@ -56,6 +56,7 @@ class account_statistics_plugin : public steemit::app::plugin
 };
 
 struct account_stats_bucket_object
+   : public abstract_object< account_stats_bucket_object >
 {
    fc::time_point_sec   open;                                     ///< Open time of the bucket
    uint32_t             seconds = 0;                              ///< Seconds accounted for in the bucket
@@ -122,6 +123,7 @@ struct account_stats_bucket_object
 };
 
 struct account_activity_bucket_object
+   : public abstract_object< account_activity_bucket_object >
 {
    fc::time_point_sec   open;                                  ///< Open time for the bucket
    uint32_t             seconds = 0;                           ///< Seconds accounted for in the bucket
@@ -132,7 +134,9 @@ struct account_activity_bucket_object
 
 } } // steemit::account_statistics
 
-FC_REFLECT( steemit::account_statistics::account_stats_bucket_object,
+FC_REFLECT_DERIVED(
+   steemit::account_statistics::account_stats_bucket_object,
+   (graphene::db::object),
    (open)
    (seconds)
    (name)
@@ -196,3 +200,14 @@ FC_REFLECT( steemit::account_statistics::account_stats_bucket_object,
    (total_pow)
    (estimated_hashpower)
 )
+
+FC_REFLECT_DERIVED(
+   steemit::account_statistics::account_activity_bucket_object,
+   (graphene::db::object),
+
+   (open)
+   (seconds)
+   (active_market_accounts)
+   (active_forum_accounts)
+   (active_market_and_forum_accounts)
+)