New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jude #171

Merged
merged 150 commits into from Nov 12, 2015

Conversation

Projects
None yet
2 participants
@muneeb-ali
Member

muneeb-ali commented Nov 12, 2015

No description provided.

jcnelson added some commits Oct 10, 2015

Add a bunch of basic functionality unit tests. The naming is
self-explanatory--for example, name_preorder_register_update_cantforce
will create a namespace, preorder a name, register it, update it, try to
update it with a different key, and verify that the different key can't
force the update.

Each test allows the programmer to define a set of wallets filled with
btc in advance, create a blockchain by describing the order of
operations on a block-by-block basis, and then query the resulting
blockstore database to verify that each operation was processed
correctly.  The 'testlib.py' file serves as a common interface between
blockstore's API and the test logic--it will be stable for all tests, so
blockstore can evolve independently.
Fix virtualchain synchronization upcall logic so the test only needs to
call next_block() once (and doesn't need to call sync_virtualchain())
Add test that verifies that we can reconstruct the database at every
point in history, even when there are multiple names being registered, updated,
transferred, and revoked.
Add logic to next_block() to snapshot the db at each block and save its
consensus hash, as well as logic to use these snapshots to verify that
we can reconstruct the database at each block.  These tests will be
applied systemically.
On each test, verify that we can reconstruct the db at each point in
history and find the same consensus hash as was generated when the test
was running.
Identify consensus hash-covered fields; remove dead serialize() method;
allow the caller to supply the namespace_hash when building a
namespace preorder script
refactoring: remove serialize() in favor of having each operation
identify the fields in the name operation that the consensus hash must
cover.
Sync with virtualchain API; serialize name operations using their
consensus hash-covered fields; generate a virtual "expire" name
operation on each block that describes the names and namespaces that
expired on that block.
Lots of bug-fixes from testing:
* bugfix: track block_id --> [names] for *expiration*, not renewal
* bugfix: correctly deserialize name and namespace history
* add method to export the current database to a new file
* add method to strip name/namespace history
* add method to get the name as it was at a particular block number
* add method to get the name as it was at a particular block number
* add method to get the history of a name between two block numbers
* add method to get the set of all names and namespaces changed at a block number
* bugfix: only return a preorder hash if the object hasn't expired
* bugfix: off-by-one error when searching for consensus hash on NAME_UPDATE
* add method to get a name preorder by hash
* bugfix: use a name's namespace's 'lifetime' field to determine if it is expired.
* bugfix: on name expire, remember 'txid'
* bugfix: commit expired objects exactly once
* have each commit_() method for expired objects return the list of keys
to expired objects, so we can include them in the consensus hash.
* remember fee, op_fee, opcode, txid, and block_number for each operation.
* on NAME_UPDATE, remember the consensus hash
* bugfix: always remember the history of opcodes
* bugfix: don't store virtualchain fields
* bugfix: correct error message on namespace version mismatch
Bugfixes and features for adding name history and fast bootstrapping:
* all methods: caller can supply its own consensus hash
* add RPC method to get a name's blockchain-derived history
* allow some RPC methods to proceed even while we're indexing
* add method to verify that an untrusted database file is consistent
with a (block id, consensus hash) pair.
Properly construct NAME_IMPORT so we can recover its recipient and
sender; properly report namespace alterations at a particular block
(i.e. do so exactly once, not more than once).
Add tests for:
* operations on imported names
* no duplicate namespace reveals

jcnelson added some commits Oct 25, 2015

Unified logic for validating parsing and building inputs; document
consensus fields; make subsidization logic explicit; verify payload
length before parsing
Many bugfixes made during testing:
* update namerec documentation (and move it to where the name operation
consensus fields are defined)
* make state the method to sanitize history when parsing it from the db
* sanitize vtindex when loading it from the db
* rework is_name_expired() to examine the name's namespace's 'lifetime'
field and 'ready_block' field, instead of trying to track it as a
secondary index.
* eliminate 'expired' flag from namerecs
* get_nameops_at() will return the tx-ordered list of all changes of all
nameops that occur in a block, with the caveat that a name can chane
multiple times in a block.  In this situation, get_nameops_at() will
return the changes to the name in their relative order.
* namespace preorders, reveals, and name preorders expire relative to
their creation block and the length of time for which they are allowed
to exist.  There is no more 'expired' field for them.
* don't return a preorder if the name is already registered and not
expired.
* remove unused method is_name_expired_at
* restore_from_history() will return the sequence of states a name
passed through on a particular block.
* save_history() should save a list of diffs for a particular block
number.
* when a name expires, it's 'revoked' field gets reset
* no need to save expired preorders
* when registering a name that had already existed and expired, preserve
its prior history.
* store the importer script and importer address separately from the
sender and recipient for a name.
* store the history of imports for a name, if imported more than once.
* add log_prescan_find_collisions() to detect name registration and
namespace reveal collisions, and cull them.  Do away with the old
'collision' flag, since we no longer process name ops on a per-block
basis.
* when renewing, a name must be owned by the sender and recipient.
* deny updates on a name owned by someone who has exceeded their name
quota.
Multiple bugfixes made from testing:
* all methods: get rid of the pay_fee keyword argument: either a tx is subsidized or
it isn't.
* blockstore_name_transfer: coerce boolean for keepdata
* blockstore_name_renew: if the renewal address is not given, then
derive it from the private key.
* when recreating a registration virtualchain op from the name db,
derive the sender and recipient from the name's history.
* when recreating a transfer virtualchain op from the name db, derive
the keepdata flag from the 'op' bytecode
* when recreating an import virtualchain op from the name db, derive the
sender an recipient from the importer fields (not the history).
* when verifying a database, be sure to use a name's relative sequence
of changes to find the right point in history from which to recreate the
virtualchain op that created the name as it was.
* add 'importdb' operation for importing an untrusted database using a
trusted consensus hash and block number.
Add RPC calls for:
* getting all nameops at a block number
* getting the names owned by an address (#157)
Add command-line options for:
* setting the working directory
* rebuilding a database, i.e. in order to regenerate the consensus hash
listing from a known-good database, or to verify that a database is
consistent with a known-good consensus hash.
Misc:
* remove the need for a 'testset' field in the config file
Add code to fetch and store announcements pushed by a whitelist of
blockchain IDs.  This is meant to allow the blockstore developers or a
particular blockstore operator to send notifications to other blockstore
operators (#115).
Fetch and store ANNOUNCE data when we find it; remove the expiry data
from the consensus hash (since it's unnecessary and very difficult to
recover for SNV).
Add methods for sending an ANNOUNCE to the API endpoint; add methods to
get name operations and name operation hashes at different points in
time (to support SNV); factor out name record history restoration and
consensus field restoration from database verification logic (in
preparation to adding an "fsck" operation).

muneeb-ali added a commit that referenced this pull request Nov 12, 2015

@muneeb-ali muneeb-ali merged commit bbe37a4 into master Nov 12, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment