Skip to content

Commit

Permalink
Add tstore backend and plugin architecture.
Browse files Browse the repository at this point in the history
This diff rewrites much of the politeia codebase. A summary of the major
changes are listed below.

**politeiad tstore backend**

A new backend was added to politeiad that is referred to as the tstore backend.
It combines a trillian log, commonly referred to as a tlog, and a key-value
store. The tstore backend differs from the git backend in three main ways.

1. Scalability

	The git backend relied on git repos saved to the file system to store data.
	This approach cannot be scaled out since it is constrained to the file system
	of a single politeiad instances. The tstore backend moves the data into an
	actual database, allowing for much greater scalability.

2. Separate the timestamps and the data blobs

	The git backend would timestamp the git commit hashes onto the decred
	blockchain. These commit hashes are merkle roots of all the files in the
	commit. The timestamp and the data blobs in this setup are interconnected. If
	you delete a data blob from the commit history, such as if you need to censor
	a public comment, you destroy the ability to recreate the merkle path for
	other data blobs included in that commit merkle root, thus destroying you're
	ability to prove inclusion of data in a decred timestamp. Your only option is
	to remove the data from the git repo in a new commit, but the censored data
	remains in the git history. The tstore backend seperates out the timestamps
	from the data blobs. A merkle tree of the data is stored in a tlog instance
	while the data itself is stored in a key-value database. This means we can
	delete data from the database, such as if you need to censor a public
	comment, without impacting any of the timestamps or our ability to prove
	the inclusion of a piece of data in a timestamp.

3. Ability to retrieve timestamp inclusion proofs

	The tstore backend gives us the ability to retrieve an inclusion proof for
	any piece of data in the backend, providing cryptographic proof that the data
	blob was included in a dcr timestamp. There was no easy way to do this in the
	git backend.

**politeiad v2 API**

A politeiad v2 API was added. This API is a simplified version of the politeiad
v1 API. The tstore backend relies solely on the politeiad v2 API.

**politeiad plugin architecture**

A formalized plugin architecture was added to the tstore backend.

A politeiad plugin extends a record with additional functionality. Some
examples of this include adding comments to a record, adding the ability to run
a dcr ticket vote on a record, and adding record validation that is specific to
the decred proposal system.

This plugin architecture allows politeiad to become a configurable system. If a
user wants to runs a politeiad instance that allows for specific file types and
record comments, they can do so by simply adjusting the politeiad
configuration. No new code is required unless they want to add in more
granular, application specific validation.

The end goal is to eventually allow plugin code to exist outside of the
politeia repository that can be dynamically loaded at runtime. While this is
not possible yet, this diff lays the foundation for doing this in the future.

**politeiawww APIs**

The politeiawww APIs have been rewritten to be generic, application agnostic
APIs. These APIs were formally tied to specific politeia applications such as
the decred proposal system and the decred contractor management system. You
would use a proposal route to submit new proposals and a separate invoice
route to submit cms invoices. These application specific routes have been
replaced with more generic APIs. For example, there now exists a single set
of routes for submitting all records, regardless of whether that record is a
proposal or an invoice. The application specific validation has been pushed
into the politeiad plugins. The validation that is specific to proposals is
now performed in a proposal system politeiad plugin.

The politeiawww APIs now map to the politeiad plugins. For example, the
politeiad comments plugin has a corresponding politeiawww API that can be used
for submitting comments on any record, regardless of whether its a proposal
comment or invoice comment.

These application agnostic APIs allow politeia to become a configurable system.
You configure the politeiad plugins for your specific use case and you
automatically get the corresponding politeiawww API routes.
  • Loading branch information
lukebp committed Mar 29, 2021
1 parent 51a60c4 commit 2e41e04
Show file tree
Hide file tree
Showing 345 changed files with 46,843 additions and 39,921 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ politeiad/politeiad
politeiawww/cmd/politeiawww_dbutil/politeiawww_dbutil
politeiawww/cmd/politeiawww_refclient/politeiawww_refclient
politeiawww/cmd/politeiawwwcli/politeiawwwcli
politeiawww/cmd/pictl/pictl
politeiawww/cmd/politeiawwwtest/politeiawwwtest
politeiawww/cmd/politeiavoter/politeiavoter
politeiawww/politeiawww
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
ISC License

Copyright (c) 2013-2019 The btcsuite developers
Copyright (c) 2015-2020 The Decred developers
Copyright (c) 2015-2021 The Decred developers

Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
Expand Down
Loading

0 comments on commit 2e41e04

Please sign in to comment.