Browse files

Documentation cleanup (#484)

* Minor cleanup in top-level dir. More to come.

* Cleanup of all Documentation/ files besides

* Fix documentation headers in

* Rearrange a couple steps in instructions
  • Loading branch information...
rtannenbaum authored and holachuy committed Jul 20, 2018
1 parent 2301bb8 commit 65636e8a9e65b97d1dc9c077275e44214b144e04
@@ -1,5 +1,5 @@
# Contributors Code of Conduct
The Data Transfer Project team are committed to preserving and fostering a diverse, welcoming and open
The Data Transfer Project team is committed to preserving and fostering a diverse, welcoming and open
community. The Data Transfer Project is open to contributors from all walks of life and should be a
harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender
identity and expression, level of experience, nationality, personal appearance, race, religion, or
@@ -18,8 +18,4 @@ again.
## Code reviews
All submissions, including submissions by project members, require review. We
use Gerrit pull requests for this purpose. Consult
[Data Transfer Project website]( for more
information on using pull requests.
## This is not an official Google product
use GitHub pull requests for this purpose.
@@ -1,5 +1,5 @@
# Developers Guide
# Developer Guide
## Communication
@@ -12,7 +12,7 @@
## Where to start
### Understading the project
### Understanding the project
* Read the [Technical Overview]( page, particularly the architecture section
* Read the latest white papers at [](
@@ -23,19 +23,19 @@
### Running the project
* See the instructions below to run locally
## First Run/Setup
### From top level directory
From the top level data-transfer-project directory:
* Install Git: `sudo apt-get install git-all`
* Install Gradle: `sudo apt-get install gradle`
* Install NVM: `curl -o- | bash`
* Install Node: `nvm install node`
* Install Angular: `npm install -g @angular/cli`
* [Install Docker](
* Install the Angular CLI `./gradlew client:installLocalAngularCli`
* In the `client/` directory, run `npm install`
## IntelliJ setup
### IntelliJ setup
The following instructions work for IntelliJ IDEA version 2017.2.6.
#### Import the project
@@ -77,34 +77,9 @@ limitations under the License.
* It should prompt you to select the new Copyright profile
* The copyright should be added to all new files from now on (note: it might be collapsed so not immediately obvious)
## Acquire Keys
To run an instance of DTP you need to have API keys from the services you wish to interact with.
Please see the [acquire keys]( document for points on how to do this.
## Starting Docker
You only need to do this once.
* `sudo service docker start`
## Building/Running locally
The following builds and optionally runs the demo server (including Worker, API and UI) locally
* NOTE: The first time you run you need to configure your credentials by copying
distributions/demo-server/env.secrets.template to env.secrets and inserting the API keys
and secrets for the services you with to interact with.
## Running locally with Docker
* `./gradlew -PcloudType=local :distributions:demo-server:dockerize`
* This copies over LOCAL settings (configured in .gradle/properties.gradle) using
the LocalCloud implementation
* This will also build the docker image.
* `docker run --rm -p 3000:443 -p 5005:5005 -p 8080:8080 --env-file distributions/demo-server/env.secrets --name dtp-demo datatransferproject/demo`
* This will run the demo server image that was just created on localhost:8080
* You should now be able to access the web application at https://localhost:3000
* The API is accessible via https://localhost:8080/_/listDataTypes
* A java debugger can be connected via port 8080
* You can interact with the docker image via `docker exec -it dtp-demo <command>`
See [Running Locally]( for instructions.
## Deploying in production
@@ -113,12 +88,15 @@ A demo distribution for Google Cloud Platform is available at
A demo Azure distribution is also in development.
These distributions have :dockerize tasks which generate dockerfiles and
build images, similar to our local demo.
## Build Problem F.A.Q
### AutoValue errors
If you get an error `error: duplicate class... final class AutoValue_...` it is indicative of your IDE and gradle clashing. To fix it you want to delete the referenced build/classes directory.
## Misc Updates
## Misc updates
There is a rest version of the client and demo-server in progress. While it is not complete yet and lacks some of the
features in the http based client, it is still runnable and works for all services that implement oauth2.
@@ -74,7 +74,7 @@ include ':extensions:data-transfer:portability-data-transfer-foo'
7. Create FooTransferExtension extending TransferExtension
* See class javadoc for [TransferExtension]( for full documentation
* See class javadoc for [TransferExtension]( for full documentation
* Best practices:
@@ -90,7 +90,7 @@ include ':extensions:data-transfer:portability-data-transfer-foo'
Auth Extensions are located in the [extensions/auth module](, where they are organized by service provider
1. See class javadoc for the [AuthServiceExtension]( for full documentation
1. See class javadoc for the [AuthServiceExtension]( for full documentation
* This class should... (TODO: add more content)
* Best practices:
* Validate the AuthServiceExtension isn’t initialized more than once in a running binary
@@ -103,7 +103,7 @@ Auth Extensions are located in the [extensions/auth module](
3. Create the test directory for the FooAuthDataGeneratorTest
* `$mkdir -p extensions/auth/portability-auth-instagram/src/test/java/org/dataportabilityproject/auth/foo`
* `$mkdir -p extensions/auth/portability-auth-instagram/src/test/java/org/datatransferproject/auth/foo`
4. Create FooAuthDataGeneratorTest with appropriate tests
@@ -1,6 +1,9 @@
# API Keys
To use the Data Transfer Project each instance needs to get its own set of API keys
for each service they want to interact with. This has many reasons and benefits:
To run the Data Transfer Project, each hosted instance needs its own API keys
for each service it will facilitate transfers with. These keys allow the
DTP instance to interact with these services' APIS.
This has many reasons and benefits:
- **Isolation** Each instance of DTP having its own API keys means that each
instance is isloated from the others. So quota and abuse issues in one
@@ -61,4 +64,4 @@ for a template.
When running on a cloud provider your cloud implementation wil override
to read stored credentials from your cloud provider.
to read stored credentials from your cloud provider.
Oops, something went wrong.

0 comments on commit 65636e8

Please sign in to comment.