Skip to content

Commit

Permalink
Documentation cleanup (#484)
Browse files Browse the repository at this point in the history
* Minor cleanup in top-level dir. More to come.

* Cleanup of all Documentation/ files besides Integration.md

* Fix documentation headers in Developer.md

* Rearrange a couple steps in instructions
  • Loading branch information
rtannenbaum authored and holachuy committed Jul 20, 2018
1 parent 2301bb8 commit 65636e8
Show file tree
Hide file tree
Showing 16 changed files with 182 additions and 351 deletions.
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Contributors Code of Conduct
The Data Transfer Project team are committed to preserving and fostering a diverse, welcoming and open
The Data Transfer Project team is committed to preserving and fostering a diverse, welcoming and open
community. The Data Transfer Project is open to contributors from all walks of life and should be a
harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender
identity and expression, level of experience, nationality, personal appearance, race, religion, or
Expand Down
6 changes: 1 addition & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,4 @@ again.
## Code reviews

All submissions, including submissions by project members, require review. We
use Gerrit pull requests for this purpose. Consult
[Data Transfer Project website](https://github.com/google/data-transfer-project/) for more
information on using pull requests.

## This is not an official Google product
use GitHub pull requests for this purpose.
44 changes: 11 additions & 33 deletions Documentation/Developer.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

# Developers Guide
# Developer Guide

## Communication

Expand All @@ -12,7 +12,7 @@

## Where to start

### Understading the project
### Understanding the project
* Read the [Technical Overview](Overview.md) page, particularly the architecture section
* Read the latest white papers at [datatransferproject.dev](https://datatransferproject.dev)

Expand All @@ -23,19 +23,19 @@
### Running the project
* See the instructions below to run locally


## First Run/Setup

### From top level directory
From the top level data-transfer-project directory:
* Install Git: `sudo apt-get install git-all`
* Install Gradle: `sudo apt-get install gradle`
* Install NVM: `curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash`
* Install Node: `nvm install node`
* Install Angular: `npm install -g @angular/cli`
* [Install Docker](https://docs.docker.com/install/)
* Install the Angular CLI `./gradlew client:installLocalAngularCli`
* In the `client/` directory, run `npm install`

## IntelliJ setup
### IntelliJ setup
The following instructions work for IntelliJ IDEA version 2017.2.6.

#### Import the project
Expand Down Expand Up @@ -77,34 +77,9 @@ limitations under the License.
* It should prompt you to select the new Copyright profile
* The copyright should be added to all new files from now on (note: it might be collapsed so not immediately obvious)

## Acquire Keys
To run an instance of DTP you need to have API keys from the services you wish to interact with.
Please see the [acquire keys](Keys.md) document for points on how to do this.

## Starting Docker
You only need to do this once.
* `sudo service docker start`

## Building/Running locally
The following builds and optionally runs the demo server (including Worker, API and UI) locally

* NOTE: The first time you run you need to configure your credentials by copying
distributions/demo-server/env.secrets.template to env.secrets and inserting the API keys
and secrets for the services you with to interact with.
## Running locally with Docker

* `./gradlew -PcloudType=local :distributions:demo-server:dockerize`
* This copies over LOCAL settings (configured in .gradle/properties.gradle) using
the LocalCloud implementation
* This will also build the docker image.

* `docker run --rm -p 3000:443 -p 5005:5005 -p 8080:8080 --env-file distributions/demo-server/env.secrets --name dtp-demo datatransferproject/demo`
* This will run the demo server image that was just created on localhost:8080


* You should now be able to access the web application at https://localhost:3000
* The API is accessible via https://localhost:8080/_/listDataTypes
* A java debugger can be connected via port 8080
* You can interact with the docker image via `docker exec -it dtp-demo <command>`
See [Running Locally](RunningLocally.md) for instructions.

## Deploying in production

Expand All @@ -113,12 +88,15 @@ A demo distribution for Google Cloud Platform is available at

A demo Azure distribution is also in development.

These distributions have :dockerize tasks which generate dockerfiles and
build images, similar to our local demo.

## Build Problem F.A.Q

### AutoValue errors
If you get an error `error: duplicate class... final class AutoValue_...` it is indicative of your IDE and gradle clashing. To fix it you want to delete the referenced build/classes directory.

## Misc Updates
## Misc updates
There is a rest version of the client and demo-server in progress. While it is not complete yet and lacks some of the
features in the http based client, it is still runnable and works for all services that implement oauth2.

Expand Down
6 changes: 3 additions & 3 deletions Documentation/Integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ include ':extensions:data-transfer:portability-data-transfer-foo'

7. Create FooTransferExtension extending TransferExtension

* See class javadoc for [TransferExtension](https://github.com/google/data-transfer-project/blob/master/portability-spi-transfer/src/main/java/org/dataportabilityproject/spi/transfer/extension/TransferExtension.java) for full documentation
* See class javadoc for [TransferExtension](https://github.com/google/data-transfer-project/blob/master/portability-spi-transfer/src/main/java/org/datatransferproject/spi/transfer/extension/TransferExtension.java) for full documentation

* Best practices:

Expand All @@ -90,7 +90,7 @@ include ':extensions:data-transfer:portability-data-transfer-foo'

Auth Extensions are located in the [extensions/auth module](https://github.com/google/data-transfer-project/tree/master/extensions/auth), where they are organized by service provider

1. See class javadoc for the [AuthServiceExtension](https://github.com/google/data-transfer-project/blob/master/portability-spi-api/src/main/java/org/dataportabilityproject/spi/api/auth/extension/AuthServiceExtension.java) for full documentation
1. See class javadoc for the [AuthServiceExtension](https://github.com/google/data-transfer-project/blob/master/portability-spi-api/src/main/java/org/datatransferproject/spi/api/auth/extension/AuthServiceExtension.java) for full documentation
* This class should... (TODO: add more content)
* Best practices:
* Validate the AuthServiceExtension isn’t initialized more than once in a running binary
Expand All @@ -103,7 +103,7 @@ Auth Extensions are located in the [extensions/auth module](https://github.com/g

3. Create the test directory for the FooAuthDataGeneratorTest

* `$mkdir -p extensions/auth/portability-auth-instagram/src/test/java/org/dataportabilityproject/auth/foo`
* `$mkdir -p extensions/auth/portability-auth-instagram/src/test/java/org/datatransferproject/auth/foo`

4. Create FooAuthDataGeneratorTest with appropriate tests

Expand Down
9 changes: 6 additions & 3 deletions Documentation/Keys.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# API Keys
To use the Data Transfer Project each instance needs to get its own set of API keys
for each service they want to interact with. This has many reasons and benefits:
To run the Data Transfer Project, each hosted instance needs its own API keys
for each service it will facilitate transfers with. These keys allow the
DTP instance to interact with these services' APIS.

This has many reasons and benefits:

- **Isolation** Each instance of DTP having its own API keys means that each
instance is isloated from the others. So quota and abuse issues in one
Expand Down Expand Up @@ -61,4 +64,4 @@ for a template.

When running on a cloud provider your cloud implementation wil override
[AppCredentialStore](../portability-spi-cloud/src/main/java/org/dataportabilityproject/spi/cloud/storage/AppCredentialStore.java)
to read stored credentials from your cloud provider.
to read stored credentials from your cloud provider.
Loading

0 comments on commit 65636e8

Please sign in to comment.