Skip to content

feat: add support for new DSN string #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 5, 2022
Merged

feat: add support for new DSN string #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f5f9fd0
feat: add support for new DSN string
christyjacob4 Nov 5, 2022
3ff9b6e
Update README.md
christyjacob4 Nov 5, 2022
0e5df89
Update README.md
christyjacob4 Nov 5, 2022
57be4d1
Update app/http.php
christyjacob4 Nov 5, 2022
3dd4b27
feat: fix errors
christyjacob4 Nov 5, 2022
520aedd
fix: linter
christyjacob4 Nov 5, 2022
7cb74a1
Merge remote-tracking branch 'origin/dev' into add-dsn-support
Meldiron Nov 5, 2022
e73fbf4
feat: review comments
christyjacob4 Nov 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 1 addition & 21 deletions .env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,6 @@
OPEN_RUNTIMES_EXECUTOR_ENV=development
OPEN_RUNTIMES_EXECUTOR_RUNTIMES=php-8.0
OPEN_RUNTIMES_EXECUTOR_STORAGE_DEVICE=Local
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_REGION=us-east-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_REGION=us-east-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_REGION=us-west-004
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_REGION=eu-central-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_REGION=eu-central-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_BUCKET=
OPEN_RUNTIMES_CONNECTION_STORAGE=file://localhost
OPEN_RUNTIMES_EXECUTOR_MAX_TIMEOUT=900
OPEN_RUNTIMES_EXECUTOR_BUILD_TIMEOUT=900
OPEN_RUNTIMES_EXECUTOR_CPUS=0
Expand Down
46 changes: 4 additions & 42 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,27 +50,7 @@ services:
environment:
- OPEN_RUNTIMES_EXECUTOR_ENV
- OPEN_RUNTIMES_EXECUTOR_RUNTIMES
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DEVICE
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_BUCKET
- OPEN_RUNTIMES_CONNECTION_STORAGE
- OPEN_RUNTIMES_EXECUTOR_MAX_TIMEOUT
- OPEN_RUNTIMES_EXECUTOR_BUILD_TIMEOUT
- OPEN_RUNTIMES_EXECUTOR_CPUS
Expand Down Expand Up @@ -99,27 +79,7 @@ volumes:
```
OPEN_RUNTIMES_EXECUTOR_ENV=development
OPEN_RUNTIMES_EXECUTOR_RUNTIMES=php-8.0
OPEN_RUNTIMES_EXECUTOR_STORAGE_DEVICE=Local
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_REGION=us-east-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_REGION=us-east-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_REGION=us-west-004
OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_REGION=eu-central-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_BUCKET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_ACCESS_KEY=
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_SECRET=
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_REGION=eu-central-1
OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_BUCKET=
OPEN_RUNTIMES_CONNECTION_STORAGE=file://localhost
OPEN_RUNTIMES_EXECUTOR_MAX_TIMEOUT=900
OPEN_RUNTIMES_EXECUTOR_BUILD_TIMEOUT=900
OPEN_RUNTIMES_EXECUTOR_CPUS=0
Expand All @@ -134,6 +94,8 @@ OPEN_RUNTIMES_EXECUTOR_DOCKER_HUB_USERNAME=
OPEN_RUNTIMES_EXECUTOR_DOCKER_HUB_PASSWORD=
```

> `OPEN_RUNTIMES_CONNECTION_STORAGE` takes a DSN string that represents a connection to your storage device. If you would like to use your local filesystem, you can use `file://localhost`. If using S3 or any other provider for storage, use a DSN of the following format `s3://access_key:access_secret@host:port/bucket_name?region=us-east-1`

> Docker compose names networks based on directory you are in. If your folder is not caled `executor`, you may need to change value of `OPEN_RUNTIMES_EXECUTOR_NETWORK` variable.

4. Start Docker container:
Expand Down
65 changes: 31 additions & 34 deletions app/http.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
use Utopia\Validator\Range;
use Utopia\Validator\Text;
use Utopia\Pools\Pool;
use Utopia\DSN\DSN;
use Utopia\Registry\Registry;

use function Swoole\Coroutine\batch;
Expand Down Expand Up @@ -165,45 +166,41 @@ function logError(Throwable $error, string $action, Logger $logger = null, Utopi

function getStorageDevice(string $root): Device
{
switch (App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_DEVICE', Storage::DEVICE_LOCAL)) {
case Storage::DEVICE_LOCAL:
default:
return new Local($root);
$connection = App::getEnv('OPEN_RUNTIMES_CONNECTION_STORAGE', '');

$acl = 'private';
$device = '';
$accessKey = '';
$accessSecret = '';
$bucket = '';
$region = '';

try {
$dsn = new DSN($connection);
$device = $dsn->getScheme();
$accessKey = $dsn->getUser() ?? '';
$accessSecret = $dsn->getPassword() ?? '';
$bucket = $dsn->getPath() ?? '';
$region = $dsn->getParam('region');
} catch (\Exception $e) {
Console::warning('Defaulting to Local storage due to error: ' . $e->getMessage());
$device = 'Local';
}

switch ($device) {
case Storage::DEVICE_S3:
$s3AccessKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_ACCESS_KEY', '');
$s3SecretKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_SECRET', '');
$s3Region = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_REGION', '');
$s3Bucket = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_BUCKET', '');
$s3Acl = 'private';
return new S3($root, $s3AccessKey, $s3SecretKey, $s3Bucket, $s3Region, $s3Acl);
return new S3($root, $accessKey, $accessSecret, $bucket, $region, $acl);
case Storage::DEVICE_DO_SPACES:
$doSpacesAccessKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_ACCESS_KEY', '');
$doSpacesSecretKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_SECRET', '');
$doSpacesRegion = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_REGION', '');
$doSpacesBucket = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_BUCKET', '');
$doSpacesAcl = 'private';
return new DOSpaces($root, $doSpacesAccessKey, $doSpacesSecretKey, $doSpacesBucket, $doSpacesRegion, $doSpacesAcl);
return new DOSpaces($root, $accessKey, $accessSecret, $bucket, $region, $acl);
case Storage::DEVICE_BACKBLAZE:
$backblazeAccessKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_ACCESS_KEY', '');
$backblazeSecretKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_SECRET', '');
$backblazeRegion = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_REGION', '');
$backblazeBucket = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_BUCKET', '');
$backblazeAcl = 'private';
return new Backblaze($root, $backblazeAccessKey, $backblazeSecretKey, $backblazeBucket, $backblazeRegion, $backblazeAcl);
return new Backblaze($root, $accessKey, $accessSecret, $bucket, $region, $acl);
case Storage::DEVICE_LINODE:
$linodeAccessKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_ACCESS_KEY', '');
$linodeSecretKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_SECRET', '');
$linodeRegion = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_REGION', '');
$linodeBucket = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_BUCKET', '');
$linodeAcl = 'private';
return new Linode($root, $linodeAccessKey, $linodeSecretKey, $linodeBucket, $linodeRegion, $linodeAcl);
return new Linode($root, $accessKey, $accessSecret, $bucket, $region, $acl);
case Storage::DEVICE_WASABI:
$wasabiAccessKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_ACCESS_KEY', '');
$wasabiSecretKey = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_SECRET', '');
$wasabiRegion = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_REGION', '');
$wasabiBucket = (string) App::getEnv('OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_BUCKET', '');
$wasabiAcl = 'private';
return new Wasabi($root, $wasabiAccessKey, $wasabiSecretKey, $wasabiBucket, $wasabiRegion, $wasabiAcl);
return new Wasabi($root, $accessKey, $accessSecret, $bucket, $region, $acl);
case Storage::DEVICE_LOCAL:
default:
return new Local($root);
}
}

Expand Down
1 change: 1 addition & 0 deletions composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"utopia-php/logger": "0.3.*",
"utopia-php/cli": "0.13.*",
"utopia-php/storage": "0.11.*",
"utopia-php/dsn": "0.1.*",
"utopia-php/registry": "0.5.*",
"utopia-php/preloader": "0.2.*",
"utopia-php/system": "dev-feat-cpu-usage as 0.5.0",
Expand Down
49 changes: 48 additions & 1 deletion composer.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 1 addition & 21 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,27 +35,7 @@ services:
environment:
- OPEN_RUNTIMES_EXECUTOR_ENV
- OPEN_RUNTIMES_EXECUTOR_RUNTIMES
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DEVICE
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_S3_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_DO_SPACES_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_BACKBLAZE_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_LINODE_BUCKET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_ACCESS_KEY
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_SECRET
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_REGION
- OPEN_RUNTIMES_EXECUTOR_STORAGE_WASABI_BUCKET
- OPEN_RUNTIMES_CONNECTION_STORAGE
- OPEN_RUNTIMES_EXECUTOR_MAX_TIMEOUT
- OPEN_RUNTIMES_EXECUTOR_BUILD_TIMEOUT
- OPEN_RUNTIMES_EXECUTOR_CPUS
Expand Down