Environment variable example #11

danleecpi · 2020-07-10T10:43:51Z

Noticed in the AWS environment variable example you have AWS_BUCKET_NAME="feral-backups" twice.

AWS_ACCESS_KEY=
AWS_SECRET_KEY=
AWS_REGION="us-west-1"
AWS_BUCKET_NAME="feral-backups"
AWS_BUCKET_NAME="feral-backups"
AWS_BUCKET_SYNC_PREFIX="craft-sync/craft-test"
AWS_BUCKET_BACKUP_PREFIX="craft-backups/craft-test"

I'm setting up syncing for Backblaze what would the example be for that?

timmyomahony · 2020-07-10T10:46:18Z

Thanks @danleecpi I've fixed that typo now. I have a guide here for Backblaze setup here which should help:

https://github.com/weareferal/craft-remote-backup/wiki/Backblaze-B2

timmyomahony · 2020-07-10T10:47:12Z

Here's the environment variables:

https://github.com/weareferal/craft-remote-backup/wiki/Backblaze-B2#create-environment-variables

BACKBLAZE_B2_MASTER_KEY_ID="..."
BACKBLAZE_B2_MASTER_APP_KEY="..."
BACKBLAZE_B2_BUCKET_NAME="feral-backups"
BACKBLAZE_B2_BUCKET_PATH="craft-backups/craft-test"

Bear in mind you can call the variables whatever you like as long as you reference them correctly from the Control Panel settings page.

danleecpi · 2020-07-10T11:11:59Z

Thanks,

I'm trying to run the following locally and on the server:
composer require gliterd/backblaze-b2:^1.3.0

I get the following error:

Problem 1
    - Installation request for gliterd/backblaze-b2 ^1.3.0 -> satisfiable by gliterd/backblaze-b2[1.3.0].
    - Conclusion: remove nesbot/carbon 2.36.0
    - Conclusion: don't install nesbot/carbon 2.36.0

timmyomahony · 2020-07-10T11:23:43Z

Interesting. I haven't run into that before.

It looks like the gliterd/backblaze-b2 package requires an older version of nesbot/carbon (~1.20). You must have another dependency that requires a newer version of nesbot/carbon (looks like 2.36.X is the latest) causing the conflict.

Can you show your composer.json?

One option here is to fork the gliterd/backblaze-b2 to include a more up-to-date version of carbon.

danleecpi · 2020-07-10T11:26:29Z

Here is the composer.json file contents:

{
  "name": "nystudio107/craft",
  "description": "nystudio107 Craft 3 CMS scaffolding project",
  "version": "1.0.15",
  "keywords": [
    "craft",
    "cms",
    "craftcms",
    "project"
  ],
  "license": "MIT",
  "homepage": "https://craftcms.com/",
  "type": "project",
  "support": {
    "email": "info@nystudio107.com",
    "issues": "https://github.com/nystudio107/craft/issues",
    "source": "https://github.com/nystudio107/craft",
    "docs": "https://github.com/nystudio107/craft"
  },
  "minimum-stability": "RC",
  "require": {
    "benjamminf/craft-embedded-assets": "dev-master",
    "carlcs/craft-redactorcustomstyles": "^3.0",
    "charliedev/section-field": "1.1.0",
    "clubstudioltd/craft-asset-rev": "v6.0.2",
    "craftcms/aws-s3": "1.2.2",
    "craftcms/cms": "3.4.26",
    "craftcms/feed-me": "4.2.3",
    "craftcms/redactor": "2.6.1",
    "craftcms/webhooks": "2.3.2",
    "doublesecretagency/craft-digitaldownload": "^2.1",
    "enupal/snapshot": "1.2.8",
    "ether/simplemap": "3.8.4.1",
    "fruitstudios/linkit": "1.1.11",
    "jalendport/craft-readtime": "1.6.0",
    "la-haute-societe/craft-page-exporter": "1.1.7",
    "mundschenk-at/php-typography": "^6.4",
    "mustasj/expanded-search": "1.3.1",
    "nav33d/craft-activitylog": "1.0.1",
    "nystudio107/craft-cookies": "^1.1",
    "nystudio107/craft-eagerbeaver": "^1.0.0",
    "nystudio107/craft-imageoptimize": "1.6.14",
    "nystudio107/craft-minify": "^1.2.5",
    "nystudio107/craft-retour": "3.1.39",
    "nystudio107/craft-scripts": "^1.2.4",
    "nystudio107/craft-seomatic": "3.3.8",
    "nystudio107/craft-typogrify": "1.1.18",
    "nystudio107/craft-webperf": "1.0.18",
    "nystudio107/craft3-multi-environment": "^1.0.5",
    "ostark/craft-async-queue": "2.1.1",
    "pennebaker/craft-architect": "2.3.3",
    "solspace/craft-freeform": "3.7.0",
    "superbig/craft-vzaddress": "^2.0",
    "topshelfcraft/supersort": "3.0.1",
    "vaersaagod/dospaces": "1.1.2",
    "verbb/field-manager": "2.1.6",
    "verbb/image-resizer": "2.0.9",
    "verbb/super-table": "2.5.1",
    "vlucas/phpdotenv": "^2.4.0",
    "weareferal/remote-backup": "1.1.0",
    "yiisoft/yii2-redis": "^2.0.6"
  },
  "autoload": {
    "psr-4": {
      "modules\\sitemodule\\": "modules/sitemodule/src/"
    }
  },
  "config": {
    "optimize-autoloader": true,
    "sort-packages": true,
    "platform": {
      "php": "7.2"
    }
  },
  "scripts": {
    "post-root-package-install": [
      "@php -r \"file_exists('.env') || copy('.env.example', '.env');\""
    ],
    "post-create-project-cmd": [
      "@php craft setup/welcome",
      "@php nys-setup welcome"
    ],
    "post-update-cmd": [
      "@php nys-setup update"
    ],
    "post-install-cmd": [
      "@php nys-setup update"
    ]
  }
}

timmyomahony · 2020-07-10T11:54:40Z

@danleecpi it looks like there is a working fork with updated dependencies here sevend93/backblaze-b2.

Could you remove gliterd/backblaze-b2 and try with sevend93/backblaze-b2?

danleecpi · 2020-07-10T13:31:53Z

Yes that seems to install fine!

timmyomahony · 2020-07-10T13:32:38Z

Great, let me know if you've any other issues with it

danleecpi · 2020-07-10T14:32:35Z

Thanks @timmyomahony

Just run my first test. I get the following error:

Class 'BackblazeB2\Client' not found

This is my setup.

I have defined my environment variables in the env file and aliases in general.php config file.

env file

// Backblaze Settings
    'BACKBLAZE_B2_MASTER_KEY_ID' => '###',
    'BACKBLAZE_B2_MASTER_APP_KEY' => '###',
    'BACKBLAZE_B2_BUCKET_NAME' => 'live-backup',
    'BACKBLAZE_B2_BUCKET_BACKUP_PREFIX' => 'craft-backup',
    'BACKBLAZE_B2_BUCKET_SYNC_PREFIX' => 'craft-sync',

general.php

'@backblazeMasterKey' => getenv('BACKBLAZE_B2_MASTER_KEY_ID'),
'@backblazeAppKey' => getenv('BACKBLAZE_B2_MASTER_APP_KEY'),
'@backblazeBucket' => getenv('BACKBLAZE_B2_BUCKET_NAME'),
'@backblazeBackup' => getenv('BACKBLAZE_B2_BUCKET_BACKUP_PREFIX'),
'@backblazeSync' => getenv('BACKBLAZE_B2_BUCKET_SYNC_PREFIX'),

I then used the aliases in the plugin settings.

timmyomahony · 2020-07-10T14:44:03Z

Is that first image from the Craft queue? Are you on the same machine you installed the sevend93/backblaze-b2 library on or is this in staging/production?

That error would suggest that it's an issue with the instance not finding the backblaze-b2 library (it doesn't look like an issue with your credentials). Might be a composer/composer.lock issue with the library not actually being installed?

As an aside, you shouldn't needed env vars AND aliases. You can just reference the env vars directly into the control panel ($BACKBLAZE_B2_MASTER_KEY_ID).

danleecpi · 2020-07-10T14:57:46Z

Yes, the first image is from the Craft queue.
Yes I am on the same machine where sevend93/backblaze-b2 was installed.

I can see it in the composer.lock file and composer.json files:

Just tried installing again... this is what I get:

  - Installing sevend93/backblaze-b2 (1.3.2): Loading from cache
Package charliedev/section-field is abandoned, you should avoid using it. No replacement was suggested.
Writing lock file
Generating optimized autoload files
Deprecation Notice: Class weareferal\RemoteSync\assets\RemoteSyncsettings\RemoteSyncSettingAsset located in ./vendor/weareferal/remote-sync/src/assets/remotesyncsettings/RemoteSyncSettingAsset.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class weareferal\RemoteSync\assets\RemoteSyncutility\RemoteSyncUtilityAsset located in ./vendor/weareferal/remote-sync/src/assets/remotesyncutility/RemoteSyncUtilityAsset.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class superbig\vzaddress\assetbundles\VzAddress\VzAddressAsset located in ./vendor/superbig/craft-vzaddress/src/assetbundles/vzaddress/VzAddressAsset.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class modules\sitemodule\assetbundles\SiteModule\SiteModuleAsset located in ./modules/sitemodule/src/assetbundles/sitemodule/SiteModuleAsset.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class lhs\craftpageexporter\PageExporterVariable located in ./vendor/la-haute-societe/craft-page-exporter/src/variables/PageExporterVariable.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class Solspace\Freeform\Library\Codepack\Exceptions\CodepackException located in ./vendor/solspace/craft-freeform/src/Library/Codepack/Exceptions/CodePackException.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201

timmyomahony · 2020-07-10T15:51:02Z

That's very strange. And you can see it in your vendor folder too? I'm going to change the Backblaze approach to use the AWS SDK (like Digital Ocean and AWS S3) over the weekend. That should fix any issues you're having

danleecpi · 2020-07-11T07:28:01Z

Amazing - Ill give it a go on Monday

danleecpi · 2020-07-13T10:52:57Z

Hi @timmyomahony,

I'm guessing this is still in progress?

timmyomahony · 2020-07-13T11:32:28Z

Yep, just working on adding a new release now

timmyomahony · 2020-07-13T13:52:08Z

OK, 1.2.0 should have a better Backblaze approach (using the AWS SDK). Here's an updated guide:

https://github.com/weareferal/craft-remote-backup/wiki/Backblaze-B2

You will need to install:

composer require aws/aws-sdk-php:^3

and also note the variables will be a little different than before:

BACKBLAZE_B2_MASTER_KEY_ID="..."
BACKBLAZE_B2_MASTER_APP_KEY="..."
BACKBLAZE_B2_REGION_NAME="us-west-002" # don't include the "s3"
BACKBLAZE_B2_BUCKET_NAME="craft-backups"
BACKBLAZE_B2_BUCKET_PATH="mysite"

I've been testing this new release today so let me know of any issues

danleecpi · 2020-07-13T16:18:08Z

I have installed and tested. I have double-checked the credentials.

I received this error when using the alias.

Status | Failed
-- | --
Backblaze B2 Error (Check credentials)

I then tried adding the values in direct to the settings. I then received this error.

Backblaze B2 Error ("Malformed Access Key Id")

I checked the Access Key Id in backblaze again and it is correct.

I'm a bit stumped. :)

timmyomahony · 2020-07-13T16:27:40Z

I got that error myself before. Did you go back and re-create the app key (outlined here)? Backblaze has two types of keys:

"master" app keys
regular app keys

In this case, we need to use regular app keys that have been granted access to your new bucket (previously you could just use a master app key). If you have the wrong key set up then it could be showing this error as it can't access the bucket via the S3 API.

timmyomahony · 2020-07-13T16:30:19Z

I can replicate that error message when I use my master app key instead of a regular app key. I get the error:

2020-07-13 17:29:03 [-][1][-][trace][yii\base\InlineAction::runWithParams] Running action: weareferal\remotebackup\controllers\RemoteBackupController::actionListDatabases()
2020-07-13 17:29:04 [-][1][-][error][Aws\S3\Exception\S3Exception] exception 'Aws\S3\Exception\S3Exception' with message 'Error executing "ListObjects" on "https://craft-backup.s3.us-west-002.backblazeb2.com/?prefix=craft-remote-sync%2Fcraft-test&encoding-type=url"; AWS HTTP error: Client error: `GET https://craft-backup.s3.us-west-002.backblazeb2.com/?prefix=craft-remote-sync%2Fcraft-test&encoding-type=url` resulted in a `403 Forbidden` response:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
    <Code>InvalidAccessKeyId</Code>
    <Message>Malform (truncated...)
 InvalidAccessKeyId (client): Malformed Access Key Id - <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
    <Code>InvalidAccessKeyId</Code>
    <Message>Malformed Access Key Id</Message>
</Error>

EDIT: I've updated the guide to mention this:

https://github.com/weareferal/craft-remote-backup/wiki/Backblaze-B2#malformed-access-key-id

danleecpi · 2020-07-14T07:38:20Z

Thanks @timmyomahony

If we have to use the regular app keys I'd suggest changing the label name on the fields settings.

I copied in the regular app id and key and now I get a new error message. Do you know what it means?

Backblaze B2 Error ("The request body was too small")

timmyomahony · 2020-07-14T09:24:54Z

Good idea @danleecpi, I've updated the labels on the backblaze settings page as well as in the docs:

https://github.com/weareferal/craft-remote-backup/wiki/Backblaze-B2

Hopefull that will make it clearer.

Regarding that error message, I haven't seen that before. Can you post the full stack trace? I would guess that it's related to the upload of the backup once it's been created. Are you backing up the database or the volumes (or both)?

danleecpi · 2020-07-14T09:51:09Z

@timmyomahony what is the best way to send you the log files?

timmyomahony · 2020-07-14T09:57:12Z

Via email to timmy@weareferal.com, or you can find me on the Craft Discord username timmy also

timmyomahony · 2020-07-14T17:57:59Z

@danleecpi I'm going to close this now - hopefully we got the issue sorted for good! If it's still an issue let me know.

(Recreating the B2 bucket as well as the app key seemed to sort the issue)

danleecpi · 2020-07-14T18:32:35Z

Thanks! Seems to be working.

timmyomahony closed this as completed Jul 10, 2020

timmyomahony reopened this Jul 10, 2020

timmyomahony mentioned this issue Jul 10, 2020

Move Backblaze to AWS-compliant API #12

Closed

timmyomahony added the bug Something isn't working label Jul 14, 2020

timmyomahony closed this as completed Jul 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Environment variable example #11

Environment variable example #11

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

timmyomahony commented Jul 10, 2020 •

edited

danleecpi commented Jul 10, 2020 •

edited

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 11, 2020

danleecpi commented Jul 13, 2020

timmyomahony commented Jul 13, 2020

timmyomahony commented Jul 13, 2020

danleecpi commented Jul 13, 2020

timmyomahony commented Jul 13, 2020

timmyomahony commented Jul 13, 2020 •

edited

danleecpi commented Jul 14, 2020

timmyomahony commented Jul 14, 2020

danleecpi commented Jul 14, 2020

timmyomahony commented Jul 14, 2020

timmyomahony commented Jul 14, 2020

danleecpi commented Jul 14, 2020

Environment variable example #11

Environment variable example #11

Comments

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

timmyomahony commented Jul 10, 2020 • edited

danleecpi commented Jul 10, 2020 • edited

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

This is my setup.

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 10, 2020

timmyomahony commented Jul 10, 2020

danleecpi commented Jul 11, 2020

danleecpi commented Jul 13, 2020

timmyomahony commented Jul 13, 2020

timmyomahony commented Jul 13, 2020

danleecpi commented Jul 13, 2020

timmyomahony commented Jul 13, 2020

timmyomahony commented Jul 13, 2020 • edited

danleecpi commented Jul 14, 2020

timmyomahony commented Jul 14, 2020

danleecpi commented Jul 14, 2020

timmyomahony commented Jul 14, 2020

timmyomahony commented Jul 14, 2020

danleecpi commented Jul 14, 2020

timmyomahony commented Jul 10, 2020 •

edited

danleecpi commented Jul 10, 2020 •

edited

timmyomahony commented Jul 13, 2020 •

edited