Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure: malformed authorization header when authenticating with AZURE_STORAGE_ACCESS_KEY #1322

Closed
Mahagon opened this issue Aug 17, 2022 · 11 comments
Assignees

Comments

@Mahagon
Copy link

Mahagon commented Aug 17, 2022

Postgres

Issue description

Describe your problem

After updating from wal-g 1.1 to 2.0 authentication fails with AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_ACCESS_KEY. So wal-g can't access the Azure Storage Account.

Errormessage:

403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

maybe this has something to do with the issue? I'm unsure about that:
https://github.com/wal-g/wal-g/pull/1135/files#diff-0cb36811c5ab07631529c9e82697352bb620d50cf5f88e3e5503bb3e85441caaR261

Please provide steps to reproduce

  • Define PGUSER, PGPASSWORD, AZURE_STORAGE_ACCOUNT, AZURE_STORAGE_ACCESS_KEY, WALG_AZ_PREFIX, PGHOST, PGPORT environment variables
  • run /usr/local/bin/wal-g wal-verify integrity timeline --json

Please add config and wal-g stdout/stderr logs for debug purpose

logs
DEBUG: 2022/08/17 15:38:03.986865 --- COMPILED ENVIRONMENT VARS ---
DEBUG: 2022/08/17 15:38:03.986937 AWS_ACCESS_KEY=
DEBUG: 2022/08/17 15:38:03.986952 AWS_ACCESS_KEY_ID=
DEBUG: 2022/08/17 15:38:03.986953 AWS_CA_BUNDLE=
DEBUG: 2022/08/17 15:38:03.986955 AWS_CONFIG_FILE=
DEBUG: 2022/08/17 15:38:03.986956 AWS_DEFAULT_OUTPUT=
DEBUG: 2022/08/17 15:38:03.986957 AWS_DEFAULT_REGION=
DEBUG: 2022/08/17 15:38:03.986958 AWS_ENDPOINT=
DEBUG: 2022/08/17 15:38:03.986959 AWS_PROFILE=
DEBUG: 2022/08/17 15:38:03.986959 AWS_REGION=
DEBUG: 2022/08/17 15:38:03.986960 AWS_ROLE_SESSION_NAME=
DEBUG: 2022/08/17 15:38:03.986961 AWS_S3_FORCE_PATH_STYLE=
DEBUG: 2022/08/17 15:38:03.986962 AWS_SECRET_ACCESS_KEY=
DEBUG: 2022/08/17 15:38:03.986964 AWS_SECRET_KEY=
DEBUG: 2022/08/17 15:38:03.986965 AWS_SESSION_TOKEN=
DEBUG: 2022/08/17 15:38:03.986967 AWS_SHARED_CREDENTIALS_FILE=
DEBUG: 2022/08/17 15:38:03.986968 AZURE_BUFFER_SIZE=
DEBUG: 2022/08/17 15:38:03.986969 AZURE_ENDPOINT_SUFFIX=
DEBUG: 2022/08/17 15:38:03.986971 AZURE_ENVIRONMENT_NAME=
DEBUG: 2022/08/17 15:38:03.986972 AZURE_MAX_BUFFERS=
DEBUG: 2022/08/17 15:38:03.986974 AZURE_STORAGE_ACCESS_KEY=REDACTED
DEBUG: 2022/08/17 15:38:03.986975 AZURE_STORAGE_ACCOUNT=REDACTED
DEBUG: 2022/08/17 15:38:03.986976 AZURE_STORAGE_SAS_TOKEN=
DEBUG: 2022/08/17 15:38:03.986978 BG_MON_COMMIT=5d5002836118e4baf4b356f859566c56433e811f
DEBUG: 2022/08/17 15:38:03.986979 DEMO=false
DEBUG: 2022/08/17 15:38:03.987101 GCS_CONTEXT_TIMEOUT=
DEBUG: 2022/08/17 15:38:03.987103 GCS_ENCRYPTION_KEY=
DEBUG: 2022/08/17 15:38:03.987105 GCS_MAX_CHUNK_SIZE=
DEBUG: 2022/08/17 15:38:03.987107 GCS_MAX_RETRIES=
DEBUG: 2022/08/17 15:38:03.987108 GCS_NORMALIZE_PREFIX=
DEBUG: 2022/08/17 15:38:03.987109 GOMAXPROCS=
DEBUG: 2022/08/17 15:38:03.987111 GOOGLE_APPLICATION_CREDENTIALS=
DEBUG: 2022/08/17 15:38:03.987112 HOME=/root
DEBUG: 2022/08/17 15:38:03.987114 HOSTNAME=postgres-postgres-0
DEBUG: 2022/08/17 15:38:03.987116 HTTP_EXPOSE_EXPVAR=
DEBUG: 2022/08/17 15:38:03.987118 HTTP_EXPOSE_PPROF=
DEBUG: 2022/08/17 15:38:03.987119 HTTP_LISTEN=
DEBUG: 2022/08/17 15:38:03.987121 KUBERNETES_PORT=tcp://10.140.0.1:443
DEBUG: 2022/08/17 15:38:03.987122 KUBERNETES_PORT_443_TCP=tcp://10.140.0.1:443
DEBUG: 2022/08/17 15:38:03.987123 KUBERNETES_PORT_443_TCP_ADDR=10.140.0.1
DEBUG: 2022/08/17 15:38:03.987125 KUBERNETES_PORT_443_TCP_PORT=443
DEBUG: 2022/08/17 15:38:03.987126 KUBERNETES_PORT_443_TCP_PROTO=tcp
DEBUG: 2022/08/17 15:38:03.987128 KUBERNETES_SERVICE_HOST=10.140.0.1
DEBUG: 2022/08/17 15:38:03.987129 KUBERNETES_SERVICE_PORT=443
DEBUG: 2022/08/17 15:38:03.987131 KUBERNETES_SERVICE_PORT_HTTPS=443
DEBUG: 2022/08/17 15:38:03.987145 LC_ALL=en_US.utf-8
DEBUG: 2022/08/17 15:38:03.987146 LOG_ENV_DIR=/run/etc/log.d/env
DEBUG: 2022/08/17 15:38:03.987229 OS_AUTH_URL=
DEBUG: 2022/08/17 15:38:03.987230 OS_PASSWORD=
DEBUG: 2022/08/17 15:38:03.987232 OS_REGION_NAME=
DEBUG: 2022/08/17 15:38:03.987234 OS_TENANT_NAME=
DEBUG: 2022/08/17 15:38:03.987236 OS_USERNAME=
DEBUG: 2022/08/17 15:38:03.987237 PAM_OAUTH2=v1.0.1
DEBUG: 2022/08/17 15:38:03.987239 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/14/bin
DEBUG: 2022/08/17 15:38:03.987240 PATRONIVERSION=2.1.4
DEBUG: 2022/08/17 15:38:03.987267 PGBACKREST_STANZA=main
DEBUG: 2022/08/17 15:38:03.987268 PGDATA=/home/postgres/pgdata/pgroot/data
DEBUG: 2022/08/17 15:38:03.987270 PGDATABASE=
DEBUG: 2022/08/17 15:38:03.987272 PGHOME=/home/postgres
DEBUG: 2022/08/17 15:38:03.987274 PGHOST=postgres-postgres
DEBUG: 2022/08/17 15:38:03.987276 PGLOG=/home/postgres/pgdata/pgroot/pg_log
DEBUG: 2022/08/17 15:38:03.987278 PGPASSFILE=
DEBUG: 2022/08/17 15:38:03.987279 PGPASSWORD=REDACTED
DEBUG: 2022/08/17 15:38:03.987280 PGPORT=5432
DEBUG: 2022/08/17 15:38:03.987282 PGROOT=/home/postgres/pgdata/pgroot
DEBUG: 2022/08/17 15:38:03.987283 PGSSLMODE=
DEBUG: 2022/08/17 15:38:03.987285 PGUSER=postgres
DEBUG: 2022/08/17 15:38:03.987287 PG_AUTH_MON_COMMIT=437435b4e2de32a820e86973f6934ec849a768e0
DEBUG: 2022/08/17 15:38:03.987288 PG_MON_COMMIT=54fbdcc3cfe7e2a626bd96dda644d9a0c6866b58
DEBUG: 2022/08/17 15:38:03.987289 PG_PERMISSIONS_COMMIT=314b9359e3d77c0b2ef7dbbde97fa4be80e31925
DEBUG: 2022/08/17 15:38:03.987291 PG_READY_RENAME=
DEBUG: 2022/08/17 15:38:03.987292 PG_TM_AUX_COMMIT=6c012d38a4c1b0ba4a36952d60b0ce3a22ac9c3d
DEBUG: 2022/08/17 15:38:03.987294 PLANTUNER_COMMIT=800d81bc85da64ff3ef66e12aed1d4e1e54fc006
DEBUG: 2022/08/17 15:38:03.987295 PLPROFILER=REL4_1
DEBUG: 2022/08/17 15:38:03.987297 POD_NAME=postgres-postgres-0
DEBUG: 2022/08/17 15:38:03.987299 POD_NAMESPACE=test
DEBUG: 2022/08/17 15:38:03.987300 POSTGIS_LEGACY=3.2
DEBUG: 2022/08/17 15:38:03.987301 POSTGIS_VERSION=3.2
DEBUG: 2022/08/17 15:38:03.987303 POSTGRES_EXPORTER_PORT=tcp://10.141.82.49:9187
DEBUG: 2022/08/17 15:38:03.987304 POSTGRES_EXPORTER_PORT_9187_TCP=tcp://10.141.82.49:9187
DEBUG: 2022/08/17 15:38:03.987306 POSTGRES_EXPORTER_PORT_9187_TCP_ADDR=10.141.82.49
DEBUG: 2022/08/17 15:38:03.987307 POSTGRES_EXPORTER_PORT_9187_TCP_PORT=9187
DEBUG: 2022/08/17 15:38:03.987309 POSTGRES_EXPORTER_PORT_9187_TCP_PROTO=tcp
DEBUG: 2022/08/17 15:38:03.987310 POSTGRES_EXPORTER_SERVICE_HOST=10.141.82.49
DEBUG: 2022/08/17 15:38:03.987311 POSTGRES_EXPORTER_SERVICE_PORT=9187
DEBUG: 2022/08/17 15:38:03.987313 POSTGRES_EXPORTER_SERVICE_PORT_METRICS=9187
DEBUG: 2022/08/17 15:38:03.987315 POSTGRES_PASSWORD=REDACTED
DEBUG: 2022/08/17 15:38:03.987316 POSTGRES_POSTGRES_PORT=tcp://10.140.0.23:5432
DEBUG: 2022/08/17 15:38:03.987318 POSTGRES_POSTGRES_PORT_5432_TCP=tcp://10.140.0.23:5432
DEBUG: 2022/08/17 15:38:03.987319 POSTGRES_POSTGRES_PORT_5432_TCP_ADDR=10.140.0.23
DEBUG: 2022/08/17 15:38:03.987320 POSTGRES_POSTGRES_PORT_5432_TCP_PORT=5432
DEBUG: 2022/08/17 15:38:03.987322 POSTGRES_POSTGRES_PORT_5432_TCP_PROTO=tcp
DEBUG: 2022/08/17 15:38:03.987323 POSTGRES_POSTGRES_REPL_PORT=tcp://10.141.146.11:5432
DEBUG: 2022/08/17 15:38:03.987326 POSTGRES_POSTGRES_REPL_PORT_5432_TCP=tcp://10.141.146.11:5432
DEBUG: 2022/08/17 15:38:03.987327 POSTGRES_POSTGRES_REPL_PORT_5432_TCP_ADDR=10.141.146.11
DEBUG: 2022/08/17 15:38:03.987328 POSTGRES_POSTGRES_REPL_PORT_5432_TCP_PORT=5432
DEBUG: 2022/08/17 15:38:03.987330 POSTGRES_POSTGRES_REPL_PORT_5432_TCP_PROTO=tcp
DEBUG: 2022/08/17 15:38:03.987331 POSTGRES_POSTGRES_REPL_SERVICE_HOST=10.141.146.11
DEBUG: 2022/08/17 15:38:03.987333 POSTGRES_POSTGRES_REPL_SERVICE_PORT=5432
DEBUG: 2022/08/17 15:38:03.987334 POSTGRES_POSTGRES_REPL_SERVICE_PORT_POSTGRESQL=5432
DEBUG: 2022/08/17 15:38:03.987335 POSTGRES_POSTGRES_SERVICE_HOST=10.140.0.23
DEBUG: 2022/08/17 15:38:03.987337 POSTGRES_POSTGRES_SERVICE_PORT=5432
DEBUG: 2022/08/17 15:38:03.987338 POSTGRES_POSTGRES_SERVICE_PORT_POSTGRESQL=5432
DEBUG: 2022/08/17 15:38:03.987340 POSTGRES_USER=postgres
DEBUG: 2022/08/17 15:38:03.987342 PROFILE_MODE=
DEBUG: 2022/08/17 15:38:03.987343 PROFILE_PATH=
DEBUG: 2022/08/17 15:38:03.987345 PROFILE_SAMPLING_RATIO=
DEBUG: 2022/08/17 15:38:03.987347 RW_DIR=/run
DEBUG: 2022/08/17 15:38:03.987349 S3_CA_CERT_FILE=
DEBUG: 2022/08/17 15:38:03.987350 S3_ENDPOINT_PORT=
DEBUG: 2022/08/17 15:38:03.987352 S3_ENDPOINT_SOURCE=
DEBUG: 2022/08/17 15:38:03.987353 S3_LOG_LEVEL=
DEBUG: 2022/08/17 15:38:03.987355 S3_MAX_PART_SIZE=
DEBUG: 2022/08/17 15:38:03.987356 S3_MAX_RETRIES=
DEBUG: 2022/08/17 15:38:03.987358 S3_RANGE_BATCH_ENABLED=
DEBUG: 2022/08/17 15:38:03.987359 S3_RANGE_MAX_RETRIES=
DEBUG: 2022/08/17 15:38:03.987361 S3_SSE=
DEBUG: 2022/08/17 15:38:03.987362 S3_SSE_C=
DEBUG: 2022/08/17 15:38:03.987363 S3_SSE_KMS_ID=
DEBUG: 2022/08/17 15:38:03.987365 S3_STORAGE_CLASS=
DEBUG: 2022/08/17 15:38:03.987366 S3_USE_LIST_OBJECTS_V1=
DEBUG: 2022/08/17 15:38:03.987368 S3_USE_YC_SESSION_TOKEN=
DEBUG: 2022/08/17 15:38:03.987370 SET_USER=REL3_0_0
DEBUG: 2022/08/17 15:38:03.987390 SSH_PASSWORD=
DEBUG: 2022/08/17 15:38:03.987391 SSH_PORT=
DEBUG: 2022/08/17 15:38:03.987392 SSH_PRIVATE_KEY_PATH=
DEBUG: 2022/08/17 15:38:03.987394 SSH_USERNAME=
DEBUG: 2022/08/17 15:38:03.987395 TIMESCALEDB=1.7.5 2.3.1 2.7.0
DEBUG: 2022/08/17 15:38:03.987396 TOTAL_BG_UPLOADED_LIMIT=32
DEBUG: 2022/08/17 15:38:03.987398 UPLOAD_CONCURRENCY=
DEBUG: 2022/08/17 15:38:03.987399 WALE_ENV_DIR=/run/etc/wal-e.d/env
DEBUG: 2022/08/17 15:38:03.987400 WALE_GPG_KEY_ID=
DEBUG: 2022/08/17 15:38:03.987402 WALE_S3_PREFIX=
DEBUG: 2022/08/17 15:38:03.987403 WALE_VERSION=1.1.1
DEBUG: 2022/08/17 15:38:03.987404 WALG_ALIVE_CHECK_INTERVAL=
DEBUG: 2022/08/17 15:38:03.987406 WALG_AZURE_BUFFER_SIZE=
DEBUG: 2022/08/17 15:38:03.987407 WALG_AZURE_MAX_BUFFERS=
DEBUG: 2022/08/17 15:38:03.987408 WALG_AZ_PREFIX=azure://backups
DEBUG: 2022/08/17 15:38:03.987409 WALG_COMPRESSION_METHOD=lz4
DEBUG: 2022/08/17 15:38:03.987410 WALG_CSE_KMS_ID=
DEBUG: 2022/08/17 15:38:03.987412 WALG_CSE_KMS_REGION=
DEBUG: 2022/08/17 15:38:03.987414 WALG_DELTA_FROM_NAME=
DEBUG: 2022/08/17 15:38:03.987415 WALG_DELTA_FROM_USER_DATA=
DEBUG: 2022/08/17 15:38:03.987417 WALG_DELTA_MAX_STEPS=0
DEBUG: 2022/08/17 15:38:03.987418 WALG_DELTA_ORIGIN=
DEBUG: 2022/08/17 15:38:03.987419 WALG_DISK_RATE_LIMIT=
DEBUG: 2022/08/17 15:38:03.987421 WALG_DOWNLOAD_CONCURRENCY=10
DEBUG: 2022/08/17 15:38:03.987423 WALG_FETCH_TARGET_USER_DATA=
DEBUG: 2022/08/17 15:38:03.987424 WALG_FILE_PREFIX=
DEBUG: 2022/08/17 15:38:03.987426 WALG_GPG_KEY_ID=
DEBUG: 2022/08/17 15:38:03.987427 WALG_GS_PREFIX=
DEBUG: 2022/08/17 15:38:03.987429 WALG_INTEGRITY_MAX_DELAYED_WALS=0
DEBUG: 2022/08/17 15:38:03.987430 WALG_LIBSODIUM_KEY=
DEBUG: 2022/08/17 15:38:03.987431 WALG_LIBSODIUM_KEY_PATH=
DEBUG: 2022/08/17 15:38:03.987433 WALG_LIBSODIUM_KEY_TRANSFORM=none
DEBUG: 2022/08/17 15:38:03.987434 WALG_LOG_LEVEL=DEVEL
DEBUG: 2022/08/17 15:38:03.987436 WALG_NETWORK_RATE_LIMIT=
DEBUG: 2022/08/17 15:38:03.987437 WALG_PGP_KEY=
DEBUG: 2022/08/17 15:38:03.987439 WALG_PGP_KEY_PASSPHRASE=
DEBUG: 2022/08/17 15:38:03.987440 WALG_PGP_KEY_PATH=
DEBUG: 2022/08/17 15:38:03.987441 WALG_PG_WAL_SIZE=16
DEBUG: 2022/08/17 15:38:03.987443 WALG_PREFETCH_DIR=
DEBUG: 2022/08/17 15:38:03.987445 WALG_PREVENT_WAL_OVERWRITE=false
DEBUG: 2022/08/17 15:38:03.987447 WALG_S3_CA_CERT_FILE=
DEBUG: 2022/08/17 15:38:03.987448 WALG_S3_MAX_PART_SIZE=
DEBUG: 2022/08/17 15:38:03.987451 WALG_S3_PREFIX=
DEBUG: 2022/08/17 15:38:03.987453 WALG_S3_SSE=
DEBUG: 2022/08/17 15:38:03.987454 WALG_S3_SSE_C=
DEBUG: 2022/08/17 15:38:03.987455 WALG_S3_SSE_KMS_ID=
DEBUG: 2022/08/17 15:38:03.987457 WALG_S3_STORAGE_CLASS=
DEBUG: 2022/08/17 15:38:03.987458 WALG_SENTINEL_USER_DATA=
DEBUG: 2022/08/17 15:38:03.987459 WALG_SERIALIZER_TYPE=json_default
DEBUG: 2022/08/17 15:38:03.987461 WALG_SKIP_REDUNDANT_TARS=false
DEBUG: 2022/08/17 15:38:03.987462 WALG_SLOTNAME=
DEBUG: 2022/08/17 15:38:03.987464 WALG_SSH_PREFIX=
DEBUG: 2022/08/17 15:38:03.987465 WALG_STATSD_ADDRESS=
DEBUG: 2022/08/17 15:38:03.987466 WALG_STOP_BACKUP_TIMEOUT=
DEBUG: 2022/08/17 15:38:03.987468 WALG_STORAGE_PREFIX=
DEBUG: 2022/08/17 15:38:03.987469 WALG_STORE_ALL_CORRUPT_BLOCKS=false
DEBUG: 2022/08/17 15:38:03.987470 WALG_STREAM_CREATE_COMMAND=
DEBUG: 2022/08/17 15:38:03.987472 WALG_STREAM_RESTORE_COMMAND=
DEBUG: 2022/08/17 15:38:03.987473 WALG_SWIFT_PREFIX=
DEBUG: 2022/08/17 15:38:03.987474 WALG_TAR_DISABLE_FSYNC=false
DEBUG: 2022/08/17 15:38:03.987476 WALG_TAR_SIZE_THRESHOLD=1073741823
DEBUG: 2022/08/17 15:38:03.987478 WALG_UPLOAD_CONCURRENCY=16
DEBUG: 2022/08/17 15:38:03.987479 WALG_UPLOAD_DISK_CONCURRENCY=1
DEBUG: 2022/08/17 15:38:03.987480 WALG_UPLOAD_QUEUE=2
DEBUG: 2022/08/17 15:38:03.987482 WALG_UPLOAD_WAL_METADATA=NOMETADATA
DEBUG: 2022/08/17 15:38:03.987483 WALG_USE_COPY_COMPOSER=false
DEBUG: 2022/08/17 15:38:03.987484 WALG_USE_RATING_COMPOSER=false
DEBUG: 2022/08/17 15:38:03.987486 WALG_USE_REVERSE_UNPACK=false
DEBUG: 2022/08/17 15:38:03.987487 WALG_USE_WAL_DELTA=false
DEBUG: 2022/08/17 15:38:03.987489 WALG_VERIFY_PAGE_CHECKSUMS=false
DEBUG: 2022/08/17 15:38:03.987491 WALG_WITHOUT_FILES_METADATA=false
DEBUG: 2022/08/17 15:38:03.987492 WAL_ARCHIVING_EXPORTER_PORT=tcp://10.141.142.220:9104
DEBUG: 2022/08/17 15:38:03.987493 WAL_ARCHIVING_EXPORTER_PORT_9104_TCP=tcp://10.141.142.220:9104
DEBUG: 2022/08/17 15:38:03.987495 WAL_ARCHIVING_EXPORTER_PORT_9104_TCP_ADDR=10.141.142.220
DEBUG: 2022/08/17 15:38:03.987496 WAL_ARCHIVING_EXPORTER_PORT_9104_TCP_PORT=9104
DEBUG: 2022/08/17 15:38:03.987498 WAL_ARCHIVING_EXPORTER_PORT_9104_TCP_PROTO=tcp
DEBUG: 2022/08/17 15:38:03.987499 WAL_ARCHIVING_EXPORTER_SERVICE_HOST=10.141.142.220
DEBUG: 2022/08/17 15:38:03.987501 WAL_ARCHIVING_EXPORTER_SERVICE_PORT=9104
DEBUG: 2022/08/17 15:38:03.987503 WAL_ARCHIVING_EXPORTER_SERVICE_PORT_METRICS=9104
INFO: 2022/08/17 15:38:03.997469 Current WAL segment: 00000042000000000000007C
INFO: 2022/08/17 15:38:04.086484 Building check runner: integrity
ERROR: 2022/08/17 15:38:04.126527 Failed to build check runner integrity:%!(EXTRA *errors.withStack=Failed to initialize timeline history map: Error during .history file '00000042.history' downloading.: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
github.com/wal-g/wal-g/pkg/storages/azure.(*Folder).ReadObject
        /home/runner/work/wal-g/wal-g/pkg/storages/azure/folder.go:365
github.com/wal-g/wal-g/internal.TryDownloadFile
        /home/runner/work/wal-g/wal-g/internal/fetch_helper.go:61
github.com/wal-g/wal-g/internal.findDecompressorAndDownload
        /home/runner/work/wal-g/wal-g/internal/fetch_helper.go:190
github.com/wal-g/wal-g/internal.DownloadAndDecompressStorageFile
        /home/runner/work/wal-g/wal-g/internal/fetch_helper.go:171
github.com/wal-g/wal-g/internal/databases/postgres.getHistoryFileFromStorage
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/timeline_history_record.go:127
github.com/wal-g/wal-g/internal/databases/postgres.GetTimeLineHistoryRecords
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/timeline_history_record.go:88
github.com/wal-g/wal-g/internal/databases/postgres.createTimelineSwitchMap
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/timeline_history_record.go:71
github.com/wal-g/wal-g/internal/databases/postgres.NewIntegrityCheckRunner
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/integrity_check_runner.go:56
github.com/wal-g/wal-g/internal/databases/postgres.BuildWalVerifyCheckRunner
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/wal_verify_handler.go:120
github.com/wal-g/wal-g/internal/databases/postgres.HandleWalVerify
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/wal_verify_handler.go:147
github.com/wal-g/wal-g/cmd/pg.glob..func20
        /home/runner/work/wal-g/wal-g/cmd/pg/wal_verify.go:49
github.com/spf13/cobra.(*Command).execute
        /home/runner/work/wal-g/wal-g/vendor/github.com/spf13/cobra/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
        /home/runner/work/wal-g/wal-g/vendor/github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
        /home/runner/work/wal-g/wal-g/vendor/github.com/spf13/cobra/command.go:902
github.com/wal-g/wal-g/cmd/pg.Execute
        /home/runner/work/wal-g/wal-g/cmd/pg/pg.go:45
main.main
        /home/runner/work/wal-g/wal-g/main/pg/main.go:8
runtime.main
        /opt/hostedtoolcache/go/1.17.10/x64/src/runtime/proc.go:255
runtime.goexit
        /opt/hostedtoolcache/go/1.17.10/x64/src/runtime/asm_amd64.s:1581
Azure error : Unable to download blob wal_005/00000042.history.lz4.
github.com/wal-g/wal-g/pkg/storages/storage.NewError
        /home/runner/work/wal-g/wal-g/pkg/storages/storage/errors.go:27
github.com/wal-g/wal-g/pkg/storages/azure.NewFolderError
        /home/runner/work/wal-g/wal-g/pkg/storages/azure/folder.go:52
github.com/wal-g/wal-g/pkg/storages/azure.(*Folder).ReadObject
        /home/runner/work/wal-g/wal-g/pkg/storages/azure/folder.go:365
github.com/wal-g/wal-g/internal.TryDownloadFile
        /home/runner/work/wal-g/wal-g/internal/fetch_helper.go:61
github.com/wal-g/wal-g/internal.findDecompressorAndDownload
        /home/runner/work/wal-g/wal-g/internal/fetch_helper.go:190
github.com/wal-g/wal-g/internal.DownloadAndDecompressStorageFile
        /home/runner/work/wal-g/wal-g/internal/fetch_helper.go:171
github.com/wal-g/wal-g/internal/databases/postgres.getHistoryFileFromStorage
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/timeline_history_record.go:127
github.com/wal-g/wal-g/internal/databases/postgres.GetTimeLineHistoryRecords
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/timeline_history_record.go:88
github.com/wal-g/wal-g/internal/databases/postgres.createTimelineSwitchMap
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/timeline_history_record.go:71
github.com/wal-g/wal-g/internal/databases/postgres.NewIntegrityCheckRunner
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/integrity_check_runner.go:56
github.com/wal-g/wal-g/internal/databases/postgres.BuildWalVerifyCheckRunner
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/wal_verify_handler.go:120
github.com/wal-g/wal-g/internal/databases/postgres.HandleWalVerify
        /home/runner/work/wal-g/wal-g/internal/databases/postgres/wal_verify_handler.go:147
github.com/wal-g/wal-g/cmd/pg.glob..func20
        /home/runner/work/wal-g/wal-g/cmd/pg/wal_verify.go:49
github.com/spf13/cobra.(*Command).execute
        /home/runner/work/wal-g/wal-g/vendor/github.com/spf13/cobra/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
        /home/runner/work/wal-g/wal-g/vendor/github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
        /home/runner/work/wal-g/wal-g/vendor/github.com/spf13/cobra/command.go:902
github.com/wal-g/wal-g/cmd/pg.Execute
        /home/runner/work/wal-g/wal-g/cmd/pg/pg.go:45
main.main
        /home/runner/work/wal-g/wal-g/main/pg/main.go:8
runtime.main
        /opt/hostedtoolcache/go/1.17.10/x64/src/runtime/proc.go:255
runtime.goexit
        /opt/hostedtoolcache/go/1.17.10/x64/src/runtime/asm_amd64.s:1581)
Traceback (most recent call last):
  File "wal-archiving-exporter.py", line 70, in 
    wal_backup_status = wal_verify()
  File "wal-archiving-exporter.py", line 30, in wal_verify
    '--json'
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/local/bin/wal-g', 'wal-verify', 'integrity', 'timeline', '--json']' returned non-zero exit status 1.
@x4m
Copy link
Collaborator

x4m commented Aug 18, 2022

Hi! Thanks for reporting this. I think @serprex is our top specialist in Azure. Philip, can you please take a look?

@serprex
Copy link
Contributor

serprex commented Aug 18, 2022

@Mahagon have you tried with the latest master? I expect #1282 would fix this

@Mahagon
Copy link
Author

Mahagon commented Aug 19, 2022

Yes this seems to work
compiled the main branch and it runs without an error 👍

@x4m
Copy link
Collaborator

x4m commented Aug 19, 2022

So, do we need a pre-release for WAL-G+Azure users?

@serprex
Copy link
Contributor

serprex commented Aug 19, 2022

@x4m What's blocking rolling a 2.1 release?

@x4m
Copy link
Collaborator

x4m commented Aug 20, 2022 via email

@serprex
Copy link
Contributor

serprex commented Aug 21, 2022

There's at least support for using managed identity with Azure backend

Looking at changes made since May 20th, a few bug fixes like #1292

@siegenthalerroger
Copy link

Is there reason that a 2.0.1 patch release isn't an option? This is fairly major imo.

@usernamedt
Copy link
Member

There's at least support for using managed identity with Azure backend

Looking at changes made since May 20th, a few bug fixes like #1292

Yes, there are also multiple improvements for the Greenplum backups. I guess we can cut a 2.0.1 release.

@x4m what do you think?

@x4m
Copy link
Collaborator

x4m commented Aug 22, 2022 via email

@usernamedt
Copy link
Member

usernamedt commented Aug 25, 2022

TWIMC, the WAL-G v2.0.1 is now available on the Releases page

@serprex serprex closed this as completed Aug 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants