Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[S3] API returns 206 instead of 500 if chunk not found on volume for partial requests #1746

Closed
kmlebedev opened this issue Jan 14, 2021 · 1 comment

Comments

@kmlebedev
Copy link
Contributor

kmlebedev commented Jan 14, 2021

Describe the bug
S3 API returns 206 instead of 500 if chunk not found on volume for partial requests

System Setup
2.20 from master

Expected behavior
S3 API returns 500 like not partial requests

Additional context

local-k8s-compose.yml:

version: '2'

services:
  master:
    image: chrislusf/seaweedfs:local
    ports:
      - 9333:9333
      - 19333:19333
    command: "master -ip=master"
  volume:
    image: chrislusf/seaweedfs:local
    ports:
      - 8080:8080
      - 18080:18080
    command: "volume -mserver=master:9333 -port=8080 -ip=volume"
    depends_on:
      - master
  mysql:
    image: percona/percona-server:5.7
    ports:
      - 3306:3306
    volumes:
      - ./seaweedfs.sql:/docker-entrypoint-initdb.d/seaweedfs.sql
    environment:
      - MYSQL_ROOT_PASSWORD=secret
      - MYSQL_DATABASE=seaweedfs
      - MYSQL_PASSWORD=secret
      - MYSQL_USER=seaweedfs
  filer:
    image: chrislusf/seaweedfs:local
    ports:
      - 8888:8888
      - 18888:18888
    environment:
      - WEED_MYSQL_HOSTNAME=mysql
      - WEED_MYSQL_PORT=3306
      - WEED_MYSQL_DATABASE=seaweedfs
      - WEED_MYSQL_USERNAME=seaweedfs
      - WEED_MYSQL_PASSWORD=secret
      - WEED_MYSQL_ENABLED=true
      - WEED_LEVELDB2_ENABLED=false
    command: '-v 9 filer -master="master:9333"'
    depends_on:
      - master
      - volume
      - mysql
  ingress:
    image: jwilder/nginx-proxy:alpine
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - ./nginx/proxy.conf:/etc/nginx/proxy.conf
  s3:
    image: chrislusf/seaweedfs:local
    ports:
      - 8333:8333
    command: '-v 9 s3 -filer="filer:8888"'
    depends_on:
      - master
      - volume
      - filer
    environment:
      - VIRTUAL_HOST=ingress
      - VIRTUAL_PORT=8333
  registry:
    image: registry:2
    environment:
      REGISTRY_HTTP_ADDR: "0.0.0.0:5001" # seaweedfs s3
      REGISTRY_LOG_LEVEL: "debug"
      REGISTRY_STORAGE: "s3"
      REGISTRY_STORAGE_S3_REGION: "us-east-1"
      REGISTRY_STORAGE_S3_REGIONENDPOINT: "http://ingress"
      REGISTRY_STORAGE_S3_BUCKET: "registry"
      REGISTRY_STORAGE_S3_ACCESSKEY: "some_access_key1"
      REGISTRY_STORAGE_S3_SECRETKEY: "some_secret_key1"
      REGISTRY_STORAGE_S3_V4AUTH: "true"
      REGISTRY_STORAGE_S3_SECURE: "false"
      REGISTRY_STORAGE_S3_SKIPVERIFY: "true"
      REGISTRY_STORAGE_S3_ROOTDIRECTORY: "/"
      REGISTRY_STORAGE_DELETE_ENABLED: "true"
      REGISTRY_STORAGE_REDIRECT_DISABLE: "true"
      REGISTRY_VALIDATION_DISABLED: "true"
    ports:
      - 5001:5001
    depends_on:
      - s3
      - ingress
  1. docker-compose -f local-k8s-compose.yml -p seaweedfs up

  2. echo "127.0.0.1 ingress" >> /etc/hosts

  3. aws --profile local --endpoint http://ingress/ s3 mb s3://registry2

  4. docker tag chrislusf/seaweedfs:local localhost:5001/chrislusf/seaweedfs:local && docker push localhost:5001/chrislusf/seaweedfs:local

  5. docker exec -it seaweedfs_volume_1 sh
    > for f in $(ls /data/registry_*.dat);do echo > $f;done

  6. docker pull

docker pull localhost:5001/chrislusf/seaweedfs:local
Error response from daemon: received unexpected HTTP status: 500 Internal Server Error

nginx log
ingress_1 | nginx.1 | ingress 172.19.0.8 - - [14/Jan/2021:10:10:25 +0000] "GET /registry/docker/registry/v2/repositories/chrislusf/seaweedfs/_manifests/tags/local/current/link HTTP/1.1" 206 0 "-" "aws-sdk-go/1.15.11 (go1.11.2; linux; amd64)"
7. s3 cp

download failed: s3://registry/docker/registry/v2/repositories/chrislusf/seaweedfs/_manifests/tags/local/current/link to ../../tmp/link An error occurred (500) when calling the GetObject operation (reached max retries: 0): Internal Server Error

nginx log

ingress_1   | nginx.1    | ingress 172.19.0.1 - - [14/Jan/2021:10:13:06 +0000] "GET /registry/docker/registry/v2/repositories/chrislusf/seaweedfs/_manifests/tags/local/current/link HTTP/1.1" 500 0 "-" "aws-cli/2.1.10 Python/3.9.1 Darwin/20.2.0 source/x86_64 prompt/off command/s3.cp"
	HTTP/1.1 206 Partial Content
	Server: nginx/1.19.3
	Date: Thu, 14 Jan 2021 07:58:16 GMT
	Content-Length: 71
	Connection: keep-alive
	Accept-Ranges: bytes
	Access-Control-Expose-Headers: Content-Disposition
	Content-Disposition: inline; filename="link"
	Content-Range: bytes 0-70/71
	Etag: "3cf4380cf06df2d2cba1fcc7a8992a23"
	Last-Modified: Thu, 14 Jan 2021 07:52:38 GMT
	X-Amz-Storage-Class: STANDARD
@kmlebedev kmlebedev changed the title S3 API returns 206 instead of 500 if chunk not found on volume for partial requests [S3] API returns 206 instead of 500 if chunk not found on volume for partial requests Jan 14, 2021
@chrislusf
Copy link
Collaborator

Thanks for reporting! Added a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants