Files S3 - JSON file type - zero bytes response

[emmems]

Hi! 👋

I have pocketbase connected to S3 (local works flawesly) via R2 CloudFlare.

The situation is:

1. uploading file to database with extension json/gltf (application/json)
2. then I have link to it: http://localhost:8090/api/files/tdf4vk1tlytuzua/t3t2t8whz4fp41u/my_file_gHctbnx2m7.json
3. I'm doing this request:

GET http://localhost:8090/api/files/tdf4vk1tlytuzua/t3t2t8whz4fp41u/my_file_gHctbnx2m7.json

4. That is my response:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Cache-Control: max-age=2592000, stale-while-revalidate=86400
Content-Disposition: attachment; filename=my_file_gHctbnx2m7.json
Content-Length: 0
Content-Security-Policy: default-src 'none'; media-src 'self'; style-src 'unsafe-inline'; sandbox
Content-Type: application/json
Last-Modified: Wed, 15 Mar 2023 08:49:29 GMT
Vary: Origin
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Date: Wed, 15 Mar 2023 08:55:23 GMT

<Response body is empty>

Additionally:
👍 Images are working just fine
it doesn't matter if it is localhost or deployed on server.

GUI logs are returning 200.

My setup:

• Go 1.20

github.com/pocketbase/dbx v1.10.0
github.com/pocketbase/pocketbase v0.13.4

If someone knows why it might not work (who is/was working on it) then let me know and I can fix it by myself 😄

[ganigeorgiev]

I'm not able to reproduce the issue and json files are uploaded and downloaded for me.

How did you upload the file?

Are you sure that your S3 settings are correct?

Does the file exist in the S3 storage?

Could you provide a minimal reproducible repo?

[emmems]

Wow I'm impressed how fast you replied man, wow really appreciate it! Thank you :)

• How did you upload the file?
  Just clicking "Upload new file", choosing file, "Save changes".

• Are you sure that your S3 settings are correct?
  Green dot is there, images are working just fine so I assume it is okay

• Does the file exist in the S3 storage?
  Yes, one of screenshots below shows that :)

• Could you provide a minimal reproducible repo?
  Yes, sure :) Here you are:

func main(){
	app := pocketbase.NewWithConfig(&pocketbase.Config{
		DefaultDebug:         false,
		DefaultDataDir:       "pb_data",
		DefaultEncryptionEnv: "./pb_encrypted/",
		HideStartBanner:      false,
		DataMaxOpenConns:     0,
		DataMaxIdleConns:     0,
		LogsMaxOpenConns:     0,
		LogsMaxIdleConns:     0,
	})

	if err := app.Start(); err != nil {
		log.Fatalln("Failed to start pocketbase:", err)
	}
}

2. run go run main.go serve
   

3. create account

4. go to file storage and fill R2 Cloudflare data.
   

5. create simple table -> upload .json file.
   
   

That's it.

S3 screenshot:

• File downloaded directly from S3 is fine.

This is file returned by pocketbase:

Weird situation I guess.

[ganigeorgiev]

Hm, it could be something specific to Cloudflare R2.

I've tested it once again with the Vultr and Wasabi S3 offerings and it is working fine for me.

Your example config looks OK, I'm just not sure what ./pb_encrypted/ is in the above (also note that disabling the db pool limits could lead to excessive memory usage and "Too many open files" errors since with every db request it will open a new db connection).

Is "Force path-style addrressing" necessary for Cloudflare R2?

Is the empty file content reproducible only with JSON files?

Is the error reproducible with the prebuilt executable?

[emmems]

Hi!

I'm sorry but I didn't have time to check that further but now I'm back!

I will firstly answer to questions:

• Force path-style addrressing <- works same with/without this setting.
• Is the empty file content reproducible only with JSON files? JSON/SVG as I discovered
• Is the error reproducible with the prebuilt executable? - yes, 0.14.0 as well.

To be honest I think this is not related to pocketbase itself but to library "gocloud.dev/blob/s3blob".

Why? When I checked amazon s3 it works, linode works fine as well. I have checked nodejs s3 provider on cloudflare and it works fine as well.

The issue is here:
/tools/filesystem/filesystem.go

func (s *System) Serve(res http.ResponseWriter, req *http.Request, fileKey string, name string) error {
	br, readErr := s.bucket.NewReader(s.ctx, fileKey, nil)

where br.Size() returns somehow 0 size :/ if this is Cloudflare R2 and json/svg files.

[ganigeorgiev]

I'm not sure how to reproduce it.

When I checked amazon s3 it works, linode works fine as well. I have checked nodejs s3 provider on cloudflare and it works fine as well.

I'm confused. Do you mean that AWS S3 and Linode works with PocketBase, but only R2 Cloudflare has the zero bytes issue? If that's the case, then maybe there is something with the Cloudflare S3 compatibility API that is causing it to fail.

[emmems]

Ohh it means:

linode S3, AWS S3- nodejs aws-s3 lib, pocketbase works ✅

cloudflare r2 - nodejs aws-s3 lib works ✅

cloudflare r2 - pocketbase doesn't work ✅ :/ it returns zero bytes.

So I'm assuming R2 has some issue with their compatibility API :/

[ganigeorgiev]

Just for info in case someone stumble here - I've created a R2 account and was able to reproduce the above issue with the aws-sdk-go pacakge. Upgrading to aws-sdk-go-v2 seems to resolve it (currently the changes are only in the develop branch).