Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

s3fs-fuse with ecryptfs? #166

Closed
stevenshiau opened this issue Apr 10, 2015 · 26 comments
Closed

s3fs-fuse with ecryptfs? #166

stevenshiau opened this issue Apr 10, 2015 · 26 comments

Comments

@stevenshiau
Copy link

Dear maintainer,
I encountered an issue when using ecryptfs with s3fs. Here is how the issue could be reproduced. First put the password file in /tmp/p.txt, and its content is like:

passphrase_passwd=mypasswd

The script "s3fs-ecryptfs.sh" for all the steps:

#!/bin/bash
s3fs steven-ocs /home/partimag/
rm -rf /home/partimag/test-s3 /tmp/ecryptfs_mnt
mkdir /home/partimag/test-s3 /tmp/ecryptfs_mnt
mount -t ecryptfs /home/partimag/test-s3 /tmp/ecryptfs_mnt -o ecryptfs_cipher=aes,ecryptfs_key_bytes=16,no_sig_cache,ecryptfs_enable_filename_crypto=n,ecryptfs_passthrough,key=passphrase:passphrase_passwd_file=/tmp/p.txt
echo "It is OK" > /tmp/ecryptfs_mnt/ok.txt
md5sum /tmp/ecryptfs_mnt/ok.txt 
umount /tmp/ecryptfs_mnt
mount -t ecryptfs /home/partimag/test-s3 /tmp/ecryptfs_mnt -o ecryptfs_cipher=aes,ecryptfs_key_bytes=16,no_sig_cache,ecryptfs_enable_filename_crypto=n,ecryptfs_passthrough,key=passphrase:passphrase_passwd_file=/tmp/p.txt
md5sum /tmp/ecryptfs_mnt/ok.txt 
umount /tmp/ecryptfs_mnt
md5sum /home/partimag/test-s3/ok.txt 
umount /home/partimag/

The results for the above script:

root@vivid:~# ./s3fs-ecryptfs.sh 
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
f313be4f2ce0aa98c9412964e7f0b86e  /tmp/ecryptfs_mnt/ok.txt
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
853a955ca26b15106dc3a32e4098ebaf  /tmp/ecryptfs_mnt/ok.txt
853a955ca26b15106dc3a32e4098ebaf  /home/partimag/test-s3/ok.txt

As you can see, the md5sum of "/tmp/ecryptfs_mnt/ok.txt" in the 2nd mounting is different from the 1st one, and it's the same with encrypted file "/home/partimag/test-s3/ok.txt". Therefore you can see ecryptfs failed in decryption.
On the other hand, I tried the nearly same steps, and the only difference is to mount a local partition instead of S3 resource. Here the script "local-ecryptfs.sh" is:

#!/bin/bash
mount /dev/sdc1 /home/partimag/
rm -rf /home/partimag/test-s3 /tmp/ecryptfs_mnt
mkdir /home/partimag/test-s3 /tmp/ecryptfs_mnt
mount -t ecryptfs /home/partimag/test-s3 /tmp/ecryptfs_mnt -o ecryptfs_cipher=aes,ecryptfs_key_bytes=16,no_sig_cache,ecryptfs_enable_filename_crypto=n,ecryptfs_passthrough,key=passphrase:passphrase_passwd_file=/tmp/p.txt
echo "It is OK" > /tmp/ecryptfs_mnt/ok.txt
md5sum /tmp/ecryptfs_mnt/ok.txt 
umount /tmp/ecryptfs_mnt
mount -t ecryptfs /home/partimag/test-s3 /tmp/ecryptfs_mnt -o ecryptfs_cipher=aes,ecryptfs_key_bytes=16,no_sig_cache,ecryptfs_enable_filename_crypto=n,ecryptfs_passthrough,key=passphrase:passphrase_passwd_file=/tmp/p.txt
md5sum /tmp/ecryptfs_mnt/ok.txt 
umount /tmp/ecryptfs_mnt
md5sum /home/partimag/test-s3/ok.txt 
umount /home/partimag/

As you can see, the only difference is in the 1st line.
Now the results for the script "local-ecryptfs.sh":

root@vivid:~# ./local-ecryptfs.sh 
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
f313be4f2ce0aa98c9412964e7f0b86e  /tmp/ecryptfs_mnt/ok.txt
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
f313be4f2ce0aa98c9412964e7f0b86e  /tmp/ecryptfs_mnt/ok.txt
81543ce01876e7e2fdc17228e731a4ba  /home/partimag/test-s3/ok.txt

As you can see, the md5sum of "/tmp/ecryptfs_mnt/ok.txt" in the 2nd mounting is the same with the 1st one. This makes sense. Therefore you can see ecryptfs worked in decryption.
Any thought? Did I miss something? Or by nature the 2 file systems conflict with each other?
Thank you very much.

Steven.
stevenshiau added a commit to stevenshiau/drbl that referenced this issue Apr 11, 2015
@stevenshiau
Avoid AWS S3 with ecryptfs.
ad9f99b 
A mechanism to avoid AWS S3 with ecryptfs was added because there is
an issue: s3fs-fuse/s3fs-fuse#166
stevenshiau added a commit to stevenshiau/drbl that referenced this issue Apr 16, 2015
@stevenshiau
drbl-functions: avoiding cloudfuse with ecryptfs.
2414c06 
A mechanism to avoid cloudfuse with ecryptfs was added because there is
an similar issue as this s3fs-fuse/s3fs-fuse#166
stevenshiau added a commit to stevenshiau/clonezilla that referenced this issue Apr 16, 2015
@stevenshiau
ocs-functions: avoiding cloudfuse with ecryptfs.
d8379ee 
A mechanism to avoid cloudfuse with ecryptfs was added because there
is an similar issue as s3fs-fuse/s3fs-fuse#166
@ggtakec
Copy link
Member

ggtakec commented Apr 18, 2015

I have not been geting this reason yet.
I tried mounting s3 my bucket on the directory which is mounted by s3fs, but s3fs could get same file.(please see fallowing example.)

example for my test:

  1. /mnt/s3 is mounted by s3fs from my bucket.
  2. /mnt/s3/test directory exists.
  3. /mnt/s3/test is mounted by another s3fs process from same bucket.
  4. makes file in /mnt/s3/test/mytestfile.(ex. "echo TEST > /mnt/s3/test/mytestfile")
  5. compared /mnt/s3/test/mytestfile and /mnt/s3/mytestfile, BUT the result is same file.

I think it seems that nested mounting by s3fs does with no problem.
So this problem has occurred in the case of using eCryptfs.
If you can, please s3fs run with "-d" or "-f" option, it puts s3fs's debugging messages.
We need more informaion to solve this issue.

Thanks in advance for your assistance.

@stevenshiau
Copy link
Author

Thanks.
I ran s3fs with "-d -f", and the output:

root@vivid:~# s3fs -f -d steven-ocs /home/partimag/
    set_moutpoint_attribute(3530): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
s3fs_init(2713): init
s3fs_check_service(3070): check services.
    CheckBucket(2538): check a bucket.
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/
    RequestPerform(1600): HTTP response code 400
    RequestPerform(1614): HTTP response code 400 was returned, returing EIO.
CheckBucket(2590): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-northeast-1'</Message><Region>ap-northeast-1</Region><RequestId>66F18F33B85BA5AB</RequestId><HostId>vzEIq1yncMkQO6jwdZlpC1QVrYTzfFyLBIP+5A2WG6C0p67Pv0uuuFcDYjGpfxFkFa/19YVloxs=</HostId></Error>
s3fs_check_service(3103): Could not connect wrong region us-east-1, so retry to connect region ap-northeast-1.
    CheckBucket(2538): check a bucket.
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/
    RequestPerform(1600): HTTP response code 200
s3fs_getattr(722): [path=/test-s3]
    HeadRequest(2124): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355367][hit count=0]
s3fs_opendir(2128): [path=/test-s3][flags=231424]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355367][hit count=1]
s3fs_readdir(2276): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355367][hit count=2]
  list_bucket(2321): [path=/test-s3]
    ListBucketRequest(2597): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1000&prefix=test-s3/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
  readdir_multi_head(2197): [path=/test-s3/][list=0]
    Request(3639): [count=1]
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355368][hit count=0]
s3fs_opendir(2128): [path=/test-s3][flags=231424]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355367][hit count=3]
s3fs_getattr(722): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355368][hit count=4]
s3fs_readdir(2276): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355368][hit count=5]
  list_bucket(2321): [path=/test-s3]
    ListBucketRequest(2597): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1000&prefix=test-s3/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
  readdir_multi_head(2197): [path=/test-s3/][list=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355368][hit count=1]
    Request(3639): [count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355368][hit count=2]
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355368][hit count=6]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355368][hit count=3]
s3fs_unlink(931): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355368][hit count=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355368][hit count=8]
    DeleteRequest(1922): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 204
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(722): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355368][hit count=9]
s3fs_rmdir(966): [path=/test-s3]
  list_bucket(2321): [path=/test-s3]
    ListBucketRequest(2597): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
    DeleteRequest(1922): [tpath=/test-s3/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/
    RequestPerform(1600): HTTP response code 204
    DelStat(370): delete stat cache entry[path=/test-s3/]
    HeadRequest(2124): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3_$folder$]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
s3fs_getattr(722): [path=/test-s3]
    HeadRequest(2124): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3_$folder$]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2321): [path=/test-s3]
    ListBucketRequest(2597): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
s3fs_mkdir(903): [path=/test-s3][mode=0755]
    HeadRequest(2124): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3_$folder$]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2321): [path=/test-s3]
    ListBucketRequest(2597): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
  create_directory_object(877): [path=/test-s3][mode=0755][time=1429355370][uid=0][gid=0]
    PutRequest(2291): [tpath=/test-s3/]
    PutRequest(2305): create zero byte file object.
    PutRequest(2415): uploading... [path=/test-s3/][fd=-1][size=0]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/
    RequestPerform(1600): HTTP response code 200
    DelStat(370): delete stat cache entry[path=/test-s3]
s3fs_getattr(722): [path=/test-s3]
    HeadRequest(2124): [tpath=/test-s3]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355370][hit count=0]
s3fs_getattr(722): [path=/]
s3fs_access(2776): [path=/test-s3][mask=W_OK ]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355370][hit count=1]
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355370][hit count=2]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/ok.txt/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt/
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/ok.txt_$folder$]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt_%24folder%24
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2321): [path=/test-s3/ok.txt]
    ListBucketRequest(2597): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/ok.txt/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
s3fs_create(841): [path=/test-s3/ok.txt][mode=100644][flags=193]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355370][hit count=3]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/ok.txt/]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt/
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2124): [tpath=/test-s3/ok.txt_$folder$]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt_%24folder%24
    RequestPerform(1600): HTTP response code 404
    RequestPerform(1624): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2321): [path=/test-s3/ok.txt]
    ListBucketRequest(2597): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/ok.txt/
    RequestPerform(1600): HTTP response code 200
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2417): contents_xp->nodesetval is empty.
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355371][hit count=4]
    create_file_object(800): [path=/test-s3/ok.txt][mode=100644]
    PutRequest(2291): [tpath=/test-s3/ok.txt]
    PutRequest(2305): create zero byte file object.
    PutRequest(2415): uploading... [path=/test-s3/ok.txt][fd=-1][size=0]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355371][hit count=5]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355371][hit count=0]
s3fs_release(2090): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_open(1912): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355371][hit count=6]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355371][hit count=0]
s3fs_open(1912): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355371][hit count=7]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=0]
s3fs_release(2090): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_flush(2018): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355371][hit count=8]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=1]
s3fs_flush(2018): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=9]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=3]
    PutRequest(2291): [tpath=/test-s3/ok.txt]
    PutRequest(2415): uploading... [path=/test-s3/ok.txt][fd=7][size=12288]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
s3fs_release(2090): [path=/test-s3/ok.txt][fd=7]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_open(1912): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=10]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=0]
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=11]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=1]
s3fs_flush(2018): [path=/test-s3/ok.txt][fd=6]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=12]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=3]
s3fs_release(2090): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(722): [path=/]
s3fs_getattr(722): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=13]
s3fs_access(2776): [path=/test-s3][mask=W_OK ]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=14]
s3fs_open(1912): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=15]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=0]
    GetObjectRequest(2513): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=1]
    PreGetObjectRequest(2429): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetObjectRequest(2528): downloading... [path=/test-s3/ok.txt][fd=6]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 206
s3fs_open(1912): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=16]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=0]
    GetObjectRequest(2513): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355372][hit count=1]
    PreGetObjectRequest(2429): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetObjectRequest(2528): downloading... [path=/test-s3/ok.txt][fd=7]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 206
s3fs_release(2090): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=17]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=0]
s3fs_flush(2018): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355372][hit count=18]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=1]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=2]
s3fs_release(2090): [path=/test-s3/ok.txt][fd=7]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355373][hit count=19]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=0]
s3fs_open(1912): [path=/test-s3/ok.txt][flags=32768]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355373][hit count=20]
    HeadRequest(2124): [tpath=/test-s3/ok.txt]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=0]
    GetObjectRequest(2513): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=1]
    PreGetObjectRequest(2429): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetObjectRequest(2528): downloading... [path=/test-s3/ok.txt][fd=6]
    RequestPerform(1584): connecting to URL http://steven-ocs.s3.amazonaws.com/test-s3/ok.txt
    RequestPerform(1600): HTTP response code 206
s3fs_getattr(722): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355373][hit count=21]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=2]
s3fs_flush(2018): [path=/test-s3/ok.txt][fd=6]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429355373][hit count=22]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=3]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429355373][hit count=4]
s3fs_release(2090): [path=/test-s3/ok.txt][fd=6]
s3fs_destroy(2756): destroy

@ggtakec
Copy link
Member

ggtakec commented Apr 18, 2015

Thanks for quickly replying.
I saw your s3fs logs, but any error is not in it.(some error in logs are by getting file/directory attributes for finding it, then those are normally message on s3fs)
When my test results, the error in the nest of s3fs (FUSE) did not occur, and the error did not occur in the case when you do not use a s3fs.

I think that In order to find the cause we must look the other way.
I try to find how to test for finding the reason, please you try to think about it.

Thanks in advance for your help.

@stevenshiau
Copy link
Author

Hi Takeshi,
Thanks. For the tests recently here, I found if I enabled the option "kernel_cache" (s3fs steven-ocs /home/partimag/ -o kernel_cache), it works. However, it only works for small files (~ bytes). When I really put big files (>MB), the "kernel_cache" option does not help.
If I enable the cache dir, yes, it works. i.e.
s3fs steven-ocs /home/partimag/ -o use_cache=/tmp
However, this will have another issue because the /tmp/ will be full eventually and the files uploading eventually fails.

Steven.

@ggtakec
Copy link
Member

ggtakec commented Apr 18, 2015

@stevenshiau I'm sorry to know no detail about eCryptfs.
Probably the cause of this issue is because s3fs does not support the extended attributes (xattr).
I will try to support it, please wait by supporting. Please see #169

Regards,

@stevenshiau
Copy link
Author

Hi Takeshi,
Got it. Thank you very much.

Steven.

@ggtakec
Copy link
Member

ggtakec commented Apr 20, 2015

@stevenshiau I created new branch "xattr" which supported extended attributes FUSE api.(posted same comment #169)
Please try to use and test xattr branch codes.

Thanks in advance for your assistance.

@stevenshiau
Copy link
Author

@ggtakec Thanks. I tested the new branch "xattr", still the results are the same:

Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
f313be4f2ce0aa98c9412964e7f0b86e  /tmp/ecryptfs_mnt/ok.txt
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
61287cbb7f35a2f07d6c3829ca278c3a  /tmp/ecryptfs_mnt/ok.txt
61287cbb7f35a2f07d6c3829ca278c3a  /home/partimag/test-s3/ok.txt

The debug info are:

    set_moutpoint_attribute(3928): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
s3fs_init(3104): init
s3fs_check_service(3461): check services.
    CheckBucket(2525): check a bucket.
    insertV4Headers(1961): computing signature [GET] [/] [] []
    url_to_host(99): url is http://s3.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3.amazonaws.com/
    RequestPerform(1648): HTTP response code 400 was returned, returing EIO.
CheckBucket(2563): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-northeast-1'</Message><Region>ap-northeast-1</Region><RequestId>91BD977E6B98EAF5</RequestId><HostId>P2lmtZn3GcyxpP5OZDAOUrkKq+NO78AAW1ITRmREtsocoYBxEmuqZKcmmhj4KctuCs5w5UaRepk=</HostId></Error>
s3fs_check_service(3494): Could not connect wrong region us-east-1, so retry to connect region ap-northeast-1.
    CheckBucket(2525): check a bucket.
    insertV4Headers(1961): computing signature [GET] [/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/
    RequestPerform(1636): HTTP response code 200
s3fs_getattr(734): [path=/]
s3fs_getattr(734): [path=/test-s3]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619657][hit count=0]
s3fs_opendir(2140): [path=/test-s3][flags=231424]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619657][hit count=1]
s3fs_readdir(2288): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619657][hit count=2]
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1000&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1000&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
  readdir_multi_head(2209): [path=/test-s3/][list=0]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    Request(3518): [count=1]
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619658][hit count=0]
s3fs_opendir(2140): [path=/test-s3][flags=231424]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619657][hit count=3]
s3fs_getattr(734): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619658][hit count=4]
s3fs_readdir(2288): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619658][hit count=5]
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1000&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1000&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
  readdir_multi_head(2209): [path=/test-s3/][list=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619658][hit count=1]
    Request(3518): [count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619658][hit count=2]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619658][hit count=6]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619658][hit count=3]
s3fs_unlink(943): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619658][hit count=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619658][hit count=8]
    DeleteRequest(1983): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [DELETE] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 204
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_rmdir(978): [path=/test-s3]
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    DeleteRequest(1983): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [DELETE] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 204
    DelStat(370): delete stat cache entry[path=/test-s3/]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
s3fs_getattr(734): [path=/test-s3]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
s3fs_mkdir(915): [path=/test-s3][mode=0755]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
  create_directory_object(889): [path=/test-s3][mode=0755][time=1429619659][uid=0][gid=0]
    PutRequest(2310): [tpath=/test-s3/]
    PutRequest(2324): create zero byte file object.
    insertV4Headers(1961): computing signature [PUT] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    PutRequest(2416): uploading... [path=/test-s3/][fd=-1][size=0]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 200
    DelStat(370): delete stat cache entry[path=/test-s3]
s3fs_getattr(734): [path=/test-s3]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619660][hit count=0]
s3fs_getattr(734): [path=/]
s3fs_access(3167): [path=/test-s3][mask=W_OK ]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619660][hit count=1]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619660][hit count=2]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3/ok.txt]
    ListBucketRequest(2570): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/ok.txt/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
s3fs_create(853): [path=/test-s3/ok.txt][mode=100644][flags=193]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619660][hit count=3]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3/ok.txt]
    ListBucketRequest(2570): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/ok.txt/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619660][hit count=4]
    create_file_object(812): [path=/test-s3/ok.txt][mode=100644]
    PutRequest(2310): [tpath=/test-s3/ok.txt]
    PutRequest(2324): create zero byte file object.
    insertV4Headers(1961): computing signature [PUT] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    PutRequest(2416): uploading... [path=/test-s3/ok.txt][fd=-1][size=0]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619660][hit count=5]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=0]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=6]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=0]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=7]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=1]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=2]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=8]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=0]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=9]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=1]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=10]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=3]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=11]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=4]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=5]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=12]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=6]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=7]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=13]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=8]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=9]
    PutRequest(2310): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [PUT] [/test-s3/ok.txt] [] [f82dc4d6303f0bc588414b51f35018e9eb2ed45031bbdfb5a704cf4b6e67f260]
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    PutRequest(2416): uploading... [path=/test-s3/ok.txt][fd=7][size=12288]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
s3fs_release(2102): [path=/test-s3/ok.txt][fd=7]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=14]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=0]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=15]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=1]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=6]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=16]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619661][hit count=3]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/]
s3fs_getattr(734): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=17]
s3fs_access(3167): [path=/test-s3][mask=W_OK ]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=18]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=19]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=0]
    GetObjectRequest(2500): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=1]
    PreGetObjectRequest(2430): [tpath=/test-s3/ok.txt][start=0][size=12288]
    insertV4Headers(1961): computing signature [GET] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    GetObjectRequest(2515): downloading... [path=/test-s3/ok.txt][fd=6]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 206
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=user.ecryptfs][value=0xb34004e0][size=16]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619661][hit count=20]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=3]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=21]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=0]
    GetObjectRequest(2500): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=1]
    PreGetObjectRequest(2430): [tpath=/test-s3/ok.txt][start=0][size=12288]
    insertV4Headers(1961): computing signature [GET] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    GetObjectRequest(2515): downloading... [path=/test-s3/ok.txt][fd=7]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 206
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=user.ecryptfs][value=0xb34004e0][size=4096]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=22]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=3]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=23]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=4]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=24]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=1]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=7]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=25]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=0]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32768]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=26]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=0]
    GetObjectRequest(2500): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=1]
    PreGetObjectRequest(2430): [tpath=/test-s3/ok.txt][start=0][size=12288]
    insertV4Headers(1961): computing signature [GET] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    GetObjectRequest(2515): downloading... [path=/test-s3/ok.txt][fd=6]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 206
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=27]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=2]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=6]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429619662][hit count=28]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=3]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429619662][hit count=4]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
s3fs_destroy(3147): destroy

Anything else I need to test?
Thanks for all your efforts.

Steven.

@ggtakec
Copy link
Member

ggtakec commented Apr 21, 2015

Hi, Steven

Thanks for replying and testing quickly.
I updated xattr branch codes, please try and check it.
And please see my comment on #169 as same as this issue.

Regards,

@stevenshiau
Copy link
Author

@ggtakec
Thanks for your updates. I have tested it again. The results are the same.

Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
f313be4f2ce0aa98c9412964e7f0b86e  /tmp/ecryptfs_mnt/ok.txt
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
c2a77aac8f1d043b5f3de42afc7dd151  /tmp/ecryptfs_mnt/ok.txt
c2a77aac8f1d043b5f3de42afc7dd151  /home/partimag/test-s3/ok.txt

root@debian:~# s3fs -d -f steven-ocs /home/partimag/
    set_moutpoint_attribute(3928): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
s3fs_init(3104): init
s3fs_check_service(3461): check services.
    CheckBucket(2525): check a bucket.
    insertV4Headers(1961): computing signature [GET] [/] [] []
    url_to_host(99): url is http://s3.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3.amazonaws.com/
    RequestPerform(1648): HTTP response code 400 was returned, returing EIO.
CheckBucket(2563): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-northeast-1'</Message><Region>ap-northeast-1</Region><RequestId>62D7351DB27DDC37</RequestId><HostId>C8GYVB43pbEqe86Sgz0NlbXQGuZA6MWBcdm6fH3xAI/JYtvZ4fD3dGVFjzI5IdwB6Dkaa2KurYY=</HostId></Error>
s3fs_check_service(3494): Could not connect wrong region us-east-1, so retry to connect region ap-northeast-1.
    CheckBucket(2525): check a bucket.
    insertV4Headers(1961): computing signature [GET] [/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/
    RequestPerform(1636): HTTP response code 200
s3fs_getattr(734): [path=/test-s3]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=0]
s3fs_opendir(2140): [path=/test-s3][flags=231424]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=1]
s3fs_readdir(2288): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=2]
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1000&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1000&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
  readdir_multi_head(2209): [path=/test-s3/][list=0]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    Request(3518): [count=1]
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668024][hit count=0]
s3fs_opendir(2140): [path=/test-s3][flags=231424]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=3]
s3fs_getattr(734): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=4]
s3fs_readdir(2288): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=5]
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1000&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1000&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
  readdir_multi_head(2209): [path=/test-s3/][list=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668024][hit count=1]
    Request(3518): [count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668024][hit count=2]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=6]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668024][hit count=3]
s3fs_unlink(943): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668024][hit count=8]
    DeleteRequest(1983): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [DELETE] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 204
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_rmdir(978): [path=/test-s3]
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    DeleteRequest(1983): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [DELETE] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 204
    DelStat(370): delete stat cache entry[path=/test-s3/]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
s3fs_getattr(734): [path=/test-s3]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
s3fs_mkdir(915): [path=/test-s3][mode=0755]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3]
    ListBucketRequest(2570): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
  create_directory_object(889): [path=/test-s3][mode=0755][time=1429668026][uid=0][gid=0]
    PutRequest(2310): [tpath=/test-s3/]
    PutRequest(2324): create zero byte file object.
    insertV4Headers(1961): computing signature [PUT] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    PutRequest(2416): uploading... [path=/test-s3/][fd=-1][size=0]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 200
    DelStat(370): delete stat cache entry[path=/test-s3]
s3fs_getattr(734): [path=/test-s3]
    HeadRequest(2157): [tpath=/test-s3]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668026][hit count=0]
s3fs_getattr(734): [path=/]
s3fs_access(3167): [path=/test-s3][mask=W_OK ]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668026][hit count=1]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668026][hit count=2]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3/ok.txt]
    ListBucketRequest(2570): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/ok.txt/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
s3fs_create(853): [path=/test-s3/ok.txt][mode=100644][flags=193]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668026][hit count=3]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt/]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt/
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
    HeadRequest(2157): [tpath=/test-s3/ok.txt_$folder$]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt_$folder$] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt_%24folder%24
    RequestPerform(1658): HTTP response code 404 was returned, returning ENOENT
  list_bucket(2333): [path=/test-s3/ok.txt]
    ListBucketRequest(2570): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [GET] [delimiter=/&max-keys=1&prefix=test-s3/ok.txt/] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com?delimiter=/&max-keys=1&prefix=test-s3/ok.txt/
    RequestPerform(1636): HTTP response code 200
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    append_objects_from_xml_ex(2429): contents_xp->nodesetval is empty.
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668026][hit count=4]
    create_file_object(812): [path=/test-s3/ok.txt][mode=100644]
    PutRequest(2310): [tpath=/test-s3/ok.txt]
    PutRequest(2324): create zero byte file object.
    insertV4Headers(1961): computing signature [PUT] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    PutRequest(2416): uploading... [path=/test-s3/ok.txt][fd=-1][size=0]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=5]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=0]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=6]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=0]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=7]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=1]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=8]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=0]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=9]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=1]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=10]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=2]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=11]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=3]
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=12]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=4]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=13]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=5]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=6]
    PutRequest(2310): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [PUT] [/test-s3/ok.txt] [] [f672343c2c80fb5eb2ed0ddc02275b5bf5220db425b206b57f66242037da5542]
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    PutRequest(2416): uploading... [path=/test-s3/ok.txt][fd=7][size=12288]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
s3fs_release(2102): [path=/test-s3/ok.txt][fd=7]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=14]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=0]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=15]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=1]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=6]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=16]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=2]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668027][hit count=3]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/]
s3fs_getattr(734): [path=/test-s3]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=17]
s3fs_access(3167): [path=/test-s3][mask=W_OK ]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668027][hit count=18]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=19]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=0]
    GetObjectRequest(2500): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=1]
    PreGetObjectRequest(2430): [tpath=/test-s3/ok.txt][start=0][size=12288]
    insertV4Headers(1961): computing signature [GET] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    GetObjectRequest(2515): downloading... [path=/test-s3/ok.txt][fd=6]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 206
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=user.ecryptfs][value=0xb36004e0][size=16]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=20]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=2]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32770]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=21]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=0]
    GetObjectRequest(2500): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=1]
    PreGetObjectRequest(2430): [tpath=/test-s3/ok.txt][start=0][size=12288]
    insertV4Headers(1961): computing signature [GET] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    GetObjectRequest(2515): downloading... [path=/test-s3/ok.txt][fd=7]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 206
s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=user.ecryptfs][value=0xb36004e0][size=4096]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=22]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=2]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=23]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=3]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=7]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=24]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=0]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=1]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=7]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=25]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=0]
s3fs_open(1924): [path=/test-s3/ok.txt][flags=32768]
    DelStat(370): delete stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=26]
    HeadRequest(2157): [tpath=/test-s3/ok.txt]
    insertV4Headers(1961): computing signature [HEAD] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 200
    AddStat(248): add stat cache entry[path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=0]
    GetObjectRequest(2500): [tpath=/test-s3/ok.txt][start=0][size=12288]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=1]
    PreGetObjectRequest(2430): [tpath=/test-s3/ok.txt][start=0][size=12288]
    insertV4Headers(1961): computing signature [GET] [/test-s3/ok.txt] [] []
    url_to_host(99): url is http://s3-ap-northeast-1.amazonaws.com
    GetObjectRequest(2515): downloading... [path=/test-s3/ok.txt][fd=6]
    RequestPerform(1619): connecting to URL http://steven-ocs.s3-ap-northeast-1.amazonaws.com/test-s3/ok.txt
    RequestPerform(1636): HTTP response code 206
s3fs_getattr(734): [path=/test-s3/ok.txt]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=27]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=2]
s3fs_flush(2030): [path=/test-s3/ok.txt][fd=6]
    GetStat(171): stat cache hit [path=/test-s3/][time=1429668028][hit count=28]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=3]
    GetStat(171): stat cache hit [path=/test-s3/ok.txt][time=1429668028][hit count=4]
s3fs_release(2102): [path=/test-s3/ok.txt][fd=6]
s3fs_destroy(3147): destroy

Anything else I can test?
Thanks for your efforts.

Steven.

@delx
Copy link

delx commented Apr 27, 2015

Are you sure that this issue is related to the extended attributes?

@stevenshiau
Copy link
Author

@delx,
No, I have no idea about that. Ggtakec guessed that. Do you have any idea about the issue?
Thanks.

Steven.

@ggtakec
Copy link
Member

ggtakec commented Apr 28, 2015

@delx @stevenshiau
I'm not necessarily sure, but encfs is trying to get the attributes through getxattr(security.capability).

s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=security.capability][value=(nil)][size=0]

I have not found detailed operation of ecryptfs yet, but I think that a cause is that ecryptfs can not be obtained extended attributes.(Who should write this extended attribute?)

For example, #169 (about encfs) seems like this issue, it puts following log:

s3fs_getxattr(2903): [path=/test-s3/ok.txt][name=user.ecryptfs][value=0xb36004e0][size=4096]

I think we need to know how to set these extended attributes for the object which is made by ecryptfs(encfs).
(I don’t know whether this reason is related to the policy settings of the secure OS(SELinux))

I try to study about it, please wait for a while, and if you know about it please let me know.

Thanks in advance for your help.

@ggtakec ggtakec mentioned this issue Apr 28, 2015
Closed
@ggtakec
Copy link
Member

ggtakec commented Apr 28, 2015

Could you run ecryptfs with ecryptfs_xattr option?

@stevenshiau
Copy link
Author

@ggtakec,
Thanks.
When I mount ecryptfs with ecryptfs_xattr option, I got different error:

mount -t ecryptfs /home/partimag/test-s3 /tmp/ecryptfs_mnt -o ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_xattr,no_sig_cache,ecryptfs_enable_filename_crypto=n,ecryptfs_passthrough,key=passphrase:passphrase_passwd_file=/tmp/p.txt
echo "It is OK" > /tmp/ecryptfs_mnt/ok.txt

Then the error is:

# ./s3fs-ecryptfs.sh
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_xattr_metadata
  ecryptfs_passthrough
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=49138d03ae6831ac
Mounted eCryptfs
./s3fs-ecryptfs.sh: line 7: /tmp/ecryptfs_mnt/ok.txt: Operation not supported

Somehow the option "ecryptfs_xattr" prevents the file creation.
Do you have same error there?
Thanks for your efforts. Appreciate.

Steven.

@ggtakec
Copy link
Member

ggtakec commented May 10, 2015

@stevenshiau I'm sorry for replying late.

I tried to do your scenario on ubuntu. I did not get something error about ecryptfs_xattr option yet.
The root of this issue was that md5 is different by the remount.
In the second mount, it is because the file contents through the s3fs + ecryptfs is returned without　decoding it.
However, I did not know why not decode the file.
So that, we were trying to run ecryptfs with ecryptfs_xattr and s3fs is changed to add extended attribute interface.
But I found a bug about these xattr interface in s3fs. ecryptfs calls xattr interface through fuse with binary data for extended attributes, but s3fs xattr branch can not handle it now.
Then I try to fix this bug, please wait a moment.

Thanks in advance for your assistance.

@stevenshiau
Copy link
Author

@ggtakec,
Got it. Thanks for notifying me that. It's nice to know you made some progress.

Steven.

@ggtakec
Copy link
Member

ggtakec commented May 17, 2015

@stevenshiau I’m sorry, still I'm not able to solve this problem.

I fixed that s3fs has been able to receive a binary array as value for extended attribute.
But still this problem which is occurred after remounting is not fixed.

It seems that s3fs could not return the value as "system.posix_acl_default" extended attribute, but this attribute is not set by any calling.
I think this is the cause for this issue.
So I’m examining about POSIX ACL, please wait a while.

Regards,

@stevenshiau
Copy link
Author

@ggtakec No problem. I can fully understand that debugging might take a lot of time. I am a very patient user, and really glad that you keep fixing this issue.
Appreciate!

BR,
Steven.

@ggtakec
Copy link
Member

ggtakec commented Jun 6, 2015

@stevenshiau
I'm sorry for rplying late.

I fixed xattr branch now, it seems that works good.(after remount)
I was misunderstood, "system.posix_acl_default" does not affect about this issue.

I warry one, that the file size which encoded is not correct at first time before reading it.
After reading the file, the file size is correct.

Note the thing about xattr of s3fs.
s3fs can support extended attributes by put/get "x-amz-meta-xattr" header.
This header is formatted following:
x-amz-meta-xattr: url encoded string({"key":"base64(value)","key":"base64(value)",....})

@ggtakec
Copy link
Member

ggtakec commented Jun 9, 2015

At the end of this week, I will try to merge xattr branch to master branch.
If you find a problem, please let me know.

Thanks in advance for your assistance.

@ggtakec ggtakec mentioned this issue Jun 13, 2015
Closed
@stevenshiau
Copy link
Author

@ggtakec,
Great!
Sorry for the late response because I have been out of office for a few days. I should be able to do the test again when I back to office next Tuesday.

Steven.

@ggtakec ggtakec mentioned this issue Jun 20, 2015
Merged
@ggtakec
Copy link
Member

ggtakec commented Jun 20, 2015

#199 merged, master branch supports extended attributes.
I closed this issue, if you find a problem, please reopen this issue or post new issue.
Regards,

@ggtakec ggtakec closed this as completed Jun 20, 2015
@stevenshiau
Copy link
Author

Sorry for the late, and thanks for your updates. However, I tried to compile the master you just merged and did the same test as my 1st post. I still got the same issue. Did you do the similar test with ecryptfs? Does it work there?
Thank you very much.

Steven.

@ggtakec
Copy link
Member

ggtakec commented Jun 20, 2015

I’m sorry for forgetting tell you about it.
After remounting encryptfs, encryptfs can not display correct original file size still reading the file contents.
I could not know the reason about this problem.
Maybe then, it encryptfs it may have been due to the fact that use extended attributes(ecryptfs_xattr option used).
But after reading file contents at least once, encryptfs displays correct file size.
Regards,

@stevenshiau
Copy link
Author

@ggtakec,
Got it. Thanks.

Steven.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@delx @stevenshiau @ggtakec