Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

DB updates do not reflect on FS #1

Open
SaMnCo opened this Issue Sep 11, 2012 · 3 comments

Comments

Projects
None yet
2 participants

SaMnCo commented Sep 11, 2012

Hi!!

First of all thanks for your development, it is promising and the feature itself is great! Great work!

As I was playing with it I ran into a problem. I am using a Debian Wheezy for my example, and I run the following:

echo "deb http://ftp.pl.debian.org/debian/ squeeze non-free" > /etc/apt/sources.list.d/squeeze-non-free.list
apt-get update && apt-get install sun-java6-jdk
update-java-alternatives -s java-6-sun fuse-utils
cd /usr/local/src/
wget http://github.com/downloads/narkisr/couch-fuse/couch-fuse_0.4.1-1_amd64.deb
dpkg -i couch-fuse_0.4.1-1_amd64.deb
apt-get -f install
mkdir /mnt/couch
couchfuse --db mount_test --path /mnt/couch/ --host http://couchdbServer:5984/

My CouchDB server is in fact a BigCouch instance running on 4 small test servers.

This totally works without problem. Just in case I also modprobed fuse, and in effect I can see my DB in the /mnt/couch directory. I can download attachment (reaaaal fast), create new docs and so on. So far so good.

When I do something from the command line to the DB, I can see the changes on CouchDB side.
However, if I do something on CouchDB (delete doc, add attachment...) I cannot see any change on the FS side.

Any idea where this might come from? Am I messing out at install?

Let me know your thoughts...
Many many thanks in advance,
Best,
Sam

Collaborator

narkisr commented Sep 11, 2012

Hey Sam, this is a know limitation right now, I didn't find a reliable way to pass notification back to the kernel on events that happend on the couch instance, the regular flow of data in FUSE file system is:

fs actions -> fuse module -> couch-fuse -> couchdb

In this case the flow is:

couchdb -> couch-fuse -> fuse-module state

I did try to emulate events on the fs (basicly doing mkdir, delete etc..) but this is a crude method

Id be happy to get ideas on someone who implemented such a thing.

SaMnCo commented Sep 12, 2012

Hi Ronen,

Thanks for your answer, I really appreciate it.

I am no developper myself (unfortunately I stick with product management
and a bit of sysadmin ;o)) so I cannot really help you in developping the
feature.

However in our project we use CouchDB as a repository to dispatch medium
sized files accross hundreds of nodes. We use a combination of custom
daemons and rabbit-MQ to manage the notification process for nodes and
trigger replication.
This leads me to my idea for your project: we use the _changes feature of
CouchDB. Nodes can query that URL then trigger the DB replication (or
master cluster can trigger notifications to nodes)

In your case, you could potentially use that idea: your module would
"regularly" query the _change URL on server and update accordingly.

Obviously this is not the best way around as you generate traffic for the
notification process, but this would be the simplest implementation, and it
can be simply daemonized to write back to a notification file which you can
monitor with inotify.
If you then have a "fs_update" function, you can use incron to trigger the
update if you file change.

I imagine you already looked at that, but if you didn't maybe this will
give you ideas!
Let me know how it goes!
Warmest regards,
Sam

2012/9/12 ronen notifications@github.com

Hey Sam, this is a know limitation right now, I didn't find a reliable way
to pass notification back to the kernel on events that happend on the couch
instance, the regular flow of data in FUSE file system is:

fs actions -> fuse module -> couch-fuse -> couchdb

In this case the flow is:

couchdb -> couch-fuse -> fuse-module state

I did try to emulate events on the fs (basicly doing mkdir, delete etc..)
but this is a crude method

Id be happy to get ideas on someone who implemented such a thing.


Reply to this email directly or view it on GitHubhttps://github.com/narkisr/couch-fuse/issues/1#issuecomment-8474634.

SaM'n'Co
R'U Ready? I was BORN ready!

Collaborator

narkisr commented Sep 12, 2012

I am aware of the _changes api and it answers the

couchdb -> couch-fuse

Part, yet the issue of somehow updating fuse kernel module still persist :)

Ill try to look for more info about this in FUSE land

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment