CloneLocker

Disclaimer: THIS DOES NOT WORK AT ALL...well at least as of 9-Jun-2013. I mean, it will compile and print some stuff under DebugView. But it's just stubbed experimentation of my first (Windows) kernel code. It is what a friend of mine calls "copypasta" (a funny riff on spaghetti code)...trying to organize libraries and look to see what problems might arise in integration. My feasability study led me to believe what I was looking at was indeed technically possible, but would require too much work to build a reliable solution in another week's time, which is all I had.

Ordinarily I'd never share such a thing in public. But in this case the exploration was not motivated by my own interests, but a problem someone else had, that I offered to think about. Such tinkering runs the risk of being taken seriously on accident when put alongside serious work on GitHub (well, if we're going to call REBmu serious--that one's in the eye of the beholder).

I probably should have made up another GitHub account and pull a Chris Gaines! But too late. This is just something to contemplate, for someone who might want a week's worth of abandoned investigation. It might be doing more harm than good by being on the Internet vs. in the tenth external HD in the closet. We'll see. :-/

---

(Note: Lock icon borrowed/stolen from Alexandre Moore. It came up as being free for commercial or personal use in an image search for "lock icon", and actually had multiple sizes. I thought I might need a tray icon size, so I picked it from the available free options in that search engine.)

CloneLocker is exploratory research, attempting to test the feasability of attacking this scenario:

1. there is a directory of files, being shared by users over some network

2. it is a bad thing if these users step on each others work

3. the files are unwieldy and large (by the standards of your day)

4. your LAN is slow for the file sizes, and/or people are working remotely

Were it not for 3 and 4, you'd just tell someone with this problem "uh...get a faster network and centralize the files on a share". Not everyone considers this a good answer for their problem. So there have apparently been attempts to build filesystems which do this for you transparently. See for instance the Wiki Page for CacheFS (which looks to be on hold for now).

There also exists an an ecosystem of userspace network synchronization products that can implement part of the desire. If your network is slow or even zero speed (working offline) it can bring you up to date lazily. Some are free ( e.g. Unison ) and some cost money ( e.g. GoodSync ). But these tend to operate on a polling strategy. They interactively ask about reconciliation after-the-fact, instead of having a lightweight locking protocol that prevents those conflicts from happening in the first place.

So let us say you want to reimplement (or just extend) one of these user space products to do a kind of "clone locking"...but don't want to go all the way to writing a new filesystem. You'd need a monitor process running on each machine that would impose artificial read/write exclusions on the local filesystem, but based on what other users on the network are doing. So let's say user A takes a read lock on a file on his disk -- then the monitor process would communicate that fact to user B's machine. Although user B is not intentionally running any program accessing the clone of the file on his disk, his monitor process would grab the appropriate lock on it anyway...as a proxy for user A, for the duration of A's interest in the file.

But how do you write such a thing for Windows (if you're not Microsoft?) Well, you have to hook the OS to be notified of file accesses: when files are being opened, closed, read, written, etc. The existence of third-party antivirus screeners on Windows means these hooks must exist; so it has to be technically possible.

Early tests seemed to confirm this. Strangely enough the seemingly elusive hooking doesn't turn out to be especially hard in the scheme of things. It's gotten easier with the introduction of what are called miniFilter drivers.

Really all you need is a kernel mode component that passes up the filenames being looked at prior to allowing access. A timer and polling period may be necessary in the user mode process, in case the client who held the lock crashed or had the power plug pulled. There are some nuisances about installing with elevated privileges, and having cryptographically signed certificates you pay MS for or tell your users to put their machines into "test mode".

I got through all that bit and got cocky, but turns out that modifying Windows DDK samples is a hornet's nest. I had encouraging early success--by following the schematic of using the NT DDK build environment to build both a user mode program and a kernel mode program. Then I became discouraged when it seemed you couldn't use the NT DDK build process to develop a sophisticated userspace app.

UPDATE: that turns out to not be true! Inadvisable, maybe, but not impossible:

Including C++ headers in user mode programs built with NT DDK

In any case, at first I wanted to keep the user mode code "lean and mean", just a small client/server C program that kept all the lock state stored in a Common Controls ListView. But then it fell victim to concerns such as how none of the decent raw C JSON libraries for network communication support WCHAR... they're full of deprecated sprintf or other such things. The filenames aren't natural paths like "C:", it's "\Devices\Harddisk1\blah\blah". Without C++ and decent libraries, matching the user mode code and the driver code is nigh impossible.

It can be made to work, but the usermode component will be complex enough that I don't think it can be reliably written in plain C on a tight schedule. The WCHAR conversions alone are tricky, so I'd absolutely switch the build environment of the user mode code to C++ and probably Qt if I had time to pursue this any further.

In summary:

1. "Kids, don't try this at home."

2. Set up your build environment for your user mode component outside the DDK and use a modern toolset. Especially if you are on a tight schedule, don't try and write anything in Win32 C. Too much volatility.

3. Buy faster network equipment.

4. Join the revolution against software complexity.