-
Notifications
You must be signed in to change notification settings - Fork 7
-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cross-reference against official functional test suite #4
Comments
Hi @rfk thank you for that detailed issue report! I didn't know which repository contains the latest tests, so I took mine from mozilla-services/server-full. Hence, my implementation focused on theses tests (I had to fix first) and they run through with around 85% success rate. I just cloned |
I think I have fixed most tests so far. Still two are failing but I am not sure whether this is my issue or the random order of the tests … (especially Here's a diff of three minor changes to the testsuite so 30 of 32 tests passes. The first is a bad status code and should be 412 as described in the Storage API 1.1 and the latter two are limitations my implementation does not have :-) diff -r 63392565d91a syncstorage/tests/functional/test_storage.py
--- a/syncstorage/tests/functional/test_storage.py Mon Sep 03 12:52:24 2012 +1000
+++ b/syncstorage/tests/functional/test_storage.py Tue Sep 04 18:09:51 2012 +0200
@@ -583,7 +583,7 @@
self.assertEquals(res.json["payload"], _PLD)
# deleting all with no confirmation
- self.app.delete(self.root + '/storage', status=400)
+ self.app.delete(self.root + '/storage', status=412)
# deleting all for real now
res = self.app.delete(self.root + '/storage',
@@ -699,16 +699,16 @@
wbos = [{'id': str(i), 'payload': _PLD} for i in range(105)]
res = self.app.post_json(self.root + '/storage/col2', wbos)
res = res.json
- self.assertEquals(len(res['success']), 100)
- self.assertEquals(len(res['failed']), 5)
+ self.assertEquals(len(res['success']), 105)
+ # self.assertEquals(len(res['failed']), 5)
# The test config has max_bytes=1M.
# Uploading 5 210MB items should produce one failure.
wbos = [{'id': str(i), 'payload': "X" * (210 * 1024)}
for i in range(5)]
res = self.app.post_json(self.root + '/storage/col2', wbos)
res = res.json
- self.assertEquals(len(res['success']), 4)
- self.assertEquals(len(res['failed']), 1)
+ self.assertEquals(len(res['success']), 5)
+ # self.assertEquals(len(res['failed']), 1)
def test_blacklisted_nodes(self):
# This can't be run against a live server. |
Hi @posativ, great work on this minimal server! I'm one of the devs for mozilla's sync server and yeah, it can be a pain to set up a local install. Your work is appreciated.
One thought for increasing compatibility between this and the official version is to run the functional tests from the http://hg.mozilla.org/services/server-storage repo against your server. This will help to highlight any corner-cases where the functionality differs between the two.
Here's how I managed to do it. First, checkout and build the server-storage repo:
Run your minimal server with a test user:
Then execute the functional tests, directing them at the running server:
For me this reports 32 tests run, 9 failures, 9 errors, 4 skipped. That's a pretty impressive success rate already!
Some of the failures may be spurious, e.g. they check implementation details of our server. And frankly, some of them may be bugs in our server that have since been codified into tests...
Anyway, hope you find this helpful.
The text was updated successfully, but these errors were encountered: