Every repository with this icon (
Every repository with this icon (
| Description: | tiny queue system based on starling, in scala edit |
-
Changing from absolute to relative timeouts in {operate,peek,remove}{React,Receive}
1 comment Created 3 months ago by blairRan into a bug yesterday with some code using peekReceive(). It was passing 15,000 millseconds to the method as a timeout, which was interpreted as being 15,000 milliseconds past January 1st, 1970, so of course, the call didn't block at all and the code calling it consumed 100% of the CPU.
Given that all the methods (that I'm aware of) in the Java world take a relative timeout time and not the absolute time when the timeout should expire (including Object#wait() and Scala's actor's methods), I suggest moving the methods to using that. This will also have the improvement of reducing the number of system calls it takes, both in the calling code and in Kestrel itself.
Personally speaking, I would rather take longer for a test suite to run than to have an unobvious API plus additional work that the server needs to perform in a high-load environment.
If changing the code is acceptable, I'll code it up.
Comments
-
Module python-memcached uses memcache ability to add meta-data (flags) to store type of data, and uses it for parsing.
For example it sets flags=0x1 to say that data is pickled and should be unpickled.
Kestrel silently drops flags so meta-data is lost which leads us to unexpected behavior: we write python tuple "(1000, 100)", we get "(I1000
I100
tp1
." as a result.As far as I understand Kestrel code, to achieve this functionality we should:
- add .flags member to QItem
- move instantiation of QItem from PersistentQueue.add to KestrelHandler.set
- use item.flags in formatting response in KestrelHandler.get
If these are acceptable changes I'm going to code them and present patch for review shortly.
Comments
-
The memcache protocol allows you set a flag byte during a set, kestrel ignores this and defaults it to 0.
Many languages uses a flag to denote the data is serialized so that the client can then deserialize it.
Comments
-
Update kestrel to utilize the binary memcache protocol
0 comments Created about 1 month ago by SuhailThere's proof out there that the gap in performance between the text based protocol and the binary one widens with more concurrently connected clients.
The binary protocol also has and will continue to have more clients available for it which helps kestrel's adoption.
Comments
-
I'm not sure if this is something to add into Kestrel itself, but tight Nagios integration would be a huge plus :)
Comments
-
It seems that Kestrel (1.2) doesn't reliably respond with stats when asked, for example, using my Python memcached client I call get_stats() (sends the "stats" command) which sporadically returns results, i.e.:
>>> pprint(c.get_stats()) [('127.0.0.1:22133 (1)', {'bytes': '0', 'bytes_read': '12', 'bytes_written': '1745', 'cmd_get': '0', 'cmd_peek': '0', 'cmd_set': '0', 'curr_connections': '1', 'curr_items': '0', 'get_hits': '0', 'get_misses': '0', 'queue_MapUrl_age': '0', 'queue_MapUrl_bytes': '0', 'queue_MapUrl_discarded': '0', 'queue_MapUrl_expired_items': '0', 'queue_MapUrl_items': '0', 'queue_MapUrl_logsize': '8798', 'queue_MapUrl_mem_bytes': '0', 'queue_MapUrl_mem_items': '0', 'queue_MapUrl_open_transactions': '0', 'queue_MapUrl_total_items': '34', 'queue_MapUrl_waiters': '0', 'queue_Recompute_age': '0', 'queue_Recompute_bytes': '0', 'queue_Recompute_discarded': '0', 'queue_Recompute_expired_items': '0', 'queue_Recompute_items': '0', 'queue_Recompute_logsize': '591', 'queue_Recompute_mem_bytes': '0', 'queue_Recompute_mem_items': '0', 'queue_Recompute_open_transactions': '0', 'queue_Recompute_total_items': '3', 'queue_Recompute_waiters': '0', 'queue_default_age': '0', 'queue_default_bytes': '0', 'queue_default_discarded': '0', 'queue_default_expired_items': '0', 'queue_default_items': '0', 'queue_default_logsize': '0', 'queue_default_mem_bytes': '0', 'queue_default_mem_items': '0', 'queue_default_open_transactions': '0', 'queue_default_total_items': '0', 'queue_default_waiters': '0', 'queue_kestrel.log_age': '0', 'queue_kestrel.log_bytes': '0', 'queue_kestrel.log_discarded': '0', 'queue_kestrel.log_expired_items': '0', 'queue_kestrel.log_items': '0', 'queue_kestrel.log_logsize': '0', 'queue_kestrel.log_mem_bytes': '0', 'queue_kestrel.log_mem_items': '0', 'queue_kestrel.log_open_transactions': '0', 'queue_kestrel.log_total_items': '0', 'queue_kestrel.log_waiters': '0', 'time': '1259186080', 'total_connections': '2', 'total_items': '0', 'uptime': '443', 'version': '1.2'})] >>> pprint(c.get_stats()) [('127.0.0.1:22133 (1)', {})]I've jumped into the debugger and it looks like Kestrel just isn't returning anything to the command.
Comments
-
Kestrel does not work on uni-processor boxes
2 comments Created 22 days ago by hugohallqvistIt seems that kestrel does not work properly on uniprocessor boxes, as maxThreads is initialized to 2, and a sanity check fails inside ThreadPoolExecutor in the actors library. The condition is maximumPoolSize < corePoolSize. corePoolSize is derived from minNumThreads which is initialized to 4 statically in FJTaskScheduler2.scala.
I've fixed it in my tree, here:
http://github.com/hugohallqvist/kestrel/commit/35b2999636c0835c7b8b94d69b8af6cfd5ed0920
Comments
I had a similar issue that I believe was a regression with Scala 2.7.7, I changed ivy/ivy.xml to use 2.7.5 (see: SHA: 54713ab) instead and everything works fine now.
hugohallqvist
Thu Dec 03 12:51:42 -0800 2009
| link
Too bad I didn't find your changes before wasting a couple of hours debugging... :-)












agreed, the API would be better this way. (i left it as absolute time because i was in a hurry, not because i like it that way, so your way sounds better.)