Lua Log Access / Date Manipulation #372

Open
dlecocq opened this Issue Mar 6, 2012 · 7 comments

Projects

None yet

4 participants

@dlecocq
dlecocq commented Mar 6, 2012

I've been really enjoying the Lua support in Redis. It takes what is already a stellar system and makes it better.

That said, there are two things that I'm wondering couldn't be made available:

  1. Support for outputting to logs. I completely understand the reasons you have to removing access to the filesystem, but I could certainly make debugging easier, though I know that debugging support has been mentioned elsewhere. Perhaps a write-only file that can be configured to a path other than the redis log? Or maybe just to the redis log? Just a thought.
  2. Support for system time access. Again, I understand the reason that this isn't supported, but it would be extremely useful to have access to a synchronized clock between clients for locking mechanisms. Perhaps a arguments into scripts with a keyword like <<TIME>> could be replaced with the current system time, so that the system time when it was processed can be conveyed correctly to the lua script, the AOF and any connected slaves.
  3. Edit: This is less of an issue, but because os.date and date don't seem to be available, it makes it difficult to do date manipulation in these scripts. I say it's less of an issue because most date manipulation could be done on the client side, but one of the really appealing aspects of these lua scripts is language agnosticism, and reusing a lot of code. Any date manipulation would then have to be ported. Again, not a huge deal, but kind of a bummer.

I had posted a comment to this effect in the Disqus comments under EVAL, but I wasn't sure if that was the right context for a request / whether it would get read. If you did see it, though, please forgive the double-posting.

@antirez
Owner
antirez commented Mar 7, 2012

On Wed, Mar 7, 2012 at 12:39 AM, dlecocq
reply@reply.github.com
wrote:

I've been really enjoying the Lua support in Redis. It takes what is already a stellar system and makes it better.

That said, there are two things that I'm wondering couldn't be made available:

  1. Support for outputting to logs. I completely understand the reasons you have to removing access to the filesystem, but I could certainly make debugging easier, though I know that debugging support has been mentioned elsewhere. Perhaps a write-only file that can be configured to a path other than the redis log? Or maybe just to the redis log? Just a thought.

redis.log(LOG_NOTICE, "Hello world") :)

I forgot to document it...

  1. Support for system time access. Again, I understand the reason that this isn't supported, but it would be extremely useful to have access to a synchronized clock between clients for locking mechanisms. Perhaps a arguments into scripts with a keyword like <<TIME>> could be replaced with the current system time, so that the system time when it was processed can be conveyed correctly to the lua script, the AOF and any connected slaves.

That's more problematic than that, example:

if os.time() % 2 then
redis.set("foo","bar")
end

This is a time dependent script, how to replicate / AOF that correctly
is not trivial.
Replacing the time at source level as you suggest may work but kills
caching of scripts.

The only thing that we may do is to provide the TIME command in Redis
returning the server unix time, and mark this command as "random" in
the command table, so that scripts will not accept write commands
after this command, exactly as we already do with other commands.

Cheers,
Salvatore

I had posted a comment to this effect in the Disqus comments under EVAL, but I wasn't sure if that was the right context for a request / whether it would get read. If you did see it, though, please forgive the double-posting.


Reply to this email directly or view it on GitHub:
#372

Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

@agladysh
agladysh commented Mar 7, 2012

My 2c: Subset of os.date()+os.time() — without access to the current timestamp — could be useful, and, IMO, worth to expose. Otherwise users would have to reinvent the wheel when working with UNIX timestamp — and is not that easy to get properly right.

@antirez
Owner
antirez commented Mar 7, 2012

Alexander: it's not a bad idea, but this requires that we switch from "make ansi" to "make posix" or something like that, not sure about the full consequences of this move. Otherwise we need to modify a bit the lua distribution, making the upgrade to next versions a bit more complex.

@agladysh
agladysh commented Mar 7, 2012

Um, what about redis.time() and redis.date()?

@antirez
Owner
antirez commented Mar 7, 2012

probably better, just ripping the implementation from the Lua source...

@catwell
catwell commented Mar 7, 2012

I might use that too. I do not really care how it is implemented. Switching to "make posix" would not bother me too much but redis.{time,date}() is fine too.

@dlecocq
dlecocq commented Mar 7, 2012

First off, glad to know about redis.log! Thanks!

Rather than source substitution which, of course, ruins caching, I was suggesting argument substitution. So that the <<TIME>> placeholder in the invocation

EVALSHA deadbeef... 2 key1 key2 arg1 arg2 "<<TIME>>"

would get translated to

EVALSHA deadbeef... 2 key1 key2 arg1 arg2 1331135179

at command parsing time. This would allow the AOF to contain the timestamp at which the command was executed, and maintain the original parameters, but still allow for functional access to the current time in the script. Conceivably, this could even be made more generalizable with a different sandboxing for the arguments from the script itself, so that certain additional features could be performed when evaluating the arguments, the results of which could be stored in place of that argument. For example,

EVALSHA deadbeef... 0 "lua:os.time()" "lua:..."
-- Would get parsed as
EVALSHA deadbeef... 0 "1331135179" "<result of second chunk>"

I don't know if that's a good idea, but it would be more generalizable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment