Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Commits on Aug 30, 2013
  1. Merge branch 'add_run_job_epoch'

    authored
    Conflicts:
    	gearman/connection.py
    	gearman/connection_manager.py
    	gearman/protocol.py
    	gearman/worker.py
  2. @Shrews

    Do not attempt to register connections not yet established

    Shrews authored committed
    When using multiple job servers, some connections may not yet be
    established (not been chosen yet), so their socket is still invalid.
    This would cause poller.register() to throw an exception.
  3. @Shrews

    Handle SSL WANT_READ/WANT_WRITE exceptions and drain SSL buffer.

    Shrews authored committed
    SSL has in internal read buffer that needs to be drained. Since
    we, by default, read in 4K chunks, anything larger would have left
    data to be read that may not have been processed properly.
    
    Also, we need to handle SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE
    exceptions around the recv() and send() calls, otherwise we would
    think we are disconnected and cause all sorts of problems.
  4. @Shrews

    Require cert validation of server.

    Shrews authored committed
  5. Add SSL support.

    David Shrewsbury authored committed
    This adds support for SSL connections to an SSL enabled job server.
    To use SSL, three SSL-related files must be given when defining a
    job server connection:
    
      keyfile - File with the private key
      certfile - File with the public key
      ca_certs - File with the CA public cert(s)
    
    GearmanConnectionManager has been modified to simultaneously
    support non-SSL and SSL connections alike. Nothing changes for
    non-SSL connections:
    
      GearmanClient(['localhost:4730'])
    
    SSL connections require that the host list use a dictionary to
    define the connection:
    
      s1 = { 'host': 'localhost',
             'port': 4730,
             'keyfile': '/path/to/keyfile',
             'certfile': '/path/to/certfile',
             'ca_certs': '/path/to/ca_certs' }
      GearmanClient([s1])
    
    Currently, all three SSL related files MUST be given and the port
    must be specified in the dictionary.
  6. @Shrews

    Import JOB_UNKNOWN at top-level.

    Shrews authored committed
  7. @malor

    Use array.array('c') as a type of incomming buffer

    malor authored committed
    A connection incoming buffer is extended using += operator. This leads
    to unnecessary memory allocations and significant performance drops when
    transferring big amounts of data (i. e. a few megabytes). Using of the array('c')
    (an array of bytes) data type enables us to handle incoming data efficiently.
  8. @kylemcc

    update connection multiplexing code to use epoll when available, fall…

    kylemcc authored committed
    … back to select when not
  9. @klange

    Revert the reconnect branch

    klange authored committed
  10. @klange

    rename reconnect to maybe_reconnect

    klange authored committed
  11. @klange

    Only reconnect if a given time has passed

    klange authored committed
    This is a per-connection timeout, default is 60 seconds.
  12. @klange

    Make the reconnect safe when we have data to send.

    klange authored committed
    If the outgoing buffer isn't empty, we're still sending
    a command. We should wait to restart the connection until
    we've given the buffer a chance to empty and a response to come
    through from the daemon.
  13. @klange

    Reset connections during poll_until timeout

    klange authored committed
    This is a bit of a hack, but:
        When poll_connections_until_stopped's timer triggers and the loop
        ends, this will reconnect connected connections when
        establish_connection is called, rather than just continuing.
    
        This might break things, and that would be terrible; but if it
        doesn't, then life is good and this should fix the odd problem of
        having connections we can't seem to write to, leaving us with only
        one connection to the server that actually gets used.
  14. Add more checking for null and non-ascii chars.

    Eskil Olsen authored committed
    The protocol says;
    "Arguments given in the data part are separated by a NULL byte, and
    the last argument is determined by the size of data after the last
    NULL byte separator. All job handle arguments must not be longer than
    64 bytes, including NULL terminator."
    
    This adds a check to ensure that all arguments, but the last, do
    not contain \0.
    
    I do not check the args pass encode('ascii'). It would make sense,
    since it's not known what will happen inside the bowels of gearmand.
    It uses memcmp for the most part, and we have to assume the
    queue storage can handle utf8, as long as it does not contain
    \0.
  15. Revert "Deal with unicode params in a conservative way."

    Eskil Olsen authored committed
    This reverts commit 0faa3ad.
  16. Revert "Fix for unicode handling and test thereof"

    Eskil Olsen authored committed
    This reverts commit 9d73fb2.
  17. @klange
  18. Fix for unicode handling and test thereof

    Eskil Olsen authored committed
  19. @rhettg

    Deal with unicode params in a conservative way.

    rhettg authored committed
  20. @rhettg

    after_job() hook

    rhettg authored committed
  21. @patricklucas

    Use weakrefs to avoid leaks

    patricklucas authored committed
    Dictionaries in client and client_handler keep references to
    GearmanJobRequest objects, but aren't explicitly removed until they
    either fail or complete.
    
    This never occurs when wait_until_complete=False, so using
    WeakKeyDictionary and WeakValueDictionary ensures the request objects
    can be garbage collected.
  22. Let gearmand handle job uniques and don't stuff the data into the uni…

    Eskil Olsen authored committed
    …que on '-'
Commits on Jun 5, 2013
  1. @jturmel

    Add support for SUBMIT_JOB_EPOCH

    jturmel authored
    * submit_job now accepts a 'when_to_run' parameter to take advantage of
      Gearman's ability to schedule jobs to be run in the future
Commits on Feb 14, 2013
  1. @malor

    Use array.array('c') as a type of incomming buffer

    malor authored
    A connection incoming buffer is extended using += operator. This leads
    to unnecessary memory allocations and significant performance drops when
    transferring big amounts of data (i. e. a few megabytes). Using of the array('c')
    (an array of bytes) data type enables us to handle incoming data efficiently.
Commits on Feb 6, 2013
  1. @klange

    Merge pull request #30 from kylemcc/epoll

    klange authored
    update connection multiplexing code to use epoll when available, fall back to select when not available
  2. @kylemcc
Commits on Feb 5, 2013
  1. @klange

    Revert the reconnect branch

    klange authored
Commits on Oct 9, 2012
  1. @klange
Commits on Oct 2, 2012
  1. @klange

    Only reconnect if a given time has passed

    klange authored
    This is a per-connection timeout, default is 60 seconds.
Commits on Sep 25, 2012
  1. @klange

    Make the reconnect safe when we have data to send.

    klange authored
    If the outgoing buffer isn't empty, we're still sending
    a command. We should wait to restart the connection until
    we've given the buffer a chance to empty and a response to come
    through from the daemon.
Commits on Sep 24, 2012
  1. @klange

    Reset connections during poll_until timeout

    klange authored
    This is a bit of a hack, but:
        When poll_connections_until_stopped's timer triggers and the loop
        ends, this will reconnect connected connections when
        establish_connection is called, rather than just continuing.
    
        This might break things, and that would be terrible; but if it
        doesn't, then life is good and this should fix the odd problem of
        having connections we can't seem to write to, leaving us with only
        one connection to the server that actually gets used.
Commits on Sep 5, 2012
  1. Add more checking for null and non-ascii chars.

    Eskil Olsen authored
    The protocol says;
    "Arguments given in the data part are separated by a NULL byte, and
    the last argument is determined by the size of data after the last
    NULL byte separator. All job handle arguments must not be longer than
    64 bytes, including NULL terminator."
    
    This adds a check to ensure that all arguments, but the last, do
    not contain \0.
    
    I do not check the args pass encode('ascii'). It would make sense,
    since it's not known what will happen inside the bowels of gearmand.
    It uses memcmp for the most part, and we have to assume the
    queue storage can handle utf8, as long as it does not contain
    \0.
Commits on Sep 4, 2012
  1. Revert "Deal with unicode params in a conservative way."

    Eskil Olsen authored
    This reverts commit 0faa3ad.
  2. Revert "Fix for unicode handling and test thereof"

    Eskil Olsen authored
    This reverts commit 9d73fb2.
Commits on Aug 22, 2012
  1. @klange
Something went wrong with that request. Please try again.