Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAQ file broker client (10_2_X) #23947

Merged
merged 20 commits into from Jul 25, 2018
Merged

DAQ file broker client (10_2_X) #23947

merged 20 commits into from Jul 25, 2018

Commits on Jul 23, 2018

  1. reorganizing source and service to support both old locking and new f… 

    …ile service which will distribute input files
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    ded3225 View commit details
    Browse the repository at this point in the history

  2. 2nd part of modifications to support new FFF file service interface

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    0ce25b3 View commit details
    Browse the repository at this point in the history

  3. handle max LS (cloud stopping) and updates to the API

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    28d786f View commit details
    Browse the repository at this point in the history

  4. fixes to file locking and changes to the file service protocol

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    742ae6e View commit details
    Browse the repository at this point in the history

  5. back off mechanism to reduce number of open sockets in BU service

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    602af5a View commit details
    Browse the repository at this point in the history

  6. allow stop LS 0 

    Signed-off-by: Srecko Morovic <srecko.morovic@cern.ch>
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    0f0e8bd View commit details
    Browse the repository at this point in the history

  7. logging for stop LS

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    7ade6d4 View commit details
    Browse the repository at this point in the history

  8. pid added to HTTP attributes (for possible use in renaming) 

    catch file not exists and check for run directory on BU
handle NORUN
fix READY with no file current LS
handle JSON file move by server to subdirectory with prefix
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    26b08bf View commit details
    Browse the repository at this point in the history

  9. support for optional unlocking 

    support for socket keep alive (experimental)
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    76b9f15 View commit details
    Browse the repository at this point in the history

  10. rename fileService to fileBroker 

    fix undefined variable value when no raw file
use shared lock for server contact and promote it to exclusive if EoLS
file(s) have to be written
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    969847d View commit details
    Browse the repository at this point in the history

  11. write EoLS files before index files from next LS are written locally

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    406703b View commit details
    Browse the repository at this point in the history

  12. remove shared lock implementation which doesn't work correctly

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    958ca03 View commit details
    Browse the repository at this point in the history

  13. removing unused lock file function

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    aa41c87 View commit details
    Browse the repository at this point in the history

  14. proper error recovery (avoiding assertions, segmentation faults) in c… 

    …ase of deletion of the run directory
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    7e85cf1 View commit details
    Browse the repository at this point in the history

  15. Use HTTP 1.1 as it will be needed for keep alive

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    7ef6400 View commit details
    Browse the repository at this point in the history

  16. instead of boost::filesystem high-level functions using low level file 

    access to copy JSON file from ramdisk. This allows better diagnosis of
problems (such as recently seen bad file descriptor problem in some
tests). This is enabled only for File Broker, while NFS locking code
remains unchanged (as it is going to be phased out).
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    163d1da View commit details
    Browse the repository at this point in the history

  17. fix for keep-alive

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    bac7414 View commit details
    Browse the repository at this point in the history

  18. 2nd round of keep-alive fixes 

    keep alive by default
close socket on error
NB: in tests it turns out that read_until reads the whole response. this
is however not
always guaranteed and updates will be needed to read based on HTTP
Content-Length
    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    1ee9ac4 View commit details
    Browse the repository at this point in the history

  19. catching const& exceptions

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    c807ce6 View commit details
    Browse the repository at this point in the history

  20. remove double const

    @smorovic
    smorovic committed Jul 23, 2018
    Copy the full SHA
    09f835c View commit details
    Browse the repository at this point in the history