Skip to content

Archive Trac docs abaco

Jump to bottom
madscatt edited this page Jun 20, 2026 · 1 revision

Docs Abaco

Legacy Trac archive page imported from docs_abaco. Source: https://genapp.rocks/wiki/wiki/docs_abaco. Review age, links, and examples before treating as current.

abaco integration notes

Related wiki pages

sharelatex

  • [http://sharelatex.tacc.utexas.edu/project]

abaco links

agave links

other links

notes

  • gadocker.tacc:~/abaco/node has test js for running abaco

  • gadocker.tacc:~/github/abaco/notes has additional info, thoughts

  • gadocker.tacc:~/github/agave-cli/bin has scripts with curl examples

  • genapp:languages:docker is the target language for generating modules as containers

    • i believe we are generating abaco compatible module containers
    • what about pushing them to dockerhub?
    • works with genapp:languages:nodeapi ?

done

  • verify abaco execution in gadocker.tacc:~/abaco/node

    • in ~/github/abaco/node
      • there are .php and .js versions for most of these || script || args || returns || notes || || gettoken || || refresh-token & access-token || needs username, password, api_key, api_secret setup || || registeractor || access-token registerjsonstring || || || getactors || access-token || all registered actor info || || || deleteactor || access-token actor:id || ||
        || getactor || access-token actor:id || || || startexecution || access-token actor:id message || body, url for messages & execution:id || message is json string to input || || getexecution || access-token actor:id execution:id || msg + json with logs: || get execution status, if COMPLETE, logs: has the output || || getexecutions || access-token actor:id || JSON of all historical and current executions || get all executions for actor || || getmessages || access-token actor:id || ? || not sure the utility of this||

  • verify genapp:docker module container execution

    • abaco uses environment variable MSG to run {{{docker run --env MSG='{}' image-id}}}
    • tested ok
    • genapp uses appconfig:resources
   "resources" {
       "docker-local" : "docker run -v __rundir__:/genapp/run --network host --user www-data ehb1/genapp___application__:__menu:id__-__menu:modules:id__ /genapp/bin/__menu:modules:id__"
      ...
   }
  • in this case, the prefix ehb1/ is required since directives:dockerhub:user is set, o.w. skip the ehb1/ prefix
    • N.B. the prefix must match
    • TODO: later we should remove this restriction {{{__~dockerhub:user{dockerhub:user/}}}} ?
  • push to dockerhub
    • directives.json
    • setting directives:dockerhub:user forces language:docker to push images
    , "dockerhub" : {
        "user" : "ehb1"
# optionally specify an id, otherwise, hostname will be used
# define id as an empty string if you want no id
# N.B. user/id duplicates will clobber, so ensure your combo is unique for each genapp instance!
#       ,"id" : "xyz"
    }

- **N.B.**: if you have more than one docker username, you should make sure you are logged into the correct account before running genapp

  • automate docker push during build of language:docker

  • user/genapp_{id_}application:menu-module

  • possible with docker api ? https://docs.docker.com/registry/spec/api/

  • run genapp generated module container on abaco

    • [done] first test registry of containers
      • e.g. {{{./registeractor.js 4f3f41607872dd6cb67e9b85553df99e '{"image":"ehb1/genapp_demo:simulate-denergy","name":"ehb1/genapp_demo:simulate-denergy","description":"ehb1/genapp_demo:simulate-denergy"}'}}}
      • success :)

  • run via abaco

    • abacorun.php / abacolib.php
    • [done] 1st convert gettoken.js etc to .php versions
    • [done] test via command line
    • finish integration into genapp
      • messaging
      • are ports open? - should be via atmosphere
      • works, but needs to listen on 0.0.0.0 ? - otherwise no messages
        • perhaps something else odd? with setup
        • udplistenhost? (would need tcp likely also)?
    • [done] appconfig:resource:abaco:"abaco/abacorun"

to do

  • renew token if needed
  • figure out file staging
  • perhaps a general staging tool (for use with airavata, seedme2 etc)

later todo

  • remove the appconfig:resource:docker-local restriction {{{__~dockerhub:user{dockerhub:user/}}}} ?
  • add command line argument for target language(s) to genapp
  • better group errors from build_docker.pl into genapp output
  • cancel abaco run?
    • delete execution
  • stateless actors will spawn workers
  • so when this is ready:
    1. register or put upon genapp docker push
    • put to actor force=true forces a refresh the image… payload as originally
    1. reuse at execution time
  • global actors for short jobs
  • potentially scaled.

resource:abaco notes

  1. [done] need api_key & api_secret setup
  2. gettoken.js (this can be rerun, if existing, will return it, if not will generate a new one)
  • if timed out, might need a new one
  • check for expired token or figure out how to renew early
  • jobs can be arbitrarily long, so we should probably expect to request every time and retry if expired
  1. register actor
  • perhaps actor:description or actor:image can be used to store additional info (user?, uuid of job?)
  1. wait until actor ready (getactor status:READY)
  2. startexecution / get execution id or failed
  3. getexecution (poll? - later message? pubsub? )
  4. when getexecution is complete "logs" will have the output
  5. delete actor
> That's a good question. Usually people send notifications from the actor itself or from a second actor who's only role is to send notifications to external agents. The first actor would message the notifications actor at the proper time.
>
> One feature we're likely to add soon that would help here is that of actor linking; if actor A is linked to actor B (set by the owner at registration) then Abaco itself would automatically send actor B a message whenever an actor A execution completed or actor A's status changed (i.e., from READY to ERROR). If this is something you would use we could get it added pretty quickly.
>
> We don't currently have a formal event system capable of external notifications for Abaco. It's something we are starting to work on this year but we want it to be a centralized service capable of solving the problem for all our platforms, including Agave, Jupyter, etc. So my guess would be it likely won't be ready until the later part of 2019.

I think I'll work with it as-is right now and notify from the actor.
I like the idea of an additional actor B for notifications, esp. if ERROR,
as actor A might not be capable of sending at that point
And also since this would seem to scale better (possibility of fewer external
communications on a busy application with lots of actors via aggregating).

Once I get the basic job runs working via genapp,  will consider actor linking
and let you know.  First impression is that I will ask for it...

ref info

  • maybe thru env variable? MSG ?
    • e.g. resource prefix: docker run genapp_application_menu_module ./executable JSON
    • or docker run --env MSG=JSON genapp_application_menu_module
  • {{{docker run genapp___appplication_____menu_____module__ /genapp/bin/module JSON}}}
  • {{{docker run -it -v /tmp:/genapp/run --user www-data genapp_demo_simulate_energy /bin/bash}}}

GenApp Wiki

Clone this wiki locally