Archive Trac docs port pool

Docs Port Pool

Legacy Trac archive page imported from docs_port_pool. Source: https://genapp.rocks/wiki/wiki/docs_port_pool. Review age, links, and examples before treating as current.

port pool notes

issues

• job is running, we can have ports

• after job is finished, the server closes, need to start job from pickle

  • this needs to be figured out in detail

• if we permanently assign port to job, how do we ever get it back?

  • have to dynamically start from pickle with a new port?
  • how is that going to work with the UI
    • UI needs to be informed of the port
    • redirect via a php routine from job id for frame?
      • if job is running, use that port, if not, start pickle server for plot?
      • https://stackoverflow.com/questions/768431/how-to-make-a-redirect-in-php
  • iframe target is php server with job id and field name
  • force iframe reload
    • https://stackoverflow.com/questions/2064850/how-to-refresh-an-iframe-using-javascript
    • https://stackoverflow.com/questions/86428/what-s-the-best-way-to-reload-refresh-an-iframe

• if we have a pool assigned to active users watching the iframe, how do we release resources?

  • messaging subscribe/unsubscribe?

  • pickle server per subscriber for finished jobs

  • on subscribe:

    • if job has servers
    • if job is running - redirect to job's servers
    • if job is not running - find job's pickle and start server

  • on unsubscribe

    • if job has servers
    • if job is running - do nothing
    • if job is not running - stop server

  • if browser closes, make sure unsubscribe?

  • https://www.w3schools.com/jsref/event_onunload.asp

  • https://stackoverflow.com/questions/20853142/trying-to-detect-browser-close-event

background

• some modules fields might need to startup their own server
  • e.g. matplotlib
• we will have a mongo'd pool of servers
  • with timestamp, job info
• scanner daemon to clean up
  • or maybe simpler check at the time of assigning

assumptions

• eventually this will get tunneled to to the same port on the host server
• currently works only on localhost jobs
• https will need multiplexing

structures

• appconfig, convert resources to "run" versions and add the optional key resources:xyz:userports
  • e.g to open ports on the localhost
  • remote clusters will need tunneling

 ,"resources" : {
   "local" : {
     "run" : ""
     ,"userports" : "8000-10000 12000-16000"
    }
    ...
  }

• json input object to modules gets per field ports when field is port assigned
  • inserted by base.php using module json
  • e.g. with two output fields (id1, id2) requiring ports,

  ,"_userports" : {
     "ports" : {
        "id1" : 8000
        ,"id2" : 8001
     }
  }

• mongo record global.userports
  • _id:port, app:application jobid:uuid timestamp:lastrefresh

todo

• matplotlib api

• architecture

• handle on pickle: make pickle, try to load pickle and show

  • https://stackoverflow.com/questions/7290370/store-and-reload-matplotlib-pyplot-object
  • https://stackoverflow.com/questions/31729948/matplotlib-how-to-show-a-figure-that-has-been-closed
  • [done] saving figure of pickle
  • --> next load and show
  • alternative save/load info http://fredborg-braedstrup.dk/blog/2014/10/10/saving-mpl-figures-using-pickle/
  • probably need to understand what plt.figure() actually does, does it give us a new figure? might need to pass figure handle
  • q's plt vs figures, multiple figures etc
  • stop backend_webagg.py's print message
  • https://stackoverflow.com/questions/11130156/suppress-stdout-stderr-print-from-python-functions

• handle on ws subscriptions

  • subscribe-server.php
  • edits to msg-wsserver.php
  • are we keeping good track?
  • close, unload etc?

• assign ports

  • getports.php

• get jobs running

  • base.php
  • jobstate-server.php
  • jobserver.php
  • add elements e.g. matpolotlib.output