TODO

* -*- outline -*-

* TODO file

* Rob Peters <https://github.com/rjpcal/>

* "Make it run, make it run correctly, make it run fast"

* (05-Dec-2016) Make signal not have to derive from nub::volatile_object

* (23-Nov-2016) Make concrete_iter work with normal iterator pair semantics

* (02-Nov-2016) Use std::chrono

* (21-Nov-2002) Need a better way to pass enums to/from Tcl

  Who uses the various link*() functions now?

    gxtcl.cc GxShapeKit GxCache GxScaler GxAligner
    gltcl.cc theAttribMap
    soundtcl.cc

    gxpointset.cc has Gfx::Canvas::VertexStyle expsed as an int Field

  Maybe the simplest way is this. Make a big ugly global associative
  array of enum names. Buyer beware of name clashes. Then we have the
  string->int translators look for a string in the biguglyarray if the
  initial int extraction fails. That would be pretty darn simple.

  Downside would be that we wouldn't have any reverse lookup
  (i.e. int->string); i.e. given a command-name, figure out which
  enums it can accept. Also, we couldn't return values to Tcl in their
  string form.

  Let's see. We could have one enum-map per tcl::pkg. Then, have a
  biguglyglobalvariable that points to the current enum-map. This enum
  map would be used in the string->int translations. Plus the tcl::pkg
  could have an auxiliary command to translate int values into string
  form. The advantage here would be that we could assume that each
  integer value is unique in a given enum-map. (Hmmm... will that
  really be true for the GL enums?)

  Maybe I should just bite the bullet and implement a real Enum type,
  which would be like an int/string union, that could be passed
  to/from C++ functions. The callee's would be responsible for knowing
  where to find an EnumList (or EnumTranslator) that could do the
  translation. Then we could get the string values out to the IO files
  (but do we want that?).

** (7-Oct-2004) Hmm... what about using Tcl arrays to define enums...
   the array name could be the enum type, and the element names could
   be the enum element names?

* (19-Nov-2004) Fix redundancy of GxShapeKit::category() redundancy

  This is duplicated (and serialized twice) in several base classes,
  including Face, MorphyFace, and Fish.

* (20-Oct-2004) make media/imgfile not depend on nub::log

* (15-Oct-2004) make a DebugScheduler that can be used in TimingHdlr

  DebugScheduler stores a priority_queue, and schedules itself in the
  Tcl event loop with Tcl_DoWhenIdle().

* (13-Oct-2004) try to minimize global data masquerading as singletons

** obj_db::instance() is OK because that is really a conceptual singleton

** tcl::Interp  (from tcl::event_loop::interp())
   --> used in Toglet::Toglet (passed to tcl::TkWidget)
   --> used in tcl::TimerSchedulerToken::dummyCallback (passed to handleLiveException)
   --> used in EventResponseHdlr::Impl::Impl (saved as member variable)
   --> used in ExptDriver::Impl::Impl (saved as member variable)
   --> used in GenericEvent::GenericEvent (passed to tcl::ProcWrapper)

** Toglet::getCurrent() and GLCanvas::getCurrent()
   Most of this stuff has to do with the fact that we sometimes need a
   canvas to compute a bounding-box, in particular when we have a
   graphic object whose native size is in pixels rather than in world
   coordinates -- this would be GxPixmap, Gabor, GaborArray,
   AglRasterFont, GlxRasterFont.

   However, this creates a cumbersome burden for objects that don't
   need a world/screen mapping in order to determine their bbox; in
   particular it makes a pain for GxScaler.

   GxScaler should be doing its hard work in draw() and
   getBoundingCube(), where we already have a canvas/bbox available;
   instead, it's currently doing hard work in setWidth() and
   scaledWidth().

   We have a weird situation where a GxShapeKit can't say how
   wide/tall it is without knowing on which canvas it will be drawn.

   The underlying problem is having to call getBoundingBox() outside
   the context of a draw operation.

   Better name for "world" coordinates would be "model" coordinates.
   Better name for "screen" coordinates would be "device" coordinates.

   --> used in toglettcl.cc::currentCanvas (used as Tcl command)
   --> used in toglettcl.cc::clear_on_exit (used for setVisibility(false))
   --> used in GxScaler::setWidth (passed to getBoundingBox())
   --> used in GxScaler::setHeight (passed to getBoundingBox())
   --> used in GxScaler::scaledWidth (passed to getBoundingBox())
   --> used in GxScaler::scaledHeight (passed to getBoundingBox())

** obj_factory::instance()

   This one is a bit of a problem, because some objects rely in
   principle on getting the global current tcl::Interp. We can fix
   that by binding a particular tcl::Interp to the construction
   routine that is registered with the factory, but then we have a
   factory that depends on a particular tcl::Interp. Does that mean
   that there should instead be one factory per tcl::Interp? That
   seems to be elevating the implementation detail of the tcl::Interp
   to too high of a design status.

   --> used in obj_mgr::new_obj() / new_typed_obj()
       --> used in objNew(), objNewArgs(), objNewArr() tcl cmds
       --> used io::legacy_reader::read_weak_object()
       --> used io::object_map::fetch_object() (used by various readers)
       --> used in TimingHdlr::addEventByName()
   --> used in various tcl pkg init routines
   --> used in tcl::def_creator()

** Strategy:

   (1) Avoid storing the result of the singleton-getters.
   (2) Pass the buck back out to callers.

* (9-Oct-2004) Fix GLCanvas::grabPixels() to work with 16-bit framebuffer

  This is a problem under my YellowDog Linux install. Should be able
  to use glReadPixels with GL_UNSIGNED_SHORT_5_6_5 or similar.

  This means including a "real" bytesPerPixel functionality, that
  doesn't just assume rgba==24 or rgba==32.

  Problem areas:

  GLCanvas::isRgba()
  GLCanvas::isColorIndex()
    --> GlxOpts::rgbaFlag

  GLCanvas::bitsPerPixel()
    --> GlWindowInterface::bitsPerPixel()
        --> GlxWrapper::bitsPerPixel()
            --> XVisualInfo::depth
        --> AglWrapper::bitsPerPixel()
            --> aglDescribePixelFormat()

  Need BmapData to be able to handle bitsperpixel values other than 8
  and 24.

* (17-Jul-2004) Clean according to the way of UNIX!

  more code reuse!!! less reinventing-the-wheel!!!

  figure out how to use boost

    boost::format -- 34KLOC
    boost::iostreams -- could replace rutz::stream_buffer

  more consistent logging+debugging

* (6-Oct-2004) fix font metrics for Aqua

* (8-Jun-2004) Change EventResponseHdlr::feedbackMap to be a proc

* (2-Jun-2004) Toglet::size doesn't work for resizing

  works OK unless the toplevel window has been manually resized
  (e.g. by dragging in the window manager)

* (1-Jun-2004) Serialization goodies

** Allow default values to be skipped in XML format.

   We don't want to rely on the default values themselves to be stored
   in the source code, since the source code might use different
   default values from time to time. Instead we should have one set of
   default values per XML file. This could be a big savings in file
   size, since e.g. hundreds or thousands of GxShapeKit objects in an
   experiment will probably all have the same values.

   This would require first writing the default values to the XML file
   the first time they are seen. Then on subsequent attempts to write
   the corresponding values, a check is made to see if the value is
   the default, and if so, the element is not written to the output.

   At what API level does default-value handling appear? Can it be
   hidden entirely within a given pair of concrete io::reader
   io::writer classes? Or does there have to be an API entry at the
   level of the abstract io::reader + io::writer interfaces? Probably
   yes to the latter, since somewhere the top-level client code has to
   specify what the actual default values are.

** Enumerations

** Native array/sequence handling

   Or, at least let concrete io::reader io::writer subclasses
   determine if they can handle arrays specially or if they will just
   rely on the standard implementation provided by
   read_utils+write_utils.

** Binary data handling

   Via base64? This would allow GxPixmaps to embed their bitmaps in
   the serialization output.

** Specialized URL handling

   So that filenames aren't just treated as plain-old-strings, but get
   tagged specially. This could then be extended to allow internet
   resources, etc. Could also be extended to allow filenames to be
   stored as relative paths relative to either a homedir or some other
   specified base dir. Then when the archive is later deserialized we
   can accomodate a change in the base dir.

   Or maybe allow filenames to contain internal references to
   environment variables using either bash or tcl syntax; then we
   could have

      ${HOME}/somedir/somefile     or    $::env(HOME)/somedir/somefile
      ${IMAGELIB}/png/foo.png      or    $::env(IMAGELIB)/png/foo.png

   The type of filename could be stored as an XML attribute, either
   "absolute" or "relative", then with another optional attribute of
   "relative-to" or "base".

      <filename type="relative" base="$::env(HOME)">somedir/somefile</filename>

* (11-Oct-1999) wrapper for Tk widgets

  Could write a Widget class that encapsulates the details of Tk
  widgets. Then we could extend with subclasses... for example one
  subclass could be designed to edit Property's of another class.

  Should make some wrappers for other Tk widgets?

* (22-Nov-2002) Make tcl::Command and tcl::pkg into Util::Object subclasses?
  ... for the ultimate in reflexion!

* (13-Sep-2001) Nested containment of FieldContainer's
  in addition to inheritance.

  OK, the main issue here is how to do the "cast" that is required in
  all of the field implementations. We need to be able to cast from a
  FieldContainer* to a C*. This is easy with inheritance, since we can
  just use a dynamic cast. But, with containment, we need to be able
  to "cast" to a subobject. Probably the easiest way is to provide a
  virtual function in FieldContainer that provides access to a child,
  if any. Then we can have a wrapper cast function that first tries
  dynamic_cast, and if that fails, tries to cast a child subobject to
  the proper type.

  Aha! And of course this should be recursive.

  One problem is that this kind of things means that a given object's
  FieldMap could change if its child object changes. How do we keep
  Tcl, for example, abreast of such changes?

  Another approach... allow FieldMap to be a recursive structure. Then
  we could have named subobjects in a FieldMap, and could specify
  subfields by using composite names with a hierarchical part
  separator, such as "aligner.mode". How to pair the submaps with the
  subobjects? Answer: have a special Field type that represents the
  link to the submap.

* (25-Jan-2002) Logging goodies

  log levels, and multiple log targets with separate log levels for
  each?

* (17-Jan-2002) get automated dependencies working

  eliminate some of the cruft from the main executable and make things
  more modular+loadable

  OK, in order to allow easy building of test driver executables, we
  need to be able to figure out which libraries + object files are
  required by the test script, and specify this information in a
  makefile-compatible format.

  (1-done) Figure out on which .o files a given .cc file depends. OK,
           ldeps::get_batch can do this now.

  (2-done) Figure out where those .o files are to be found (as
           individual .o files, in a static archive, in a dynamic
           library). OK, we handle this with a regexp mapping in
           linkdeps.tcl.

  (3-done) Figure out what to do with a new .o file (keep it separate,
           or put it in a dynamic library). OK, this is handled by
           build_lib_rules.tcl.

  (4) specify these results to a makefile

  -- Aha! General principle: need a generic mapping from #include
     directives to link options

     -- local #include's just involve adding a .o file

     -- std library #include's require nothing, since it is
        automatically linked

     -- other #include's require some kind of user defined map, such
        as <engine.h> requires libmatlabengine

  -- Start simple: let's get a script to build the LIBDEEP... targets
     and dependencies for us. The only thing that the rest of the
     Makefile will depend on is the $(PROJECT_LIBS) variable.

* (24-Aug-2001) How to get animation working?

  I have a tcl::timer for generic timer callbacks now, but the problem
  is: how to inform the animated thing that it is time to stop drawing
  itself, for example when the Widget switches to displaying something
  else? I guess the appropriate model is for GxNode's the
  sigNodeChanged signal to indicate they need to be redrawn (e.g. for
  animation), and for the Widget to connect to that signal to listen
  for changes. Later on, we can try to optimize the redraws so that in
  an animation only the changed things are redrawn. OK, I've got this
  working now to animate the bounding boxes... but the next question:
  how to avoid excessive updates if multiple objects are on the screen
  at once? We need only one timer per scene, rather than one timer per
  object. [3-Sep-2001] Aha! GxNode's should just emit change events as
  necessary, everyone at their own rate... But, the Widget should also
  have a timer callback going, so that it redisplays itself only at
  some maximum frame rate. When it gets signals saying that GxNode's
  have changed, it just remembers this fact until its next redisplay
  comes around, rather than immediately redisplaying the scene. Widget
  could have a maxRefreshRate attrib, instead of allowRefresh; then
  "maxRefreshRate 0" would be equivalent to "allowRefresh false".

* (6-Jun-2001) refactoring Trial/TimingHdlr/ResponseHandler

  I'm getting the feeling these are starting to become inherently buggy.

  For example, need to make sure I haven't broken anything by changing
  trHaltExpt() to not call trAbortTrial() and trEndTrial().

  Need to have a useful way of debugging TimingHdlr/Trial sequences.

* (30-Nov-2000) a event type to install a GxNode?
  Now that screen-update is better controlled, we could introduce a
  new TrialEvent type that puts a GxNode into the Widget... this would
  give a way to implement visual feedback in an experiment.

* (21-Sep-2000) writer subclass for matlab export

  A new subclass of writer could handle exporting data in a format
  suitable for Matlab use, perhaps. Aha! A generic FormatWriter could
  take arguments describing which fields are to go into the file. In
  fact, we could just give it a format string like this:

  fwriter.setFormat("category eyeDistance eyeHeight noseLength mouthHeight");

  and it would filter out any other data. The implementation would
  actually be similar to AsciiStreamReader, using a std::map to store
  attributes as they come in from writeTo(), then generating output
  from the attributes in the map using the format string. Easier would
  be to store a map from field names to integers, where the integers
  are indices into a vector of strings that will be printed.

    MapType::iterator ii = attrib_map.find(field_name);
    if (ii != attrib_map.end())
      {
        int i = (*ii).second;
        char buf[4096];
        ostrstream ost(buf, 4096);
        ost << val << '\0';
        fields[i] = buf;
      }

  then later:

    for (int i = 0; i < fields.size(); ++i)
      itsBuf << fields[i] << sep;

  Not too much more effort would be needed to allow simple parsing of
  the format string, so that we could include both literal text, as
  well as fields to be substituted, like this:

  fwriter.setFormat("c: $category ed: $eyeDistance eh: $eyeHeight");

  ... on second thought, this might actually be quite a bit of
  work. The FormatWriter would still be quite useful even if it only
  allowed field substitutions. We start out using the '$' syntax,
  though, to allow for future expansion.

* (1-Nov-2000) Event handling by a chain of command
  passing either up or down the experiment hierarchy (i.e. Expt -->
  Block --> Trial --> EventResponseHdlr or the reverse direction).

* (22-May-2000) destroy the pause keybinding when an experiment is done
  The problem is, ExptDriver shouldn't know about the names of the Tcl
  commands that run it... therefore we don't want ExptDriver to
  create/destroy bindings to "Expt::pause".

* (15-May-2000) GOAL #5 Improve the overall timing accuracy of the software.

** (29-Nov-1999) adjust TrialEvent internal delays
   TrialEvent's could internally adjust their requested delays so that
   the actual delays converge to the requested delays. This would
   require some care to ensure that we don't send the entire system
   into an unstable oscillation. We'd maintain two values:
   itsIdealDelay, and itsRequestedDelay. The ideal delay remains
   constant, but we modify the requested delay based on experience to
   try to get the actual delay to meet the ideal delay. There are
   several strategies we could use to make these adjustments-- one
   would be to try to minimize the total timing error of the
   TrialEvent, another would be to minimize the average absolute value
   of the timing error. [Later] We could also control the timing more
   globally... where a block or experiment knows how long it is
   supposed to last, then timer events report to the block how long
   they requested, and how long they actually lasted. Then the block
   could adjust the requested times for future trial events in order
   to compensate.

** (14-May-2000) Implemented error-checking and adjustment in TrialEvent
   This gives very good accuracy (+/-1msec average per trial) over an
   experiment. Still need to control this error-checking from
   script-level, and to provide a hint to the TrialEvent at
   construction time.

** (15-May-2000) integrated timing info
   Need to include info about timing throughout the experiment object
   hierarchy, from Experiment, Block, and Trial, to TimingHdlr. Each
   object should know its allotted time, its expected error, and
   should compute its actual error and use this to give feedback to
   someone in order to improve future timing. Experiment and Block
   don't have too much control, but they can keep an overall view of
   how the timing is going, and perhaps issue a request for future
   trials to compensate their timing by a particular scale factor in
   order to adjust for past errors.

** (15-May-2000) Trial's should know their desired length
   and give feedback to their TimingHdlr's so that the TimingHdlr's
   can do a multiplicative scaling to compensate.

* (29-Nov-1999) How to automate the data analysis?

  Need to bridge the gap between output of groovx program running
  experiment, and necessary input to Matlab to process data. I think
  this is a possible route:

        1) expt completes and saves files

        2) a groovx script runs and processes the data further,
           generating the data files that matlab needs, and also
           **generating a Matlab script file (.m file)** that we can
           run to process the data.

        3) Run the .m file to finish the data processing.

  What input does this process need? We need to know the location of
  the pairs/triads experiment file, and of the classification
  experiment files.

  Breakdown of classification process:

  1) make incidence matrices (see make_inc_mat_mfaces_s50.m as
     example) from resp files.

  2) run models (see run_mfaces_models.m as example)

  Important data:

  - base directory which contains subjects' individual directories
  - name of subject (encoded as directory for subject)
  - date of pairs experiment (to create pairs directory based on date)
  - name for type of objects
  - names of response files in probe experiments

  Use case:
  1) run experiments + collect multiple data files from several subjects
  2) run mds analysis on results of similarity tasks
  3) compile subjects' overall categorization results across all clones
  4) fit various models to the categorization results

* (17-Nov-1999) automate the experiment-running process

  [later] Aha! I see that this is just yet another level of
  abstraction on top of block/experiment etc. In fact, there might be
  two abstractions: one which defines a series of experiments to be
  run together as a session (ExperimentSession?), in which the next
  experiment might depend on the results of the previous one, and a
  higher level abstraction which defines a series of sessions
  (ExperimentSeries) that constitute an entire experiment, lasting
  perhaps several weeks. An ExperimentSession probably requires only
  one process, whereas an ExperimentSeries requires multiple processes
  to be run, over the course of many days/weeks (and therefore
  requires the help of some persistent data in the form of files in
  order to maintain itself over the entire duration).

  1) 'log in' a subject ==> change to the correct directory

  2) run an experiment

  3) move the experiment data files to appropriate names and chmod
     appropriately

     --> this requires ExptDriver() to provide the names of its most
     recently saved files

  4) possibly repeat steps 2+3

     --> We can store a file that contains a list of experiment
     commands that should be issued. A separate file, called
     'next_expt' can be updated to always contain the command needed
     to run the next experiment. When one experiment is completed, the
     driver rewrites the 'next_expt' file appropriately.

  5) exit

* (17-Nov-1999) matlab automation for data analysis?
by opening a pipe to a Matlab process from within Tcl?

* (29-Sep-1999) Avoid unnecessary sendChangeMsg()'s
  It would be nice, in the Value/Property scheme, to be able to avoid
  triggering sendChangeMsg() when the value doesn't actually change
  (e.g if the new value was the same as the current value, or was out
  of bounds).

* (15-Jun-1999) Bitmaps need to have attribs for glPixelZoom()
  and other operations that can affect pixel rendering... should these
  be placed in a separate Position subclass?

* (May-1999) extract experiment+trial statistics