ms-rfc-3.txt

.. _rfc3:

==============================================
MS RFC 3: Feature Layer Plug-in Architecture
==============================================

:Date:  2005/08/22
:Author: Jani Averbach
:Contact: javerbach at extendthereach.com
:Status: Adopted
:Version: MapServer 4.8

Abstract Solution
-----------------

Implement a virtual table structure for ``layerObj``. This structure
will contain function pointers for all layer specific operations.
This structure will be populated when the layer is opened or first
time accessed.  After that all back-end format specific layer
operations will happen through this vtable which is cached in the
``layerObj`` struct.


Technical Solution
------------------

All file names and numbers are against released MapServer 4.6.0 source
code.

1. Add new field to the ``layerObj``. It will be pointer to the vtable,
   which will contain function pointers for this layer.

2. Add the virtual table architecture

   The vtable will be initialized when the layer is accessed at the
   first time. 
   The vtable will be populated with dummies when it is created and
   these dummies will implement feasible default actions if there is
   any (e.g. do nothing) or return error by default.


   2.1. Standard functions which are found currently from ``maplayer.c``

        2.1.1. InitItemInfo

               ::

                 int (*LayerInitItemInfo)(layerObj *layer);

                             
        2.1.2. FreeItemInfo

               ::

                  void (*LayerFreeItemInfo)(layerObj *layer);


        2.1.3. Open

               ::

                  int (*LayerOpen)(layerObj *layer);

               Currently there are two layers which accept more than
               the generic layer arg for LayerOpen function:

               :: 

                  int msOGRLayerOpen(layerObj *layer, 
                                     const char *pszOverrideConnection);
                  int msWFSLayerOpen(layerObj *lp, 
                                     const char *pszGMLFilename, 
                                     rectObj *defaultBBOX);
                                     
               However, these are called from ``msLayerOpen`` with
               ``NULL`` args, so I think that proposed interface for
               this virtual table function should be fine.


        2.1.4. IsOpen

               ::

                  int (*LayerIsOpen)(layerObj *layer);


        2.1.5. WhichShapes

               ::

                  int (*LayerWhichShapes)(layerObj *layer, 
                                          rectObj rect);

        2.1.6. NextShape

               ::

                  int (*LayerNextShape)(layerObj *layer, shapeObj *shape);


        2.1.7. GetShape

               ::

                  int (*LayerGetShape)(layerObj *layer, shapeObj *shape, 
                                       int tile, long record);


        2.1.8. LayerClose

               ::

                  int (*LayerClose)(layerObj *layer);


        2.1.9. LayerGetItems

               ::

                  int (*LayerGetItems)(layerObj *layer);


        2.1.10. GetExtent

                ::

                   int (*LayerGetExtent)(layerObj *layer, 
                                         rectObj *extent);


        2.1.11. GetAutoStyle

                ::

                   int (*LayerGetAutoStyle)(mapObj *map, layerObj *layer,
                                            classObj *c, int tile, 
                                            long record);
                


   2.2.  New functions and/or fields for vtable

         2.2.1. CloseConnection

                This function is used to actually close the
                connection, in case that layer implements some kind of
                connection pooling by its own.  If layer doesn't use
                any connection pooling, this function should be
                implemented as no-op. Caller should first call layer's
                "close" function, and finally at the very end
                ``CloseConnection``.

                The signature of function is

                ::

                   int (*LayerCloseConnection)(layerObj *layer);


                And the main place where this function will be called,
                is ``mapfile.c: 4822``

                ::
                   
                   void msCloseConnections(mapObj *map)

                This function is needed because e.g. ``POSTGIS`` is
                implementing this usage pattern at the moment
                ``maplayer.c:599``

                ::

                    void msLayerClose(layerObj *layer)
                    ...
                   /*
                    * Due to connection sharing, we need to close the results
                    * and free the cursor, but not close the connection.
                    */
                    msPOSTGISLayerResultClose(layer);


         2.2.2. SetTimeFilter

                This function is used to create a time filter for
                layer. At the moment we have three special cases
                (``maplayer.c: 1635``):

                1. ``POSTGIS`` with it's own function
                2. Layers with backticks delimited expressions
                3. Layers without backticks

                The idea is provide a generic helper function,

                ::

                   int makeTimeFilter(layerObj *lp,
                                      const char *timestring,
                                      const char *timefield,
                                      const bool bBackTicks)

                And the actual layer's ``SetTimeFilter`` could use the
                above, or implement something totally different as
                ``POSTGIS`` is doing at the moment.

                The signature for layer's vtable function is

                ::

                   int (*LayerSetTimeFilter)(layerObj *lp, 
                                             const char *timestring, 
                                             const char *timefield);



   2.3. Extra functions to add to the vtable

        2.3.1. FLTApplyFilterToLayer (``mapogcfilter.c: 1084``)

               This is the main filter interface for layers.  We will
               provide two helper functions, one for "SQL" case and
               the another for "non-SQL" case, and set all layers,
               except ``POSTGIS``, ``ORACLESPATIAL`` and ``OGR`` to
               call directly this "non-SQL" version of this helper
               function (else-branch of if).

               ``ORACLESPATIAL``, ``POSTGIS`` and ``OGR`` could use
               "SQL" version of the helper function (actual if-branch)
               if the conditions for this are met, otherwise they will
               use Non-SQL version of the function.


        2.3.2. layerObj->items allocation

               There will be vtable function for allocation ``items`` and
               initialization for ``numitems``. If layer has some
               special needs for these objects it can override default
               function.

               The signature for function will be:

               ::

                  int (*CreateItems)(layerObj *) 

               which does the allocation and set numitems to correct
               value.

        2.3.3. msCheckConnection (``mapfile.c: 4779``)

               This API is deprecated. It is called only from
               ``msMYGISLayerOpen``.  We will not provide this
               functionality through vtable.


   2.4. Interface functions for internal layers

        We have to add some new interface functions to access layers.

        2.4.1 Function interface to initialize internal layer type

              We need a per layer type a function which will be called
              when the layer's vtable has to be initialized. These
              functions will be

              ::
                 
                 int msXXXInitializeLayerVirtualTable(layerObj *)

              where XXX will be name of the layer. This function is
              called anytime when the vtable isn't initialized and the
              layer is accessed at the first time.


        2.4.2 Function interface to change the connectiontype of layer
              
              To change the connection type of layer, it has to be
              done by function interface. Accessing directly
              ``connectiontype`` field is not supported anymore.

              To change the connectiontype and to connect to the new
              layer, there will be following interface function

              ::

                 int msConnectLayer(int connectiontype,
                                    const char *library_str)

              where ``connectiontype`` is the type of new layer, and
              ``library_str`` is the name of library which will
              prodive functionality for this new layer.  For internal
              layer types this second argument is ignored.



3. Remove unwanted interfaces
   
   Frank Warmerdam proposed [FW1]_ that we remove all layer specific
   interface functions from ``map.h``.

      I see each "built-in" module such as mapsde.c providing a
      registration function such as "msSDEInitializeLayerVirtualTable"
      so that none of the layer type specific definitions need to
      appear in map.h any more.

   .. [FW1] 
      | ``Frank Warmerdam on the mapserver-dev:``
      | ``Message-Id: <smtpd.490f.4303f2ee.d8246.1@mtaout-c.tc.umn.edu>``
      | ``Date:       Wed, 17 Aug 2005 22:31:09 -0400``
      | ``Subject:    Re: MapServer  Plug-in Infastructure: RFC and PATCH``




Files and objects affected
--------------------------

This proposal will affect at least following files and objects:

* ``map.h``

  * ``layerObj`` will contain new fields. There will be a new
    object ``vtableObj`` in the ``map.h``.


* ``maplayer.c`` 

  * Various changes, layer specific ``switch``-statements will go
    away, vtable handling and layers vtable initialization will be
    added.


* ``mapfile.c``

  * Cleaning of  ``msCheckConnection``
  * Vtable for ``msCloseConnection``


* ``mapogcfilter.c``

  * Remove layer-logic from ``FLTApplyFilterToLayer``


* ``mapXXX.c``, where ``XXX`` is the name of layer.

  * Add new initialization function
  * Add all new interface functions
  * Fix existing interface functions, if needed / add wrappers for
    ``msOGRLayerOpen`` and ``msWFSLayerOpen``. 



Backwards compatibility issues
------------------------------

This is binary and source code level backward incompatible change. The
proposal will remove some previously public functions, and add new
field(s) to the ``layerObj`` struct. 

This proposal is not MapScript backward compatible, it will break
scripts which change directly ``connectiontype`` field in
``layerObj``.

The proposal is MAP-file backward compatible.


Implementation Issues
---------------------

* Biggest problem is probably that the author has ignored or missed
  something by oversight which will show up during implementation.
  However, there is a prototype implementation of external plug-in
  architecture which works at the moment and is based on ideas
  presented in this proposal.  So there is some real life experience
  that this architecture change is feasible thing to do.

* I also like to note that this proposal won't remove all layer
  specific code from MapServer e.g. ``WMF``, ``WMS`` and ``GRATICULE``
  are used as special cases from place to place.



Bug ID
------

Bug 1477_
 
.. _1477: https://github.com/mapserver/mapserver/issues/1477



Voting history
--------------

Vote proposed by Jani Averbach on 9/19/2005, result was +4
(3 non-voting members).

Voting +1: Howard Butler, Frank Warmerdam, Yewondwossen Assefa, Daniel Morissette

Proposal passes and will move forward.


Open questions
--------------

* How do we like to expose layer's virtual table to the layers. We
  have at least two different routes to go:

  * expose it as a struct, layers will fill vtable pointers by
    accessing directly struct's field.

  * expose it as a complete opaque type, vtable pointers will be set
    by accessing functions ``setLayerOpenFP``, ``setLayerCloseFP`` and
    so on.

  The advance of second option is that this way we could easily add
  new functions to the struct if we refactor code more or found some
  logic which is ignored by oversight in this proposal.


* Are there any special issues with the raster query layer support
  which is handled via the layer API?



.. Local Variables: 
   mode: text
   End: