notes.pict3d

   
# Notes on pict3d

The definitive source code is at https://pkgs.racket-lang.org/package/pict3d
Pict3d has its own opengl binding that's for typed racket.
See source code for this.

Default:
  git://github.com/jeapostrophe/pict3d
for racket 6.1.1. AND 6.2
  git://github.com/jeapostrophe/pict3d#for-racket-v6.2.1

Package description at https://pkgs.racket-lang.org/package/pict3d

# Notes here on figuring out pict3d.


What is has to do:

make 3D objects
set attributes
render.

Rendering seems to involve;
* making the objects into triangle meshes
* creating appropriate openGL buffers
* writing the mesh data into them

The git repository commit messages keep saying
that various kinds of documentation
have been completed, but I can't find any of this documentation anywhere.

next TODO:

figure out how it writes triangles or meshes into the buffers
figure out how it constructs these buffers
figure out how it passes the buffers to openGL
figure out the shaders.
figure out how it communicates shader variables to the shaders.

This stuff seems to be done in several passes.

pict3d seems to be on github at https://github.com/ntoronto/pict3d

ASK: Is there another repository that's, meybe, more up-to-date and maintained by the racket project?  YES:  https://pkgs.racket-lang.org/package/pict3d

Is it specific to racket?

Directory structure there:

* pict3d
  * private
    * engine
      * draw
        * draw-pass.rkt
	* draw-passes.rkt
	* merge-passes.rkt
	* types.rkt
      * scene
        * scene-query.rkt
	* scene.rkt
	* tags.rkt
	* types.rlt
      * shader
        * serialize-vertices.rkt
	* shader-code.rkt
      * draw.rkt
      * scene.rkt
      * shader.rkt
      * types.rkt
      * utils.rkt
    * gl
      * buffer.rkt
      * cached-vector.rkt
      * context.rkt
      * face.rkt
      * framebuffer.rkt
      * object.rkt
      * program.rkt
      * renderbuffer.rkt
      * texture.rkt
      * typed-context.rkt
      * untyped-context.rkt
      * vertex-array.rkt
    * gui
      * shape
      * utils
      * indicators.rkt
      * invisible-context.rkt
      * master-context.rkt
      * parameters.rkt
      * pict3d-bitmap.rkt
      * pict3d-canvas.rkt
      * pict3d-combinators.rkt
      * pict3d-draw.rkt
      * pict3d-snip.rkt
      * pict3d-struct.rkt
      * typed-pict3d-canvas.rkt
      * typed-pict3d-combinators.rkt
      * typed-user-types.rkt
      * untyped-master-context.rkt
      * untyped-pict3d-bitmap.rkt
      * untyped-pit3d-canvas.rkt
      * user-types.rkt
    * math
      * fl.rkt
      * fl2.rkt
      * fl3.rkt
      * fl4.rkt
      * flaffine3.rkt
      * fllinear3.rkt
      * flplane3.rkt
      * flprojective3.rkt
      * flrect3.rkt
      * fls3.rkt
      * flv3-merge.rkt
      * flv3.rkt
      * flv4.rkt
      * maybe-flrect3.rkt
    * shape
      * sphere
        * ge_30.rkt
	* ge_32.rkt
	* sphere-type.rkt
      * composite.rkt
      * cylinder.rkt
      * directional-light.rkt
      * disk.rkt
      * frozen-scene.rkt
      * point-light.rkt
      * polygon.rkt
      * rectabgle.rkt
      * sphere.rkt
      * triangle-mesh.rkt
      * triangle-outline.rkt
      * types.rkt
    * soup
      * deform.rkt
      * geometry.rkt
      * group.rkt
      * mesh.rkt
      * query.rkt
      * types.rkt
    * universe
      * big-bang.rkt
    * engine.rkt
    * ffi-types.rkt
    * ffi.rkt
    * gl.rkt
    * lazu-gui.rkt
    * math.rkt
    * memo.rkt
    * shape.rkt
    * soup.rkt
    * untyped-utils.rkt
    * utils.rkt
  * scribblings
  * tests
  * engine.rkt
  * info.rkt
  * main.rkt
  * universe.rkt
* typed
  * opengl
    * generated -- the types of OpenGL API call as represented in this typed Racket bindig,
    * specfiles -- the files from which this API was generated.  Modified, because there seem to be errors in the official originals.
    * untyped.rkt
      This seems to be a module of macros used to define the various OpenGL functions for Racket, as well as some functions for manupulating OpenGL buffers and pointers.
    * ...

I find myself curious about pict3d-bitmap within the gui directory.
I suspect it's just something available for ordinary 2D user interfacing, but I don't really see why pict3d would need a user interface library at all.

Also a number of files seem to be available in typed and untyped versions.

And I suspect things in private/gl are important.
Not to mention things in shape, suh as triangle-mesh, triangle-outline, and polygon.

I wonder where quad is defined.  It seems to be in private/shape/polygon.rkt.  At least, that contains make-quad-shape, quad-shape, make-quad-shape, quad-shape->triangle-mesh-shape, quad-shape-mesh, set-quad-shape-color, set-quad-shape-emitted, set-quad-shape-material, ger-quad-shape-passes. get-quad-shape-bboc, quad-shaoe-transform, quad-shaoe-ray-intersect, quad-shaoe-extract-faces, quad-shaoe-tesselate, and quad-shape-deform.  quad-shaoe-functions seems to wrap these all together.

So it looks as if quad-shape is a good example of a shape to use as an example toward understanding.  Gathering together the quad-shape-funtions looks like the method table for an object.  I guess I shoudl try and figure out just what the conventions are for this.

polygon.rkt seems n=mostly to be concerned with quads, in fact.
and it uses
  * triangle-mesh.rkt, and:
  * "../math.rkt"		       
  * "../memo.rkt"
  * "../engine.rkt"
  * "../soup.rkt"
  * "../utils.rkt"
  * "types.rkt"
suggesting these have some particular importance.
Triangle-mesh obviously because everything, including quads, have to be reduced to triangles when rendered

triangle-mesh.rkt

uses
  * "../math.rkt"
  * "../memo.rkt"
  * "../engine.rkt"
  * "../soup.rkt"
  * "../utils.rkt"
  * "types.rkt"

and it gives me clues about the "passes" in the crey rare comments in this code.

  * pass1: meterial
  * pass2: color
  * triangler mesh shape passes

and operations that seem not to be passes:

  * bounding box (is this a pass?)
  * transform
  * ray intersection
  * tesselation
  * warp

and a lot gets wrapped up as triangle-mesh-shape-functions.


triangle-outline.rkt

I'm guessing this describes one triangle at a time instead of doing entore meshes.

Uses:
  * typed/opengl
  * "../math.rkt"
  * "../memo.rkt"
  * "../engine.rkt"
  * "../soup.rkt"
  * "../utils.rkt"
  * "types.rkt")

The shape seems to consist of vertices, corners, edges, width, and back?
  Vertices wouold seem to be the obvious thing.
  corners an edges are bookean, suggesting that the indicate whether corners and edges are to be explicitly drawn.
  width is a flonum, not lear what it's the width of.
  back? is probably something about face orientation.
triangle outline shape is also a shape, whihc means it has all the properties of a general shape.

There are functions to set the color, the 'emitted' (whatever that is), the material.

pass 1 material

functions for vertex-attributes, fragment-attributes, vertex code, and a lot more,

pass 2 color

functions for draw vertex attributes, draw fragment attributes,
These see to be something that piles up attribute names for opengl shaders to ues
      
Triangle mesh shpe passes

and a bunch of miscellany handlinf
* bounding box
* transform
* deform
* and a collection of all these functions.


Current source repository turns out to be https://pkgs.racket-lang.org/package/pict3d

I'll switch to that.


## 2019 11 18

I'll have a look at opengl, in pict3d/typed/opengl

### typed-opengl.scrbl

It looks as if is going to be documentation, and in a sense it is.
It includes a lot of sections that generate calling conventions for only the "non-extendion" OpenGL procedures.  You still need to find the official opengp documentation elsewhere to know what they *d8*.
However, the wrapper procedures *do* heck for errors where that is allowed, so you don't have to do that youeself.

It also describes functions for manipulating homogenous vectors.  OpenGL likes thee a lot; they are the principak means of communcating with the GPU.

### typed/opengl/generated/gl_specs.inc

typed/opengl/generated/gl_specs.inc is an inclusion that includes all the definitions of opengl-defined names,  I suspect this, too, is automatically generated.  THough ite revision message is "initial commit"/.

### pict3D/typed/opengl.rkt
  This provides ann the definitions intended for the Racket openGL user.  It does this by passing on definitions from opengl/typed.gkt and opengl/utils.rkt.

#### pict3d/typed/opengl/typed.rkt

This provides the types of the Racket opengl functions.  It dies this by including generated/gl-specr.inc after defining a bunch of macros that are heavily used.  It does the include with an (include "generated/gl_specs.inc") and the included file has no special #lang intro.

TODO: Can I do similar with my scribble problems?
  
#### pict3d/typed/opengl/readspec.rkt

This reads the standards-ommittee-provided .spec files from the specfiles directory and generates the Racket versins of them.

#### pict3d/typed/opengl/utils.rkt

This provies a bunch of typed functions that deal in strings, GL versions, innteger types, extensions, vectors, textures, shaders, bitmaps.

* This is the first time I've seen code in the pict3D repository that I've seen code relating to textures.
  * It looks as if textures were once disabled because of a problem with typed racket.  THey got re-enabled "two years ago" (as of Nov 18, 2019).

2019 12 20

this utils.rkt looks to ne==be the next thing I hve to figure out in bottom-up order.

It provides:

gl-get-string
         gl-version
         gl-shading-language-version
         gl-extensions
         gl-has-extension?
         gl-version-at-least?
         gl-core-profile?
         gl-type?
         gl-vector->type
         gl-vector->length
         gl-vector->type/cpointer
         gl-vector->type/cpointer/length
         gl-type->ctype
         gl-type-sizeof
         gl-type-alignof
         gl-vector-sizeof
         gl-vector-alignof
         bitmap->texture
         load-texture
         load-shader
         create-program

(: split-spaces (-> String (Pairof String (Listof String))))
??

(: gl-get-string (->* [Integer] [Natural] String))
returns a string describing some aspect of the GL implementation.
See https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetString.xhtml

gl-version-hash

(: parse-version-string (-> String Natural))
    converts a erino-tring to a natural numner.
gl-ersion
    returns the versino as a natural number

(: gl-shading-language-version-hash (HashTable GL-Context<%> Natural))

(: parse-shading-language-version-string (-> String Natural))

(: gl-shading-language-version (-> Natural))

(: gl-extensions-hash (HashTable GL-Context<%> (Setof Symbol)))

(: gl-extensions (-> (Setof Symbol)))

(: gl-has-extension? (-> Symbol Boolean))

(: gl-version-at-least? (-> Natural Boolean))

(: gl-core-hash (HashTable GL-Context<%> Boolean))
;;

(: gl-core-profile? (-> Boolean))
??

(define gl-types
  (make-hasheqv 
   (list (cons GL_UNSIGNED_BYTE _uint8)
         (cons GL_BYTE _sint8)
         (cons GL_UNSIGNED_SHORT _uint16)
         (cons GL_SHORT _sint16)
         (cons GL_UNSIGNED_INT _uint32)
         (cons GL_INT _sint32)
         (cons GL_FLOAT _float)
         (cons GL_DOUBLE _double)))
a hashtable assigning types to the integer GL uses to identify types.
?? Are these Racket types?

(: gl-type? (-> Integer Boolean))
Sees if the integer is a key in the gl-types hashtable.

(: gl-vector->type (-> GLVector Integer))
?? Seems to provide an OpenGL type encoded as integer from the GLVector.
?? What is a GLVector?  Presumable a Racket handle for something GL knows?

(: gl-vector->length (-> GLVector Index))

(: gl-vector->type/cpointer (-> GLVector (Values Integer CPointer)))



(: gl-vector->type/cpointer/length (-> GLVector (Values Integer CPointer Index)))

(: gl-type->ctype (-> Integer CType))
(: gl-type-sizeof (-> Integer Index))
(: gl-type-alignof (-> Integer Index))
(: gl-vector-sizeof (-> GLVector Index))
(: gl-vector-alignof (-> GLVector Index))

;;; Utility functions for dealing with textures

;; Convert argb -> rgba, and convert to pre-multiplied alpha.
;; (Non-premultiplied alpha gives blending artifacts and is evil.)
;; Modern wisdom is not to convert to rgba but rather use 
;; GL_BGRA with GL_UNSIGNED_INT_8_8_8_8_REV. But that turns out not
;; to work on some implementations, even ones which advertise
;; OpenGL 1.2 support. Great.

(: argb->rgba! (-> Bytes Void))
Rearranges the pixel format in an array pf bytes, presumably representing pixels.

(: bitmap->texture (-> (Instance Bitmap%) [#:mipmap Any] [#:repeat (U 'none 'x 'y 'both)] Natural))
;; Convert a Racket bitmap into an OpenGL texture (with lots of default settings)
TODO: thi is a long function with subfunctions.  Probably worth looking at as an example of dealing with OpenGL compleications.

(: load-texture (-> Path-String [#:mipmap Any] [#:repeat (U 'none 'x 'y 'both)] Natural))

;;; Utility functions for dealing with shaders
TODO: figure out what this all means.

(: get-shader-parameter (-> Natural Integer Integer))
(: get-shader-info-log (-> Natural String))
(: get-program-parameter (-> Natural Integer Integer))
(: get-program-info-log (-> Natural String))
(: load-shader-source (-> Natural Input-Port Void))
(: load-shader (-> (U Path-String Input-Port) Natural Natural))
(: create-program (-> Natural * Natural))

TODO: Picr3D uses shaders.  Where are they?  Can they be jiggered to talk to textures?  Are they perhaps already capable of dealing with textures, although the higher-level code doesn't do anything with textures?

pict3d/private/engine/shader/shader.rkt seems to contain the shaders, or perhaps, components of shaders still to be put together.
And the shader code seems related to .../shader/serialize-vertices.rkt

TODO: Why is shader.rkt writen in plain racket, but shader-code in tyepd racket?

2019 12 21

#### pict3d/pict3d/private/engine/shader/serialize-vertices.rkt

A bunch of functions to put different things into buffers.
They appear to check for buffer overflow, but don't do anything about it
except for raising an error,
Except for serialize-byte and serialize-bytes, which do no such check.
They are presumably intended to be called only from the other functions that do check.

#### pict3d/pict3d/private/engine/shader/shader-code.rkt

The real work of opengl seems to be done in shaders nowadays.
So pict3d contains a raft of functions to keep track of shaders, and a library of specific shaders.

There are a series of attribute functions, presumably to give GL shader attributes names and define them to openGL.

Then there's a struct partial-code.  It seems to provide some kind of a manifest for shader code, to be used by Racket to keep track of what it's doing.
partial-code does seem to be a tree struture, in that it "includes" a list of partial-code.  It might contain the actual code in its definitions field, which is a Listof String.

There are functions to recursively run through the partial-code tree structure and collect things from it.

There are to have a lot of shader code fragments, stored as text, coded as strings delimited by #<<code and terminated by just the word code.  These code pieces seem to be built into something like large-scale abstract syntax, presumaby to be unravelled before being passed to OpenGL.

The tree traversal would appear to be done the functions
  (: partial-code-all-includes (-> partial-code (Listof partial-code)))
  
  (: partial-code->string/no-includes (-> partial-code String))
  
  (: partial-code->string (-> partial-code String))
  (: shader-code->string (-> shader-code String))
  (: standard-uniform->pair (-> standard-uniform (Pair String Symbol)))
  (: partial-code-standard-uniform-pairs (-> partial-code (Listof (Pair String Symbol))))
  (: partial-code-all-in-attributes (-> partial-code (Listof attribute)))
  (: partial-code-plain-in-attributes (-> partial-code (Listof attribute)))
  (: vertex-code-vao-struct (-> vertex-code vao-struct))

2019 12 29

Then there are "convenience functions"

that make partial code, make-vertex-code, make-geometry-code, make-fragment-code

And then the "programs"

the actual struct declarations for program code, and make-program-code.
TODO: where is the structure declaration for the other code?

And then a big list (yes, a list) for program-code-shaders.

And a "standard shader library"

Here's where we find the actual code blocks.



The standard shader library:

Each entry seems to be the value of a Racket identifier, and contains
  * a string that is presumably a name used for identification and lookup by the Racket code,
  * a list of pieces o actual shader code as a code block (presumably a string).

Many have Racket names constructed by replacing underscores by hyphens and appending "-code".
The 
* matrix-code -- Shader code fragments for matrix handling
  * "matrix"
    rows2mat4x3, a2p, linear_inverse, affine_inverse

Next are a bunch of definitions defining shader code.
Most has a Racket name consisting of the shader-code name with underscores replaced by hyphens and followed by "-code".

* model-vertex-code
  * "model-vertex"
  * get_model_transform
* rect-code
  * "rect"
  * struct rect
* output-2d-rect-vertex-code
  * "output-2d-rect-vertex"
    output_2d_rect_vertex

Similarly,

* output-unit-cube-vertex-code
* output-unit-quad-vertex-code

* float-constants-code
  pos-infinity, neg-infinty, epsilon
* impostor-bounds-code -- What are immpostor bounds?
  Seems to be various upper and lower bounds, clipping, etc.
  But what is the 8-element array 'vs' for?
  
* output-impostor-vertex-code -- ???

* pack-unpack-normal-code -- These all seem to be coordinate transformations
  * pack-unpack-normal
    signNotZero
    float32x3_to_oct -- oct seems to be an octahedron, possibly a stand-in for a sphere.
    oct_to_float32x3
    snorm12x2_to_unorm8x3
    unorm8x3_to_snorm12x2
    oct_to_float32x3
    snorm12x2_to_unorm8x3
    unorm8x3_to_snorm12x2
    unpack_normal
    pack_normal

* depth-fragment-code
  * "depth-fragment"
  float get_frag_depth, get_view_depth

* surface-code
  "surface"
  struct surface

* get-surface-fragment-code
  "get-surface-fragment"
  get_surface

Different kinds of surface reflectivity

* light-fragment-code
  frag_coord_to_position, attenuate_invsqr_quad, specular, output_light
* output-mat-fragment-code
  output_mat
* output-opaq-fragment-code
  output_opaq
* output-tran-fragment-code
  output_tran(
* unpack-emitted-code
  unpack_emitted
* ray-trace-fragment-code "ray-trace-fragment"
  frag_coord_to_direction, unit_sphere_intersect, unit_disk_intersect, unit_cylinder_intersect
* rgb-hsv-code
  rgb_to_hsv, hsv_to_rgb


2019 12 30

It looks like I'm really going to have to learn more about the shader language to understand how it is embedded into Racket.

2020 02 04

Happy New Year

Time to look at Racket packages again.  THis tim searching for ones that relate to opengl.

get-bonus: an ancient looking experiment in 2-D video games by Jay McCarthy.
    fails to compile

glm: mthematics for GLSL, the OpenGL shading language.  A lot of vector and matrix math.

glsl: the OpenGL shader language.  Compiles and passes tests.  Has a missing dependency.  Not clear what this does.  It seems to define a concrete and/or abstract syntax, likely for glsl.  Not clear what this is for; opengl takes shader code as strings, so why would Racket need shader parse trees?

graphics-engine:  has failing tests.  Not aat all clear what this is supposed to do.  The documentation just says "Helo world".  The first example I looked at doesn'f seem to require anythong within the graphics-engine package.  But it is an example how to draw a triangle.
  It depeds on glsl, so maybe it'a a clus as to what glsl does.
  
mode-lambda: a 2d sprite-based graphics engine.  Passes its tests, a blessed relief.  And there's actually documentation.

opengl:  Compiles, but has failing test (possibly because an automated tester doesn't have any graphics output mechanism?) Machine-generated OpenGl bindings,  Contains some utilities for managing bitmaps and the like,
  TODO:  Is this just an untyed version of opengl/typed foud in pict3d?

pict3d:  Compiles, but ails tests. 3d graphis base on solid modelling, but without intersection of difference, and without textures.

pict3d-die-cut:  Generates text and 2-D shapes from paths.  Presumably makes them for use in pict3d.

pict3d-orig: same as pict3d, but a much older version.

vulcan:  low-level bindig for vulkan, a new graphics system to replace opegl.


I might possible learn to use some 3D in Racket by looking at the examples in the opengl package.  It might lead to insights for the documentatino and use of pict3d.  Or it might completely obviate this pict3d project.

-- hendrik

I wonder why pict doesn't seem to be on the list of Racket packages on https://pkgs.racket-lang.org/index.html