Plan out API for dataframe.

doc/nnc-cnnp.rst

@@ -9,3 +9,12 @@ Model
 -----
 
 **Model** is the core abstraction in **common neural network primitives (CNNP)** interface. It can represent both a layer or a group of layers. An ordinary neural network layer contains parameters, and applies the parameters to input neurons to generate activations in output neurons. The **model** abstraction goes beyond one input and one output. It can take multiple sets of input neurons, and generate activations on multiple sets of output neurons. The main difference between a **model** and a **command** in a concrete graph is that the **model** contains states (parameters).
+
+The **model** itself is incredibly flexible. You don't need to know any shape of the inputs or outputs to compose a model. **Models** are composable. The most simple way to compose a new model from a list of models is to use :cpp:func:`ccv_cnnp_sequential_new`. This function takes a list of models, and runs activations through them sequentially (the input of current model is the output of the model prior). Alternatively, :cpp:func:`ccv_cnnp_model_new` can compose a new model out of a set of model inputs and outputs. More on this in `Model IO`_.
+
+The **model** concept is meta in a sense that a **model** is not materialized until you call :cpp:func:`ccv_cnnp_model_compile` with tensor inputs / outputs parameters. Internally, this method will materialize **model** into a symbolic graph that has proper shape. After a **model** compiled, either evaluate the **model** against inputs or train the **model** is possible.
+
+Model IO
+--------
+
+Composing a **model** with :cpp:func:`ccv_cnnp_model_new` requires model inputs and model outputs. The concept of model inputs / outputs is remarkably similar to tensor symbols. In this case, it is broader. Ordinarily, :cpp:func:`ccv_cnnp_input` gives a :cpp:any:`ccv_cnnp_model_io_t` that represents a tensor as input. When :cpp:func:`ccv_cnnp_model_apply` called with a **model** and set of inputs, its :cpp:any:`ccv_cnnp_model_io_t` output represents a set of tensors generated by applying inputs against the said **model**. Thus, :cpp:any:`ccv_cnnp_model_io_t` can conceptually both be a single tensor and a set of tensors. For the given model inputs and outputs, a set of **models** that are used to generate the outputs from the inputs can be traced to compose a new **model**. This also means a composed **model** can be used to compose a more complex **model**. In this way, the model IO abstraction is very natural to compose ever complex **models**.

doc/nnc-dy.rst

@@ -17,7 +17,7 @@ NNC's **dynamic graph execution** design will attempt to address above problems
 Naming The Variable
 -------------------
 
-Like in most frameworks, **dynamic graph execution** in NNC operates at variables. **Dynamic graph** executes command on a set of input variables, writes the result to a set of output variables. Variables can be inspected anytime with ``ccv_nnc_tensor_from_variable``. The underlying tensor may not be allocated when the variable is created. ``ccv_nnc_tensor_variable_t`` is an opaque structure and its inner work shouldn't be of an interest to users.
+Like in most frameworks, **dynamic graph execution** in NNC operates at variables. **Dynamic graph** executes command on a set of input variables, writes the result to a set of output variables. Variables can be inspected anytime with :cpp:func:`ccv_nnc_tensor_from_variable`. The underlying tensor may not be allocated when the variable is created. :cpp:any:`ccv_nnc_tensor_variable_t` is an opaque structure and its inner work shouldn't be of an interest to users.
 
 Tracing The Operation
 ---------------------
@@ -31,12 +31,12 @@ Upon *automatic differentiation*, no tape is used (or, the **symbolic graph** se
 Optimizations
 -------------
 
-In **PyTorch**, there is a need to ``requires_grad`` such that the framework knows which variable should be discarded to save memory. If it is not done carefully, the memory usage can grow unbounded. **Dynamic graph** here provides ``ccv_nnc_tensor_variable_free`` where when a tensor variable is freed, we will release its memory when it is safe. This method meant to hook up with object finalization methods in host languages (C++'s destructor, Objective-C's ``dealloc``, ``deinit`` in Swift, ``finalize`` in Java, ``tp_dealloc`` in Python).
+In **PyTorch**, there is a need to ``requires_grad`` such that the framework knows which variable should be discarded to save memory. If it is not done carefully, the memory usage can grow unbounded. **Dynamic graph** here provides :cpp:func:`ccv_nnc_tensor_variable_free` where when a tensor variable is freed, we will release its memory when it is safe. This method meant to hook up with object finalization methods in host languages (C++'s destructor, Objective-C's ``dealloc``, ``deinit`` in Swift, ``finalize`` in Java, ``tp_dealloc`` in Python).
 
 Interoperability (Not Ready)
 ----------------------------
 
-There are some sticky issues with interoperability between **static graph** (the **symbolic graph** we formed by hand) with **dynamic graph**. The way they interoperate is through ``CCV_NNC_CUSTOM_FORWARD`` / ``CCV_NNC_CUSTOM_BACKWARD`` functions. When a **static graph** includes a **dynamic graph**, its tape needs to book-keeping for the **dynamic graph**. When a **dynamic graph** includes a **static graph**, it also needs to create a tape at that point for the execution. All these implies significant changes for the ``ccv_nnc_tensor_tape_t`` implementation to accommodate these new requirements.
+There are some sticky issues with interoperability between **static graph** (the **symbolic graph** we formed by hand) with **dynamic graph**. The way they interoperate is through ``CCV_NNC_CUSTOM_FORWARD`` / ``CCV_NNC_CUSTOM_BACKWARD`` functions. When a **static graph** includes a **dynamic graph**, its tape needs to book-keeping for the **dynamic graph**. When a **dynamic graph** includes a **static graph**, it also needs to create a tape at that point for the execution. All these implies significant changes for the :cpp:any:`ccv_nnc_tensor_tape_t` implementation to accommodate these new requirements.
 
 Future Optimizations
 --------------------

doc/nnc-schd.rst

@@ -1,7 +1,7 @@
 NNC Static Schedule A Graph
 ===========================
 
-A concrete graph runs in topological order sequentially when you invoke ``ccv_nnc_graph_run``. Thus, all dependencies executed before the subsequent command got executed. This default behavior doesn't leverage massive parallelism built in today's CPU / GPU environment, leaves too much computation power on the table.
+A concrete graph runs in topological order sequentially when you invoke :cpp:func:`ccv_nnc_graph_run`. Thus, all dependencies executed before the subsequent command got executed. This default behavior doesn't leverage massive parallelism built in today's CPU / GPU environment, leaves too much computation power on the table.
 
 NNC supports to static schedule a graph such that all the parallelism are utilized. That's been said, **static scheduling** shouldn't be confused with **automatic parallelization**. The later term suggests to turn a non-parallelizable graph into a parallelizable ones by adding additional data transfers or graph transformations. **Static scheduling** considers dependencies of a command, and try to schedule independent commands onto different streams.
 

lib/nnc/ccv_nnc.h

@@ -1973,10 +1973,47 @@ void ccv_nnc_dynamic_graph_dot(const ccv_nnc_dynamic_graph_t* const graph, const
  * 5. All these can be done efficiently, on a scale of hundreds of Gigabytes data.
  */
 
+/**
+ * @defgroup level_5_dataframe Dataframe API
+ * @{
+ */
+
+/**
+ * A data enumeration function to supply data for a given row index (and length).
+ */
+typedef void (*ccv_cnnp_column_data_enum_f)(const int row_idx, const int row_size, void** const data, const int column_idx, void* const context);
+/**
+ * Column data.
+ */
+typedef struct {
+	ccv_cnnp_column_data_enum_f data_enum;
+	void* context;
+} ccv_cnnp_column_data_t;
+/**
+ * An opaque structure point to the dataframe object.
+ */
 typedef struct ccv_cnnp_dataframe_s ccv_cnnp_dataframe_t;
-CCV_WARN_UNUSED(ccv_cnnp_dataframe_t*) ccv_cnnp_dataframe_new(void);
+/**
+ * Create a dataframe object with given column data.
+ * @param column_data The column data that can be loaded.
+ * @param column_size The size of column data array.
+ */
+CCV_WARN_UNUSED(ccv_cnnp_dataframe_t*) ccv_cnnp_dataframe_new(const ccv_cnnp_column_data_t* const column_data, const int column_size);
+/**
+ * Derive a new column out of existing columns in the dataframe.
+ * @param dataframe The dataframe object that contains existing columns.
+ * @param column_idxs The columns that will be used to derive new column.
+ * @param column_idx_size The size of existing columns array.
+ * @param context The context that can be used to generate new column.
+ */
+CCV_WARN_UNUSED(int) ccv_cnnp_dataframe_map(ccv_cnnp_dataframe_t* const dataframe, const int* const column_idxs, const int column_idx_size, void* const context);
+/**
+ * Free the dataframe object.
+ */
 void ccv_cnnp_dataframe_free(ccv_cnnp_dataframe_t* const dataframe);
 
+/** @} */
+
 /**
  * @page model Model
  *
@@ -2001,6 +2038,11 @@ void ccv_cnnp_dataframe_free(ccv_cnnp_dataframe_t* const dataframe);
  * evaluation path.
  */
 
+/**
+ * @defgroup level_5_model Model API
+ * @{
+ */
+
 /**
  * model type is an abstract type, you won't interact with a naked model ever.
  */
@@ -2222,4 +2264,6 @@ CCV_WARN_UNUSED(ccv_cnnp_model_t*) ccv_cnnp_flatten(void);
 
 /** @} */
 
+/** @} */
+
 #endif

lib/nnc/ccv_nnc_symbolic_graph_parallel.c

@@ -119,6 +119,7 @@ void ccv_nnc_symbolic_graph_data_parallel(ccv_nnc_symbolic_graph_t* const graph,
 		}
 	} ccv_nnc_graph_visit_endfor
 	const int parallel_count = parallel;
+	assert(parallel_count > 1);
 	// Now, actually create these tensors.
 	if (!graph->data_parallel.tensor_symbol_idx)
 		graph->data_parallel.tensor_symbol_idx = (int*)ccmalloc(sizeof(int) * (parallel_count - 1) * tensor_symbol_size);
@@ -176,7 +177,7 @@ void ccv_nnc_symbolic_graph_data_parallel(ccv_nnc_symbolic_graph_t* const graph,
 	// dup_exec_idx is the array starts with 0 here.
 	for (i = 0; i < (parallel_count - 1) * graph_exec_symbol_size; i++)
 		dup_exec_idx[i] = -1;
-	int max_io_size = 0;
+	int max_io_size = 1;
 	for (i = 0; i < dup_execs->rnum; i++)
 	{
 		const int d = *(int*)ccv_array_get(dup_execs, i);