[API Change, OPTIMIZATION] Only process conns that need to be processed

The API is simplified: do not expose the user code to several
queues.  A "connection queue" is now an internal concept.
The user processes connections using the single function
lsquic_engine_process_conns().  When this function is called,
only those connections are processed that need to be processed.
A connection needs to be processed when:

    1. New incoming packets have been fed to the connection.
    2. User wants to read from a stream that is readable.
    3. User wants to write to a stream that is writeable.
    4. There are buffered packets that can be sent out.  (This
       means that the user wrote to a stream outside of the
       lsquic library callback.)
    5. A control frame (such as BLOCKED) needs to be sent out.
    6. A stream needs to be serviced or delayed stream needs to
       be created.
    7. An alarm rings.
    8. Pacer timer expires.

To achieve this, the library places the connections into two
priority queues (min heaps):

    1. Tickable Queue; and
    2. Advisory Tick Time queue (ATTQ).

Each time lsquic_engine_process_conns() is called, the Tickable
Queue is emptied.  After the connections have been ticked, they are
queried again: if a connection is not being closed, it is placed
either in the Tickable Queue if it is ready to be ticked again or
it is placed in the Advisory Tick Time Queue.  It is assumed that
a connection always has at least one timer set (the idle alarm).

The connections in the Tickable Queue are arranged in the least
recently ticked order.  This lets connections that have been quiet
longer to get their packets scheduled first.

This change means that the library no longer needs to be ticked
periodically.  The user code can query the library when is the
next tick event and schedule it exactly.  When connections are
processed, only the tickable connections are processed, not *all*
the connections.  When there are no tick events, it means that no
timer event is necessary -- only the file descriptor READ event
is active.

The following are improvements and simplifications that have
been triggered:

    - Queue of connections with incoming packets is gone.
    - "Pending Read/Write Events" Queue is gone (along with its
      history and progress checks).  This queue has become the
      Tickable Queue.
    - The connection hash no longer needs to track the connection
      insertion order.

APIs.txt

@@ -127,44 +127,35 @@ drive QUIC connections:
        ea_packets_out callback.
 
 
-Connection Queues
------------------
-
-Each connection lives in one or more queues.  These are:
-
-    - "All" queue.  This is not really a queue, but rather a hash where
-      connections can be looked up by ID.  It is possible for a connection
-      to exist *outside* of this queue: this happens when the connection is
-      closed, but still has packets to send.  In this case, the connection
-      is present in
-    - Outgoing queue.  This queue contains connections which have packets
-      to send.  The connection in this queue are ordered by priority: the
-      connection that has gone longest without sending is first.
-    - Incoming queue.  This queue contains connections that have incoming
-      packets.
-    - Pending RW Events queue.  (See Connection Read-Write Events).
-    - Advisory Tick Time queue.  This queue is used when packet pacing is
-      turned on (see es_pace_packets option).
-
-Each of these queues can be processed by a specialized function.  They are,
-respectively:
-
-    - lsquic_engine_proc_all()
-    - lsquic_engine_send_unsent_packets()
-    - lsquic_engine_process_conns_with_incoming()
-    - lsquic_engine_process_conns_with_pend_rw()
-    - lsquic_engine_process_conns_to_tick()
-
-Processing, or "ticking," a connection removes it from Incoming, Pending
-RW Events, and Advisory Tick Time queues.  The connection gets placed
-onto these queues as necessary.
-
-A simple approach is to
-    - Read packets from socket, give it to the engine using
-      lsquic_engine_packet_in(), and call
-      lsquic_engine_process_conns_with_incoming(); and
-    - Call lsquic_engine_proc_all() every few dozen milliseconds.
+Connection Management
+---------------------
 
+A connections needs to be processed once in a while.  It needs to be
+processed when one of the following is true:
+
+    - There are incoming packets;
+    - A stream is both readable by the user code and the user code wants
+      to read from it;
+    - A stream is both writeable by the user code and the user code wants
+      to write to it;
+    - User has written to stream outside of on_write() callbacks (that is
+      allowed) and now there are packets ready to be sent;
+    - A timer (pacer, retransmission, idle, etc) has expired;
+    - A control frame needs to be sent out;
+    - A stream needs to be serviced or created.
+
+Each of these use cases is handled by a single function:
+
+    lsquic_engine_process_conns()
+
+The connections to which the conditions above apply are processed (or
+"ticked") in the least recently ticked order.  After calling this function,
+you can see when is the next time a connection needs to be processed using
+
+    lsquic_engine_earliest_adv_tick()
+
+Based on this value, next event can be scheduled (in the event loop of
+your choice).
 
 Connection
 ----------
@@ -235,12 +226,3 @@ Versions
 
 QUIC version are listed in enum lsquic_version.  To specify a list of
 versions, they are usually placed in a bitmask, e.g. es_versions.
-
-
-Connection Read-Write Events
-----------------------------
-
-TODO.
-
-(Do not worry about it if you are not writing to streams outside
-of on_write() callback.)

CHANGELOG

@@ -1,3 +1,61 @@
+2018-04-09
+
+    [API Change, OPTIMIZATION] Only process conns that need to be processed
+
+    The API is simplified: do not expose the user code to several
+    queues.  A "connection queue" is now an internal concept.
+    The user processes connections using the single function
+    lsquic_engine_process_conns().  When this function is called,
+    only those connections are processed that need to be processed.
+    A connection needs to be processed when:
+
+        1. New incoming packets have been fed to the connection.
+        2. User wants to read from a stream that is readable.
+        3. User wants to write to a stream that is writeable.
+        4. There are buffered packets that can be sent out.  (This
+           means that the user wrote to a stream outside of the
+           lsquic library callback.)
+        5. A control frame (such as BLOCKED) needs to be sent out.
+        6. A stream needs to be serviced or delayed stream needs to
+           be created.
+        7. An alarm rings.
+        8. Pacer timer expires.
+
+    To achieve this, the library places the connections into two
+    priority queues (min heaps):
+
+        1. Tickable Queue; and
+        2. Advisory Tick Time queue (ATTQ).
+
+    Each time lsquic_engine_process_conns() is called, the Tickable
+    Queue is emptied.  After the connections have been ticked, they are
+    queried again: if a connection is not being closed, it is placed
+    either in the Tickable Queue if it is ready to be ticked again or
+    it is placed in the Advisory Tick Time Queue.  It is assumed that
+    a connection always has at least one timer set (the idle alarm).
+
+    The connections in the Tickable Queue are arranged in the least
+    recently ticked order.  This lets connections that have been quiet
+    longer to get their packets scheduled first.
+
+    This change means that the library no longer needs to be ticked
+    periodically.  The user code can query the library when is the
+    next tick event and schedule it exactly.  When connections are
+    processed, only the tickable connections are processed, not *all*
+    the connections.  When there are no tick events, it means that no
+    timer event is necessary -- only the file descriptor READ event
+    is active.
+
+    The following are improvements and simplifications that have
+    been triggered:
+
+        - Queue of connections with incoming packets is gone.
+        - "Pending Read/Write Events" Queue is gone (along with its
+          history and progress checks).  This queue has become the
+          Tickable Queue.
+        - The connection hash no longer needs to track the connection
+          insertion order.
+
 2018-04-02
 
     - [FEATURE] Windows support

include/lsquic.h

@@ -178,9 +178,6 @@ struct lsquic_stream_if {
 /** By default, infinite loop checks are turned on */
 #define LSQUIC_DF_PROGRESS_CHECK    1000
 
-/** By default, Pending RW Queue infinite loop checks are turned on: */
-#define LSQUIC_DF_PENDRW_CHECK      10
-
 /** By default, read/write events are dispatched in a loop */
 #define LSQUIC_DF_RW_ONCE           0
 
@@ -331,23 +328,6 @@ struct lsquic_engine_settings {
      */
     unsigned        es_progress_check;
 
-    /**
-     * A non-zero value enables internal checks to identify suspected
-     * infinite loops in Pending RW Queue logic.  The value of this
-     * setting is the number of times a connection on Pending RW Queue
-     * is allowed to be processed without making progress before it is
-     * banished from Pending RW Queue.
-     *
-     * Progress is considered to have happened if any of the following
-     * occurs:
-     *   - User reads data, FIN, or new error (due to a reset) from a
-     *     stream.
-     *   - A new stream-related frame is packetized.
-     *
-     * The defaut value is @ref LSQUIC_DF_PENDRW_CHECK.
-     */
-    unsigned        es_pendrw_check;
-
     /**
      * A non-zero value make stream dispatch its read-write events once
      * per call.
@@ -364,19 +344,15 @@ struct lsquic_engine_settings {
 
     /**
      * If set, this value specifies that number of microseconds that
-     * functions @ref lsquic_engine_proc_all(),
-     * @ref lsquic_engine_process_conns_with_incoming(),
-     * @ref lsquic_engine_process_conns_to_tick(), and
-     * @ref lsquic_engine_process_conns_with_pend_rw() are allowed
-     * to spend before returning.
+     * @ref lsquic_engine_process_conns() is allowed to spend before
+     * returning.
      *
      * This is not an exact science and the connections must make
      * progress, so the deadline is checked after all connections get
      * a chance to tick and at least one batch of packets is sent out.
      *
      * When processing function runs out of its time slice, immediate
-     * calls to @ref lsquic_engine_has_pend_rw() and
-     * @ref lsquic_engine_has_unsent_packets() return false.
+     * calls to @ref lsquic_engine_has_unsent_packets() return false.
      *
      * The default value is @ref LSQUIC_DF_PROC_TIME_THRESH.
      */
@@ -503,50 +479,11 @@ lsquic_engine_packet_in (lsquic_engine_t *,
         void *peer_ctx);
 
 /**
- * Process all connections.  This function must be called often enough so
+ * Process tickable connections.  This function must be called often enough so
  * that packets and connections do not expire.
  */
 void
-lsquic_engine_proc_all (lsquic_engine_t *engine);
-
-/**
- * Process connections that have incoming packets.  Call this after adding
- * one or more incoming packets using lsquic_engine_packet_in().
- */
-void
-lsquic_engine_process_conns_with_incoming (lsquic_engine_t *);
-
-/**
- * Process connections in Advisory Tick Time queue whose tick times are in
- * the past.
- */
-void
-lsquic_engine_process_conns_to_tick (lsquic_engine_t *);
-
-/**
- * Returns true if engine has connections that have pending read or write
- * events.
- *
- * Connections with pending read or write events are those that have at
- * least one stream whose state changed outside of the regular callback
- * mechanism.  The simplest example is writing directly to the stream
- * object when data comes in.
- *
- * A call to @ref lsquic_engine_proc_all,
- * @ref lsquic_engine_process_conns_with_incoming,
- * @ref lsquic_engine_process_conns_to_tick, or
- * @ref lsquic_engine_process_conns_with_pend_rw removes processed connection
- * from Pending RW queue.
- */
-int
-lsquic_engine_has_pend_rw (lsquic_engine_t *);
-
-/**
- * Process connections that have pending read or write events (@see
- * lsquic_engine_has_pend_rw for description).
- */
-void
-lsquic_engine_process_conns_with_pend_rw (lsquic_engine_t *);
+lsquic_engine_process_conns (lsquic_engine_t *engine);
 
 /**
  * Returns true if engine has some unsent packets.  This happens if
@@ -895,10 +832,10 @@ void
 lsquic_conn_abort (lsquic_conn_t *c);
 
 /**
- * Returns true if there is a connection on the Advisory Tick Time queue,
- * false otherwise.  If true, `diff' is set to the difference between
- * the earliest advisory tick time and now.  If the former is in the past,
- * the value of `diff' is negative.
+ * Returns true if there are connections to be processed, false otherwise.
+ * If true, `diff' is set to the difference between the earliest advisory
+ * tick time and now.  If the former is in the past, the value of `diff'
+ * is negative.
  */
 int
 lsquic_engine_earliest_adv_tick (lsquic_engine_t *engine, int *diff);

src/liblsquic/CMakeLists.txt

@@ -52,6 +52,7 @@ SET(lsquic_STAT_SRCS
     lsquic_hpack_enc.c
     lsquic_xxhash.c
     lsquic_buf.c
+    lsquic_min_heap.c
     )
 
 

src/liblsquic/lsquic_alarmset.c

@@ -52,3 +52,25 @@ lsquic_alarmset_ring_expired (lsquic_alarmset_t *alset, lsquic_time_t now)
             }
         }
 }
+
+
+lsquic_time_t
+lsquic_alarmset_mintime (const lsquic_alarmset_t *alset)
+{
+    lsquic_time_t expiry;
+    enum alarm_id al_id;
+
+    if (alset->as_armed_set)
+    {
+        expiry = UINT64_MAX;
+        for (al_id = 0; al_id < MAX_LSQUIC_ALARMS; ++al_id)
+            if ((alset->as_armed_set & (1 << al_id))
+                                && alset->as_expiry[al_id] < expiry)
+            {
+                expiry = alset->as_expiry[al_id];
+            }
+        return expiry;
+    }
+    else
+        return 0;
+}

src/liblsquic/lsquic_alarmset.h

@@ -67,4 +67,7 @@ lsquic_alarmset_init_alarm (lsquic_alarmset_t *, enum alarm_id,
 void
 lsquic_alarmset_ring_expired (lsquic_alarmset_t *, lsquic_time_t now);
 
+lsquic_time_t
+lsquic_alarmset_mintime (const lsquic_alarmset_t *);
+
 #endif

src/liblsquic/lsquic_attq.c

@@ -26,7 +26,6 @@ struct attq
 {
     struct malo        *aq_elem_malo;
     struct attq_elem  **aq_heap;
-    lsquic_time_t       aq_min;
     unsigned            aq_nelem;
     unsigned            aq_nalloc;
 };
@@ -91,17 +90,6 @@ attq_verify (struct attq *q)
 #endif
 
 
-int
-attq_maybe_add (struct attq *q, struct lsquic_conn *conn,
-                                            lsquic_time_t advisory_time)
-{
-    if (advisory_time < q->aq_min)
-        return 1;
-    else
-        return attq_add(q, conn, advisory_time);
-}
-
-
 static void
 attq_swap (struct attq *q, unsigned a, unsigned b)
 {
@@ -273,13 +261,3 @@ attq_next_time (struct attq *q)
     else
         return NULL;
 }
-
-
-lsquic_time_t
-attq_set_min (struct attq *q, lsquic_time_t new_min)
-{
-    lsquic_time_t prev_value;
-    prev_value = q->aq_min;
-    q->aq_min = new_min;
-    return prev_value;
-}

src/liblsquic/lsquic_attq.h

@@ -27,11 +27,6 @@ attq_create (void);
 void
 attq_destroy (struct attq *);
 
-/* Return 1 if advisory_time is too small, 0 on success, -1 on failure */
-int
-attq_maybe_add (struct attq *, struct lsquic_conn *,
-                                            lsquic_time_t advisory_time);
-
 /* Return 0 on success, -1 on failure (malloc) */
 int
 attq_add (struct attq *, struct lsquic_conn *, lsquic_time_t advisory_time);
@@ -48,7 +43,4 @@ attq_count_before (struct attq *, lsquic_time_t cutoff);
 const lsquic_time_t *
 attq_next_time (struct attq *);
 
-lsquic_time_t
-attq_set_min (struct attq *, lsquic_time_t new_min);
-
 #endif