Ongoing rewrite of the local cache code

It's a bit smarter now. Addresses a bunch of issues at once, though it
still needs several tweaks and testing:

- #78: Provide a dedicated namespace for each RRDP notification, to
  prevent malicious RPPs from overriding files from other RPPs.
- #79: RRDP session and serial are no longer cached in RAM; they're
  extracted from cached notification files as they are needed.
  This prevents all RRDP from being considered outdated during startup.
- #80: rsync-strategy has been removed.
- #81: The cache now retains RRDP files.

The refactor has been more intrusive than intended. I've been retouching
the core loop and rrdp/https code, which has yielded the following
further disinfections:

- #77: Refactor the HTTP code so 304 is handled as success, despite no
  file modifications having been made.
- It seems the old code was refusing to download RPPs via RRDP when said
  RPP wasn't also (unrelatedly) served via rsync. This seemed to stem
  from an old RFC misunderstanding from the previous developer.
- I've deprecated `rsync.priority` and `rrdp.priority`, mostly just to
  simplify the code. I haven't seen anyone using these config fields,
  and I think SIAs and/or randomness should be the ones to decide which
  protocol is preferred for a given RPP, not Fort's admin.
- However, I have also decided to deprecate `shuffle_tal_uris`, because
  I also suspect it's completely unused, and would like to hear some
  complaints otherwise.
- Deprecated `rsync.arguments-flat`, because non-recursive rsyncs are
  not needed anymore.
- Since RRDP files are no longer deleted immediately after use, the
  `DEBUG_RRDP` compilation has lost its purpose, so I deleted it.
- The code was using `HASH_ADD_STR` on strings contained outside of the
  node structure. This is illegal according to uthash's documentation,
  and might have induced some crashes in the past.

.gitignore

@@ -95,6 +95,9 @@ config.sub
 *.trs
 test-driver
 
+# Developer playground
+sandbox/
+
 # Temporal files
 *~
 tmp

deconf.sh

@@ -4,4 +4,6 @@ git clean -dfx \
 	-e .settings \
 	-e .metadata \
 	-e Debug/ \
-	-e tmp
+	-e tmp/ \
+	-e sandbox/
+

src/Makefile.am

@@ -14,18 +14,15 @@ fort_SOURCES += cert_stack.h cert_stack.c
 fort_SOURCES += common.c common.h
 fort_SOURCES += config.h config.c
 fort_SOURCES += daemon.h daemon.c
-fort_SOURCES += delete_dir_daemon.h delete_dir_daemon.c
 fort_SOURCES += extension.h extension.c
 fort_SOURCES += file.h file.c
 fort_SOURCES += init.h init.c
-fort_SOURCES += internal_pool.h internal_pool.c
 fort_SOURCES += json_parser.c json_parser.h
 fort_SOURCES += line_file.h line_file.c
 fort_SOURCES += log.h log.c
 fort_SOURCES += nid.h nid.c
 fort_SOURCES += notify.c notify.h
 fort_SOURCES += output_printer.h output_printer.c
-fort_SOURCES += random.h random.c
 fort_SOURCES += resource.h resource.c
 fort_SOURCES += rpp.h rpp.c
 fort_SOURCES += sorted_array.h sorted_array.c
@@ -35,7 +32,6 @@ fort_SOURCES += thread_var.h thread_var.c
 fort_SOURCES += json_handler.h json_handler.c
 fort_SOURCES += validation_handler.h validation_handler.c
 fort_SOURCES += validation_run.h validation_run.c
-fort_SOURCES += visited_uris.h visited_uris.c
 
 fort_SOURCES += asn1/content_info.h asn1/content_info.c
 fort_SOURCES += asn1/decode.h asn1/decode.c
@@ -49,6 +45,9 @@ fort_SOURCES += types/serial.h types/serial.c
 fort_SOURCES += types/uri.h types/uri.c
 fort_SOURCES += types/vrp.c types/vrp.h
 
+fort_SOURCES += cache/local_cache.c cache/local_cache.h
+fort_SOURCES += cache/tmp.c cache/tmp.h
+
 fort_SOURCES += config/boolean.c config/boolean.h
 fort_SOURCES += config/filename_format.h config/filename_format.c
 fort_SOURCES += config/log_conf.h config/log_conf.c
@@ -66,6 +65,7 @@ fort_SOURCES += crypto/hash.h crypto/hash.c
 
 fort_SOURCES += data_structure/array_list.h
 fort_SOURCES += data_structure/common.h
+fort_SOURCES += data_structure/path_builder.h data_structure/path_builder.c
 fort_SOURCES += data_structure/uthash.h
 
 fort_SOURCES += http/http.h http/http.c
@@ -91,9 +91,6 @@ fort_SOURCES += rrdp/rrdp_loader.h rrdp/rrdp_loader.c
 fort_SOURCES += rrdp/rrdp_objects.h rrdp/rrdp_objects.c
 fort_SOURCES += rrdp/rrdp_parser.h rrdp/rrdp_parser.c
 
-fort_SOURCES += rrdp/db/db_rrdp.h rrdp/db/db_rrdp.c
-fort_SOURCES += rrdp/db/db_rrdp_uris.h rrdp/db/db_rrdp_uris.c
-
 fort_SOURCES += rsync/rsync.h rsync/rsync.c
 
 fort_SOURCES += rtr/err_pdu.c rtr/err_pdu.h

src/abbreviations.txt

@@ -1,7 +1,5 @@
 This file lists the abbreviations used through the code.
 (Standard C, dependency and RFC-defined abbreviations are generally excluded.)
-If you find an abbreviation that is not listed here, feel free to report it as
-a bug.
 
 addr: address
 addr4: IPv4 address
@@ -10,11 +8,14 @@ be: Big Endian
 cert: certificate
 certstack: certificate stack
 db: database
+dl: download
 eof: end of file
 err: error
 fd: File Descriptor (see `man 2 accept`)
 guri: global URI
 hdr: header
+hh: hash (table) hook
+ht: hash table
 id: identifier
 len: length
 max: maximum
@@ -27,9 +28,10 @@ ptr: pointer
 refget: reference get (+1 to reference counter)
 refput: reference put (-1 to reference counter)
 rk: Router Key
-rpp: Repository Publication Point
+rpp: Repository Publication Point (RFC 6481)
 str: string
 tmp: temporal
+ts: timestamp
 uint: unsigned int
 vrp: Validated ROA Payload (RFC 6811)
 vrps: Validated ROA Payloads (VRP database)