Add support for aggregation keys

In the DTrace port to BPF, the initial support for aggregations
had the kernel maintaining aggregations in a per-CPU BPF map that
was statically allocated and managed.  Each aggregation had its
own region of per-CPU memory.  That is, run-time aggregation keys
were not supported.

To support aggregation keys, switch to a per-CPU BPF hash map.
The hash map key comprises:

*)  the aggregation variable ID

*)  the aggregation keys (if any)

*)  a 64-bit zero (to conform to dynamic variable "tuples")

On the producer (dt_cg.c) side:

*)  Use the existing dt_cg_arglist() to process the
    aggregation keys and produce the BPF hash map key.

*)  Call the new dt_get_agg() to get a pointer to the
    aggregation data in the BPF hash map.  The function call
    will create the element if needed, increment the data
    counter, and advance the pointer to the actual data.

*)  Push this pointer to the stack.

*)  Call the aggregation function: compute the value
    to be aggregated and pop the stack for the pointer
    to where the aggregation data is.

*)  Record the aggregation description (e.g., sizes of
    keys and data) if not already done.  This step is
    performed during code generation, though it does not
    generate code per se.

On the consumer (dt_aggregate.c) side, to snapshot aggregation data:

*)  Use dt_bpf_map_next_key() to walk the keys of the
    BPF hash map.

*)  For each hash map key, read the aggregation data on all CPUs.

*)  For each CPU with a nonzero data counter value, check
    dt_aggregate's user-space hash table for this key-specific
    aggregation.  Initialize or aggregate the user-space data
    accordingly.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>

bpf/Build

@@ -25,6 +25,7 @@ bpf_dlib_SOURCES = \
 	agg_lqbin.c agg_qbin.c \
 	basename.S \
 	dirname.S \
+	get_agg.c \
 	get_bvar.c \
 	get_dvar.c \
 	index.S \

bpf/get_agg.c

@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ */
+#include <linux/bpf.h>
+#include <stdint.h>
+#include <bpf-helpers.h>
+#include <dt_dctx.h>
+
+#ifndef noinline
+# define noinline	__attribute__((noinline))
+#endif
+
+/*
+ * Get a pointer to the data storage for an aggregation.  Regular aggregations
+ * are stored as an indexed aggregation with an empty key.  The 'id' parameter
+ * is the aggregtion id; 'ival' is the initial value for min/max aggregations
+ * (and otherwise 0).  The 'dflt' parameter is a pointer to a zero-filled area
+ * of memory that is at least the size of the largest aggregation.
+ */
+noinline uint64_t *dt_get_agg(const dt_dctx_t *dctx, uint32_t id,
+			      const char *key, uint64_t ival, const char *dflt)
+{
+	uint64_t	*valp;
+
+	/* place the variable ID at the beginning of the key */
+	*(uint32_t *)key = id;
+
+	/* try to look up the key */
+	valp = bpf_map_lookup_elem(dctx->agg, key);
+
+	/* if not found, create it */
+	if (valp == 0) {
+		/* start with all zeroes */
+		if (bpf_map_update_elem(dctx->agg, key, dflt, BPF_ANY) < 0)
+			return 0;
+
+		valp = bpf_map_lookup_elem(dctx->agg, key);
+		if (valp == 0)
+			return 0;
+
+		/* ival is nonzero only for min() and max() */
+		if (ival)
+			valp[1] = ival;
+	}
+
+	/* increment the data counter */
+	valp[0] += 1;
+
+	/* advance past the data counter */
+	valp += 1;
+
+	return valp;
+}