DPAA2: Release new buffers based on buffer pool utilization.

dpaa2_ni driver is attached with a limited (DPAA2_NI_BUFS_INIT) number
of buffers released to the buffer pool. It might not be a problem when
several channels per network interface are available. HoneyComb is a
good example with 5600 = 16 channel * 350 buffers, but each
network interface on Ten64 will have 350 = 1 channel * 350 buffers
released to the buffer pool initially. Stress test and a huge amount
of ingress traffic can lead to the pool depletion and dropped frames
in this case.

This patch allows to monitor buffer pool using dpaa2_ni_bp_task(),
detect free buffers left with help of a new software portal command
(DPAA2_SWP_QUERY_BP) and release new ones if necessary, i.e.
pool will be doubled in size when amount of free buffers is
less then 25% of the previous pool capacity.

State of the buffer pool can be examined using sysctl:

	dev.dpaa2_ni.X.stats.buf_free
	dev.dpaa2_ni.X.stats.buf_num

where the first one denotes number of free buffers detected right
_before_ the last grow of the pool (0 means pool didn't grow), and the
second one denotes current size of the pool.

sys/dev/dpaa2/dpaa2_bp.h

@@ -1,7 +1,7 @@
 /*-
  * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
  *
- * Copyright (c) 2022 Dmitry Salychev <dsl@mcusim.org>
+ * Copyright (c) 2022 Dmitry Salychev
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -45,6 +45,15 @@ struct dpaa2_bp_attr {
 	uint16_t		 bpid;
 };
 
+/**
+ * @brief Configuration/state of the buffer pool.
+ */
+struct dpaa2_bp_conf {
+	uint8_t			 bdi;
+	uint8_t			 state; /* bitmask */
+	uint32_t		 free_bufn;
+};
+
 /**
  * @brief Software context for the DPAA2 Buffer Pool driver.
  */

sys/dev/dpaa2/dpaa2_io.c

@@ -1,7 +1,7 @@
 /*-
  * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
  *
- * Copyright (c) 2021 Dmitry Salychev <dsl@mcusim.org>
+ * Copyright (c) 2021-2022 Dmitry Salychev
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -355,6 +355,17 @@ dpaa2_io_conf_wq_channel(device_t iodev, struct dpaa2_io_notif_ctx *ctx)
 	return (0);
 }
 
+/**
+ * @brief Query current configuration/state of the buffer pool.
+ */
+static int
+dpaa2_io_query_bp(device_t iodev, uint16_t bpid, struct dpaa2_bp_conf *conf)
+{
+	struct dpaa2_io_softc *sc = device_get_softc(iodev);
+
+	return (dpaa2_swp_query_bp(sc->swp, bpid, conf));
+}
+
 /**
  * @brief Release one or more buffer pointers to the QBMan buffer pool.
  */
@@ -533,6 +544,7 @@ static device_method_t dpaa2_io_methods[] = {
 	/* QBMan software portal interface */
 	DEVMETHOD(dpaa2_swp_enq_multiple_fq,	dpaa2_io_enq_multiple_fq),
 	DEVMETHOD(dpaa2_swp_conf_wq_channel,	dpaa2_io_conf_wq_channel),
+	DEVMETHOD(dpaa2_swp_query_bp,		dpaa2_io_query_bp),
 	DEVMETHOD(dpaa2_swp_release_bufs,	dpaa2_io_release_bufs),
 
 	DEVMETHOD_END