Update NEWS and dtrace.spec for errata release 2.0.0-1.13.2

Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
oracle · Nov 10, 2023 · 3c47de5 · 3c47de5
1 parent 347ad8a
commit 3c47de5
Show file tree

Hide file tree

Showing 3 changed files with 174 additions and 96 deletions.
diff --git a/NEWS b/NEWS
@@ -1,3 +1,61 @@
+2.0.0-1.13.2 (Nov 10th, 2023)
+-----------------------------
+
+Thirteenth errata of the standalone userspace implementation.
+
+** This is a pre-release with limited functionality. **
+
+New features:
+
+ - DTrace can now be used for tracing with upstream kernels, albeit with some
+   limitations.
+
+ - The ip provider has been implemented.
+
+ - The trunc() action has been implemented.
+
+ - The pcap() action has been implemented.
+
+ - The inet_ntoa6() subroutine has been implemented.
+
+ - The inet_ntop() subroutine has been implemented.
+
+ - Support for modules.builtin.ranges data from the kernel has been added.
+   This is the new way to determine module name association for kernel symbols
+   that are built into the kernel.  Support for kallmodsyms is retained for
+   kernel that do not support modules.builtin.ranges yet.
+
+ - A BTF-to-CTF convertor has been added to support using DTrace with kernels
+   that do not provide CTF data.  Note that BTF is currently more limited than
+   CTF.  E.g. BTF does not provide datatype information for kernel variables.
+
+Bugfixes:
+
+ - Struct and union member access in alloca()-allocated memory no longer cause
+   a BPF verifier violation.
+
+ - Birfield offset calculations have been corrected.
+
+ - Disassembler output for endianness conversion instructions has been
+   corrected.
+
+ - Bounds checking of array datatypes of size 0 or 1 in the kernel is now
+   skipped because they are commonly used in the kernel as anchors for
+   dynamically sized arrays.
+
+ - Zero constants will now be checked at compile time wherever NULL pointer
+   argument checking is done.
+
+ - Uprobes are now created using the offset in the inode rather than based on
+   an absolute address.
+
+ - Building in various forms of kernel builds is now more streamlined.
+
+Internal changes:
+
+ - The dependency on waitfd() has been replaced with a mechanism that does not
+   depend on this system call.
+
 2.0.0-1.13.1 (Jun 7th, 2023)
 ----------------------------
 

diff --git a/README.md b/README.md
@@ -7,10 +7,7 @@ visibility for our work and to make it even easier for people to access the
 source.  We also use this repository to work with developers in the Linux
 community.
 
-The main development branch is [dev](https://github.com/oracle/dtrace-utils/tree/dev).
-
-A small number of kernel patches are needed, which are [here](https://github.com/oracle/dtrace-linux-kernel).
-Branches named starting v2/* are suitable (see 2.1.3 below for more details).
+The main development branch is [devel](https://github.com/oracle/dtrace-utils/tree/devel).
 
 We provide prebuilt x86_64 and aarch64 DTrace userspace packages for Oracle
 Linux 9 (UEK7 kernel), Oracle Linux 8 (UEK7 or UEK6 kernel), and Oracle Linux 7
@@ -36,15 +33,18 @@ Source code for the UEK kernel is available on github in the
 * [1. License](#1-license)
 * [2. How to Build DTrace](#2-how-to-build-dtrace)
   * [2.1. Prerequisites](#21-prerequisites)
-     * [2.1.1. Step 1: Either install a recent-enough binutils and GCC...](#211-step-1-either-install-a-recent-enough-binutils-and-gcc)
-     * [2.1.2. Step 2: ... or install the Compact Type Format (CTF) libraries (deprecated)](#212-step-2-or-install-the-compact-type-format-ctf-libraries-deprecated)
-     * [2.1.3. Step 3: Build a Kernel with add-on features that DTrace uses](#213-step-3-build-a-kernel-with-add-on-features-that-dtrace-uses)
-     * [2.1.4. Step 4: Get the Other Necessary Packages](#214-step-4-get-the-other-necessary-packages)
+    * [2.1.1. Verify that binutils is recent enough...](#211-verify-that-binutils-is-recent-enough)
+    * [2.1.2. ... or install a recent-enough binutils](#212-or-install-a-recent-enough-binutils)
+    * [2.1.3. Install other necessary packages](#213-install-other-necessary-packages)
   * [2.2. Build DTrace](#22-build-dtrace)
 * [3. Testing](#3-testing)
 * [4. How to Run DTrace](#4-how-to-run-dtrace)
 * [5. Questions](#5-questions)
 * [6. Pull Requests and Support](#6-pull-requests-and-support)
+* [7. Building Previous Releases of DTrace](#7-building-previous-releases-of-dtrace)
+  * [7.1. Prerequisites](#71-prerequisites)
+  * [7.2. Build a kernel with add-on features that DTrace uses](#72-build-a-kernel-with-add-on-features-that-dtrace-uses)
+  * [7.3. Build DTrace](#73-build-dtrace)
 
 ## 1. License
 
@@ -62,99 +62,28 @@ kernel versions.
 Please read this section carefully before moving over to the build
 documentation to ensure your environment is properly configured.
 
-#### 2.1.1. Step 1: Either install a recent-enough binutils and GCC...
+#### 2.1.1. Verify that binutils is recent enough...
 
 DTrace uses a type introspection system called CTF.  This is supported by
 upstream GCC and GNU Binutils.  Make sure you have binutils 2.36 or later
-installed, and trunk GCC or GCC 12 (which can be configured with a --prefix
-specific to itself to avoid disturbing the system compiler: just point
-HOSTCC and CC at that GCC when compiling the kernel and DTrace).
-
-If your distro provides a binutils development package, you need to install
-that too (we need to link against libctf.so).
-
-#### 2.1.2. Step 2: ... or install the Compact Type Format (CTF) libraries (deprecated)
-
-If you can't update your toolchain, you can alternatively install
-[libdtrace-ctf](https://github.com/oracle/libdtrace-ctf).  However, this library
-is deprecated and cannot always read CTF produced by the GCC/binutils combination
-above.
-
-If you're building a kernel newer than 5.16, you must use the toolchain combination,
-as the old libdtrace-ctf-using CTF converter in the kernel tree has been removed
-in favour of something much smaller using Binutils's libctf.  Equally, if you're
-compiling the kernel using GCC 11 or higher, you must use the toolchain approach
-(which means installing GCC master or GCC 12), since the old converter cannot
-handle the DWARF emitted by GCC 11.
-
-If you're using libdtrace-ctf, the kernel build additionally depends on the
-following packages that may not be installed on the system yet.  The table below
-gives the package names for Debian and Oracle Linux as examples.
-
-| Prerequisite | Debian           | Oracle Linux          |
-|:-------------|:-----------------|:----------------------|
-| glib2        | libglib2.0-dev   | glib2-devel           |
-| libdw        | libdw-dev        | elfutils-devel        |
-| libelf       | libelf-dev       | elfutils-libelf-devel |
-
-#### 2.1.3. Step 3: Build a kernel with add-on features that DTrace uses
-
-DTrace currently depends on a few extra kernel features that are not available
-in the upstream kernel:
-
-- CTF type information extraction
-- /proc/kallmodsyms
-
-As noted above, patches that implement these features are available from the
-v2/* branches in our Linux kernel repository for features that DTrace uses:
-https://github.com/oracle/dtrace-linux-kernel
-
-The customary way to obtain these patches is to clone an appropriate upstream
-kernel version (5.8.1 or later) and to merge our
-[updated tree](https://github.com/oracle/dtrace-linux-kernel/v2/5.8.1) into it.
-The patch set is quite small, but may need some tweaking if it is being merged
-into a newer tree.  We occasionally push a new patched tree to our github repo,
-so do look around to see if there is a better match.
-
-Oracle Linux's [UEK7](https://github.com/oracle/linux-uek/tree/uek7/ga)
-provides a simple alternative if you don't want to patch your own kernel.
-
-Building of a patched kernel from sources is also straightforward.
-Please consult `Documentation/process/changes.rst` in the kernel source tree
-to ensure all required dependencies are installed first.  The following steps
-should build and install the kernel.
+installed.
 
-```
-# Start with your preferred .config options.  Features DTrace needs should
-# enable themselves automatically.
-make olddefconfig
-make
+If your distro provides libctf.so in a binutils development package, you need
+to install that too.
 
-# This step will produce vmlinux.ctfa, which holds all CTF data for the kernel
-# and its modules.  If it doesn't work, you don't have a suitable toolchain
-# and/or are missing dependencies needed for the older libdtrace-ctf-based
-# tools.
-make ctf
+#### 2.1.2. or install a recent-enough binutils
 
-# Install with root privileges.
-sudo make INSTALL_MOD_STRIP=1 modules_install
-sudo make install
-```
+If your distro provides binutils 2.36 or later, you should install it.  If not,
+you can build your own local copy (which can be configured with a --prefix
+specific to itself to avoid disturbing the distro version).
 
-It is preferred (but not required) to reboot into your new kernel before
-trying to build DTrace.  (If you don't reboot into it, you need to specify
-some extra options when running make.)
+#### 2.1.3. Install other necessary packages
 
-#### 2.1.4. Step 4: Get the Other Necessary Packages
-
-A few remaining packages are required either for building or at runtime.  They
+A few other packages are required, either for building or at runtime.  They
 should be part of most Linux distributions today and can be installed via the
 package manager of your distro.  The table below gives package names for Debian
 and Oracle Linux.
 
-We assume that the packages needed to build the kernel remain installed so they
-are not repeated here.
-
 For building:
 
 | Prerequisite       | Debian                      | Oracle Linux              |
@@ -170,6 +99,8 @@ For building:
 | wireshark          | wireshark                   | wireshark                 |
 | valgrind           | valgrind                    | valgrind-devel            |
 | fuse3 or fuse      | libfuse3-dev or libfuse-dev | fuse3-devel or fuse-devel |
+| kernel headers     | linux-headers-<kver>-<arch> | kernel-uek-devel-<kver>   |
+|                    | linux-headers-<kver>-common |                           |
 
 At runtime:
 
@@ -180,9 +111,8 @@ At runtime:
 
 ### 2.2. Build DTrace
 
-The simplest way of building DTrace is booting into your new kernel with the
-add-on features DTrace expects and issuing the following commands from the
-DTrace source tree:
+The simplest way of building DTrace is done by issuing the following commands
+from the DTrace source tree:
 
 ```
 make
@@ -205,7 +135,9 @@ the `include/uapi/linux/dtrace` are compatible with the utils version being
 built).
 
 You can point at a different kernel version with the KERNELS variable, e.g.
+```
 make KERNELS="5.16.8"
+```
 as long as the source tree that kernel was built with remains where it was
 when that kernel was installed.
 
@@ -234,7 +166,9 @@ Logs from test runs are stored in `test/log/<run_number>/`.
 The most recent logs are also available by via the `test/log/current/` symbolic link.
 
 The testsuite itself has more dependencies that need to be installed.
-The full list is available in the [dtrace.spec](dtrace.spec) file.
+The full list is available in the
+[dtrace.spec](https://github.com/oracle/dtrace-utils/blob/devel/dtrace.spec)
+file.
 
 ## 4. How to run DTrace
 
@@ -248,9 +182,81 @@ which is installed in /usr/**bin**/dtrace.
 For questions, please check the
 [dtrace-devel mailing list](https://oss.oracle.com/mailman/listinfo/dtrace-devel).
 
+NEW: We have a #linux-dtrace IRC channel on irc.libera.chat, you can
+ask questions there as well.
+
 ## 6. Pull Requests and Support
 
-We currently do not accept pull requests via GitHub, please contact us via the mailing list above. 
+We currently do not accept pull requests via GitHub, please contact us
+via the mailing list or IRC channel listed above.
 
 The source code for DTrace is published here without support. Compiled binaries are provided as part of Oracle Linux, which is [free to download](http://www.oracle.com/technetwork/server-storage/linux/downloads/index.html), distribute and use.
 Support for DTrace is included in Oracle Linux support subscriptions. Individual packages and updates are available on the [Oracle Linux yum server](https://yum.oracle.com). 
+
+## 7. Building Previous Releases of DTrace
+
+For versions of DTrace prior to 2.0.0-1.13.2 a small number of kernel patches
+patches are needed, which are
+[here](https://github.com/oracle/dtrace-linux-kernel).  Branches named
+starting v2/* are suitable.
+
+To build such versions of dtrace follow the steps below.
+
+### 7.1. Prerequisites
+
+See [above](#21-prerequisites).
+
+At this point, you will need an additional step:
+
+### 7.2. Build a kernel with add-on features that DTrace uses
+
+DTrace 2.0.0-1.13.1 and earlier depend on a few extra kernel features that are
+not available in the upstream kernel:
+
+- CTF type information extraction
+- /proc/kallmodsyms
+- New system call: waitfd()
+
+As noted above, patches that implement these features are available from the
+v2/* branches in our Linux kernel repository for features that DTrace uses:
+https://github.com/oracle/dtrace-linux-kernel
+
+The customary way to obtain these patches is to clone an appropriate upstream
+kernel version (5.8.1 or later) and to merge our
+[updated tree](https://github.com/oracle/dtrace-linux-kernel/tree/v2/5.8.1)
+into it.  The patch set is quite small, but may need some tweaking if it is
+being merged into a newer tree.  We occasionally push a new patched tree to our
+github repo, so do look around the repo to see if there is a branch with better
+match.
+
+Oracle Linux's [UEK7](https://github.com/oracle/linux-uek/tree/uek7/ga)
+provides a simple alternative if you don't want to patch your own kernel.
+
+Building of a patched kernel from sources is also straightforward.
+Please consult `Documentation/process/changes.rst` in the kernel source tree
+to ensure all required dependencies are installed first.  The following steps
+should build and install the kernel.
+
+```
+# Start with your preferred .config options.  Features DTrace needs should
+# enable themselves automatically.
+make olddefconfig
+make
+
+# This step will produce vmlinux.ctfa, which holds all CTF data for the kernel
+# and its modules.  If it doesn't work, you don't have a toolchain that is
+# recent enough.
+make ctf
+
+# Install with root privileges.
+sudo make INSTALL_MOD_STRIP=1 modules_install
+sudo make install
+```
+
+It is preferred (but not required) to reboot into your new kernel before
+trying to build DTrace.  (If you don't reboot into it, you need to specify
+some extra options when running make.)
+
+### 7.3. Build DTrace
+
+See [above](#22-build-dtrace).
diff --git a/dtrace.spec b/dtrace.spec
@@ -85,7 +85,7 @@ Conflicts:    systemtap-sdt-devel
 Provides:     systemtap-sdt-devel
 Summary:      DTrace user interface.
 Version:      2.0.0
-Release:      1.13.1%{?dist}
+Release:      1.13.2%{?dist}
 Source:       dtrace-%{version}.tar.bz2
 BuildRoot:    %{_tmppath}/%{name}-%{version}-build
 ExclusiveArch:    x86_64 aarch64
@@ -249,7 +249,21 @@ fi
 %{_libdir}/dtrace/testsuite
 
 %changelog
-* Wed Jun 07 2023 Kris Van Hees <kris.vna.hees@oracle.com> - 2.0.0-1.13.1
+* Thu Nov  9 2023 Kris Van Hees <kris.vna.hees@oracle.com> - 2.0.0-1.13.2
+- Support using DTrace with upstream kernels.
+- Implement provider: ip.
+- Implement actions: trunc(), pcap().
+- Implement subroutines: inet_ntoa6().
+- Implement subroutines: inet_ntop(). (Eugene Loh)
+- Support modules.builtin.ranges for builtin module-symbol association.
+- Provide a BTF-to-CTF convertor to provide (limited) kernel type information
+  when CTF is not available.
+- Remove dependency on waitfd(). (Nick Alcock)
+- Various testsuite fixes and improvements.
+  (Nick Alcock, Eugene Loh, Kris Van Hees)
+- Various code improvements. (Nick Alcock, Eugene Loh, Kris Van Hees)
+
+* Wed Jun  7 2023 Kris Van Hees <kris.vna.hees@oracle.com> - 2.0.0-1.13.1
 - Restart dtprobed when upgrading DTrace.
 - Report and clean up orphaned tracing events after each test.