adding smeagle stability test example

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

smeagle/README.md

@@ -0,0 +1,10 @@
+# Smeagle Compat
+
+This is a dummy example of using Smeagle for Stability and Compatibility tests,
+as outlined in Matt's Binary-Level ABI Compatibility Model.
+
+ - [stability-tests](stability): compares two versions of a library to itself
+ - *compatibility-tests*: compares a binary and contender library (not added yet)
+
+All data is generated from smeagle's facts in the [artifacts](https://github.com/buildsi/build-abi-containers-results/tree/main/artifacts/smeagle)
+repository.

smeagle/stability-tests/README.md

@@ -0,0 +1,45 @@
+# Stability Tests
+
+This is an example of parsing Smeagle output, the two .yaml files in this folder,
+to calculate a stability test. The libraries are very small so we don't need ASP
+or anything optimized for this.
+
+## Algorithm
+
+- Define A as [libmath-v1.0.0.so.yaml](libmath-v1.0.0.so.yaml)
+- Define B as [libmath-v2.0.0.so.yaml](libmath-v2.0.0.so.yaml)
+
+
+A and B are two instances of the same library. Can B replace A?
+
+- We’ll define Exported(BinSet), Imported(BinSet), Dependencies(Bin) as:
+  - Let Exported(BinSet) be a set of interface/type/locations triples exported by a set of binaries: {i, type, loc : abi_location(bin, i, "Export", type, loc)  bin  BinSet,}.
+  - Let Imported(BinSet) be a set of interface/type/location triples imported by some binary: {i, type, loc : abi_location(bin, i, "Import", type, loc)  bin  BinSet}.
+  - Let Dependencies(Bin) be the set of binaries that binary Bin depends on. 
+- We’ll say B can safely replace A if:
+  - Exported({A}) Exported({B}) -and- Imported({B}) Imported({A}) Exported(Dependencies(B))
+
+## Usage
+
+```bash
+#                           Library A              Library B
+$ ./run-stability-test test libmath-v1.0.0.so.yaml libmath-v2.0.0.so.yaml
+```
+
+And the result
+
+```bash
+$ ./run-stability-test test libmath-v1.0.0.so.yaml libmath-v2.0.0.so.yaml 
+INFO:root:Checking if Exported({A}) ⊆ Exported({B}): yes
+INFO:root:Checking if Imported({B}) ⊆ Imported({A}): no
+Interface(InterfaceName='_ZN11MathLibrary10Arithmetic3AddEii', InterfaceType='function', Name='a', Type='Integer32', Location='%rdi', Direction='import')
+Interface(InterfaceName='_ZN11MathLibrary10Arithmetic3AddEii', InterfaceType='function', Name='b', Type='Integer32', Location='%rsi', Direction='import')
+
+```
+
+## Notes
+
+ - This is currently not accounting for dependencies - note that the second rule above includes looking at `Exported(Dependencies(A))` and we don't have that context here.
+ - This is also not doing any intelligent comparison of attributes, but just looking for the entire interface definition as presented (as the model describes)
+ - This script can be modified as the ability to define interfaces other than functions is added.
+ - The data files from Smeagle 0.0.0 were modified to have quotes around variables (and this bug has been [fixed](https://github.com/buildsi/Smeagle/pull/35))

smeagle/stability-tests/libmath-v1.0.0.so.yaml

@@ -0,0 +1,52 @@
+library: "/opt/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/mathclient-1.0.0-eztzlw3my5mrzohcujj3qamrznq2kraf/lib/libmath.so"
+locations: 
+  - function:
+      name: _ZN11MathLibrary10Arithmetic8SubtractEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import
+  - function:
+      name: _init
+      parameters:
+  - function:
+      name: _ZN11MathLibrary10Arithmetic8MultiplyEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import
+  - function:
+      name: _fini
+      parameters:
+  - function:
+      name: _ZN11MathLibrary10Arithmetic6DivideEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import
+  - function:
+      name: _ZN11MathLibrary10Arithmetic3AddEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import

smeagle/stability-tests/libmath-v2.0.0.so.yaml

@@ -0,0 +1,52 @@
+library: "/opt/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.5.0/mathclient-2.0.0-hvlamaz4jdwsmi5gds2pmzaa3um3frnm/lib/libmath.so"
+locations: 
+  - function:
+      name: _ZN11MathLibrary10Arithmetic8SubtractEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import
+  - function:
+      name: _ZN11MathLibrary10Arithmetic3AddEii
+      parameters:
+        - name: a
+          type: Integer32
+          location: "%rdi"
+          direction: import
+        - name: b
+          type: Integer32
+          location: "%rsi"
+          direction: import
+  - function:
+      name: _fini
+      parameters:
+  - function:
+      name: _init
+      parameters:
+  - function:
+      name: _ZN11MathLibrary10Arithmetic6DivideEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import
+  - function:
+      name: _ZN11MathLibrary10Arithmetic8MultiplyEdd
+      parameters:
+        - name: a
+          type: Float64
+          location: "%xmm0"
+          direction: import
+        - name: b
+          type: Float64
+          location: "%xmm1"
+          direction: import

smeagle/stability-tests/run-stability-test

@@ -0,0 +1,206 @@
+#!/usr/bin/env python3
+
+import argparse
+import shutil
+import logging
+import tempfile
+import jsonschema
+import os
+import re
+import json
+import calendar
+import subprocess
+import time
+import shutil
+import sys
+import yaml
+
+from dataclasses import dataclass
+
+logging.basicConfig(level=logging.INFO)
+
+# We want the root
+here = os.path.abspath(os.path.dirname(__file__))
+
+
+def read_yaml(filename):
+    with open(filename, "r") as fd:
+        content = yaml.load(fd, Loader=yaml.FullLoader)
+    return content
+
+
+def write_file(content, filename):
+    with open(filename, "w") as fd:
+        fd.write(content)
+
+
+@dataclass
+class Interface:
+    """An interface currently mostly refers to a function
+    """
+
+    InterfaceName: str
+    InterfaceType: str
+    Name: str = None
+    Type: str = None
+    Location: str = None
+    Direction: str = None
+
+
+class StabilityTest:
+    """
+    Determine if two libraries A and B are stable, meaning library B can
+    replace library A
+    """
+
+    def __init__(self, A, B):
+        self.A = A
+        self.B = B
+
+    def is_compatible(self):
+        """
+        Determine if the libraries are compatible
+        """
+        self.check_exported_subset()
+        self.check_imported_subset()
+
+    def check_exported_subset(self):
+        """
+        Check for overlap of exported interfaces
+        """
+        missing = self._check_subset(self.A.Exported, self.B.Exported)
+        self._print_check("Checking if Exported({A}) ⊆ Exported({B})", missing)
+
+    def _print_check(self, message, missing):
+        """
+        Print missing imports or exports to the terminal
+        """
+        # None missing == one is a subset of the other (yes), missing == not a subset (no)
+        status = "no" if missing else "yes"
+        logging.info("%s: %s" % (message, status))
+        if missing:
+            for m in missing:
+                print(m)
+            print()
+
+    def check_imported_subset(self):
+        """
+        Check for overlap of imported interfaces
+        """
+        # This isn't entirely correct - needs to be U Exported(Dependencies(B))
+        missing = self._check_subset(self.B.Imported, self.A.Imported)
+        self._print_check("Checking if Imported({B}) ⊆ Imported({A})", missing)
+
+    def _check_subset(self, A, B):
+        """
+        Common function to compare two lists.
+        
+        The A and B might not be relevant, but should be provided in the
+        correct order to the function.
+        """
+        missing_in_b = []
+        for iface in A:
+            if iface not in B:
+                missing_in_b.append(iface)
+        return missing_in_b
+
+
+class Library:
+    def __init__(self, yaml_file):
+        if not os.path.exists(yaml_file):
+            sys.exit("%s does not exist" % yaml_file)
+        self.data = read_yaml(yaml_file)
+        self.Imported = []
+        self.Exported = []
+        self.yaml_file = os.path.abspath(yaml_file)
+        self.parse()
+
+    def __str__(self):
+        return os.path.basename(self.yaml_file)
+
+    def __repr__(self):
+        return self.__str__()
+
+    def parse(self):
+        """
+        Parse interface entries (e.g., functions) into appropriate attributes.
+        """
+        for entry in self.data.get("locations", {}):
+            for interface_type, data in entry.items():
+
+                # Create interface entries for each parameter. This will skip
+                # functions that don't have any parameters - we might want
+                # to account for those in the model.
+                params = data.get("parameters")
+                if not params:
+                    continue
+
+                for param in params:
+                    direction = param.get("direction")
+
+                    # We can't include anything in the model without a direction
+                    if not direction or direction not in ["import", "export"]:
+                        continue
+
+                    # The set to add to depending on the direction
+                    group = getattr(self, "%sed" % direction.capitalize())
+                    group.append(
+                        Interface(
+                            data["name"],
+                            interface_type,
+                            param["name"],
+                            param["type"],
+                            param["location"],
+                            direction,
+                        )
+                    )
+
+    def Dependencies(self):
+        """We currently don't have a representation of dependencies in Smeagle
+        Let Dependencies(Bin) be the set of binaries that binary Bin depends on.
+        """
+        return set()
+
+
+def get_parser():
+    parser = argparse.ArgumentParser(description="Smeagle Stability Test")
+
+    description = "actions for Smeagle stability test"
+    subparsers = parser.add_subparsers(
+        help="run-tests actions",
+        title="actions",
+        description=description,
+        dest="command",
+    )
+
+    # Run a complete test, which includes building the test container
+    test = subparsers.add_parser("test", help="test two libraries for compatibility")
+    test.add_argument("inputs", help="inputs to run", nargs=2)
+    return parser
+
+
+def main():
+    """
+    Entrypoint for running stability test.
+    """
+    parser = get_parser()
+
+    def help(return_code=0):
+        parser.print_help()
+        sys.exit(return_code)
+
+    # If an error occurs while parsing the arguments, the interpreter will exit with value 2
+    args, extra = parser.parse_known_args()
+    if not args.command:
+        help()
+
+    libA = Library(args.inputs[0])
+    libB = Library(args.inputs[1])
+
+    # Determine if B can safely replace A
+    test = StabilityTest(A=libA, B=libB)
+    test.is_compatible()
+
+
+if __name__ == "__main__":
+    main()