Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Add JGF reader support #521
This PR adds JGF reader support. This should not be merged before PR #520.
@@ Coverage Diff @@ ## master #521 +/- ## ========================================== - Coverage 76.38% 76.03% -0.35% ========================================== Files 59 69 +10 Lines 6221 6481 +260 ========================================== + Hits 4752 4928 +176 - Misses 1469 1553 +84
Refactor resource_graph_db_t into its own directory (resource/store). This will replace the same named class in resource/schema/resource_graph.hpp when integrated into the rest of the system. Add two load() methods into this class which takes in an object of the base reader class to decode the input string into its resource graph. How it is decoded is delegated down to the reader object whose runtime type must be a derived class of the base reader class. Introduce resource_graph_metadata_t that contains metadata about the resource graph. This helps simplify reader class methods. Add readers/resource_reader_base.[hpp|cpp] which define the base reader interfaces. They include a few pure virtual abstract interfaces that upcoming derived reader classes must override. The main methods of this class are unpack() and unpack_at(). The unpack() method deserializes the given string into a resource graph. The unpack_at() method unpacks the given string into a resource graph and then grafts the top-level vertices to an existing vertex in the resource graph.
Introduce resource_reader_grug_t by deriving it from resource_reader_base_t. Refactor GRUG reader code into resource_reader_grug_t and implement the unpack() method. Adjust this code to wean off of the old resource_graph_db_t and make use of resource_graph_t and resource_graph_metadata_t instead. The unpack_at() method is unsupported. Resource whitelist is not supported either, and this is indicated by returning false from is_whitelist_supported() method.
Introduce resource_reader_hwloc_t by deriving it from resource_reader_base_t. Refactor Hwloc reader code into resource_reader_hwloc_t and implement the unpack() and unpack_at() methods. Adjust this code to wean off of the old resource_graph_db_t and make use of resource_graph_t and resource_graph_metadata_t instead. Add a bit better error message support for hwloc API calls from the original hwloc reader code.
Introduce resource_reader_jgf_t by deriving it from resource_reader_base_t. First, fetch each JGF node from the "nodes" key from the JGF string, and add a vertex of the resource graph data while building vmap that keeps track of the JGF nodeid-vertex pair using that info. Next, fetch each JGF node from the "edges" key and add an edge of the resource graph data using that info in combination with the source and target vertices provided by the vmap. As part of resource graph vertex and edge population, we fill in graph infrastructure information such as idata.member_of as well as resource graph metadata (e.g., roots of each detected subsystem). Detect each unique subsystem and its root vertex by using the paths field of each JGF node. If the paths field of a JGF node contains a path with one slash (e.g., "containment": "/cluster0"), deem it as the root of that subsystem. Implement unpack() method while leaving unpack_at() method unsupported. Hence, is_whitelist_supported() returns false.
Instantiate an object of a derived resource reader class that corresponds to the input string (reader type name) and return it as the object of the resource reader base class. Allow the client code to be written as the base class, making it future proof. Use shared_ptr class to enhance safety.
Integrate the new resource reader scheme into the entire resource infastructure and the existing tests. - Drop the old reader/generator code in generators/gen.[hpp|cpp]. - Delete the resource_graph_db_t class from schema/resource_graph.hpp as the new definitions in store/resource_graph_store.[hpp|cpp] will be wired in. - Update resource/Makefile.am with respect to the above removals and the new resource reader files that were previously added. Allow the new code to make into the resource convenient library. Add jansson as an extra dependency as required by the JGF reader as well. - Change the command line options for resource-query and the load options for resource match module. For resource-query, --grug and --hwloc-xml are deprecated and --load-file and --load-format are added instead. --hwloc-whitelist is now changed to --load-whitelist. For the resource module, grug-conf, hwloc-xml, and hwloc-whitelist have been deprecated. Instead, load-file, load-format, and load-whitelist are used. - Adjust resource-query and resource module code with respect to the new resource_graph_db_t and resource_graph_metadata_t definitions. - populate_resource_db in resource-query and resource module code now makes use of the new resource reader support. - Update all of the sharness test cases to use the new cli and module-load options.
Add 3 machine definitions written in JGF in data/resource/jgfs (in parallel to data/resource/grugs). Will be used by future JGF reader sharness tests. hwloc_4core.json contains a very small machine configuration with a total of 1 compute node with 4 compute cores. power.json describes the same system specified by data/resource/grugs/power.graphml. tiny.json describes the same system specified by data/resource/grugs/tiny.graphml When power.json or tiny.json is loaded into resource-query and resource module, these programs must behave exactly same as they were loaded with the corresponding GRUG files. In fact, future test codes will be written based ont this property.
Copied from t3001-resource-basic.t. Only change the machine specification from GRUG to JGF. Because we use the exactly same resource-query commands, the test outputs must match with the results of t3001-resource-basic.t.
Copied from t3010-resource-power.t. Only change the machine specification from GRUG to JGF. Because we use the exactly same resource-query commands, the test outputs must match with that of t3010-resource-power.t. Adjust this test code to ensure the JGF reader works correctness when there are multiple subsystems in the JGF spec (i.e. containment and power).
Add simple tests to ensure the JGF reader works when used within the resource module as well.