Skip to content

Commit

Permalink
utils/hwloc-calc: cleanup topology options management
Browse files Browse the repository at this point in the history 
Enforce those options (--restrict/-i/--disallowed) at the
beginning of the command-line instead of reupdating the
topology later if options and locations are mixed.

Try to document this but we don't show a precise error message
if --restrict/-i/... is given too late.

Signed-off-by: Valentin Hoyet <valentin.hoyet@inria.fr>
Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
  • Loading branch information
@bgoglin
Valentin Hoyet authored and bgoglin committed May 15, 2020
1 parent 990e6a5 commit 9ae74eb
Show file tree
Hide file tree
Showing 2 changed files with 136 additions and 140 deletions.
104 changes: 56 additions & 48 deletions utils/hwloc/hwloc-calc.1in
View file
Expand Up @@ -12,7 +12,7 @@ hwloc-calc \- Operate on cpu mask strings and objects
.SH SYNOPSIS
.
.B hwloc-calc
[\fIoptions\fR] \fI<location1> [<location2> [...] ]
[\fItopology options\fR] [\fIoptions\fR] \fI<location1> [<location2> [...] ]
.
.PP
Note that hwloc(7) provides a detailed explanation of the hwloc system
Expand All @@ -22,8 +22,61 @@ it should be read before reading this man page.
.\" **************************
.\" Options Section
.\" **************************
.SH TOPOLOGY OPTIONS
.
All topology options must be given before all other options.
.
.TP 10
\fB\-\-no\-smt\fR, \fB\-\-no\-smt=<N>\fR
Only keep the first PU per core in the input locations.
If \fI<N>\fR is specified, keep the <N>-th instead, if any.
PUs are ordered by physical index during this filtering.
.TP
\fB\-\-restrict\fR <cpuset>
Restrict the topology to the given cpuset.
.TP
\fB\-\-restrict\fR nodeset=<nodeset>
Restrict the topology to the given nodeset, unless \fB\-\-restrict\-flags\fR specifies something different.
.TP
\fB\-\-restrict\-flags\fR <flags>
Enforce flags when restricting the topology.
Flags may be given as numeric values or as a comma-separated list of flag names
that are passed to \fIhwloc_topology_restrict()\fR.
Those names may be substrings of actual flag names as long as a single one matches,
for instance \fBbynodeset,memless\fR.
The default is \fB0\fR (or \fBnone\fR).
.TP
\fB\-\-disallowed\fR
Include objects disallowed by administrative limitations.
.TP
\fB\-i\fR <file>, \fB\-\-input\fR <file>
Read topology from XML file <file> (instead of discovering the
topology on the local machine). If <file> is "\-", the standard input
is used. XML support must have been compiled in to hwloc for this
option to be usable.
.TP
\fB\-i\fR <directory>, \fB\-\-input\fR <directory>
Read topology from <directory> instead of discovering the topology
of the local machine.
On Linux, the directory may contain the topology files
gathered from another machine topology with hwloc-gather-topology.
On x86, the directory may contain a cpuid dump gathered
with hwloc-gather-cpuid.
.TP
\fB\-i\fR <specification>, \fB\-\-input\fR <specification>
Simulate a fake hierarchy (instead of discovering the topology on the
local machine). If <specification> is "node:2 pu:3", the topology will
contain two NUMA nodes with 3 processing units in each of them.
The <specification> string must end with a number of PUs.
.TP
\fB\-\-if\fR <format>, \fB\-\-input\-format\fR <format>
Enforce the input in the given format, among \fBxml\fR, \fBfsroot\fR,
\fBcpuid\fR and \fBsynthetic\fR.
.
.SH OPTIONS
.
All these options must be given after all topology options above.
.
.TP 10
\fB\-p\fR \fB\-\-physical\fR
Use OS/physical indexes instead of logical indexes for both input and output.
Expand Down Expand Up @@ -104,58 +157,12 @@ while a comma is used to separate indexes
\fB\-\-single\fR
Singlify the output to a single CPU.
.TP
\fB\-\-no\-smt\fR, \fB\-\-no\-smt=<N>\fR
Only keep the first PU per core in the input locations.
If \fI<N>\fR is specified, keep the <N>-th instead, if any.
PUs are ordered by physical index during this filtering.
.TP
\fB\-\-taskset\fR
Display CPU set strings in the format recognized by the taskset command-line
program instead of hwloc-specific CPU set string format.
This option has no impact on the format of input CPU set strings,
both formats are always accepted.
.TP
\fB\-\-restrict\fR <cpuset>
Restrict the topology to the given cpuset.
.TP
\fB\-\-restrict\fR nodeset=<nodeset>
Restrict the topology to the given nodeset, unless \fB\-\-restrict\-flags\fR specifies something different.
.TP
\fB\-\-restrict\-flags\fR <flags>
Enforce flags when restricting the topology.
Flags may be given as numeric values or as a comma-separated list of flag names
that are passed to \fIhwloc_topology_restrict()\fR.
Those names may be substrings of actual flag names as long as a single one matches,
for instance \fBbynodeset,memless\fR.
The default is \fB0\fR (or \fBnone\fR).
.TP
\fB\-\-disallowed\fR
Include objects disallowed by administrative limitations.
.TP
\fB\-i\fR <file>, \fB\-\-input\fR <file>
Read topology from XML file <file> (instead of discovering the
topology on the local machine). If <file> is "\-", the standard input
is used. XML support must have been compiled in to hwloc for this
option to be usable.
.TP
\fB\-i\fR <directory>, \fB\-\-input\fR <directory>
Read topology from <directory> instead of discovering the topology
of the local machine.
On Linux, the directory may contain the topology files
gathered from another machine topology with hwloc-gather-topology.
On x86, the directory may contain a cpuid dump gathered
with hwloc-gather-cpuid.
.TP
\fB\-i\fR <specification>, \fB\-\-input\fR <specification>
Simulate a fake hierarchy (instead of discovering the topology on the
local machine). If <specification> is "node:2 pu:3", the topology will
contain two NUMA nodes with 3 processing units in each of them.
The <specification> string must end with a number of PUs.
.TP
\fB\-\-if\fR <format>, \fB\-\-input\-format\fR <format>
Enforce the input in the given format, among \fBxml\fR, \fBfsroot\fR,
\fBcpuid\fR and \fBsynthetic\fR.
.TP
\fB\-q\fR \fB\-\-quiet\fR
Hide non-fatal error messages.
It mostly includes locations pointing to non-existing objects.
Expand Down Expand Up @@ -192,7 +199,8 @@ Different input lines will be processed separately.
.
.PP
Command-line arguments and options are processed in order.
For instance, it means that changing the type of input indexes
First topology configuration options should be given.
Then, for instance, changing the type of input indexes
with \fB\-\-li\fR or changing the input topology with \fB\-i\fR
only affects the processing the following arguments.
.
Expand Down
172 changes: 80 additions & 92 deletions utils/hwloc/hwloc-calc.c
View file
Expand Up @@ -17,10 +17,17 @@

void usage(const char *callname __hwloc_attribute_unused, FILE *where)
{
fprintf(where, "Usage: hwloc-calc [options] <location> ...\n");
fprintf(where, "Usage: hwloc-calc [topology options] [options] <location> ...\n");
fprintf(where, " <location> may be a space-separated list of cpusets or objects\n");
fprintf(where, " as supported by the hwloc-bind utility, e.g:\n");
hwloc_calc_locations_usage(where);
fprintf(where, "Input topology options (must be at the beginning):\n");
fprintf(where, " --no-smt Only keep a single PU per core\n");
fprintf(where, " --restrict [nodeset=]<bitmap>\n");
fprintf(where, " Restrict the topology to some processors or NUMA nodes.\n");
fprintf(where, " --restrict-flags <n> Set the flags to be used during restrict\n");
fprintf(where, " --disallowed Include objects disallowed by administrative limitations\n");
hwloc_utils_input_format_usage(where, 10);
fprintf(where, "Conversion options:\n");
fprintf(where, " [default] Report the combined input locations as a CPU set\n");
fprintf(where, " --number-of <type|depth>\n"
Expand All @@ -45,13 +52,6 @@ void usage(const char *callname __hwloc_attribute_unused, FILE *where)
fprintf(where, " --sep <sep> Use separator <sep> in the output\n");
fprintf(where, " --taskset Use taskset-specific format when displaying cpuset strings\n");
fprintf(where, " --single Singlify the output to a single CPU\n");
fprintf(where, "Input topology options:\n");
fprintf(where, " --no-smt Only keep a single PU per core\n");
fprintf(where, " --restrict [nodeset=]<bitmap>\n");
fprintf(where, " Restrict the topology to some processors or NUMA nodes.\n");
fprintf(where, " --restrict-flags <n> Set the flags to be used during restrict\n");
fprintf(where, " --disallowed Include objects disallowed by administrative limitations\n");
hwloc_utils_input_format_usage(where, 10);
fprintf(where, "Miscellaneous options:\n");
fprintf(where, " -q --quiet Hide non-fatal error messages\n");
fprintf(where, " -v --verbose Show verbose messages\n");
Expand Down Expand Up @@ -231,7 +231,6 @@ static int hwloc_calc_type_depth(hwloc_topology_t topology, const char *string,
int main(int argc, char *argv[])
{
hwloc_topology_t topology;
int loaded = 0;
unsigned long flags = 0;
unsigned long restrict_flags = 0;
char *input = NULL;
Expand Down Expand Up @@ -267,32 +266,69 @@ int main(int argc, char *argv[])

set = hwloc_bitmap_alloc();

/* don't load now, in case some options change the config before the topology is actually used */
#define ENSURE_LOADED() do { \
if (!loaded) { \
hwloc_topology_init(&topology); \
hwloc_topology_set_all_types_filter(topology, HWLOC_TYPE_FILTER_KEEP_ALL); \
hwloc_topology_set_flags(topology, flags); \
if (input) { \
err = hwloc_utils_enable_input_format(topology, flags, input, &input_format, verbose, callname); \
if (err) return EXIT_FAILURE; \
} \
err = hwloc_topology_load(topology); \
if (restrictstring) { \
hwloc_bitmap_t restrictset = hwloc_bitmap_alloc(); \
hwloc_bitmap_sscanf(restrictset, restrictstring); \
if (hwloc_topology_restrict (topology, restrictset, restrict_flags)) { \
perror("Restricting the topology"); \
/* FALLTHRU */ \
} \
hwloc_bitmap_free(restrictset); \
free(restrictstring); \
} \
if (err < 0) return EXIT_FAILURE; \
depth = hwloc_topology_get_depth(topology); \
loaded = 1; \
} \
} while (0)
while (argc >= 1) {
if (!strcmp (argv[0], "--disallowed") || !strcmp (argv[0], "--whole-system")) {
flags |= HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED;
goto next_config;
}
if (!strcmp (argv[0], "--restrict")) {
if (argc < 2) {
usage (callname, stderr);
exit(EXIT_FAILURE);
}
if(strncmp(argv[1], "nodeset=", 7))
restrictstring = strdup(argv[1]);
else {
restrictstring = strdup(argv[1]+8);
restrict_flags |= HWLOC_RESTRICT_FLAG_BYNODESET;
}
argv++;
argc--;
goto next_config;
}
if (!strcmp (argv[0], "--restrict-flags")) {
if (argc < 2) {
usage (callname, stderr);
exit(EXIT_FAILURE);
}
restrict_flags = hwloc_utils_parse_restrict_flags(argv[1]);
argc--;
argv++;
goto next_config;
}
if (hwloc_utils_lookup_input_option(argv, argc, &opt,
&input, &input_format,
callname)) {
argc--;
argv++;
goto next_config;
}

break;

next_config:
argc--;
argv++;
}

hwloc_topology_init(&topology);
hwloc_topology_set_all_types_filter(topology, HWLOC_TYPE_FILTER_KEEP_ALL);
hwloc_topology_set_flags(topology, flags);
if (input) {
err = hwloc_utils_enable_input_format(topology, flags, input, &input_format, verbose, callname);
if (err) return EXIT_FAILURE;
}
err = hwloc_topology_load(topology);
if (restrictstring) {
hwloc_bitmap_t restrictset = hwloc_bitmap_alloc();
hwloc_bitmap_sscanf(restrictset, restrictstring);
if (hwloc_topology_restrict (topology, restrictset, restrict_flags)) {
perror("Restricting the topology");
/* FALLTHRU */
}
hwloc_bitmap_free(restrictset);
free(restrictstring);
}

while (argc >= 1) {
if (*argv[0] == '-') {
Expand All @@ -305,16 +341,8 @@ int main(int argc, char *argv[])
goto next;
}
if (!strcmp (argv[0], "--disallowed") || !strcmp (argv[0], "--whole-system")) {
if (loaded) {
fprintf(stderr, "Input option %s disallowed after options using the topology\n", argv[0]);
exit(EXIT_FAILURE);
}
flags |= HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED;
goto next;
}
if (!strcmp(argv[0], "-h") || !strcmp(argv[0], "--help")) {
usage(callname, stdout);
return EXIT_SUCCESS;
fprintf(stderr, "Input option %s disallowed after options using the topology\n", argv[0]);
exit(EXIT_FAILURE);
}
if (!strcmp (argv[0], "--no-smt")) {
no_smt = 0;
Expand All @@ -324,31 +352,6 @@ int main(int argc, char *argv[])
no_smt = atoi(argv[0] + 9);
goto next;
}
if (!strcmp (argv[0], "--restrict")) {
if (argc < 2) {
usage (callname, stderr);
exit(EXIT_FAILURE);
}
if(strncmp(argv[1], "nodeset=", 7))
restrictstring = strdup(argv[1]);
else {
restrictstring = strdup(argv[1]+8);
restrict_flags |= HWLOC_RESTRICT_FLAG_BYNODESET;
}
argv++;
argc--;
goto next;
}
if (!strcmp (argv[0], "--restrict-flags")) {
if (argc < 2) {
usage (callname, stderr);
exit(EXIT_FAILURE);
}
restrict_flags = hwloc_utils_parse_restrict_flags(argv[1]);
argv++;
argc--;
goto next;
}
if (!strcmp(argv[0], "--number-of") || !strcmp(argv[0], "-N")) {
if (argc < 2) {
usage(callname, stderr);
Expand Down Expand Up @@ -454,25 +457,12 @@ int main(int argc, char *argv[])
taskset = 1;
goto next;
}
if (hwloc_utils_lookup_input_option(argv, argc, &opt,
&input, &input_format,
callname)) {
if (loaded) {
fprintf(stderr, "Input option %s \"%s\" disallowed after options using the topology\n", argv[0], argv[1]);
exit(EXIT_FAILURE);
}
argv += opt;
argc -= opt;
goto next;
}

fprintf (stderr, "Unrecognized option: %s\n", argv[0]);
usage(callname, stderr);
return EXIT_FAILURE;
}

ENSURE_LOADED();

cmdline_args++;
lcontext.topology = topology;
lcontext.topodepth = depth;
Expand All @@ -485,17 +475,16 @@ int main(int argc, char *argv[])
if (hwloc_calc_process_location_as_set(&lcontext, &scontext, argv[0]) < 0)
fprintf(stderr, "ignored unrecognized argument %s\n", argv[0]);

next:
argc--;
argv++;

if (showobjs && nodeseto) {
fprintf(stderr, "ignoring --nodeset-output when --largest output is enabled\n");
nodeseto = 0;
}
}

ENSURE_LOADED();
next:
argc--;
argv++;
}

if (numberoftype && hwloc_calc_type_depth(topology, numberoftype, &numberofdepth, "--number-of") < 0)
goto out;
Expand Down Expand Up @@ -544,9 +533,9 @@ int main(int argc, char *argv[])
char *current, *tmpline;

/* stop if line is empty */
if (!fgets(line, (int)len, stdin))
break;

if (!fgets(line, (int)len, stdin)){
break;
}
/* keep reading until we get EOL */
tmpline = line;
while (!strchr(tmpline, '\n')) {
Expand All @@ -564,7 +553,6 @@ int main(int argc, char *argv[])
break;
len *= 2;
}

/* parse now that we got everything */
current = line;
hwloc_bitmap_zero(set);
Expand Down

0 comments on commit 9ae74eb

Please sign in to comment.