Custom Meters from external source

[BenBE]

Based on several requests (cf. #406, #899^historic) and as the general case would be quite interesting, it might be good to have a generic "Custom Script Meter".

The basic idea for such a meter would be, that htop could allow for up to 4 external "meter scripts" that it would execute and run in the background. The output of the program would then be parsed into a meter as follows:

bar_range:0 1000
bar_data:666 123 42 23
bar_color:4 6 2 1
bar_label:789/1000
text:Custom meter showing 666 foo per 1000 bar. 42 baz happened.
led:666 foo / 123 bar / 42 baz

The lines have the following meaning:

• bar_range: Specifies the min and max limits of the bar meter. Values in bar_data are scaled accordingly
• bar_data: Up to seven integer values separated by space giving the data to show in the bar graph.
• bar_color: The colors for up to seven channels denoted by up to 7 integers separated by spaces (1=blue, 2=green, 3=cyan, 4=red, 5=magenta, 6=yellow/orange, 7=grey, 8=shadowed). Missing colors are defaulted to 7 (grey).
• bar_label: The text to display right-aligned in the meter. Text left-truncated at spaces if too long. Merges color with displayed bar values.
• text: Text to display in the text meter style.
• led: Text for the "large digits" display. Digits are automatically displayed as large retro 7-segment LED characters. Otherwise identical to text. If not present, caption from text is used.

The option value starts directly after the colon (:). For options giving plain numbers, these need to be separated by exactly one space, no decimal separator or thousands separator may be given. Invalid values are dropped and the whole record is replaced by its default values. Settings for bar_range and bar_color are remembered (thus only need to be given once), other values must be given at 1Hz or higher. Parsing for the values of one second ends when encountering a blank line. Last record parsed is displayed.

[fasterit]

Thanks for writing the idea up. I think this would make a lot of sense.

Just a few random thoughts from reading through your initial proposal:

variable=value is much more common than variable:value, I'd stick with "="

htop could just listen to a named pipe (FIFO) and programs sending "meter info" there would get bars or column info as requested displayed. Records need to be smaller than PIPE_BUF(512 Unix, 4096 on Linux) to allow sequenced concurrency.

I'd put everything in one line terminated by <LF> (and silently drop <CR> before <LF> for the Windows folk) to make it more a readable protocol, like:

config meter=room_temp color="red orange green blue"
data meter=room temp val=42

and for columns

config col=importance label=IMP default_color=grey
data col=importance val="1=1000, 2=555, 4700=222" color=1=red
# with 1, 2, 4700 being the PIDs where these values are show in the "IMP" column

(you see, I'd use named colors or possibly html color codes (#AABBCC) as again those are more universal than whatever a specific terminal emulator (or htop profile) considers to be "3")

config would just need a "on / off switch" and an option to set the initial umask for the FIFO
/DLange

[natoscott]

The lines have the following meaning:

* `bar_data`: Up to seven integer values separated by space giving the data to show in the bar graph.

May as well accept (parse) floating point since values ultimately become 'double's within htop?

One important consideration is counter values vs non-counter values - counters are (almost always) converted to rates (the change in value divided by the change in time) over consecutive samples, so either:

• we need some way to flag counters vs non-counters (if htop is to do the rate conversion, not the script)

• else, the script needs to be long-running so that it can perform multiple samples and provide already-rate-converted values back to htop. In this case the script would also need to be told what the sampling interval is. :|

I think the former approach makes more sense because otherwise a change in the sampling interval (e.g. through the htop UI) would require some kind of side-protocol to inform the script of the new interval.

[BenBE]

the idea is having the script to be long-running and providing htop with the values as they should be displayed. If the meter update frequency is less than 1Hz this is fine, as only the last record returned from the script is shown. And even if we wanted to communicate with the script how often we are expecting updates, we could just write update:1000 to indicate updates are expected every 1000ms.

[sevagh]

Lately I find myself having htop open in one pane, and nvidia-smi -l 1 in another, all the time, and wish I could do both (and this is the appropriate way).
Since this is marked as good-first-issue, could you give some pointers on:

• Which source code files this would most likely be implemented in

[BenBE]

Since this is marked as good-first-issue, could you give some pointers on:

• Which source code files this would most likely be implemented in

As this would become a new Meter for htop this would become a new *Meter.[ch] file in the main directory, as the interface for this would most likely be as platform independent as possible. If there are things that are to be split between different platforms you'd put a generic implementation in the eneric/` subdirectory for all platforms that can use a common implementation. E.g. reading the standard out from a sub-process can be implemented the same on all POSIX-compliant platforms, but would require a platform-specific implementation on Microsoft systems. Thus this would be a candidate for a function that might need some additional care.

To get an idea how meters typically work have a look at e.g. the CPUMeter or the MemoryMeter. For simple text meters the UptimeMeter or the SysArchMeter are good candidates for having a look at how things work. The SysArchMeter also demonstrates the aspect of splitting things across different platforms.

To have the final ExternalDataSourceMeter be as extensible as possible it's recommended to formalize its protocol in a first step. The final documentation for this protocol should land in docs/ExternalMeters.md (or similar) as part of the PR. Some of the things that such a proposal should address are outlined in the above discussion, although some aspects may come up as things are written - so no worries if the proposal has to be edited a bit after its initial draft.

When processing the data (stream) from the external source be sure to take care of defensive programming. The parser should assume malformed data coming in and treat valid data as the exception. Note: htop will often enough run as root and thus special care about code quality should be taken. Better be on the defensive side of things,

[smalinux]

Hi, I was thinking about these custom meters.
Why bar_data limited to four values only? I understand that valuably to CPUMeter represents these 4 nice colors.
But in the case of new general custom meters maybe the external sources more or less than 4 values.

Also, I think variable=value is much more common than variable:value, I think most Linux config files follow this style. for example: audit daemon config file: /etc/audit/auditd.conf

Last thing: What's the range of numbers custom Meters will take?
[Here] CPUMeters takes numbers from 1 to 'cpus'. and the rest of Meters start from "1 << 16"
I'm not sure what happened is there as I'm not familiar with that so much. but I think these keys will need to reorder.
say for example :
0 to 100 for CPUMeter.
101 to 1000 for these generic custom meters.
1001 to 2000 for current ordinary meters.
etc... I think this is easier to understand ^^"

[BenBE]

Why bar_data limited to four values only? I understand that valuably to CPUMeter represents these 4 nice colors.
But in the case of new general custom meters maybe the external sources more or less than 4 values.

Usually it's not very wise to have more than 4 values displayed in one (bar) meter as space is quite limited. Also you need colors for those values which makes those meters stand out too much if too colorful. It's not a hard limit, but a general recommendation.

Also, I think variable=value is much more common than variable:value, I think most Linux config files follow this style. for example: audit daemon config file: /etc/audit/auditd.conf

That's up to the protocol decision what we settle at. The above description mostly was some grough description of what could be.

Last thing: What's the range of numbers custom Meters will take?
[Here] CPUMeters takes numbers from zero to n. and the rest of Meters start from "1 << 16"
I'm not sure what happened is there as I'm not familiar with that so much. but I think these keys will need to reorder.
say for example :
0 to 100 for CPUMeter.
101 to 1000 for these generic custom meters.
1001 to 2000 for current ordinary meters.
etc... I think this is easier to understand ^^"

Opposite to the process fields, the meters currently have no fixed IDs. Instead Meters are identified by their MeterClass. The ID you see in those code snippets is for having multiple instances of the same meter. Don't worry about that too much yet as having more than one custom meter can be added easily later on (if the source allows for easy parameterization).

[Noctumsempra]

It would be nice if the character that makes the meter could be changed from "|" to the one the user prefers. Is there ANY way to change that ugly default pipe for a better visual character with more body defined by us? Thanks in advance.

[BenBE]

@Noctumsempra I split this into a separate issue (#647) as this issue here is not about UI, but backend functionality.

[Noctumsempra]

@Noctumsempra I split this into a separate issue (#647) as this issue here is not about UI, but backend functionality.

Thank you! And also sorry for posting in the wrong issue.

Regards.