Add ability to create Linux perf event and metric json #27
Conversation
|
I used the new script to generate the core/uncore/metric JSON files, I diff the files, and they are almost the same except for the following : |
|
Thanks for looking! On reviewing the points above I don't find any issues to address in the code. A more detailed explanation is below.
If you look at the events they have filters: { It is expected even in the old code that these are events are dropped:
This is the same as above.
I don't see this. For example, in perf/icelakex/uncore-other.json:
Agreed. You can add a sort here: to reduce this. |
|
+Kan
On 11/17/2022 1:14 AM, Ian Rogers wrote:
Thanks for looking! On reviewing the points above I don't find any
issues to address in the code. A more detailed explanation is below.
3. the following uncore events for BDX/IVT/BDW-DE/HSX/ are missed
by new scripts
"EventName": "UNC_H_ADDR_OPC_MATCH.FILT",
"EventName": "UNC_H_ADDR_OPC_MATCH.ADDR",
"EventName": "UNC_H_ADDR_OPC_MATCH.OPC",
"EventName": "UNC_H_ADDR_OPC_MATCH.AD",
"EventName": "UNC_H_ADDR_OPC_MATCH.BL",
"EventName": "UNC_H_ADDR_OPC_MATCH.AK",
If you look at the events they have filters:
{
"Unit": "HA",
"EventCode": "0x20",
"UMask": "0x3",
"EventName": "UNC_H_ADDR_OPC_MATCH.FILT",
"BriefDescription": "QPI Address/Opcode Match; Address & Opcode Match",
"PublicDescription": "UNC_H_ADDR_OPC_MATCH.FILT",
"Counter": "0,1,2,3",
"MSRValue": "0x00",
"ELLC": "0",
"Filter": "HA_AddrMatch0[31:6], HA_AddrMatch1[13:0], HA_OpcodeMatch[5:0]",
"ExtSel": "0"
},
It is expected even in the old code that these are events are dropped:
https://github.com/intel/event-converter-for-linux-perf/blob/master/uncore_csv_json.py#L244
This behavior was discussed with Kan Liang on LKM.
Add Kan to confirm if IVT/BDW-DE/HSX also needs to remove these events.
In the following LKM thread, Kan only checks the BDX for the events
with three Filters.
drop_event_filter_start = [
"ha_addrmatch",
"ha_opcodematch",
"irpfilter",
]
***@***.***/
4. the following uncore event for BDX/IVT/BDE-DE/JKT/HSX is
missed by new scripts
"EventName": "UNC_H_ADDR_OPC_MATCH.FILT",
This is the same as above.
5. the following uncore event for ICX/SNR/BDX/IVT/BDW-DE/JKT/HSX
is missed by new scripts
"EventName": "UNC_I_SNOOP_RESP.HIT_ES"
I don't see this. For example, in perf/icelakex/uncore-other.json:
I double check, it has no problem.
|{ "BriefDescription": "Responses to snoops of any type that hit E or
S line in the IIO cache", "EventCode": "0x12", "EventName":
"UNC_I_SNOOP_RESP.ALL_HIT_ES", "PerPkg": "1", "PublicDescription":
"Responses to snoops of any type (code, data, invalidate) that hit E
or S line in the IIO cache", "UMask": "0x74", "Unit": "IRP" }, |
6. Because the events are sorted, it is hard to diff, I manually
do it, so may miss something else.
Agreed. You can add a sort here:
https://github.com/intel/event-converter-for-linux-perf/blob/master/uncore_csv_json.py#L280
like:
events : list[Dict[str, str]] = sorted(list(iter), key=lambda event:
event['EventName'])
to reduce this.
Thanks, I will do that.
…
—
Reply to this email directly, view it on GitHub
<#27 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AVHPWBGMSK7W3XNMVXQA3TDWIUJATANCNFSM6AAAAAARZPVG2M>.
You are receiving this because you commented.Message ID:
***@***.***>
--
Zhengjun Xing
|
|
On 2022-11-17 3:30 a.m., Xing, Zhengjun wrote:
+Kan
On 11/17/2022 1:14 AM, Ian Rogers wrote:
>
> Thanks for looking! On reviewing the points above I don't find any
> issues to address in the code. A more detailed explanation is below.
>
> 3. the following uncore events for BDX/IVT/BDW-DE/HSX/ are missed
> by new scripts
> "EventName": "UNC_H_ADDR_OPC_MATCH.FILT",
> "EventName": "UNC_H_ADDR_OPC_MATCH.ADDR",
> "EventName": "UNC_H_ADDR_OPC_MATCH.OPC",
> "EventName": "UNC_H_ADDR_OPC_MATCH.AD",
> "EventName": "UNC_H_ADDR_OPC_MATCH.BL",
> "EventName": "UNC_H_ADDR_OPC_MATCH.AK",
>
> If you look at the events they have filters:
>
> {
> "Unit": "HA",
> "EventCode": "0x20",
> "UMask": "0x3",
> "EventName": "UNC_H_ADDR_OPC_MATCH.FILT",
> "BriefDescription": "QPI Address/Opcode Match; Address & Opcode Match",
> "PublicDescription": "UNC_H_ADDR_OPC_MATCH.FILT",
> "Counter": "0,1,2,3",
> "MSRValue": "0x00",
> "ELLC": "0",
> "Filter": "HA_AddrMatch0[31:6], HA_AddrMatch1[13:0],
> HA_OpcodeMatch[5:0]",
> "ExtSel": "0"
> },
>
> It is expected even in the old code that these are events are dropped:
> https://github.com/intel/event-converter-for-linux-perf/blob/master/uncore_csv_json.py#L244
> This behavior was discussed with Kan Liang on LKM.
>
Add Kan to confirm if IVT/BDW-DE/HSX also needs to remove these events.
We never the filters on any platforms. I think we can remove them.
Thanks,
Kan
… In the following LKM thread, Kan only checks the BDX for the events
with three Filters.
drop_event_filter_start = [
"ha_addrmatch",
"ha_opcodematch",
"irpfilter",
]
***@***.***/
> 4. the following uncore event for BDX/IVT/BDE-DE/JKT/HSX is
> missed by new scripts
> "EventName": "UNC_H_ADDR_OPC_MATCH.FILT",
>
> This is the same as above.
>
> 5. the following uncore event for ICX/SNR/BDX/IVT/BDW-DE/JKT/HSX
> is missed by new scripts
> "EventName": "UNC_I_SNOOP_RESP.HIT_ES"
>
> I don't see this. For example, in perf/icelakex/uncore-other.json:
>
I double check, it has no problem.
> |{ "BriefDescription": "Responses to snoops of any type that hit E or
> S line in the IIO cache", "EventCode": "0x12", "EventName":
> "UNC_I_SNOOP_RESP.ALL_HIT_ES", "PerPkg": "1", "PublicDescription":
> "Responses to snoops of any type (code, data, invalidate) that hit E
> or S line in the IIO cache", "UMask": "0x74", "Unit": "IRP" }, |
>
> 6. Because the events are sorted, it is hard to diff, I manually
> do it, so may miss something else.
>
> Agreed. You can add a sort here:
> https://github.com/intel/event-converter-for-linux-perf/blob/master/uncore_csv_json.py#L280
> like:
> events : list[Dict[str, str]] = sorted(list(iter), key=lambda event:
> event['EventName'])
>
> to reduce this.
>
Thanks, I will do that.
>
> —
> Reply to this email directly, view it on GitHub
> <#27 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AVHPWBGMSK7W3XNMVXQA3TDWIUJATANCNFSM6AAAAAARZPVG2M>.
> You are receiving this because you commented.Message ID:
> ***@***.***>
>
|
|
I checked all the platforms, and no other core/uncore events were missed when converted by the new scripts. |
|
So this is wrong and something that can be fixed. The old logic is here: You will see that the replacement is tagged as being only for SKL, however, you have seen that it is actually applying in the old code on SPR, SKL, SKX, HSX and CLX. The replacement is happening so that the cpu@...@ expansion of the c1 doesn't happen for an uncore event. We can do the same fix in the new code here: However, the new code has to be explicit with a rewrite for every architecture it applies to (the 5 listed above). I'll patch the original CLs, but it is also possible to fix issues like this once the code is merged, which would unblock the EBS_Mode work that's needed on the Valkyrie metrics. |
Add new script that will create the Linux perf command json files. Initially the command parses ../mapfile.csv and generates the perf version of this file in perf/mapfile.csv.
Parse the perfmon event json and generate perf style json. The output exactly matches the existing event json event except some unused, by perf/jevents.py, dictionary values are dropped. Specifically the following are no longer passed through: CollectPEBSRecord, Counter, CounterHTOff, ELLC, L1_Hit_Indication and Offcore.
Reuse the existing conversion but extend for uncore and uncore_experimental. Add necessary fix ups to keep the perf json near identical to existing pref json. Differences include: - don't remove events that have a "tbd" description - sort events - ascii-fication during reading No support currently for the CSV file customization of events.
Uncore CSV files modify or duplicate events, the duplicated events can add 'filter' values to the event that make it more specific. Metrics may also be generated. Differences from previous approach: - When a new event is created then this is updated rather than the existing event. - If an event has a filter and is in the CSV file but with no filter, then filter value isn't dropped. - Events without descriptions aren't dropped. - There is clean up of strings to remove unnecessary whitespace and avoid duplicated descriptions. - For tigerlake there are duplicated events with different units, the first rather than the second event is dropped (unit 'imc' rather than 'h_imc').
Support conversion of TMA metrics and E-core TMA metrics. Verification that events in the metric exist is performed, from this a minimal set of fixups are created and more metrics supported. For BDW-DE 3 metrics are explicitly dropped due to missing events. The valkyrie metrics aren't inserted currently.
Load extra metrics such as ICX/metrics/perf/icx_metric_perf.json. Use the save_form function so that missing events are identified.
This commit introduces a brief section describing TMA files.
This commit releases v1.07 event files for Sapphire Rapids.
This commit releases v1.16 event files for Alder Lake.
This commit releases v1.08 event files for Tiger Lake. An uncore experimental event file is also introduced.
This commit releases v1.08 event files for Sapphire Rapids.
The mapfile Version column primarily uses capital V. This commit fixes a few of my previous updates with lower case v.
Leftover from the repository rename.
|
I added the rewrites such as for SKL here: |
|
I tried the updated scripts, CLX/ICX/SPR/SKL/SKX "cpu@" issue disappeared. |
|
Agreed, the rewrite applies more than just to HSX: It looks like we could use format validation on top of event validation for metrics. Fwiw on a HSX: |
Add a script to create the Linux perf event and metric json. When the event and metric data is updated running this script will create json files for the Linux perf tool that can be mailed to the Linux kernel mailing list. A single script is used so, for example, metrics can verify events in the formula against those in the json files.