Palmetto slower after firmware update #132
Comments
|
Update: The frequency was reduced by 1/3rd. Here's the old /proc/cpuinfo: processor : 0 ... processor : 31 timebase : 512000000 And here's the new /proc/cpuinfo: processor : 0 ... processor : 31 timebase : 512000000 |
|
On Wed, 2015-04-08 at 13:59 -0700, skachm wrote:
Two things:
Cheers,
|
|
The frequency measures at 2.067 GHz for all of the cores: $ sudo ppc64_cpu --frequency I tried several options in linux power governor: performance, userspace set to max frequency, etc. without any change in the frequency. $ sudo cpufreq-info -c 0 |
|
Also the output for performance: $ sudo cpufreq-info -c 0 (Ubuntu 14.10, Palmetto without any hardware additions or changes, op-build firmware v1.0 built natively on a Palmetto.) |
|
On Thu, 2015-04-09 at 12:34 -0700, skachm wrote:
What governor is active and how did you configure it ? If it's running the "on demand" governor, it will go to the lowest You can try echo'ing "performance" in all the scaling_governor files Then make sure, using ppc64_cpu --frequency, that the frequency measured
|
|
I tried performance and userspace to 4.32 GHz, ppc64_cpu reported 2.067 GHz under both governors. |
|
On Thu, 2015-04-09 at 16:18 -0700, skachm wrote:
Ok. I observed that a while ago too but for me at least it seems fixed Cheers, |
|
On the original Palmetto code we did not have the OCC supported. The OCC ("On-Chip Controller") is a device inside the P8 module that varies the voltages and frequencies based on thermal readings, power cap settings, and Linux governor requests. The original code simply set the frequency to 3ghz, while the latest code has the OCC which can run at a large frequency range. (It looks like 4.32 to 2.067 is your processor module's range) I believe the /proc/cpuinfo represents the nominal frequency of the module that Hostboot told skiboot/Linux via the devtree. There are three frequency points that each module is qualified at: safe, nominal, turbo. Nominal is the "label" frequency. In this case the /proc/cpuinfo is showing 2ghz which is the safe frequency for all modules. Whenever the OCC encounters a problem it will force the processor to safe frequency to minimize potential of hardware damage. So, since I suspect your processor was running in safe from the boot time (based on this /proc/cpuinfo value), that would seem to indicate that the OCC failed to start during Hostboot. Do you see any indication of this in the early console output? You might see a message somewhere between "ISTEP 18.0" and "ISTEP 21.1" messages indicating we failed to start the OCC. It is also possible that the OCC is failing, for some reason, during runtime. @ozbenh mentioned seeing this before. I think when we have seen this before (OCC failing at boot) many of the cases were to outdated APSS code. The APSS is an FPGA related to the power supplies and voltage regulators (Analog Power something something). The code for this FPGA is also on github but I don't know how to update it: https://github.com/open-power/apss @guillesilva / @sbroylesibm - Is this something you can help with? I recall there were some i2c commands you could run from the BMC to check the APSS status and code level to see if it is running properly and what the state of it was. If all of the APSS code loads that Tyan originally shipped are out of date, we'd probably recommend that they put an update and instructions on their website. @Erich-Hauptli / @kyle-tyan - You might want to keep tabs on this issue. I suspect as we get more "adventurous" users, like @skachm, others are going to run into similar problems. |
|
On Thu, 2015-04-09 at 20:27 -0700, Patrick Williams wrote:
No, on any not-too-ancient kernel, it should display a frequency based
What we have observed in the past (but doesn't seem to be happening on Reading the PMSR would still return the pstate corresponding to 4.3Ghz There was no log of OCC failures from HBRT.
Is v5 still current ? One problem I've had is the BMC i2c bus getting
First make sure you have BMC and PNOR completely up to date, at least Cheers, |
|
FYI: The latest APSS level is 05, you can check it from the BMC with: Get APSS level/usr/local/bin/i2c-test -m 1 -b 2 -s 0x38 -rc 1 -d 0x01 Also, as a general sanity check on the APSS you can scan the IIC bus to the APSS to make sure the IIC slave address is responding with: Scan IIC bus, APSS is on address 0x70/usr/local/bin/i2c-test --scan -b 2 |
|
I first checked it out less than 2 months ago (around March 26th), but I checked out a fresh copy of the repo, built it, reflashed the firmware and the server seems to be working now. $ sudo ppc64_cpu --frequency |
|
Glad it is working for you. When you build a level for something other than testing, it is better to checkout a tag after the clone rather than HEAD. This way you get some content that has already been qualified by us. From: skachmSent: Monday, April 13, 2015 8:11 AMTo: open-power/op-buildReply To: open-power/op-buildCc: Patrick WilliamsSubject: Re: [op-build] Palmetto slower after firmware update (#132)I first checked it out less than 2 months ago (around March 26th), but I checked out a fresh copy of the repo, built it, reflashed the firmware and the server seems to be working now. $ sudo ppc64_cpu --frequency —Reply to this email directly or view it on GitHub. |
Changes Included for package hostboot, branch master: 4638dc5 - Dan Crowell - 2019-12-18 - Add current istep into TI SRC 75c0908 - Dan Crowell - 2019-12-17 - Switch DECONFIG over to DELAYED_DECONFIG 0921b80 - Mark Pizzutillo - 2019-12-16 - Add new DDIMM spd version 0_3 and update UTs b47fb59 - Artem Senichev - 2019-12-16 - Replace descriptions with JEDEC register names Changes Included for package pnor, branch master: 6fb8d91 - Dan - 2019-12-18 - Merge pull request open-power#132 from open-power/revert-131-python3-as-python 22d22f9 - Dan - 2019-12-18 - Revert "Allow sbeOpDistribute to decide its interpreter" 21bd41e - Dan - 2019-12-16 - Merge pull request open-power#131 from stewartsmith/python3-as-python 052e1bb - Stewart Smith - 2019-12-13 - Allow sbeOpDistribute to decide its interpreter Signed-off-by: hostboot <hostboot@us.ibm.com>
Changes Included for package hostboot, branch master: 4638dc5 - Dan Crowell - 2019-12-18 - Add current istep into TI SRC 75c0908 - Dan Crowell - 2019-12-17 - Switch DECONFIG over to DELAYED_DECONFIG 0921b80 - Mark Pizzutillo - 2019-12-16 - Add new DDIMM spd version 0_3 and update UTs b47fb59 - Artem Senichev - 2019-12-16 - Replace descriptions with JEDEC register names Changes Included for package pnor, branch master: 6fb8d91 - Dan - 2019-12-18 - Merge pull request open-power#132 from open-power/revert-131-python3-as-python 22d22f9 - Dan - 2019-12-18 - Revert "Allow sbeOpDistribute to decide its interpreter" 21bd41e - Dan - 2019-12-16 - Merge pull request open-power#131 from stewartsmith/python3-as-python 052e1bb - Stewart Smith - 2019-12-13 - Allow sbeOpDistribute to decide its interpreter Signed-off-by: hostboot <hostboot@us.ibm.com>
The Palmetto I built and upgrade to firmware 1.0 is running about 30% slower after the update. (The blackscholes benchmark that took 256s with factory firmware now takes 378s with firmware 1.0 for example.) I suspect the CPU is downclocking itself, but I'm not sure how to prevent that. Are there settings that I should tweak before building the firmware?
The text was updated successfully, but these errors were encountered: