Say NO to Paywall of PCI-E Bifurcation — Gray's grind

[GrayXu]

https://www.grayxu.cn/wiki/pcie-bifurcation/

[su-thomas]

thanks for the write up! I'm very interested in that OCP + M.2 PCIe carrier card you have. Where did you purchase it from?

[GrayXu]

AliExpress

[su-thomas]

thanks for the info, can you provide link? I'm having trouble finding it

[GrayXu]

I'm sorry, but the link I purchased at that time has also expired. The principle of these bifurcation cards is similar to PCIe Riser, and they also have a clock buffer (such as 9DB433).

You can find similar open-source PCB designs here: https://oshwhub.com/wesd/pciex16-zhuan-ocp-nvme.

[su-thomas]

I'm not able to manufacture my own one, just looking to the buy assembled product. Can you share the title of the aliexpress item from your purchase history? that might help me find another listing

[GrayXu]

Due to my location in mainland China, the situation may be different from yours. Searching on Taobao with the keyword "OCP bifurcation" will get many results, but I am not sure if they can be shipped overseas.

[su-thomas]

I'm still having trouble finding it on taobao - could you send a few links of the product results you get?

…

On Fri, 22 Sept 2023 at 19:56, Guanglei Xu ***@***.***> wrote: Due to my location in mainland China, the situation may be different from yours. Searching on Taobao with the keyword "OCP bifurcation" will get many results, but I am not sure if they can be shipped overseas. — Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEB3SFYRFS6FXMZRBVZY3RLX3VOFVANCNFSM6AAAAAARM3IGXM> . You are receiving this because you commented.Message ID: ***@***.***>

[GrayXu]

@su-thomas stuff like this, https://item.taobao.com/item.htm?spm=a21n57.1.0.0.5cbf523ck7pgh3&id=725009349117&ns=1&abbucket=7#detail
but it should be noted that most of them are products from small workshops.

[su-thomas]

thanks for that link, I've ordered one. I'd like to ask more about the OCP LSI HBA you're using - is that the ZT System model?
Does it work in IT mode?
What is the firmware?
Can you share output of sas2flash -listall ?

[GrayXu]

@su-thomas I don't know the exact model, but it should be an Inventec OCP card. I'm using it in IT IR mode with zfs.

$ ./sas2flash -listall
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

Num   Ctlr            FW Ver        NVDATA        x86-BIOS         PCI Addr
----------------------------------------------------------------------------

0  SAS2308_2(D1)   15.00.00.00    0f.00.02.03    07.35.00.00     00:01:00:00

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.

[su-thomas]

thanks for the info! my mistake on the command, actually need output of sas2flash -list if you don't mind

[GrayXu]

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

        Controller Number              : 0
        Controller                     : SAS2308_2(D1)
        PCI Address                    : 00:01:00:00
        SAS Address                    : 500a0d1-0-021f-dfe0
        NVDATA Version (Default)       : 0f.00.02.03
        NVDATA Version (Persistent)    : 0f.00.02.03
        Firmware Product ID            : 0x2714 (IR)
        Firmware Version               : 15.00.00.00
        NVDATA Vendor                  : Inventec
        NVDATA Product ID              : SAS2308
        BIOS Version                   : 07.35.00.00
        UEFI BSD Version               : N/A
        FCODE Version                  : N/A
        Board Name                     : MEZZ2308
        Board Assembly                 : N/A
        Board Tracer Number            : N/A

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.

[su-thomas]

hmm it looks like it's in IR mode, not IT. I worry how I can find IT mode firmware to flash, can't find anything about this card online

[GrayXu]

@su-thomas If you hadn't mentioned it, I probably wouldn't have figured it out lol. It is indeed in IR mode, but I've been using it directly connected all along.
There isn't much information about this card, but changing the mode doesn't necessarily require flashing the original vendor firmware.

[su-thomas]

I'm not aware of a way to change it to IT mode without flashing IT firmware. Furthermore, using it in IR typically won't give raw drive access (which is advised against for ZFS) - are you able to access drive smart data with smartctl?

[GrayXu]

1. Flashing is definitely necessary, but what I mean is you may not necessarily need to flash Inventec's firmware. And of course there is also a possibility that this card cannot be flashed. I searched and it seems that OCP form factor SAS cards are indeed rare, so there are even fewer instructions related to flashing firmware.
2. I am still using it in JBOD-like mode, without configuring logical drives, so I can access SMART data.

[su-thomas]

Do you think a 9207-8i firmware would be the most ideal one to flash?

How do you get raw drives with IR firmware?

[GrayXu]

• I guess so.
• Actually, some modern IR firmware can also provide raw access. I didn't know before that this SAS2308 chip could be used in this way. As long as you don't configure a logical volume, you can use the raw drives directly.

[gabizro]

Hello
What motherboard is Asus Z77 more exactly?
Tried this on HP8300 with I7 3770 but all I get is x16 port downgraded to x8 but no device detected also no new PCIe port appear on device manager.
Maybe I can check some BIOS modules to mod my BIOS

[GrayXu]

@gabizro
P8Z77, but I think the Q77 chipset also supports this modification. It might be an issue with your bifurcation card.
I have only succeeded in bifurcating the PCIe of the south bridge chipset (1x4->4x1) through modding BIOS.

[gabizro]

Thanks for info.
In the meantime looking for differences of 77 chipset from little info I could find Z77 say it support 1x16 or 2x8 or 1x8 2x4 but for Q77 "not specified" or 1x16
I tracked traces on board and find unpopulated resistors but once I solder one or both x16 port is switched to x8 and only device connected to x8 port is detected.
Tested with oscilloscope, in x16 mode even with no device connected there is some activity on all 16 lanes on socket but when switched to 2x8 or 1x8 2x4 only on first 8 lanes are same activity as before rest of them have no activity.

[GrayXu]

@gabizro
Regarding the chipset, I have also seen examples of successful bifurcating with Q87, so I think Q77 should be fine as well. I have also seen successful examples of modifying CFG values by changing the reserved resistors on the motherboard. However, in your specific scenario, I am not very professional about this and not sure if it is possible to determine whether the bifurcation was successful based on lane activity. It is also possible that some models of motherboards simply do not support such hacking

[gabizro]

Still looking for info I get this from some datasheet
PCI Express x16 from processors can be one PCI Express x16 or two
PCI Express x8 or One PCI Express x8 + Two PCI Express x4 by jumper setting with
C216 chipset. It only supports one PCI Express x16 with Q77 chipset

[stillyahoo]

Did you have to insulate the conductive bridges after the modification to isolate them from the socket pins? I've been looking into this for an 8th gen cpu on Q370 chipset after being directed to your findings (thank you so much for the work) and from more knowledgeable people than me the question being asked is:

"..do you know how the motherboard pulls those pins high? If it is through high-value pull-up resistors then shorting ground to them will be fine, but if it uses a transistor to pull it high then you might find yourself shorting VCC to ground..."

[GrayXu]

@stillyahoo
good question
First, I didn't make any additional changes.
Second, while different motherboards might vary in implementation, successful examples with reserved resistors on modified motherboards suggest that it should mostly be the former.