https://www.amazon.de/dp/B09MSB4TXT is working perfectly

[frankol]

thank you for your informations regarding bifurcation!
I did as you suggested and soldered the resistors and its working perfectly

[mgutt]

Could you please add some photos?

[frankol]

at the sata port is a 2tb ssd dissasembled. it fits perfectly

[xiaoran007]

Has anyone tried it on the M910X?

[badger707]

@frankol - nice, thanks for confirming!!!
I could not make it working for me initially when probing/exploring this and just give up thinking its not going to work... So by seeing your post my thinking now is that either my my PCIe/M.2 card (PH45) is faulty (aliexpress) or my PCIe riser board/card SKU is not the one what I need...
Can you reconfirm your riser card Lenovo SKU, or maybe post a label picture?
Now I have ordered another PH45 card as per your picture and will try to give it another try...

[frankol]

[...]Can you reconfirm your riser card Lenovo SKU, or maybe post a label picture?

I ordered this one on ebay: https://www.ebay.de/itm/125280817080
" PCIE16 Erweiterungsgrafikkarte für ThinkCentre M920x M720q P330 Tiny5 01AJ940"

Dont think its a original one :-), but its working

[su-thomas]

hi @frankol and @badger707 - I am curious to learn more about this. Where did the resistor diagram come from? (I do not see it in the repo).

Additionally, which resistor pads sets which CFG?

[frankol]

hi @frankol and @badger707 - I am curious to learn more about this. Where did the resistor diagram come from? (I do not see it in the repo).

Additionally, which resistor pads sets which CFG?

There is a schematics availabe. Just try to search for "IQ3X0IL.-.NM-B511.REV1.0"

Regarding the pads and their function:

[badger707]

@su-thomas schematics you can get from this repo here. However I think you were looking for Lenovo P320 (non-tiny) solution as per you post here -- if your board and chipset on your box is same then you can try steps above from @frankol, if not same -- then you'd need schematics and boardview for your specific motherboard model. Even then it may or may not work well with your board BIOS. So it depends...

[su-thomas]

thanks for the info @frankol and @badger707

out of curiosity, when you do this mod on the m920q, does the BIOS show anything different? or it just silently works?

[frankol]

it just silently works

[systemerror-codenotfound]

any idea if this works in a M910X?

[marcojk]

at the sata port is a 2tb ssd dissasembled. it fits perfectly

Hi frankol. Can I ask what's that big heatsink in the first picture for? Does it have a lenovo PN?

[frankol]

at the sata port is a 2tb ssd dissasembled. it fits perfectly

Hi frankol. Can I ask what's that big heatsink in the first picture for? Does it have a lenovo PN?

Hi marcojk, sorry its a custom build

[marcojk]

Great work!

[frankol]

My final configuration is 5x NVME (2 from the normal nvme slots, 2 from the pcie-card and 1 from the wlan port (pcie-x1) with an cable adapter) and one ssd. A awesome storage monster :-)

[dxx-223]

Hi frankol. Out of interest, which CPU are you using?

The config table says "CFG[2], CFG[6:5] & CFG[7] are relevant for H and S-processor line only",
but I'm not seeing any -H or -S processors that fit socket LGA1151.

Asking because I can't get this to work on my M920q and I'm wondering whether it's because:

1. I'm using a G5500T to test although Intel Ark claims it supports 1x16, 2x8, 1x8+2x4
2. my soldering isn't good enough
3. my motherboard is only rev 1.0 unlike yours which seems to be rev 3.0

My PCIe x16 riser, and dual M.2 m-key expansion card work in another M720q so I'm ruling them out as the issue.

Frustrating!

[frankol]

hi,
im using Intel® Core™ i5-8500T cpu.
You can test the soldering with a multimeter.
What cpu are you using in the m720q setup?

[dxx-223]

My M720q has an i7-9700T - which I'll move across to the M920q if I can get bifurcation to work.
Just to clarify: my M720q is not set up for bifurcation. I only mentioned it as a way to test the x16 riser, and dual M.2 expansion card work. The M720q is factory-standard x8 mode, with an M.2 card in the first slot in the expansion card, so using lanes 0-3.

I've just tried the M920q with a spare i5-8500T but no change.
My soldering seems to be OK after some multimeter checks. I'm using 1K 0402 resistors.

Using your picture for reference, I've performed tests indicated by the loops in pink:

Has anyone else managed to get this to work?
If so, could you please report in with with revision of your IQ3X0IL motherboard you have?

[dxx-223]

Sorry for the comment spam but it seems bifurcation is working for me.
I tested each slot of the dual M.2 expansion card with an NVMe M.2 drive and both times it worked.

What I'm really trying to get working is an ASM2142 USB 3.1 M.2 card from AliExpress.
But it never works in either slot of the dual M.2 expansion card, nor in a simple 1-slot M.2 expansion card.
Turns out it doesn't even work in the M.2 on the underside!
Maybe the USB card just doesn't play well with my motherboard, or doesn't like lane reversal. It works in the M720q.

The whole point of me trying to install this USB card is because the built-in USB root host is dedicated to a VM so I need another USB root host for the 2nd VM.

I can't use the USB from a B-key M.2 socket, or the built-in WLAN socket, or even the USB display daughter board because they all share the same USB root host that's dedicated to the 1st VM.

Hope this helps someone!

[frankol]

nice to hear bifurcation is working in general.
Regarding the usb card, you could try the wifi pcie-x1 slot with an nvme adapter, like this one https://amzn.eu/d/4ou0rEt .
Its not a good one, you see what i mean

[dxx-223]

If only I had read more of the documentation before commenting...

Regarding my earlier comment:

The config table says "CFG[2], CFG[6:5] & CFG[7] are relevant for H and S-processor line only",
but I'm not seeing any -H or -S processors that fit socket LGA1151.

H and S-Processor lines aren't about CPU suffixes (e.g. 8100H) but more about the package / socket:

[zzs95]

Has anyone tried it on the M910X?

any idea if this works in a M910X?

I designed a tester PCB of LGA1151. Use a multimeter to measure the pin positions according to the solder joints of the CFG resistor on the m910x.

This measurement seems reasonable. CFG5 and CFG6 are in the same position as m920q, and CFG2 is in a similar position.

This M2 was inset in the first slot of the adapter (0-3 lanes). It's matched with the PCIE mapping.

Edit:
I tried with PH45, however 2nd slot still didn't work. The conclusion of my attempts is: m910x is able to split lanes but cannot recognize the second device. I would try another kind of adapter for 1 NVME M2 and 1 SATA M2. And the SATA port in adapter will connect to the soft SATA cable that the motherboard comes with, through the male-esata extender.

Edit 2:
1 NVME M2 plus 1 SATA M2 method work fine without bifurcation. It's a easy way to and 2 drives.
systemerror-codenotfound did the verification on m910x. The resistor points are correct and the m910x can do bifurcation.

[su-thomas]

Due to reversing the PCIe lanes, is it expected something like a single x1 card would no longer work? If someone could test, that would be helpful. I'm having issues when testing a x1 2.5Gbe card after this bifurcation modification.

[dxx-223]

Due to reversing the PCIe lanes, is it expected something like a single x1 card would no longer work? If someone could test, that would be helpful. I'm having issues when testing a x1 2.5Gbe card after this bifurcation modification.

I see the same problem with my two M.2 USB host cards, which want either x1 and x2 lanes.
Neither work in the PH45 adapter BUT they do individually work in a simple, passive single-socket M.2 riser like this one:

(Illustrative: not exact same model that I'm using)

My conclusion is that clock-buffer-only bifurcation adapter cards like the PH45 only really work for NVMe drives for some reason that I don't understand.

I'm about to buy a cheap dual M.2 adapter card with a real PCI-PCI bridge chip, e.g. ASM1182, to see my USB host cards work with that. If so, that means I'll actually need something like a Glotrends PA20 (ASM2812) or StarTech PEX8M2E2 (ASM2824) in the longer term.

ASM2812: PCIe Gen3 but only x4 lanes from upstream adapter to CPU - x4 shared across both M.2 devices
ASM2824: PCIe Gen3 and x8 lanes upstream from adapter to CPU - so full speed for both M.2 devices

There are cheaper ASM2812-based cards on AliExpress, usually with shiny red heatsink, but beware that there is next to zero gap between the M.2 sockets. People have reported that they can't fit two M.2 devices if either has a heatsink. The Glotrends PA20 has more gap but unfortunately costs way more.

I'll report back regarding ASM1182 compatibility, hopefully next week.

[su-thomas]

Due to reversing the PCIe lanes, is it expected something like a single x1 card would no longer work? If someone could test, that would be helpful. I'm having issues when testing a x1 2.5Gbe card after this bifurcation modification.

I see the same problem with my two M.2 USB host cards, which want either x1 and x2 lanes. Neither work in the PH45 adapter BUT they do individually work in a simple, passive single-socket M.2 riser like this one:

Hmm, in my case, I am using a PCIe form factor x1 card directly in the PCIe slot, and it is not being detected. Hence I'm wondering if the the PCIe lane reversal means that cards less than x4 cannot be detected because none of their lanes line up with the 0th lane, being not the first one anymore? I'm not sure personally.

[dxx-223]

Due to reversing the PCIe lanes, is it expected something like a single x1 card would no longer work? If someone could test, that would be helpful. I'm having issues when testing a x1 2.5Gbe card after this bifurcation modification.

I see the same problem with my two M.2 USB host cards, which want either x1 and x2 lanes. Neither work in the PH45 adapter BUT they do individually work in a simple, passive single-socket M.2 riser like this one:

Hmm, in my case, I am using a PCIe form factor x1 card directly in the PCIe slot, and it is not being detected. Hence I'm wondering if the the PCIe lane reversal means that cards less than x4 cannot be detected because none of their lanes line up with the 0th lane, being not the first one anymore? I'm not sure personally.

After some more Googling, it seems lane reversal support needs to be at both ends, so both CPU and end device. This might explain why we're both seeing issues.

I wonder if there adapters or cables that perform x4 lane reversal at the M.2 M-key socket or maybe x4x4 reversal at the PCIe slot.

Proof of concept would be via buying a PCIe x4 extension cable, desoldering all 4 lanes, then re-reversing / connecting what would be CPU lane 3 to PCIe slot lane 0. Does that sound about right?

I have a "PEXP4-SX" x4 extension adapter much like this one:

I can try to reverse one lane to see if that makes any difference.
But a concern is that it adds about 2cm between the PCIe x8 socket and whatever is plugged in, so there might not be enough room for the PH45 after.

[dxx-223]

As I suspected, the PEXP4-SX takes up too much room, such that the PH45 dual M.2 card is blocked by the metal lugs on the CPU heatsink. These lugs are to support the 2.5" drive caddy. Even if I chopped those off, the PH45 would still be blocked by the CPU fan assembly.

An alternative to the rigid extension adapter would be a cable like this:

... but I think even then, the PH45 card would sit too high, or too low, or still be blocked by the heatsink/fan assemblies.
That's without even considering the cable itself is 20cm and would take up space by being folded.
I guess you could have the cable extend out of the case but that's really ugly.

Yet another option would be to attempt lane reversal at the M.2 end.
e.g. buy an M.2 M-key x4 extension cable/adapter:

and then break into the cable halfway to swap lanes.
This would probably suffer from lack of internal space too though.

The ideal solution would be a version of the Lenovo 01AJ940 riser that performs lane reversal.

[dxx-223]

I modded my PEXP4-SX by desoldering all the socket legs that connected to PCIe lanes, then soldered wires to connect M/B socket / CPU lane 3 to PEXP4-SX lane 0. But my M920q didn't seem to detect either M.2 card, not even the one that only wants x1 lane.

Tried desoldering the PRSNT#2 for x4 lanes on PCIe pin B31 but that no effect either.
Tried all the different BIOS PCIe speed settings: auto, Gen1, Gen2, Gen3 - no effect.

It could be that my soldering isn't good enough, the PEXP4-SX is somehow broken, there's too much 'noise' on the soldered wires? Run out of ideas really.

Maybe NVMe drives work well because they always try to negotiate a full x4 and the PCIe interface chips on NVMe drives are good enough to handle lane reversal as well?

[galvesribeiro]

I'd pay some $$$ for it.

@j4cbo Yeah, put some Patreon or some other way for us to pay for it fully backed or at least contribute for the schematics/BOM and your time so we can send it to manufacture/assembly somewhere like PCBWay.

Now that you guys mentioned, yeah, I'm curious as to how all this is working. Are you embedding a PCIe switcher on the board?

@wociscz

similar as your setup 2x 8tb u.2 at the back 2x 8tb at the front, 1tb in m.2 a/e, and 10gbe sfp+ mellanox.

You are not using the original case for this right? I'm trying to imagine the U.2 drivers on the back and front AND a Mellanox in that tiny case 😄

[wociscz]

You are not using the original case for this right? I'm trying to imagine the U.2 drivers on the back and front AND a Mellanox in that tiny case

Sure. 3d printing for the rescue. Another thing which I have to solve is the power line - where to pull it from. The tiny flat sata cable is... is tiny for the purpose.

[j4cbo]

There's no PCIe switch, here's an overall block diagram of the board:

The buck converter necessary is to avoid overloading the 3.3v rail on the motherboard. There's some additional complexity in dealing with PCIe ancillary signals like CLKREQ# and PERST# (which is part of what's broken on V1), but that's the main gist of it. Once I've got the second prototype brought up I'll look at options for producing a batch of them.

[galvesribeiro]

Ahhh nice! So you are pulling 4 more lanes from the PCH. Makes sense. Cool! Looking forward for it! 😄

[struthio]

@j4cbo 12PCIe lanes ? Doesn't M920q had only 8 lanes in PCIe Slot ?

[kitor]

@j4cbo 12PCIe lanes ? Doesn't M920q had only 8 lanes in PCIe Slot ?

Did you even read this thread? It was written at least 4 times already: Riser slot has 8 PCIe lanes coming from CPU and 4 PCIe lanes coming from PCH (chipset) making it 12 total. This is even visible on the drawing just above your comment...
There were two official risers: one was x4 - used PCH lanes, x8 - used CPU lanes.

We used x8 riser and CPU lanes for bifurcation mod, which had the unfortunate issue of not working well with some devices as PCIe lanes were reversed in that configuration - which is not supported by majority of devices.

@j4cbo combined all knowledge from this thread and created a riser that both hardware reverses the lines to make it work as expected by most devices + pulled those extra PCH lanes.

BTW: @j4cbo any particular reason why x4 slot is wired to CPU instead of wiring both m.2 slots to CPU and x4 to PCH?
I guess that's just purely related to order of lanes available on riser slot? (PCH lanes available at the very end) which makes this the easiest to implement?

And BTW2: With similar approach it should be possible to make x8 (not bifurcated) + m.2 riser, right? That would be cool for M720 users - would allow them to plug 2nd NVME and keep x8 slot at the same time. And M920 too, depending on scenario :)
Not sure if there's space/possibility to incorporate this into a single PCB design - this would be cool if one can solder either x4 + m.2 + m.2 or x8 + m.2 depending on needs.

[struthio]

@j4cbo I've miss that there are additional 4 lanes from PCH. Thought 8 was all, that is why I was surprised seeing 12 used.
Nevertheless, I would love to buy this custom riser.

[galvesribeiro]

@kitor

There were two official risers: one was x4 - used PCH lanes, x8 - used CPU lanes.

Hummm. I hope the Riser I got has x8 support so it can be used x4x4 with the NVMe splitter card while we wait @j4cbo magic to happen :)

My m920q is with the guy having the solder worked out. 🤞🏻

[j4cbo]

@kitor

BTW: @j4cbo any particular reason why x4 slot is wired to CPU instead of wiring both m.2 slots to CPU and x4 to PCH? I guess that's just purely related to order of lanes available on riser slot? (PCH lanes available at the very end) which makes this the easiest to implement?

Yes, that's exactly it, the CPU lanes are at the "back" of the slot (near the back panel) and the PCH lanes are closer to the middle of the board.

And BTW2: With similar approach it should be possible to make x8 (not bifurcated) + m.2 riser, right? That would be cool for M720 users - would allow them to plug 2nd NVME and keep x8 slot at the same time. And M920 too, depending on scenario :) Not sure if there's space/possibility to incorporate this into a single PCB design - this would be cool if one can solder either x4 + m.2 + m.2 or x8 + m.2 depending on needs.

It would definitely be possible to make such a riser, though not with the same PCB. When you enable bifurcation, you also have to enable lane reversal, so my PCB rearranges the lanes on the CPU-attached PCIe x4 connector to un-swap them. A non-bifurcation version would need to route them straight through.

(Even without that, there would be signal integrity concerns. With slow signals it's fine to run a trace to two different connectors where only one is soldered in, but at high speeds that would cause signal integrity problems due to unterminated reflections. PCIe Gen 3 is ridiculously fast - signals through copper travel at about half the speed of light, so at 8 gigabit per lane, a bit is about 2cm "long". If there's no termination on the line then the signal bounces off the end and messes things up.)

[marcojk]

@j4cbo

It would definitely be possible to make such a riser, though not with the same PCB. When you enable bifurcation, you also have to enable lane reversal, so my PCB rearranges the lanes on the CPU-attached PCIe x4 connector to un-swap them. A non-bifurcation version would need to route them straight through.

In that case, as per the table on post from [zzs95], one could avoid lanes reversal by changing the CFG signals, i.e. setting them to 0:0:1, which in turn means only soldering resistor for CFG6.
Would that work?

[wociscz]

@j4cbo As your work looks promissing, is it possible to send you some $$$ for tests (also to reserve spot in the queue after you start mass producing it :-D )? Send me your paypal link then. Regards, Jan.

[kitor]

In that case, as per the table on post from [zzs95], one could avoid lanes reversal by changing the CFG signals, i.e. setting them to 0:0:1, which in turn means only soldering resistor for CFG6. Would that work?

@marcojk
No, this will not work. You have lanes 0-15 available. 0-7 are physically connected to PCIe slot, 8-15 elsewhere.
Default 1:0:1 splits this into two x8 links: 0-7 and 8-15.

Our 0:0:0 split it to three reversed links: 3-0, 7-4 and 15-8.
Your idea with 0:0:1 would give you 0-7, 8-11 and 12-15. This would split the wrong "slot".

Double reversal is unavoidable because of hardware constraints.

@j4cbo

so my PCB rearranges the lanes on the CPU-attached PCIe x4 connector to un-swap them.

Right, I forgot we swap all the lanes around, not just "second half". I guess the fact single SSD still works in bifurcated slot with simple single disk adapter gave me that wrong impression.

(Even without that, there would be signal integrity concerns. With slow signals it's fine to run a trace to two different connectors where only one is soldered in, but at high speeds that would cause signal integrity problems due to unterminated reflections.

Ah, reflections! Sure, didn't think of that.
In ye olden days I had external GPU connected to my notebook via self-made ExpressCard to PCIE x1 adaptor which used ~20cm of old 40 pin ATA cable as physical medium to connect both slots. It was janky but worked well. Obviously it didn't had the reflections problem though :)

[kitor]

Sorry for double bump but if anyone of you read only email notifications from GH, I updated the previous message as first version probably had the lanes listed wrong :)

[j4cbo]

@wociscz I've just ordered the second prototype and will keep people posted when it comes in (1-2 weeks probably). I don't want to take deposits until I'm sure I have something I can sell :)

@kitor exactly - lanes 8-15 off the CPU aren't connected at all. There are schematics linked here: badger707/m920q-dual-NVME#3

Some SSDs do support automatic lane reversal, which is why people have had success with off-the-shelf bifurcation boards like the very first post in this issue. Of the 5 SSDs I tested only 2 seem to have automatic reversal. (But I'm rearranging the lanes on my board, so any should work.)

[j4cbo]

I set up a repo for my board: https://github.com/j4cbo/tiny5-m2-riser

[systemerror-codenotfound]

Has anyone tried it on the M910X?

any idea if this works in a M910X?

I designed a tester PCB of LGA1151. Use a multimeter to measure the pin positions according to the solder joints of the CFG resistor on the m910x.

This measurement seems reasonable. CFG5 and CFG6 are in the same position as m920q, and CFG2 is in a similar position.

This M2 was inset in the first slot of the adapter (0-3 lanes). It's matched with the PCIE mapping.

Edit: I tried with PH45, however 2nd slot still didn't work. The conclusion of my attempts is: m910x is able to split lanes but cannot recognize the second device. I would try another kind of adapter for 1 NVME M2 and 1 SATA M2. And the SATA port in adapter will connect to the soft SATA cable that the motherboard comes with, through the male-esata extender.

HI,

Do you know if there is any update on that?

It would be great to get it to work on M910X as it already has 2xnvme drivers on the bottom nad it's usually cheaper than a M920q/M920X

Maybe we can even try and make another adapter for the M910X aswell to get a x8 pci + 2 x4 nvme drives

[galvesribeiro]

Just to update, I did the Dual NVMe mod on the back of the board and it worked perfectly fine. Great!

Then I added the PCIe bifurcation mod from this repo and with a riser I tried a single NVMe, it worked. When I plug on the same Riser a network card, I don't see anything detected on lspci or dmesg.

Perhaps I'll need to wait for @j4cbo's riser to use the NIC since the lanes are messed up after the bifurcation mod and my riser doesn't take care of swapping it properly and in the process, add more 2 NVMes :D

Looking forward for te new riser :)

[galvesribeiro]

From @j4cbo lspci seems like the bifurcation didn't work...

Here is mine:

[root@yoda:~] lspci
0000:00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers 
0000:00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) 
0000:00:01.2 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x4) 
0000:00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] 
0000:00:08.0 System peripheral: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model 
0000:00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller 
0000:00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM 
0000:00:14.3 Network controller: Intel Corporation Cannon Lake PCH CNVi WiFi 
0000:00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller 
0000:00:16.3 Serial controller: Intel Corporation Cannon Lake PCH Active Management Technology - SOL 
0000:00:17.0 SATA controller: Intel Corporation Cannon Lake PCH-H AHCI Controller 
0000:00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 
0000:00:1b.4 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 
0000:00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 
0000:00:1f.0 ISA bridge: Intel Corporation Q370 Chipset LPC/eSPI Controller 
0000:00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS 
0000:00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller 
0000:00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller 
0000:00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-LM 
0000:02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 
0000:03:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 
0000:04:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 

Apparently I'm missing this:

Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8)

We only have soldered 2x 1k resistors here:

Does that mean something is wrong with the bifurcation mod? Or am I missing something?

Thanks!

[kitor]

@galvesribeiro What bifurcation card do you use? That's with two M.2 SSDs on bifurcation card, right?

[galvesribeiro]

@kitor
I got the same riser as @frankol : https://pt.aliexpress.com/item/1005004901746026.html?spm=a2g0o.order_list.order_list_main.35.21efcaa44wgrQm&gatewayAdapt=glo2bra

I'm still waiting for a PH45 Dual M.2 to arrive this week so I can test with 2x M.2.

What I tested was with a single PCIe to M.2 card, which worked, and with a single Dual 10G NIC (Supermicro AOC-STGN-i2S) which doesn't work. I'm not sure why this card won't appear at all on lspci...

But regardless of the riser or the bifurcation card, shouldn't that missing PCIe controller appear on lspci as well after the mod is completed?

[kitor]

But regardless of the riser or the bifurcation card, shouldn't that missing PCIe controller appear on lspci as well after the mod is completed?

@galvesribeiro nope, see my lspci from early on when I had bifurcation card which didn't work. Just the bridge which had card inserted showed up, and just after a cold boot, reset was enough for it to disappear.

What I tested was with a single PCIe to M.2 card, which worked, and with a single Dual 10G NIC (Supermicro AOC-STGN-i2S) which doesn't work.

That's completely expected. Bifurcation mod swaps pcie lanes around and only m.2 NVME seems to support that.

You won't be plugging any regular PCIe device into a bifurcated slot unless you
a) wait for @j4cbo riser
b) go crazy, buy some "on cable" m2 -> pcie extension and solder it to swap lanes yourself.

[galvesribeiro]

nope, see my lspci from early on when I had bifurcation card which didn't work. Just the bridge which had card inserted showed up, and just after a cold boot, reset was enough for it to disappear.

Interesting... Ok. Then I'm better have lucky with this PH45 when it arrive and 🙏🏻 for it to work. 😄

That's completely expected. Bifurcation mod swaps pcie lanes around and only m.2 NVME seems to support that.

Hummmm understood. That NIC would only work with @j4cbo riser apparently since his prototype has a NIC + 2 m.2 NVMe and deal with the lane swapping. Have to wait for that.

[galvesribeiro]

Updates! The M.2 bifurcation card PH45 arrived and it just worked! 😄

You can see on the picture 2x M.2 on the back slots + 2x on the PCIe card.

I took the opportunity to use another adapter that I bought together which should adapt the Wifi card to use a 2230 NVMe (one I took from a SteamDeck). Although it work just fine (the first item from the picture, Phison), I can't find a way to boot from it, since it is "slow", not sure I'll have much use of it anyway, but just be aware if you use that adapter that it will be detected by the OS but it doesn't appear in the BIOS to boot (if you find how to make it work, please let me know!).

That made me keep using the SATA SSD (last one in the picture, Kingston) for the boot (without its case, just the PCB wrapped by some rubber tape).

Also, now I see the missing x8 PCIe Controller:

[root@yoda:~] lspci
0000:00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers 
0000:00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) 
0000:00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) 
0000:00:01.2 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x4) 
0000:00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] 
0000:00:08.0 System peripheral: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model 
0000:00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller 
0000:00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM 
0000:00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller 
0000:00:16.3 Serial controller: Intel Corporation Cannon Lake PCH Active Management Technology - SOL 
0000:00:17.0 SATA controller: Intel Corporation Cannon Lake PCH-H AHCI Controller 
0000:00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 
0000:00:1b.4 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 
0000:00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #6 
0000:00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 
0000:00:1f.0 ISA bridge: Intel Corporation Q370 Chipset LPC/eSPI Controller 
0000:00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS 
0000:00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller 
0000:00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller 
0000:00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-LM 
0000:02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 
0000:03:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 
0000:04:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 
0000:05:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO 
0000:06:00.0 Non-Volatile memory controller: Phison Electronics Corporation PS5013 E13 NVMe Controller 

Looking forward for @j4cbo's riser so we can stick a dual 10G on it 😄

[galvesribeiro]

NVM, got the M.2 2230 to boot! Just have to make sure that nothing else has the ability to boot. The machine will try everything and nothing will be found but the installed OS on that disk and will move with it :)

More on that here.

[kitor]

Ah yes, I think this was mentioned somewhere in this thread. UEFI doesn't "detect" those but they silently work.

10th reply :).

[galvesribeiro]

@kitor ah ok. I haven't understood the convo. Tought they were talking about the NVMe on the PCI slots, not the 2230 on the wifi one. The ones on the PCIe they are available to boot. It is just the WiFi slot that has this black magic 😄

[kitor]

Oh, then I somehow skipped the fact you boot from WiFi slot (which is a whole paragraph lol).

I indeed mixed two facts - one is that it won't show extra drives on info page and another is that they (in pcie slot) will show as bootable ones on boot list if efi boot stuff is present.