How to tell which chip is bad ?

[tonycstech]

IF there are any errors, how to interpret the results to a specific chip or at least, to a channel ?

[galkinvv]

In general there is no any way to tell this.
The only exception is the RX470-590 GPUs where the is known conversion for min&max addresses displayed in "Errors address range" line to a memory IC.
the 3rd less-significant digit is interesting

0x.....0.. and 0x.....8.. is B0 (U2200)
0x.....1.. and 0x.....9.. is B1 (U2300)
0x.....2.. and 0x.....A.. is A0 (U2000)
0x.....3.. and 0x.....B.. is A1 (U2100)
0x.....4.. and 0x.....C.. is C0 (U2400)
0x.....5.. and 0x.....D.. is C1 (U2500)
0x.....6.. and 0x.....E.. is D0 (U2600)
0x.....7.. and 0x.....F.. is D1 (U2700)

And even for RX470-590 this works only if exactly one chip has errors, since intermediate adresses with errors can't be easily calculatedm only the first and the last.

[tonycstech]

Great. 470-590 is a prehistoric dinosaur GPU. Basically this software is useless. Want to test AMD ? run tserver Want to test Nvidia ? run MODS.

…

On Fri, Aug 9, 2024 at 1:40 PM Vasily Galkin ***@***.***> wrote: In general there is no any way to tell this. The only exception is the RX470-590 GPUs where the is known conversion for min&max addresses displayed in "Errors address range" line to a memory IC. the 3rd less-significant digit is interesting 0x.....0.. and 0x.....8.. is B0 (U2200) 0x.....1.. and 0x.....9.. is B1 (U2300) 0x.....2.. and 0x.....A.. is A0 (U2000) 0x.....3.. and 0x.....B.. is A1 (U2100) 0x.....4.. and 0x.....C.. is C0 (U2400) 0x.....5.. and 0x.....D.. is C1 (U2500) 0x.....6.. and 0x.....E.. is D0 (U2600) 0x.....7.. and 0x.....F.. is D1 (U2700) And even for RX470-590 this works only if exactly one chip has errors, since intermediate adresses with errors can't be easily calculatedm only the first and the last. — Reply to this email directly, view it on GitHub <#47 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOUZUBQWZWTEKFVL4BE4FMDZQT5JTAVCNFSM6AAAAABMIZRCBOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRZGAYDKNY> . You are receiving this because you authored the thread.Message ID: ***@***.*** .com>

[galkinvv]

The main design goal is completely different from tserver/mods.
It is not about finding the failing channel (the manual says nothing about memory channels), it's about determine "should the VRAM on a seemingly-mostly-working GPU be suspected or not".

Compared to tserver/mods memtest_vulkan tries to be easier&quicker to start and provides heavier load => tries to find VRAM errors faster. Also, sometimes it allows ensuring that artifacts are not caused by memory: If some app is artifacting but memtest_vulkan running in parallel shows no errors - then there is no any problems with VRAM ICs or memory controller, the errors in artifacting app are caused by a core.

And if errors are found in memtest_vulkan - then its time to run mods/tserver, definitely. From my experience with NVIDIA sometimes it may be quite complex to find mods test that would detect any errors for gpus where memtest_vulkan finds presence of errors in a 5 minutes. But knowing the fact that errors are there - I perform more attempts to reproduce them inside mods.

So, actually main analogues to compare the efficiency are not tserver/mods but a "Kombustor with artifacts scanner" and "OCCT VRAM testing"

[tonycstech]

Useful information. Thank you. Customer sent me RTX 8000 claiming this application reported errors so i ran it on my system and it gave me no errors at all. Customers hardware/software is at fault in this case.

…

On Fri, Aug 9, 2024 at 2:55 PM Vasily Galkin ***@***.***> wrote: The main design goal is completely different from tserver/mods. It is not about finding the failing channel (the manual says nothing about memory channels), it's about determine "should the VRAM on a seemingly-mostly-working GPU be suspected or not". Compared to tserver/mods memtest_vulkan tries to be easier&quicker to start and provides heavier load => tries to find VRAM errors faster. Also, sometimes it allows ensuring that artifacts are not caused by memory: If some app is artifacting but memtest_vulkan running in parallel shows no errors - then there is no any problems with VRAM ICs or memory controller, the errors in artifacting app are caused by a core. And if errors are found in memtest_vulkan - then its time to run mods/tserver, definitely. From my experience with NVIDIA sometimes it may be quite complex to find mods test that would detect any errors for gpus where memtest_vulkan finds presence of errors in a 5 minutes. But knowing the fact that errors are there - I perform more attempts to reproduce them inside mods. So, actually main analogues to compare the efficiency are not tserver/mods but a "Kombustor with artifacts scanner" and "OCCT VRAM testing" — Reply to this email directly, view it on GitHub <#47 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOUZUBXXHQVXRY5STAA4LH3ZQUGDZAVCNFSM6AAAAABMIZRCBOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRZGA3DSMQ> . You are receiving this because you authored the thread.Message ID: ***@***.*** .com>

[tonycstech]

When i ran this test, it said it was using under 8GB of memory.
Why does it not use at least 90% of memory to run the test ?
Is there a way to change the amount of memory to test as well as number of iteration and time test runs ?

[galkinvv]

Regarding the size the tool itself has no memory size limitations

Last time I tested RTX 3090 - it uses 90% of memory:

1: Bus=0x01:00 DevId=0x2204   24GB NVIDIA GeForce RTX 3090
2: Bus=0x00:00 DevId=0x591D   8GB Intel(R) HD Graphics P630
Tester worker logging started at 2024-06-29T23:18:56.864092Z
Standard 5-minute test of 1: Bus=0x01:00 DevId=0x2204   24GB NVIDIA GeForce RTX 3090
      1 iteration. Passed  0.0519 seconds  written:   18.1GB 769.1GB/sec        checked:   21.8GB 768.2GB/sec

I marked in bold the 21.8GB after "checked:" - the size printed in the 1st iteration is the size being tested.

Allocating less then "All size minus 3GB" of memory is typically caused by vulkan driver specifics. Two commоn cases:

• sometimes vulkan drivers reports that most memory is already in use so memtest_vulkan assumes that alllocating too much memory would lead to conflict with other apps and would not be able to really load memory bus. So it resorts to testing lower memory size. This can be overriden by specifying the size to test explicitely - create a .bat-file near memtest vulkan with contents like: memtest_vulkan.exe 1 46000000000 - that will try to test ~46GB on GPU 1.
• the other situation is some driver limitations. Sometimes vulkan drivers fails allocating to much contigous memory and then such direct memory size specification will report allocation failure. If this is your case:
  • rename the binary memtest_vulkan -> memtest_vulkan_verbose
  • make same correction in a batch file
  • run again
  • post here the contents of memtest_vulkan.log generated on a failed 46GB test.

All of the above applies both to Windows and Linux versions, with minor adjustments regarding .bat (I'm not sure what OS you are testing RTX 8000)

Now about the iteration count

The current version of test is always infinite - it prints intermediate results after 5 minutes, but continues to run infinitely - and will report any errors if they would be found. Closing the window or pressing Ctrl+C at any moment stops testing. So it's up to you to select the time to test.

Results are always saved to a log file, so they are not lost on window close.

To be honest I something think about adding "run test for a specified time period" option - however I'm not sure how to implement asking this from user without inventing complexities or GUI. Maybe some other option in a batch file