Issue

On Windows with 6 or more GPUs some will show "Windows has stopped this device because it has reported problems. (Code 43)" status in Device Manager and won't work.

Mileage varies - on some systems with only 5 GPUs some will report "Code 43".

Code 43

Solution

6xGPU_mod application for any Windows version and any AMD driver version.

14_4-mod_v3 for Windows Vista/7/8/8.1 x64 and 14.4 AMD driver - 14-4_mod_v3 readme, 14-4_mod_v3 download.

13_12-mod_v3 for Windows Vista/7/8/8.1 x64 and 13.12 AMD driver - 13-12_mod_v3 readme, 13-12_mod_v3 download.

What is going on?

However, if AMD 12.6 driver is used all is fine. Also with AMD 13.1 driver downloaded from TechPowerUp none GPUs report "Code 43".

So, obviously the driver is to blame. The question is what exactly is causing the issue.

Turns out AMD 13.1 driver from TechPowerUp is different from one available from AMD.

 

After unpacking both 13.1 drivers, I found out that differences are mostly in inf files used for installation and initial configuration.

C7151592.inf - 13.1 from TechPowerUp

HKR,, DP_DisableHBR2, %REG_DWORD%, 1

 

HKLM,"Software\ATI Technologies\CBT",ReleaseVersion,,"9.012-121219a-151592C-ATI"

HKR,, ReleaseVersion,,"9.012-121219a-151592C-ATI"

 

HKR,, DalNumOfPathPerDpMstConnector, %REG_DWORD%, 0

HKR,, PX_AI_IndInstallSupport, %REG_DWORD%, 1

C7154961.inf - 13.1 from AMD

HKR,, DP_DisableHBR2, %REG_DWORD%, 0

 

HKLM,"Software\ATI Technologies\CBT",ReleaseVersion,,"9.012-121219a-154961C-ATI"

HKR,, ReleaseVersion,,"9.012-121219a-154961C-ATI"

12.6 from AMD

HKR,, DP_DisableHBR2, %REG_DWORD%, 1

HKR,, DalNumOfPathPerDpMstConnector, %REG_DWORD%, 0

Apparently both 12.6 from AMD and 13.1 from TechPowerUp have "DalNumOfPathPerDpMstConnector" set to "0" and "DP_DisableHBR2" - to "1".

Looks like that both those settings are related to DisplayPort, DP_DisableHBR2, DalNumOfPathPerDpMstConnector.

 

After changing both settings in AMD 13.12 driver and installing it on 6xGPU(2x6950 and 4x7950) rig none GPUs reported "Code 43" error.

13.12 modded

13.12 inf files with those changes were originally published as 13-12_mod.

 

However later it turned out that it does not work for rigs with R9 290 cards.

So some more settings were changed in the inf files -

HKR,, DisableDMACopy, %REG_DWORD%, 1

HKR,, DalEnableRemoteDisplayPath, %REG_DWORD%, 0

HKR,, DalWFDEnable, %REG_DWORD%, 0

HKR,, DDC2Disabled, %REG_DWORD%, 1

HKR,, DisableMMSnifferCode, %REG_DWORD%, 1

HKR,, Catalyst_Version,,"13.12 lbr v2"

And tested on 6xGPU rig with 4x7950 2x290 and 1x6950 GPUs -

13.12 mod v2

New mod was published as 13-12_mod_v2.

 

Later it was found that 6 R9 290 or 290x work just fine with windows 8.1 default drivers from Microsoft. And that modv2 does not work for 6x290 GPUs.

After inspecting inf files from Windows 8.1 I found out that there are several more differences -

HKR,,DalNumSupportedHDMIConnections, %REG_DWORD%, 0

;HKR,, DALRULE_ALLOWMONITORRANGELIMITMODESCRT, %REG_DWORD%, 0

DALRULE_ALLOWMONITORRANGELIMITMODESCRT was not present in Windows 8.1 inf files at all.

13.12 inf files with those new changes from Windows 8.1 inf files were published as 13-12_mod_v3 - 13-12_mod_v3 readme, 13-12_mod_v3 download.

And tested on 7x290 rig -

Windows 8.1 with 7 x AMD R9 290, 13.12 mod v3

and on 6x280 rig -

Windows 7 with 6 x AMD R9 280x, 13.12 mod v3

 

Later modv3 was applied to 14.4 driver and published as 14-4_mod_v3 - 14-4_mod_v3 readme, 14-4_mod_v3 download.

 

6xGPU_mod Application

Those settings from inf files go into the registry.

So it is possible to modify the registry instead of modifying inf files.

However, both approaches have disadvantages.

Since inf files are changed they are no longer signed and Windows by default prevents installation of unsigned drivers, so driver signature enforcement must be disabled prior modded driver installation.

Registry modifications cannot be embedded into the OS image and used for new rig deployments. Also registry changes are not permanent - after plugging new AMD GPU driver will be re-installed and changes will be gone.

 

Also I found out that simple registry file won't do.

DalNumOfPathPerDpMstConnector and DP_DisableHBR2 are stored here - [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E968-E325-11CE-BFC1-08002BE10318}\0001] where "0001" is the number of GPU, can be plenty of them. That's configuration for devices with class "Display" in the system.

DalNumOfPathPerDpMstConnector - [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\amdkmdag] - that's config for the service.

But.. also in here - [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Video\{B6C1E103-EFAA-4A8B-9FC9-1138C9E82BB1}\0000] and "B6C1E103-EFAA-4A8B-9FC9-1138C9E82BB1" does not match for two rigs I've checked. Also one of them has multiple similar keys under Control\Video.

 

However modding new driver versions turned out to be annoying so I've developed 6xGPU_mod application which modifies registry according to mod_v3(or any) rules.

Since registry modifications have disadvantages, inf files modification feature will be included in the application new versions.

Conclusion

Windows has no problems working with 6+ GPUs. However it may be tricky to get GPUs running.


Donations are welcome!

BTC: 14kzZJuzRZ27crZNdQgG2kQor6RUmtMTCL

LTC: LgVc7KdedPGZyDXHXEH9G7z6AoTmTvDdWb