Add Scrypt-Adaptive-Nfactor algorithm for mining vertcoin

[p4fg]

Vertcoin uses a adaptive nfactor version of scrypt. The current vertcoin-miner is lagging behind in functionality and everyone would benefit of a single miner.

http://vertcoin.org/
https://github.com/Bufius/vertminer-gpu

[veox]

I've actually been thinking more and more lately of a complete rewrite (most probably in Python for easier cross-platform support) that would allow extending the core functionality with pluggable modules (plug-ins). This would allow handling not-necessarily-compatible work units, user interfaces, peering protocols, devices, and whatever else the various communities come up with. (EDIT: The "core" referred to here would essentially be plumbing.)

The work done from cgminer 3.7.2 to sgminer 4.1.0 has mostly been pulling in the changes already made to other forks. Code review is a major overhead (and, in all honesty, it's getting stale as of late). So is the research needed to gracefully resolve incompatibilities. There hasn't been much of those so far, but I can see them in the distance.

That said, I do not see myself starting a rewrite right now, taking all the other projects (besides sgminer) into consideration.

[p4fg]

I have had the same ideas, It would also be kind of neat if the pools could signal/hint the type of algorithm used in the pool. That way we could have multipools mining different algorithms (scrypt-jane/scrypt/adaptive-scrypt/etc..), using one single miner...

[NNygren]

Too bad it's not as simple as dropping the .cl kernel from vertminer into sgminer and renaming it...
I wish the vertminer author would have forked sgminer in the first place, but all that ASIC BTC-only code is still there bloating it up.

[troky]

Check this https://github.com/troky/sgminer/tree/nscrypt

it works fine with --nscrypt --kernel="nscrypt"

[veox]

@troky haven't tested, just a quick review - looks good.

Perhaps it would be better to have an arged option --algo=nscrypt instead of an unarged --nscrypt?

[troky]

It was just quick port from https://github.com/Bufius/vertminer-gpu . I was thinking to remove use of --nscrypt or --algo=nscrypt completely to avoid redundancy and to add flag like use_nscrypt=true when nscrypt kernel is selected and adjust rest of code.

[troky]

error fixed in https://github.com/troky/sgminer/tree/nscrypt
it compiles in linux now.

[dlueth]

Does not seem to be fixed troky :( Error persists and make errors out with:

scrypt.c:430:65: error: macro "scrypt_1024_1_1_256_sp" passed 4 arguments, but takes just 3
scrypt.c: In function ‘scrypt_regenhash’:
scrypt.c:430: error: ‘scrypt_1024_1_1_256_sp’ undeclared (first use in this function)
scrypt.c:430: error: (Each undeclared identifier is reported only once
scrypt.c:430: error: for each function it appears in.)
scrypt.c:434:55: error: macro "scrypt_1024_1_1_256_sp" passed 4 arguments, but takes just 3

Anyway: Greatly appreciate your effort!

[troky]

My bad. Still warming up ;)
Check troky@f846f38

[dlueth]

:)

[dlueth]

Hm, line 434 in scrypt.c is the next one to error out

[troky]

Fixed in troky@850f514

[dlueth]

Edit: compiles and runs OK but is rather slow. Compared to vertminer I loose about 120 Kh/s on my 280x with adjusted settings.

[veox]

@troky, I was proposing a "generic" option like algorithm so that more algorithms could be added if needed without introducing too many flags. It can be set opt_hidden in the option configuration table so that it doesn't show in the help screen. Then it can be auto-set depending on the chosen kernel.

[troky]

@dlueth I have the same hashrate as I had in vertminer with the same configuration. Don't use xIntensity nor raw intensity yet since something is wrong with that code.
Post vertminer/sgminer settings here.

[dlueth]

No problem, here you go:

"intensity" : "13,12",
"vectors" : "1",
"worksize" : "256",
"lookup-gap" : "2",
"thread-concurrency" : "8193,4096",
"shaders" : "2048,1440",
"auto-fan" : true,
"auto-gpu" : true,
"gpu-engine" : "970-1100,765-896",
"gpu-fan" : "100",
"gpu-memclock" : "1500,1200",
"gpu-memdiff" : "0",
"gpu-powertune" : "0,0",
"gpu-vddc" : "0,1.050",
"gpu-dyninterval" : "7",
"gpu-platform" : "0",
"gpu-threads" : "2,1",
"temp-cutoff" : "90",
"temp-overheat" : "85,85",
"temp-target" : "80,75",
"temp-hysteresis" : "3",
"expiry" : "1",
"hotplug" : "0",
"log" : "5",
"no-pool-disable" : true,
"no-submit-stale" : true,
"queue" : "0",
"scan-time" : "1",
"shares" : "0"

It might look funny but the rig is running a R9 280x and an older 5850.My main reason for running sgminer over cgminer is the possibility to have different gpu-threads. The 280x prefers 2 while the 5850 will only run with 1.

For vertminer I set gpu-threads to 1 which leads to approx. 300/165 Kh/s. TC for the vertminer for the 5850 can be set to 5760.

When using your fork and I switch to gpu-threads 2,1 the 280x goes down to approx. 230Kh/s. With a TC of 5760 for the 5850 it will not start at all. Have to go down to something like 4096 for the 5850 which will lead to 95Kh/s anyway.

So, depending on which options I choose I loose between 70Kh/s and 140Kh/s compared to vertminer.

[troky]

I am using this config for both vertminer, and sgminer-nscrypt fork:

"gpu-fan" : "70,70,75",
"gpu-threads" : "1",
"gpu-engine" : "947,980,1030",
"gpu-memclock" : "1250,1350,1250",
"gpu-powertune" : "8,10,13",
"intensity" : "20,20,19",
"temp-target" : "82",
"temp-overheat" : "90",
"temp-cutoff" : "93",
"temp-hysteresis" : "2",
"worksize" : "256",
"thread-concurrency" : "24550",
"expiry" : "30",
"scan-time" : "1"

2x XFX R9 290
1x PowerColor R9 290 OC

... and I am getting the same results in both miners (415-445 KH/s).

Try not to use "shaders" config parameter and always test with SAME config.

[NNygren]

Silly me, I was syncing the main git tree so it was not grabbing the nscrypt branch when I was first trying to compile. All works very well now, good job :)

[dlueth]

@troky i would love to test with identical configs but sgminer will not even start mining on gpu1 if i do not lower its tc. Vertminer does start and run with 300/165 with the exact same config.

[troky]

@dlueth I'll go through all ported code again to check for possible problems. It is strange that the same config from vertminer doesn't work in sgminer-nscrypt since nscrypt code ported from vertminer is not modified.
Just one more thing for the reference: what vertminer version do you use and what OS?

[dlueth]

@troky hang on a second, just arrived at work where my rig is hosted and had an idea on the way which seems to proove right at first glance:

In vertminer (either official or TheKev, both give identical results) there seems to be some kind of logic that limits TC. Even though I put "5760" for the 5850 in the config the bin file states it is "4032" and "8912" for the 280x instead of "8913". So far so good, but in the end results stay more or less identical:

"intensity" : "13,12",
"vectors" : "1",
"worksize" : "256",
"lookup-gap" : "2",
"thread-concurrency" : "8192,4032",
"shaders" : "2048,1440",
"auto-fan" : true,
"auto-gpu" : true,
"gpu-engine" : "970-1100,765-896",
"gpu-fan" : "100",
"gpu-memclock" : "1500,1200",
"gpu-memdiff" : "0",
"gpu-powertune" : "0,0",
"gpu-vddc" : "0,1.050",
"gpu-dyninterval" : "7",
"gpu-platform" : "0",
"gpu-threads" : "1",
"temp-cutoff" : "90",
"temp-overheat" : "85,85",
"temp-target" : "80,75",
"temp-hysteresis" : "3",
"expiry" : "1",
"hotplug" : "0",
"log" : "5",
"no-pool-disable" : true,
"no-submit-stale" : true,
"queue" : "0",
"scan-time" : "1",
"shares" : "0"

Gives 300/165 in vertminer (official and TheKev) and 295/100 in the modified sgminer.

[dlueth]

@troky Just for the reference: I tested it on TheKev Vertminer 0.5.2 and official Vertminer 0.5.4 current beta. My Rig is running on a custom minimal mining install of Debian Squeeze with Catalyst 13.12, ADL SDK 6.0 and APP SDK 2.9

[troky]

@dlueth Please try the same config but without "shaders" : "2048,1440", line.

[dlueth]

@troky same result (might even be slightly worse at 296/95), deleted the *.bin files just to make sure nothing gets mixed up...

[troky]

I did some updates (mostly about TC calculation). Pull fresh tree, recompile, test and report back.

[troky]

@dlueth Can you build and test https://github.com/Bufius/vertminer-gpu ? I ported code from master. Maybe you used some older code/miner with better results?

[dlueth]

@troky deleted my previous answer, forgot to quit the running vertminer process...

Pulled your changes, compiled a fresh version, deleted bin files but results stay the same :(

[troky]

Nice test.
Obviously, there is no problem with kernel compilation... All params look fine.
@p4fg can you compare compiled binaries for 5850 for both vertminer and sgminer-nscrypt? They should be the same.

[troky]

@p4fg I can't see Set globalThreads to... lines in your logs. Can you post them for each card/miner as above?
vertminer should have line applog(LOG_DEBUG, "set globalThreads to %u",threads); uncommented in driver-opencl.c

[troky]

@dlueth @p4fg pull fresh sgminer-vert and try with proper "shaders" setting in configuration (e.g 1440 for 5850)
It seems that compute_shaders works only for newer cards.

[veox]

@p4fg I can't see Set globalThreads to... lines in your logs. Can you post them for each card/miner as above?

Ah, crap. I removed that ~ a week ago, because someone complained about how spammy it is. :(

[p4fg]

The compiled kernels are different, but that is no surprise as the CL-code differs (have not checked how much though)

[troky]

@p4fg Why should compiled kernel be different if they are compiled using the same params on the same machine with the same hardware?

Anyway, please test again with troky@e82f938 applied and proper shaders config

[p4fg]

Because the kernel source-file differs between my Vertminer 0.5.3 and sgminer.... :-)

Will do another test tonight!

[troky]

So use the same file.
nscrypt.cl used in sgminer is https://github.com/Bufius/vertminer-gpu/blob/master/scrypt140202.cl

[dlueth]

@troky modified sgminer: 300/129 Kh/s, vertminer 300/165 - so it is an improvement but there still hast to be some kind of difference regarding older GPUs

[p4fg]

Alrighty.. more testing going on... and more weirdness found (using same parameters for both)

Vertminer 0.5.3

4fca55e8975629fc34fe783d26855d8f  vertminer-git/scrypt140202.cl
b60d7fac5c6d8acc12c276040f796f81  scrypt140202Cypressglg2tc4032w256l8.bin
5902d1640584f29715c6a0b20da2fe09  scrypt140202Tahitiglg2tc19712w256l8.bin
49b3f52bc249f1d9527ceeeb7bd96903  scrypt140202Tahitiglg2tc8193w256l8.bin

Sgminer troky/sgminer@e82f938

4fca55e8975629fc34fe783d26855d8f  kernel/nscrypt.cl
5a736a7b0c90b574fac38a6df99bebc6  nscryptCypressglg2tc4032w256l8.bin
633c5546bffced31869cd35e2c7faac7  nscryptTahitiglg2tc19712w256l8.bin
52a4fc5096d8c973c9d3afffaad98400  nscryptTahitiglg2tc8193w256l8.bin

Same .cl-file.. different compiled binary-kernels...

@troky using shaders 1440 gives a threadcount of 11?

[20:52:39] Selected 1: Cypress
[20:52:39] Preferred vector width reported 4
[20:52:39] Max work group size reported 256
[20:52:39] Max shaders calculated 1440
[20:52:39] Max mem alloc size is 536870912
[20:52:39] Kernel nscrypt is experimental.
[20:52:39] GPU 1: selecting lookup gap of 2
[20:52:39] GPU 1: selecting thread concurrency of 11
[20:52:39] No binary found, generating from source
[20:52:39] Setting worksize to 256
[20:52:39] cl_amd_media_ops found, setting BITALIGN
[20:52:39] BFI_INT patch requiring device not found, will not BFI_INT patch
[20:52:39] CompilerOptions: -D LOOKUP_GAP=2 -D CONCURRENT_THREADS=11 -D WORKSIZE=256 -D BITALIGN -D GOFFSET

Problem seems to be this:

size_t tc_limit = cgpu->shaders * use_nscrypt ? 11 : 5;
            if (cgpu->thread_concurrency > tc_limit)
                cgpu->thread_concurrency = tc_limit;

A TC of 11 (or 5 for that matter) seems a bit low.. (and also produces tons of HW errors)

[troky]

How about this: troky@c8e4ec3

[dlueth]

@troky In my case the change did not do a thing, still at approx. 300/125 compared to 300/165

[troky]

still waiting for @p4fg test to see if TC is fixed or not.

[NNygren]

I'm having very good results on my HD7970, after playing around with different configs and builds. Finally down to 0 HW errors! Yay! :)

What I had to do was use the .cl kernel file from vertminer 5.2 and drop it into your latest build (renaming it nscrypt.cl, of course), compile without ADL, and setup my config as follows:
"intensity" : "13",
"worksize" : "256",
"kernel" : "nscrypt",
"lookup-gap" : "2",
"thread-concurrency" : "8192",
"shaders" : "2048",
"gpu-threads" : "2"

Getting a solid 380Kh/s, again with ZERO HW errors. These are results I could not achieve with any of the vertminer versions.
Last note, as ADL always tends to slow me down in ArchLinux, I used AMDOverdriveCtrl to set my clocks to 1075(GPU) and 1500(MEM)

[johncronan]

Which cl file should I be using with this? I got it to build and run, but based on the speeds it seems to still be hashing regular Scrypt. Does it come with the file I need or do I pull it out of the vertminer source?
edit: @troky I tried with the scrypt140202.cl file referenced above but I just get "Error -52: Enqueueing kernel onto command queue. (clEnqueueNDRangeKernel)" on all GPUs.

[troky]

@kylecronan rename scrypt140202.cl to nscrypt.cl and put it in kernels folder. You should also select proper kernel ("nscrypt") in config and lower TC and intensity.

[johncronan]

@troky Oh oops, I just forgot 'git checkout origin/nscrypt', but now I get ohash undeclared on line 482 of scrypt.c. Is there a particular commit I should try out? edit: okay, c8e4ec3 compiles fine, but I can't use different gpu-threads per device. That's what I'm really hoping to do. Is it possible?

[troky]

Oops. Typo fixed in troky@3212502

[johncronan]

Cool. And with ADL in there I can do the different gpu-threads. This is great, thank you!

[m330ci]

Hi Guys, pls help out a lamen. Anybody want to give a step by step dummy guide on how to get mining vertcoins here?.
Ive downloaded cudaminer, cgminer, i have my wallet, im on github with my repository folder and im stuck.
Please help.

K

[gdevenyi]

This is bug reporting mechanism for sgminer, please seek help elsewhere.

[dadiv]

@m330ci - you will find helpful Vertcoin folks here and here.

[veox]

Branch nfactor has been merged into master.

I'll add the doc label to this and keep it open for a while, seems a lot of useful information in this issue discussion.

[mrbrdo]

I guess we can close this now, since Scrypt-N has been in sgminer for quite a while now.