Pawn attack span improvement

[MJZ1977]

Remove pawn attack span for backward pawns and behind opposed pawns. This is a typical scheme in positional play and one of weaknesses of SF in multiple high level games.

STC
LLR: -2.95 (-2.94,2.94) [0.50,4.50]
Total: 66843 W: 14884 L: 14717 D: 37242
http://tests.stockfishchess.org/tests/view/5d8dcb1b0ebc590f3beb2956

LTC
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 77699 W: 12993 L: 12602 D: 52104
http://tests.stockfishchess.org/tests/view/5d8de9bc0ebc590f3beb3d00

Bench : 3965294

[MJZ1977]

Observation : I submit the patch as it was tested. But it is better I think to correct backward formulation

backward = !(neighbours & forward_ranks_bb(Them, s + Up))
&& (stoppers & (leverPush | (s + Up)));

to remove phalanx from backward pawns.

[Rocky640]

I agree with the s + Up change
I would also change opposed for a bitboard
and use bool(opposed) in the connected calculation

this would give the nicer

       if (!backward)
       {
            if(opposed)
                e->pawnAttacksSpan[Us] |= pawn_attack_span(Us, s)
                    & ~pawn_attack_span(Us, frontmost_sq(Them, opposed));
            else
                e->pawnAttacksSpan[Us] |= pawn_attack_span(Us, s);
        }

However, your backward pawn still controls 2 squares,
and for consistency it would be better always include them
This would not more expensive to compute
//line 88
e->pawnAttacksSpan[Us] = e->pawnAttacks[Us] = pawn_attacks_bb<Us>(ourPawns);

[MJZ1977]

I try this option here :
http://tests.stockfishchess.org/tests/view/5d8e20ee0ebc590f3beb6fb8

Edit : not successful

[Rocky640]

Thanks for trying.
I can't see why adding the immediate squares controlled by backwards could be bad in any way.

So I respin your last attempt, with a slightly faster version (same bench) and got a yellow.
http://tests.stockfishchess.org/tests/view/5d8e4c8b0ebc590f3bebb170

And now trying a LTC
http://tests.stockfishchess.org/tests/view/5d8e69400ebc590f3bebbfbe

EDIT: it failed. Draw your own conclusions.

[MJZ1977]

Perhaps it is because "opposed" is heavier in Bitboard formulation. Or simply bad luck ...

By the way, I don't know why checks failed. Perhaps because of the rebase ?

[Rocky640]

Yet another similar test, including the missing 2 squares controlled by each backward, is looking neutral http://tests.stockfishchess.org/tests/view/5d8f86490ebc59085c6be853

One must consider that due to some recent change, we also require outposts to be pawn protected,
so the number of positions where this will PR will help is quite reduced.

It must also be pointed out that the PR did not pass STC.
Given the additional experiments done by the author and myself in my previous posts,
I'm afraid that the LTC result of this PR was luck, and will not resist a simplification.

Finally, just to illustrate the missing squares impact that I tried to fix.
This PR wrongly considers f6 and b5 as valid outpost squares.

If he puts a minor on b5 or f6, he gets full outpost bonus.
Not too bad, search will eventually see if it is a bad idea.

However, white gets 3 reachable outpost bonus for the square f6
i could also added a 4th one on b5 by adding a Bf1.

Note also that a6 is a member of the backward bitboard, but this is ok.
Only problem is not removing squares b5, f6 and h6 from the pawn attack span.

[ddugovic]

However, white gets 3 reachable outpost bonus for the square f6

If the simplification fails, perhaps this is the reason why: the g7-pawn can only capture one of those pieces, and for the remaining pieces this would be an outpost. Doing search of forcing moves isn't necessarily a bad thing, and in endgames a runaway pawn duo can be worth a piece.

[Rocky640]

Reachable outpost are a long term feature and it cannot be good to give that bonus
when not deserved.

[snicolet]

It seems that some days to let the discussion flow is a good idea here.

[MJZ1977]

Just to be sure that we have the best patch in the PR, I run a variation with keeping opposed variable as boolean :
http://tests.stockfishchess.org/tests/view/5d91d2380ebc594fb88eb01f

Edit : not passed.
Without pawns attacked by backwards :
http://tests.stockfishchess.org/tests/view/5d91edbe0ebc594fb88eb300

[NKONSTANTAKIS]

1. The results indicate that LTC results are not lucky, but the elo gain at 60" is borderline.
2. Having extensive LTC confidence, bad STCs indicate good scaling.
3. The patch does not add a lot of complexity.
4. From a community perspective the consensus likes this patch.

On chess-related analysis of "incorrect" outposts, I have to say that its probably good to give the bonus nonetheless in order to promote the search towards such creative moves. There are numerous positions that SF is blind or takes a lot of time to find out piece sacrifices, like putting a knight on f6 when we have a pawn on e5. It will be great if SF spots them earlier, soas to be able to enter the proper variations beforehand.

IMO it would be sad to skip such a patch, I would merge it as is, but at the very least try a VLTC of the original (which had the best LTC result) to verify scaling.

[Rocky640]

I'm not against creative bonus for "reachable en prise squares" for our minor pieces on advanced ranks. for example, we can look for opportunities to create some advanced passers this way, or some important file openings, but it could be done in a simpler and more exact way. In my view, this would be separate idea, and should accordingly be tested separately.

Evaluation does not need to be accurate, but should help to track small improvements in the position.
Giving equal bonus for unsafe and safe reachable squares does not make sense to me.

In my view, this PR was interesting, but neutral and not solid enough.
Meanwhile I parked my running experiment at priority -1.

[MJZ1977]

I think that this PR can be merged : it passes LTC test and yellow STC. We can try simplifications after it has been merged.

[snicolet]

Merged via e6f4b5f, thanks!

I have committed because of the apparent Elo gain, but I agree with Alain that the code seems a little bit convoluted now, and I would certainly welcome simplifications.

In fact I always prefer when pawns.cpp exports "pure geometrical" features (ie atomic features which are direct consequences of the chess rules), and we do the smart/evaluation part in evaluate.cpp

[MichaelB7]

There could be a regression with the last 3 patches, and it may be this patch. Can anyone run a current regression test with and without this patch at LTC ? I hope I'm wrong. Thanks.
Edit - I'm seeing this at t/c 180 plus 1.8 inc, but it's not possible for me to test at a sufficient number of games locally to say for sure.

[xoto10]

Just to check, when you say last 3 patches are you comparing e6f4b5f with 70a38d7 ?

[NKONSTANTAKIS]

@MichaelB7 Why do you suspect a regression? Anyway you can safely exclude this, it doesn't change much and results are robust. If you are testing multicore I would look at vdv's patch which prunes more with more threads. Our LTC is 20", and this doesn't apply much to real time controls, so we have to trust @vondele intuition.

If you test position solving, its natural that with more pruning it will take more time. On the plus side, deeper searches should improve positional play.

[MJZ1977]

If we suspect a regression with this patch, the easiest way is to try simplifying it with [-3..1] ?

[snicolet]

I am closing this, as Alain simplification has been merged now:
#2348

[MichaelB7]

ok thanks.