Tuesday, July 25, 2006

Scientific Test, Part VI: Discussion

Target Feature Count (TFC) is a simple and stupid patzer method. Firstly, it disregards the position. Secondly, it disregards the relation of the targets other than simple tactics. Thirdly, all features are treated and counted as equal. And in the fourth instance, all positional features such as open files, ranks and diagonals, strong outposts and pawn weaknesses are completely ignored. Therefore, it cannot be expected that TFC always can find the most important target in any position.

In a quiet, equal position there are no targets that are ready to be attacked. In such a position, TFC will be of no use at all. The best moves will be moves that gain positional features, and these are not counted in TFC. Maybe it would be worthwhile do develop a similar method that could be called Positional Feature Count.

For the reason mentioned before, the TFC test was performed only with positions where Deep Shredder saw a remarkable advantage for one side.

Despite of being simple and stupid, the test scored between 85 and 90 percent both in sensitivity and specificity. This is clearly better than I expected, but of course it is far from being sufficient. If it were, chess would be simple and all patzers could become grandmasters.

Even a perfect TFC would not be able to show the best move, because in most cases there is more than one move to attack a given target. In order to find the best move, pattern recognition is strongly required. It will show by which moves in which order the attack must run.

It may be criticized that TFC is a mechanistic approach, neglecting all creativity of chess play. Well, grandmasters may be creative, let them be! But we patzers must first learn to count. That's it. My idea is that counting is just a first step. Stronger players will not just count features in a digital way, but they will integrate the values of differently weighted features to one target value.

Pure pattern recognition without any idea of a target is just trial and error. In a complicated positon, there will always be too many candidate moves, and considering all of them will bring us into time trouble, even in a slow game. The idea behind TFC is to give pattern recognition an idea where the music plays in a position.

TFC may also give a hint whether a position is tactical or not. With a total of six or more target features in a position a tactic is very likely. In the positions that I have tested, there were 7.6 target features on average, and all of these were tactical positions.

This answers the question of when to perform a TFC. It is best to do it all the time. Counting, by the way, is just a method that forces the player to look at all aspects of a position. Therefore, TFC is no wasting of time.

Monday, July 24, 2006

Scientific Test, Part V: Results

Classification of targets

TFC positive means that a target scored highest in target feature count. TFC negative means that a target scored less than highest. True means that Deep Shredder classified the target according to TFC. False means that Deep Shredder disagreed with TFC.

TFCtrue      false      
positive     6621

87 targets in 74 positions scored highest. In 61 positions, there was only one top scoring target. In 12 positions, there were two top targets, and in one position there were three top targets.

In most cases, Deep Shredder agreed with TFC, either in the positive way (a target scoring highest is to be attacked by the best move) or in the negative way (a target scoring less than highest is not to be attacked by the best move).

The TFC test has a sensitivity of 89 percent: If a target is rated top by Deep Shredder, the probability is 89% that it will score highest in TFC.

The TFC test has a specificity of 86 percent: If a target scores highest in TFC, the probability is 86% that it will be rated top by Deep Shredder.

More about the calculation of sensitivity and specificity ...

Note that the TFC test does not say anything about moves. If a target scores top there may be a number of moves to attack it, and only one of them is the best one.

Wednesday, July 19, 2006

Scientific Test, Part IV: Materials and Methods

I analyzed all the 18 games of my last two Winterthur Chess Week participations (2004 and 2005) with Deep Shredder 9 (MacOS X 10.4, 1.2 GHz PowerPC G4 processor, 512 MB RAM), running in a 5 thread analysis mode.

After every move I decided whether to do a Target Feature Count (TFC) or not: If the value of the position, as rated by Deep Shredder, did not substantially change (less than half a pawn unit) after a ply, I discarded the position and analyzed the next ply. If the positional value changed considerably, in most cases more than a pawn unit per ply, I did a TFC for this position.

This analysis is only about attacking a target, not about defending. I do this because it seems easier to classify attacking than defending moves. Of course, defense is just as important as attack.

Target Protocol: For every eligible position I recorded the three top targets together with their TFC values, the move rated as best by Deep Shredder, and the relation between this move and the targets. This relation was counted as positive if the best move was directly aimed at the target (e.g. by a capture) or if it supported such an attack by indirect means (e.g. removal of the guard). It was counted as negative if there was no relation between the best move and the target. For every position, only one positive relation was recorded (e.g. only between the best move and the main target, but not between the best move and the guard protecting the main target). Such a sub-target always had a lower TFC score than the main target. If two or more targets have been attacked by one and the same move (e.g. a Knight Fork), the key square of this attack was recorded as the «main target». If there was no such key square (e.g. in a discovered attack), the relation to the target with the highest TFC score was recorded as positive and to the targets with lower scores as negative.
  • Every target that scored highest in TFC and was positively related to the best move was recorded as true positive.
  • Every target that scored highest but was negatively related to the best move was recorded as false positive.
  • Every target that did not score highest and was negatively related to the best move was recorded as true negative.
  • Every target that did not score highest but was positively related to the best move was recorded as false negative.
I have analyzed 222 targets from 74 positions in 18 games.

Tuesday, July 18, 2006

Scientific Test, Part III: Target Feature Count

Target Feature Count is a simple method of identifying and ranking targets in a chess position. It is assumed that the target with the highest TFC value is most likely the one to be attacked by the best move, either directly or indirectly.
  • Piece value: Being a King or a Queen (add 1 point)
  • Lack of protection: Not being protected by a pawn (add 1 point), not being protected at all or only by King or Queen (add 1 point). Of course, pinned or overworked pawns or pieces must not be counted as protectors.
  • Attacks: Add 1 point for every attack, including x-rays and possible attacks with next move; add 1 extra point for attack by a piece of lesser value.
  • Capture: Add 1 point for being en prise (unprotected and can be taken with the actual move) or being trapped (unable to escape).
  • Balance: If a target is protected and attacked, calculate a sequence of captures and recaptures until the highest target feature count is reached.
  • Multiple attack: If two or more targets can be attacked by one move, sum the features of all involved targets.
Special cases
  • Threatening checkmate in one = 8 points: King (1), to be attacked (1), to be attacked by a lesser piece (1), being trapped (1), mating square unprotected by a pawn or piece (1), protected only by the King (1), attacked twice (2).
  • Queen en prise = 4 to 6 points: Queen (1), not being protected by a pawn (0) or only by a piece (1) or only by the King or not protected at all (2), being attacked (1), being attacked by a piece of lesser value (1), can be captured with the actual move (1).
  • Piece or pawn en prise = 4 points: Not being protected by a pawn (1), not being protected at all (1), being attacked (1), can be captured with actual move (1). In this case, it is irrelevant if it is taken by a piece of higher, equal or lesser value.
  • The Exchange en prise = 3 to 5 points: Rook being protected by a pawn (0) or only by a piece (1) or only by King or Queen (2), being attacked (1), being attacked by a piece of lesser value (1), can be captured with the actual move (1).
  • Check or attacking the Queen = 3 points: King/Queen (1), to be attacked (1), to be attacked by a piece of lesser value (1). Besides the target feature count, the tempo gain must also be taken into account here.
Target Feature Count is a measure of the vulnerability of a target, not of its value. For instance, a pawn en prise and a rook en prise both have the TFC value of 4, but a very different material value.

Update: This is the conclusion after a preliminary test of TFC. Further examinations are needed to find out how TFC can be fine-tuned for use in practical games.

Monday, July 17, 2006

Scientific Test, Part II: Objective

To test if the target that is attacked by the best move in a position, after a bad move of one player, can be predicted by my method of target feature count. The best move must attack the predicted target, either directly, or indirectly by removal or the guard or a similar tactic. If there are two or more «best moves» with a difference of only a small fraction of pawn unit in the computer analysis, one of these moves must be aimed at the predicted target directly or indirectly.

This test is restricted to positions immediately after a considerable shift of positional value, assessed by a strong computer chess software. All equal or quiet positions are excluded from this test.

Saturday, July 15, 2006

Scientific Test, Part I: Introduction

Success in chess, imho, is built upon four main elements:
  1. Tactical fitness (pattern recognition, calculation)
  2. Knowledge (opening repertoire, endgames, strategic rules, experience)
  3. Thought process (useful heuristic to find the best move in the given time)
  4. Mental strength (ability to cope with stress, time trouble etc.)

Tactical fitness: MDLM has described a method of gaining it. CTS is a very useful training tool. I don't see how this can further be optimized. Repetition is the key to success.

Knowledge: This is a never ending task but has to be limited. If you lack tactical fitness, even a huge amount of knowledge will not help you.

Mental strength: A mental trainer surely would help, but I have none. So I must rely on myself. Learn how to win and how to lose. Gain self-confidence. Learn patience, and all the like.

Thought process: I am convinced that in any chess position there must be an optimal search strategy (or heuristic) to find the candidate move that will, after tactic calculation prove, finally come out as best move.

I have tried to find a simple method of searching and ranking targets. My assumption is that the top ranked target must be the key to the best move. This is a hypothesis, and therefore it should be possible trying to falsify it in a scientific test. In my next posts I shall use the scheme of a scientific publication, starting with the objectives of my study, a description of my methods so that they can be replicated by fellows in a sort of peer review process, then post my results, discuss them in detail and finally come to a conclusion. There will be two possibilities:

A: If my hypothesis must be rejected, we all will be smarter than before, and the work may start again with the never ending hope to find a really useful thought process.

B: If my hypothesis cannot be falsified by the test, this will be no proof in the strict sense of Sir Carl Popper. But this won't bother me because I'll put science aside, being eager to make things change in my practical games.

Friday, July 14, 2006

Back on CTS

Yesterday I started using CTS again. Have been away only some months and see a completely different site now. Has been down for some days or weeks, I heard. Cheaters have taken the top positions. As far as I get from the message board discussion they did not necessarily exploit bugs in the software. Some seem to be memory geniuses, just learning by heart thousands of problems. What use of it? Only the infantile joy of being on top, above all the IMs?

Also CTS has slimmed down and dropped the graphics features that may have slowed down the server too much. The number of tacticians has doubled since I left, but I am still in the upper half of the list. I expected my rating to dive after my long hiatus, but this did not happen. I did 20 problems (17+ 3-) and my rating increased by 20 points. My all-time high is in the range of my RD. Anyway, it seems that the abilities acquired at CTS do not get lost quickly.

Thursday, July 13, 2006

Target Rating

Black to move. Targets ranked by a simple heuristic.

1. Pawn f2 (5 points)
Not protected by pawn, only protected by King, attacked by Knight, x-rayed by Bishop, x-rayed by Rook.

2. Queen g6 (4 points)
Being a Queen, not protected at all, not protected by a pawn, can be attacked by a Knight.

3. Knight d3 (3 points)
Not protected at all, not protected by a pawn, attacked by Queen.

4. Bishop c1, pawns b2, c7, f5, g2 (2 points)
Not protected by a pawn, attacked by a piece.

In this special case, the Target Rating heuristic leads to the best move Nxf2, capture of the 5 point target. Not bad so far. But I have to test it with a lot more positions.

Wednesday, July 12, 2006

Bottom-Up and Top-Down

The CCT heuristic (Checks, Captures, Threats) I have been using recently is essentially a bottom-up method. It is wonderful if only a few options are on the board. Just follow the principal variations and look if they bring advantage. In other words, CCT is excellent if there is a simple tactic.

I have tried out a new training method using CCT. I replay games of a collection in the analysis mode with Deep Shredder. The level of the players is between 1600 and 2000. I hide the analysis window behind the board window, but let the top line visible, which shows me the positional value, but not the moves. Whenever the positional value shifts considerably, showing that an error has been made, I look at the positon and try to find the best move.

In many cases it can be found by CCT. But in many other cases I think it is better to switch to a top-down heuristic. Look for targets that can be attacked and then figure out how the attack works best.

What I need is a Target Rating (TR) heuristic that is able to detect not only the obvious targets but also the hidden ones. The ideal thought process will be a mix of TR (identify targets and rank them according to their value) and CCT (try out the most forcing ways to attack the targets).

My idea of Target Rating is to look for target features such as «unprotected», «piece of high value», «protected only by King or Queen», «mating square» and the like. Then to add points for the features, and in the end the top target should get most points. Then use again CCT to find the best move to attack the top target.

Just an idea, unproved, not yet worked out.

Tuesday, July 11, 2006

The Queen Strikes Back

Patzer sees check, patzer gives check? Or what is the reason for always checking checks first? Simple because a tempo gain is guaranteed, such as here. Maybe in most cases patzers are right, even if they do not stumble over a mate they have not seen.

Black to move. Queen hanging.

Principal variation: 23.-Qd3+ 24.Kg1 Nb3 threatens to capture the Rook a1 or the Bishop c1 only protected by the Rook. Sanity check positive because Queen is not in danger. Black should be winning.

Monday, July 10, 2006

Neglected Sanity Check

My opening had failed and I already was in serious trouble. I hated the pin and planned to get rid of it.

White to move. No check. No capture. Threat: h3 attacking the Bishop. Principal variation: 15.h3 Bxf3 16.Qxf3. I neglected the sanity check here and was quite surprised by 16.-Nb3! In fact, even before considering h3, a safety check of my position would have told me to keep a close eye on the semi-open a-file and my unprotected rook.

Sunday, July 09, 2006

Only Static Compensation

After a non-gambit like continuation I had zero attack for the pawn, only the static compensation of the bishop pair and a doubled pawn of Black. This does not look like an Evans Gambit, does it?

White to move. No check. No capture. No threat. The position is quiet.

Positional assessment: White has the bishop pair and a better development for the pawn. Black double pawn, but semi-open file against the isolated pawn a2. Worst standing white pieces are Queen d1 and Rook f1. Rule 1: Move every piece once before moving a piece twice. Rule 2: Connect the Rooks. Both rules are in favour of a Queen move clearing the first rank.

Initial candidate moves: 14.Qc2 or 14.Qd2. The first candidate allows a strong black Knight attack on g4 and is therefore discarded.

Principal Variation with forcing moves: 14.Qd2 Bg4 (threatens to destroy the white castle) 15.Qe3 Bxf3 16.Qxf3. Quiet position. Safety check positive.

Play 14.Qd2. In this quiet position it is hard to tell if this is really the best move. Deep Shredder prefers Qc1 but at least has no refutation against Qd2.

Saturday, July 08, 2006

Tempo's Just Another Word For Having The Right To Move

Yeah, it's like Bob Dylan lyrics. But does it help? Temposchlucker, in a comment to an earlier post of mine, raised the question of what's the value of a tempo. I agree, that's the point of it.

Being White is equal to +40 Elo. This has been calculated from millions of games. In other words, half a tempo is worth 40 Elo in the opening. Having the odds of a pawn is worth +200 Elo, therefore the value of a tempo should be 80/200 = 0.4 pawn units. But I never trust a statistics that I have not faked myself, so let's have a look at good old Evans Gambit. It has a reputation of being sound, will say, about 100 percent compensation for the pawn.

Black to move. 4.-Bxb4. Bishop moves without improving the position, +1 tempo for White. 5.c3 Bc5 moving 3rd time, White has +2 tempi now. 6.d4 exd4 7.cxd4 Bb6 moving 4th time, White has +3 tempi. Balance: 3x0.4 = 1.2 pawn units for the pawn.

Or 5.-Ba5 (+2 tempi) 6.d4 exd4 (+3 tempi) 7.0-0 dxc3 (+4 tempi) 8.Qb3. Balance: White has 4 tempi (1.6 pawn units) for 3 pawns, but is about to gain back one of the pawns. In any average position this would mean the loss of 0.4 pawns or being short of a tempo. But this position is very sharp and Black must invest at least one more tempo to save his King, making the balance at least equal.

I have played so many gambits without counting the tempi. I am about to change things now. Time will tell if it helps.

Friday, July 07, 2006

Look for a Better One

If you see a good move, look for a better one! Generalkaia, in his comment to my previous post, has raised the question how I could say that 9.e5 is the best move here. I didn't prove it in my post and should try to do it now.

Of course Bxf7+ is a check and a capture, but can be ruled out quickly because it simply loses a piece for nothing.

But Qb3 is a standard move in the Evans Gambit, just as e5 is, and therefore should be considered here. Principal variation: 9.Qb3 Na5 10.Bxf7 Kf8 and now the Queen has no square to protect the Bishop. The attack on f7 is not safe and must be ruled out. Without this attack, Qb3 makes no sense and must be ruled out, too.

Now the quiet move 9.Nc3 that I actually played in the game. There are just too many quiet variations in this position to be calculated. But the positional calculation is simple. White's trump is his advanced pawn majority in the center and both sides have developed two pieces, the white bishop being more active than his black counterpart. After two quiet moves such as 9.Nc3 Nge7 the balance in the pieces has become more equalized plus the relative weight of the pawn center against the piece development has diminished. Also Black has the zwischenzug Na5 forcing the Bishop to abandon the strong diagonal a2-g8. But the most important of all: Every move that allows Black to bring his King to safety with the plan Nge7 and 0-0 kills the powerful value of the advanced pawn center.

Therefore 9.e5 seems to be definitely the best move in this position.

Thursday, July 06, 2006

Don't Lose Tempo, Gambiteer!

In gambit I have often made quiet developing moves after having given the pawn. Against weak opponents you may still win with such a strategy, but this opponent was +300 Elo!

White to move. No check. No capture. Threats: 9.Qb3 against f7 or 9.e5 to open the e-file or 9.Bg5 to attack the Queen.

Positional assessment: White has given a pawn for attack. The black King is still in the center. Therefore the white plan must be to open the center as fast as possible, making 9.e5 the top initial candidate. Non-forcing moves like 9.Nc3 allow Black to improve his position in turn, so the compensation for the pawn must diminish.

Principal variations:
9.e5 dxe5 10.d5 Na5 11.Nxe5 Nxc4 12.Nxc4 - Black trades material and has the bishop pair bonus, but remains behind in development. The open e-file against the King should give White a full compensation for the pawn.
9.e5 d5 10.Bb5 - Black keeps the e-file closed, but opens the diagonal a3-f8. White has a powerful pawn center. Should be enough compensation for the pawn.

Conclusion: 9.e5 is safe and should be played.

Wednesday, July 05, 2006

First Game Revisited

Start game two years ago at Winterthur Chess Week. The first critical situation showed up in the 12th move.

White just has moved 12.Rfe1. No check. Capture: 13.Bxf6. Threats: Q-R battery x-ray on Bishop e7, 13.d5.

Black to move. No check. No capture. Threats: 12.-h6 and 12.-Na5, to attack the Bishops. Na5 to be ruled out - move every piece once before moving a piece twice, and there is no tactic here.

First assessment: White has more forcing moves and is better developed than Black. Black must improve his position, including quiet, developing moves into his search. Worst standing piece is Bishop c8. Therefore, 12.-Bd7 is the top initial candidate of the quiet moves.

Initial candidate list:
a) h6 (threat)
b) Bd7 (quiet)

Principal variations
aa) 12.-h6 13.Bxf6 Bxf6 14.d5 exd5 15.Nd5 Qd6 16.Nxf6 Qxf6
ab) 12.-h6 13.Bh4 just parries the threat, same situation as before
ba) 12.-Bd7 13.d5 exd5 14.Bxf6 Bxf6 15.Nxd5 Qd6 16.Rad1 Rad8
bb) 12.-Bd7 13.d5 exd5 14.Nxd5 Nxd5 15.Bxd5 Bb4
All are safe so far, but Black has to be very careful, White still standing better.

Final candidates: both 12.-h6 and 12.-Bd7 are playable, but h6 is more forcing. Everything else being equal, the more forcing move is to be preferred. Sanity check positive. Play 12.-h6.

Tuesday, July 04, 2006

Reverse Engineering of my WCW Games

Three months to my next big tournament, Winterthur Chess Week (WCW). Last year I did a lot of puzzles as a preparation. The success was not bad but far from good. This year I have a different strategy. I want to have a close look at my 18 former WCW games with a method that could be called reverse engineering. Will say, disassemble the thing in order to find out the construction plan - a set of simple, logical steps leading to the best move or at least to a good, playable move. That is, my new heuristic.

The heuristic itself looks simple, but chess positions are not. Therefore it cannot be bad to apply the heuristic with well-known (Shredder-analyzed) positions from old games before using it in new games.

Puzzles? No more at the moment. Why not? The shortcoming of puzzles is that you know there must be a tactical solution. In a game nobody tells you. You need a tactic filter. And «Checks, Captures, and Threats» is just such a filter.

Monday, July 03, 2006

The Way is the Goal

Again, not the solution is important here, but the way it can be found.

White to move. No check. No capture. Looking for threats. All black pieces except the Queen look safe, so the Queen a5 is the only significant target. The only piece that can attack her is the dark-colored Bishop. But how? Bd2 is too slow. A possible initial candidate move, but not a threat and therefore to be studied later, if ever.

Bb6 would be the only forcing threat, but is not possible (yet). Assume that it has been enabled by «Superman». Then the black Queen has only two escapes: Qb4 and Qh5. Is there any move that protects b6 and prevents the queen escape? Yes, Nd5 is «Superman», so the top initial candidate has been found, threatening to win the Queen. Black is forced to capture the Knight. (The Queen has as new escape, a4, but now the Knight has a fork on c7, winning the Rook.)

Principal Variation: 11.Nd5 exd5 12.exd5 (threatens to win the Queen by discovered check) Nce7 13.d6

Black to move. No check. No capture. No threat. From White's point of view, the position is quiet and ready to be analyzed positionally.

Initial assessment: this PV is at least winning back the Knight. If the Pawn is lost on e7, the material balance has not changed since 3.c3, but the compensation has improved markedly: Open e-file with Queen x-ray to black King. Big lead in development and in space. If White manages to hold the wedge on e7, he has a winning position. Therefore, 11.Nd5 is safe and makes a final candidate move.

Can a quiet move like 11.Bd2 compare with this? Surely not.

Decision: 11.Nd5 is to be played. Sanity Check positive: No other piece is attacked (Nd5 has passed safety check before). White plays 11.Nd5!

The Superman Trick: It may help whenever a threat cannot be made with the first move. Sometimes a quiet preparation is required. But even then it is better to look at the forcing moves first, assuming that «Superman» has enabled them. If they still are not sufficient, it is a waste of time to search for «Superman».

Saturday, July 01, 2006

First Try With My New Move Search Heuristic

Okay, we all know that White can move 8.Nb5! here and probably wins. But this is not the point. Will my new method be able to find this move, even if I wipe the whole opening book out of my brain and if I restrict pattern recognition to a one-move level? That is, seeing a knight fork in one, but not a knight fork in two moves. White to move:

Bishop is hanging. This problem has to be solved at quiescence, but not before.

White to move. No check. Captures on e6 and d7 to be discarded at first sight. Threats: 8.Nb5 to capture the Queen, no other threats. Nb5 is the first initial candidate move.

Black to move. No check. One capture: 8.-Qxc4 winning the Bishop, but White, one check, 9.Nd6+ and one capture, 10. Nxc4 winning the Queen. Safety check positive: 8. Nb5 is a final candidate move.

Black has to discard his capture, leaving him with no check, no capture and no threat. Other moves: 8.-Qd8, Qb6, Qa5, Qc6, Qc5. White in any case check 9. Nd6+ and the King must move, losing his right to castle.

Principal variation: 8.Nb5 Qd8 9.Nd6+ Kf8.

White to move. No check. No capture, because any capture that removes the excellent Nd6 from the board must be discarded. No significant threats on both sides. The position is quiet and safe.

Assessment: White is still down a pawn. Compensation has improved markedly: Excellent Knight Outpost on d6, advantage in space. Black King is unsafe. Black Rooks are buried for a long time. Black is behind in development by several tempi. Big advantage for White. No other candidate move has to be considered.

Decision: 8.Nb5 is to be played. Sanity check positive: No other piece is attacked (Bc4 has passed the safety check before). White plays 8.Nb5!

Note that 8.Qe2? (actually played in the game) not even shows up in the initial candidate move list! Why not? Simply because this is a quiet and passive move and thus not worth being the top candidate in a tactical situation.