Time for Reflection

After finally finishing both the chess library and the web chess game, I do start to feel like I have created something I first thought was beyond my abilities and made something other people can use and enjoy. However, it is at this stage of the development that it is important to take a step back, try to look at the finished project with new eyes, and see what is missing, what parts are lacking, and what can be done to make it better.

It's done and working, so what more do I need to do? Show the world, ship it, everything is done, time to move on to the next project, right? This can be a tricky part of the process. Seeing your own work from someone else's point of view, as if for the first time, with a skeptical mindset, is hard, because working in it for so long you have become blind to the obvious issues. Sometimes it helps to step away from it for a week or two, but not too long, as you may never return to it.

After reviewing things as they currently are I have found some issues. The styles used when a piece is selected and when a move is animated, all need to be updated, because they look a bit off. These are minor issues and can be sorted out with some CSS adjustments.

The most important problem I have is that the computer player is very poor. I can beat it very quickly. I can move my queen next to a pawn for it to be taken, and the computer would not capture it, but make some other random move instead. When a computer is playing against the computer, it never seems to end, as if it doesn't know how to perform a checkmate.

The computer player configuration options may have all the settings I need for it to function better. I just need to work out which are the better values to use. How do I test one set of config values against a different set of values? I could manually play against the computer to see if it improves its game, but that would take forever. I could somehow rig it so that the computer plays against another chess library computer player, but that doesn't feel right. What is possible, however, is that I could have one computer player, with one set of configuration settings, play against a second computer player, with a different set of config values.

I think this idea needs to be expanded though. Instead of doing this inside the web game, I will need to use the computer-player class, which allows me to make a computer play against another computer, without working in a web browser. I could run it multiple times, changing between white and black players, and saving the results into a CSV file to be analyzed. This will be an interesting exercise.

Run a Game

I am not really sure how any of this is going to end up looking. All I have are some ideas of what to test and investigate. To start with, I want to be able to create a white and black computer player config object, change some of the settings, and then run a game between them to see what the results look like. I am putting everything into an "analyse" folder, as it is not really part of the chess library itself.

I have created a "game.js" file for the Game class. This has the static play function that will be used to run a single game of chess between the two given computer players. Each player has its own ComputerPlayer object, which is used to process their part of the whole game. Each one has their configuration set by using the config function. After this the board is created with the standard starting board layout.

The game then goes into a loop, with each player taking their turn, white then black and so on, allowing each computer to calculate their next move, adjusting the board, and moving on to the next one.

After each move is made the results are updated. I also check to see if the game has finished, which if it has then I break out of the loop and return the results. I have also added a maximum number of moves a game can go on for. Hopefully each game is no more than 100 moves long, but I need to make sure, just in case.

First Test

To make sure everything is working correctly so I can start performing multiple games, I am going to create a simple test. In the file "test-intrinsic.js" is the class TestIntrinsic with a run function that itself calls all the other testing functions, which at the moment only has the testCoefficient2vs1 function.

At the start I am setting several totals that are used to record the results of all the games played. This is followed by creating the computer player config objects, which are set to the old values, which are the current value at the moment before I work out a better range of numbers. This is so I can create the same results no matter what the newer default config values end up being.

For this test I am changing the intrinsic coefficient value from 1.0 to 2.0. This gives the board rating move weight to the value of a piece on the board than is standard.

The players are swapped around between white and black for each game, so player 1 is not playing white for each game. The game is then played with the results returned.

If the game was abandoned, then only that total is updated. If not, then the other total values are adjusted. The last part is to swap the players around.

After all the games the results are written to the console for now, instead of outputting to a CSV file. This is the first test, so I just want to make sure everything is working correctly.

After running the test, I ended up with an abandoned game. This is very disappointing. Something is wrong with the computer player logic somewhere.

Checkmate Error

The test game didn't finish, it kept going on past 100 moves and was abandoned. Why is this the case? My first area to look at is whether the computer player code handles checkmates correctly. To do this I have created the following board layout and asked the player, white in this case, to make a move.

All the white player needs to do is move the queen to the top of the board and it will win the game. But when the computer calculates the next move to make, it doesn't perform the checkmate but moves the king's knight instead. At least I can duplicate the issue and debug it to see what is going on.

I added a test for this in the "test-computer-player.js" file, setting up the same board and getting the white computer player to make a move. I then added a break point to see what was happening, ending up inside the _workoutFinalMove function, looking at the list of node moves that were created, the child nodes belonging to the root node, to see where the checkmate move was and what its board's rating value was.

The white player had 40 possible moves to choose from, all with similar board ratings (50 to 80), except for one that stood out, with the negative value -10042. This must be the queen's checkmate move, but the value should be positive not negative. There must be a bug somewhere.

The board rating class is used to give the current state of the board a rating value, where a positive amount means the board is better for the white player, and a negative one would be better for the black player. When it comes to setting the checkmate value, it checks which player it is that has been checkmated and assigns either a positive or negative value. The queen's checkmate move was being given a negative value instead of a positive one, so technically, all I need to do is reverse what it is doing at the moment.

There are times like this where you don't really understand why the issue exists, only that something needs reversing, swapping plus for minus or the other way around, and retesting to see if everything starts working correctly. However, I do feel like I need to properly understand the reason for a problem so that I can truly believe I have fixed it. Randomly changing some of the math and hoping it fixes the issue does not sit well with me.

In this case, if the player is white, it is not a positive thing to be in check or checkmate, therefore the board's rating needs to be downgraded, lowered, showing that it is losing value and would be in a worse state. Therefore, I am changing the plus and minus parts around for the check and checkmate calculation.

After making the changes and retesting, the list of nodes includes the queen's checkmate move, with the positive rating value of 10158. That problem is now sorted. However, after rerunning computer against computer test, the game was again abandoned. Still, something is wrong.

I ran a computer against computer match using the web game, so that I could see the game being played between them, and I saw that white player was chasing the king around the board, as if putting the king in check was more important than anything else, even if the king could take the piece afterwards. I believe this is down to the computer player config setting "check" being given too high a value. Instead of 100 this may need to be 10.

After using the web game to see how the new adjustment played out, the white player won after 45 moves. After rerunning the analyse tests, the first game was abandoned, and the second one was won with 99 moves. I am increasing the move count limit to 200 and see if that helps. Watching the game playout between the two computer players does not fill me with confidence, they still play poorly.

First Test Again

Now that I have sorted out the checkmate issue, and adjusted the check value, I am now rerun the same intrinsic coefficient test, but this time I am playing 10 games. The end result is interesting.

There is still one abandoned game. It does look like doubling the intrinsic coefficient does have some effect, winning 7 out of the 10 games. This is working as I was hoping it would. Comparing two different sets of computer player configurations and seeing which one wins more. Before I move on to the next test, I want to move some of the testing code into a reused function and also think about writing the results to a file to be later examined.

I have created the "test-game.js" file containing the run function, which basically does the same thing as the other test function, except it has a number of parameters that allows me to run a test with different computer player configuration settings and save the results into a CSV file.

This allows me to change the testing functions so that they are reusing the same code to run the games. Here is an example of the results from my testing.

Now that it looks like I can quickly run a number of different tests using different configuration settings, I want to run some ideas and see what the results look like. First up is changing the intrinsic coefficient values.

When both computer players have the same intrinsic coefficient value of 1 then the end result does seem predictable, with no player having any advantage. When player 1 has double the value, set to 2, it wins most of the games, with only 1 being abandoned. It does seem increasing the value of the pieces a player has with the boards rating improves its winning ability. However, it drops off when the value is 4, with more games becoming abandoned.

Each piece has an intrinsic value, which is something I could change, but to keep it fair I need to do some math. At the moment the values are queen is 8, rook is 5, bishop and knight is 4 and the pawn is 1, therefore using the default number of pieces on the board (2 rooks = 10), the final value for all pieces (for a single player) is currently 42. Therefore, if I want to change the values, it would be fairer to make sure the amounts add up to the same value.

First, on the first test I changed the values of the pieces so that the queen was given a higher level, which did give it the advantage. Giving the knight a higher value to the rook and bishop did not help at all, losing more games than winning. I also wanted to run the same tests but give each player a higher intrinsic coefficient value, to make sure these piece changes were having some real effect. It does seem that giving the queen a higher value compared to the other pieces does make a difference.

More Tests

I now want to see if the number of moves ahead the player looks into makes any difference. I can understand this is true to a point but where this point is will be interesting to find out.

The first game I reset the player to only look 1 level ahead and as you would imagine it didn't win a single game. The second game had the level increased to 3 levels. I did notice that each game took a lot longer to run. The result however was very good, with it winning all but one game (which was abandoned). This is what I expected to see. I would have tested level 4, but I have a feeling it would take hours to process.

It seems like a straightforward conclusion to say that the more moves ahead you look the better the outcome will be. It may be possible to go up to 4 levels when using the web game, as it has more threads to work with, but for these tests it doesn't seem worth it.

I now move on to testing the defence side of things. I am going to include the defence coefficient and see what the results are. But, while performing the games, it got into problems, it stopped working.

Another Issue

While playing through some computer versus computer testing, there was an error. I added a check point to see what was going on and thankfully the issue triggered again. Looking at the information available to me I saw that the board only contained a black king. This is not a state that the board should be able to get into. It was at this point I figured out my mistake.

1. A king was able to move next to another king. This is not a valid move.
2. I do not check for stalemate where neither player can win.

A king cannot move next to another king because it would be putting itself into check. I was not checking this and as a result, one king was able to capture another. Thankfully the extra code I needed to add was very simple to do.

In the "move-calculator.js" file there is the _inCheck function, which is used to see if the given move in the given board is valid or not. I have added an extra function to check the king. This is just like the other checks for the other types of pieces.

It is very similar to the other "inCheck" functions. It sets the king piece it is looking for, then gets the piece at the different locations on the board and checks if it is the other king. I also added an extra test to make sure it was working correctly. This is all that was required to fix this issue. When the solution is simple, I have a good feeling that it was the correct one.

The next part is to check if the game has ended in a stalemate. This is a little more involved. The first part is to change the Board class to have a new property stating whether the board is finished in stalemate. I have added the _stalement property, including the getters and setters, the clone and to/from code parts. I have also added the same property in the Move class.

The question is, how and where do I work out if the game has gotten into a stalemate? What are the requirements that need to be met? I am going to say that, if there are only two kings on the board, then the game is in stalemate. However, it may be possible to have pawns blocked by other pawns so they cannot move, but the king could capture them, maybe. I would imagine there are many edge cases where it is possible to have a stalemate with more than just two kings, but I am going to ignore these possibilities.

The logical location for the check is in the "move-calculator.js" file, in the _processCheck function, where it looks to see if all the moves are valid, put the player into check, and the game into checkmate. All I need to do is add an extra check to see if the game is now in a stalemate state.

For each move, if it does not put the game into checkmate, then I need to check if the game is in stalemate by calling the _inStalemate function.

The function looks at each square on the board and checks the piece. If it isn't empty and not a king, then it must be another piece, and therefore that board cannot be in a stalemate state. I like how easy this has turned out to be.

The only other place I need to make an adjustment to the code is within the "move-maker.js" file, when I am updating the current state of the board. I am copying the stalemate value from the move into the final board. That is the main chess library sorted, but I do need to also update the web game too.

Within the "chess-game.js" file, in the _updateStatus function, I check to see if the board is in stalemate and change the class name used and the status information text shown. I also hide some status elements too.

If the game is in stalemate, then I do not want the user to be able to select any of the pieces anymore. I can do this by adding some code into the square click event function and checking the stalemate status.

Back to Testing

After fixing those issues I can now return to my defence coefficient testing. I want to see what happens when I change the values. Will they make any difference?

Looking at the data, it does, but not in a good way. The first parts have the intrinsic coefficient unchanged, but in the second parts I have increased them to 2.0. It does look like the more defence I set the computer player the longer the games are and therefore more likely for them to be abandoned. I need to do the same type of tests but with the attacking side.

In the first parts I am only changing the attach coefficient, which as you can see, only makes things worse. For part 2 I am again setting the intrinsic coefficient value to 2.0, but it doesn't seem to make a difference. It does seem that adjusting the defence or attack coefficients does not improve anything. Perhaps if I look at changing the value of the pieces?

Changing the values of the defence pieces does have an effect. If the first test, where the queen is given the value of 18, it ends up losing more games, but setting the queen and knight to be 6.0, improves things. I also perform the same test but with a defence coefficient set to 2.0, which before had made things worse, and again it seems to have a bad effect. The last line shown gives information about the default piece values for comparison. Now for the same thing but with attacking.

There isn't much of a difference again. Any adjustment to the attack coefficient creates problems. None of this is looking good. It does seem that a more attacking approach has a worse outcome.

I have looked at changing a number of different settings, but they are not showing which of the configuration changes I need to make in order to build a smarter computer player. Therefore, I am going to try and combine some of the results and see if there are any improvements. Below are the configuration values I have changed.

• intrinsicCoefficient = 2.5
• intrinsicQueen = 18.0
• intrinsicRook = 3.0
• intrinsicBishop = 3.0
• intrinsicKnight = 2.0
• intrinsicPawn = 1.0
• defenceQueen = 6.0
• defenceRook = 4.0
• defenceBishop = 4.0
• defenceKnight = 6.0
• defencePawn = 1.0
• attackCoefficient = 0.5

The new configuration changes clearly beat the old default settings. This is great to see. But how does it play against a human opponent?

I played a game against the computer to test the new configuration settings to see if it played better than before. Sadly, it was still rubbish, and I found another bug. I put the computer into checkmate, and it then performed a move, when it should have done nothing.

In the "chess-game.js" file, within the _checkComputerRun function, I have added a new check to see if the game has finished in checkmate, and to stop the computer from making another move. Once the fix was added I was able to continue playing against the computer. I have also added in a check to look for the game ending in stalemate.

To see if I could improve the computer player a little more, I decided to increase the level, the number of moves ahead to look at, from 2 to 3. The first move took 2 minutes to calculate and ended up creating 420,000 nodes, which is not great. The second computer move took over 5 minutes. I increased the number of node workers from 4 to 10, hoping that it would cut down the time. Nope, that didn't work as hoped. I am thinking that level 2 is the limit, or something is wrong with the code. Maybe the browser is throttling the node workers, slowing down the processing? The third computer move contains 695,491 nodes!

After all the effort I have put into making this and it just doesn't work correctly. I am full of frustration and feelings of failure. Has it all been a total waste of time? I need to take a break to think about whether I should abandon the project and maybe consider if I should still write software for "fun"!