Kirk D. Brown — Blog

CodeCup - Training for pnn7

2026-09-07T00:00:00.000Z

CodeCup 2027 Training for pnn7

After another 30 hours of training, Fable 5 was disappointed in the results, since there wasn't any meaningful improvement made.

Fable 5 created a set of experiments to figure out the next steps rather than just relying on another training run. Those experiments tested various parameters and neural network configurations.

Fable 5 made updates to the model and generated pnn7.

Results, including pnn7, in local competition:

Points	Player
49899	pnn7
48678	pnn6
45762	pnn5
45573	pnn5c
43848	pnn4
42749	pnn3
35601	pnn2
34487	pnn1
30391	player3
28374	player2
24738	player1

pnn6 had been updated as well.

CodeCup - New Training

2026-09-06T00:00:00.000Z

Change in Training

After a change in training from win-loss records to total points scored in the competition, pnn6 finished training. The training started before the test competition and finished hours later.

Local competition points results:

Points	Player
43735	pnn6
43500	pnn5
42238	pnn5c
40944	pnn4
39706	pnn3
32965	pnn2
31923	pnn1
28205	player3
26085	player2
22599	player1

pnn5c was an adjustment made to pnn5 for testing.

pnn6 is only 235 points ahead of pnn5. This appears to be a minor improvement.

For the Next Competition

The next test competition will be on September 19.

Fable 5 was asked what improvements could be made based on the first test competition results and the latest training results.

Some tests were run to evaluate the conversion of numbers from Python training to C++ player code.

There were also some adjustments made to the C++ player code to improve its competition performance.

Another training round was started.

CodeCup - Test Competition #1 Results

2026-09-05T00:00:00.000Z

CodeCup 2027 Test Competition #1 Results

pnn5 was submitted for the first test competition as it was the best-trained player in a local competition.

There were 23 entries in the competition. Five Swiss rounds were played to get an initial ranking. Then 20 rounds were played to determine the final ranking. In each round, two games were played, once as the red player and once as the white player.

pnn5 ranked 6th in the test competition, from the 20 rounds, with a score of 6741. The first-place score was 7529, 788 points ahead of pnn5. That is 19.7 points per game to make up.

Fable 5 analyzed the results of the games and it reached the same conclusion as I did on the scoring. It also went deeper into the games and looked at the moves made by each player.

Fable 5 made guesses as to what algorithms were used by the players. I installed the Claude for Chrome extension. Fable 5 used the extension to gather data from the games. Fable 5 even updated the code to run a sample game to test adjustments against the top player.

Next Competition

The next test competition will be in two weeks, on September 19.

My thoughts were that Fable 5 might be able to make changes to the code to improve the player's performance in the next competition.

I thought that if the top players are also using AI to code their players, they might be able to improve their performance as well and possibly faster than I have been.

CodeCup - Training Updates

2026-09-04T00:00:00.000Z

CodeCup 2027 Update

Continuing with the CodeCup 2027 posts, this update reports the results of the 5th training round and an update to the training evaluations.

5th Training Round Results

Points	Player
35478	pnn5
33280	pnn4
32735	pnn3
27764	pnn2
26655	pnn1
23366	player3
21460	player2
18222	player1

pnn5 finished ahead of pnn4 by 2,198 points — a wider gap than pnn4's 545-point lead over pnn3.

Training Evaluations Update

While writing the previous post, I had AI review the post for any errors or issues. In the post, I mentioned that the pnn4 player showed a minor improvement over the pnn3 player. The AI noted that when the pnn3 and pnn4 players played each other, they tied on the win-loss record.

While the 5th training round was underway, I informed the AI that the scores were added up for each player in the competition and the ranking in the competition was determined by the total score, which is slightly different from a win-loss record used in the training rounds.

Fable 5 suggested updating the training to focus on the total score rather than the win-loss record. This update was completed, and there wasn't enough time to complete the 6th training round before the test competition on Saturday. It was taking about 3 hours to train each generation, and training 10 generations would take too long.

The update to the training will be used for all training rounds going forward.

Scoring

The scoring of a game is as follows:

Players score points based on the territory they control.
The winner gets a bonus of 200 points.
The loser gets a bonus of 100 points.
A program that makes an invalid move, crashes, exits early, or otherwise behaves incorrectly gets a score of 0 points, regardless of territory. In the competition, if a program fails in one of those ways, it gets replaced by another program so that the other player can still compete and earn points.

Fable 5 points out that the win-loss record is embedded in the scoring system in the 100-point difference between the winner and loser bonuses.

CodeCup

2026-09-02T00:00:00.000Z

CodeCup

CodeCup is an annual programming competition where you submit a code file that plays a game. You can monitor how well the code runs through the test competitions that run every three weeks and make updates. The final competition, the official one, runs in late January of a given year.

While the game changes each year, the technical constraints usually remain the same. Your program has a maximum size, a limited amount of memory, and a time limit per game.

My first competition was for 2005 with the game Lamistra, a Stratego-like game. There have been many different games played over the years including a Scotland Yard like game, a maze exploration game, quantum tic-tac-toe, a variation of Reversi, and many more.

The Challenge

The challenge each year is with the technical constraints.

The source code file has a maximum size of 1,474,560 bytes, which was the size of a floppy disk, and can use up to of 2GB of memory. And your code has 30 seconds to run. Your code cannot write files, spawn threads or processes, or connect to another server. There's no "thinking" time while your code runs. In some years, your program reads in a file.

The competition software is what your program communicates with via standard input (stdin) and standard output (stdout). Standard error (stderr) can be used to output debugging when a competition is running.

CodeCup also provides software that can be used to test your code on your local computer, called Caia.

I've mostly implemented the players in C++ and it's become my yearly C++ project.

My CodeCup History

I was informed of the competition in 2004 for the CodeCup 2005 game called Lamistra. I got hooked by both the game and the technical constraints. I've been following the competition each year since. For Lamistra, my program ranked 11th in the final competition. My best year was CodeCup 2016, placing 9th in the final competition.

CodeCup 2027 was announced in early August 2026 with the final competition set for late January 2027. The announcement time in previous years was usually in July or August with the final competition set for late January of the next year.

I would write a program and work on it most years locally. Over the years, I got into a pattern of writing a first program for a player that just made random moves, so that I got the protocol and rules of the game implemented correctly. Some years, I wouldn't even submit a program if the program wasn't completed or if it didn't work correctly.

My intent was to always improve my programs and try out different AI algorithms for playing games, but there was always a time constraint on my end.

For CodeCup 2027, the game is called Tumbleweed, an abstract game on a hexagonal grid where players place numbered stacks based on lines of sight to capture territory.

Players score points in the competition based on the amount of territory they capture and if they won, lost, or had an error. The winner gets an additional 200 points and the loser gets an additional 100 points. A program that makes an illegal move, crashes, or exits early gets 0 points.

AI Round One: Claude Code and Opus 5

In previous years, I'd download the Caia code and run my program against the three provided player programs. And typically, my programs wouldn't do well against the best of those three players.

When the competition was announced in early August 2026, I decided to use Claude Code and Opus 5 to do all of the work for me. I downloaded the Caia software for the competition and I set up a directory for Claude Code to produce programs and run the games against the players.

I was in the habit of taking parts of the C++ code I'd written in previous years to start the code, usually the utility functions for communication and some basic utility functions for the game logic. For 2027, I didn't do that. I let AI figure out everything itself.

My initial prompt in Claude Code was to look at the rules of the game and create a program that could beat each of the provided players. I set each of the provided players as a goal, so that Opus could work up from the easiest to hardest player. The prompt also said to read the rules and report back on how everything was supposed to work before jumping into coding anything. The prompt also pointed the AI to the local Caia Tumbleweed installation and told it not to modify any code inside that directory and that it could only create a symbolic link to code. It was also told how to run the code to test locally.

After a couple of minutes, Opus reported what the game was about and an implementation plan for the game. It first thought it played only one side of the two-player game, but I prompted it letting it know the program needed to be able to play as either player in the game. 15 seconds later, it updated its understanding of what was needed and started on the first goal, to beat the easiest player provided.

After 9 minutes and 50 seconds, it had generated a program that could beat the easiest player as either player in the game. The resulting program beat the easiest player 10–0 playing red and 10–0 playing white. Then I approved moving on to playing against the second provided player.

After another 41 minutes, there was a program that was very competitive against the second provided player. The program won 9–1 playing red and 8–2 playing white. Then I approved moving on to playing against the third player, the toughest player provided.

After 5 and a half minutes, there was a player that was beating player3. The heavy lifting was done against the second player. The result was the program won 9–1 as red and 10–0 as white.

After updates for each program, the final results were:

Opponent	Record	as red	as white
player1	20–0	10–0	10–0
player2	20–0	10–0	10–0
player3	19–1	9–1	10–0

This was impressive. In a couple of hours in total, Opus 5 created a program that could beat the toughest player provided by Caia. This was something that I hadn't achieved in previous years of the competition.

The program used "a classic iterative-deepening alpha-beta search with a hand-tuned positional evaluation — no MCTS, no neural net, no opening book, no transposition table."

I asked what could be done to improve the program. I also asked about other AI algorithms that I was aware of and if they'd work. And of course, it came up with a plan.

Next Steps

I was running Claude Code inside a Docker container using a sandbox (SBX) so that I wouldn't constantly be prompted by it to permit it to do things. Training a neural network would take a lot of time since Claude Code was running inside a Linux container and not on my MacBook Pro (MBP). It didn't have access to the hardware to train things more quickly. So I set up the container to be able to SSH into my MBP so that it could train on the hardware.

Claude Code managed to update the code to fix some compatibility issues with functions not being available on a Mac that were available in Linux. Then it came up with a plan to train a neural network. It wanted a specific version of Python (3.12) and PyTorch, so I installed them. It also complained that g++ wasn't installed and it was using clang++, so I installed g++ tools as well to get GCC parity with the competition server.

Claude Code would run tests and go through each of the phases it planned and reported that the game ran in the time limit. Both the clang and gcc builds fit in the normal time limit, so timing wasn't going to be an issue.

After a few iterations, it managed to train a neural network that beat each player 10–0 as the red player and 10–0 as the white player. It was creating various programs and comparing them using Elo rankings. An analysis of the Elo rankings hasn't been performed yet other than noting that the neural network was improving with each iteration.

Ultimately, it was training using Python and then updating C++ code for submission to the competition server. It was a little frustrating waiting a few weeks before submissions were opened up to see how well the code would actually work on the competition server.

I let Claude Code do its thing over the next few days with regular check-ins and monitoring the chat. Most of my prompts at this point were "What's next?" or "What's running?" If I was asked to make a decision, I typically went with what was suggested or told Claude Code to try them all out.

At one point, I noticed that the produced programs would contain a lot of bugs and I told Claude Code to double and triple check each iteration to make sure it didn't introduce new bugs.

An Opus 5 version of the program was submitted to the competition server. It was accepted and it beat the server's test player. It also ran in under 30 seconds per game, which was reassuring.

AI Round Two: Claude Desktop and Fable 5

On August 31, 2026, I switched to using Claude Desktop and Fable 5 to create a new project for CodeCup 2027. It was a shorter prompt than with Claude Code and Opus 5. Fable 5 took the prompt and started to do everything correctly the first time, including understanding that the program had to play the game as either player.

Claude Desktop and creating a new project inside it worked well. There were far fewer permission prompts and I didn't have to run an SBX sandbox for Claude Code. The other way I've run Claude Code is using a Linux VM inside Parallels and the flags to skip permission prompts. My preference at the moment is to use Claude Desktop on macOS rather than spinning up a Linux VM or running Claude Code using SBX.

Claude Desktop did find the programs created by Claude Code as those programs were linked in from the Caia directory. So it was able to run the programs and evaluate them against each other.

Claude Desktop also put the source code inside Caia's bin directory, despite being told to only link the compiled binary.

There was an initial training run on macOS that took a few hours to run. Once it completed, Fable 5 suggested continuing the training. So I continued the training for a second time, letting it run for a much longer time.

Once the second training run completed, Fable 5 evaluated the results and concluded that the approach was not good. So Fable 5 updated the approach and we retrained from the results of the first training run.

Four training runs have been made. There may be time for a couple more training runs before the first test competition.

The code files produced by Fable 5 after the first and fourth training runs were submitted to the competition server. They both beat the server's test player and ran in under 30 seconds per game.

Organization of Files

Before compiling these results, the files were reorganized. The code was moved out of the bin directory into a separate directory in order to keep the bin directory as clean as possible.

The algorithm family was put into the file names, to prevent confusion between approaches and version numbers. Versions 1 and 2 of the neural network family were recreated from the saved files.

The older versions v1 and v2 that were created were from a different family of algorithms. The focus of this project was on the neural network family.

Fable 5 Results

Here are the initial results from the Fable 5 training runs as of this writing. The provided players were run against the four neural network players generated by Fable 5 from each of the training runs.

The neural network players are named pnn1 through pnn4, where pnn stands for "pure neural network".

pnn1 results

Opponent	Record	as red	as white
player1	20–0	10–0	10–0
player2	20–0	10–0	10–0
player3	19–1	10–0	9–1
pnn2	8–12	7–3	1–9
pnn3	0–20	0–10	0–10
pnn4	1–19	1–9	0–10

pnn2 results

Opponent	Record	as red	as white
player1	20–0	10–0	10–0
player2	20–0	10–0	10–0
player3	20–0	10–0	10–0
pnn1	12–8	9–1	3–7
pnn3	1–19	1–9	0–10
pnn4	3–17	3–7	0–10

pnn3 results

Opponent	Record	as red	as white
player1	20–0	10–0	10–0
player2	20–0	10–0	10–0
player3	20–0	10–0	10–0
pnn1	20–0	10–0	10–0
pnn2	19–1	10–0	9–1
pnn4	10–10	9–1	1–9

pnn4 results

Opponent	Record	as red	as white
player1	20–0	10–0	10–0
player2	20–0	10–0	10–0
player3	20–0	10–0	10–0
pnn1	19–1	10–0	9–1
pnn2	17–3	10–0	7–3
pnn3	10–10	9–1	1–9

The competition script provided by Caia runs each program against each other player for 20 games, 10 times for each color. The overall results are the sum of the points scored:

Total Points	Player
29817	pnn4
29201	pnn3
25202	pnn2
23866	pnn1
20995	player3
19095	player2
16044	player1

We can see the improvements of the neural network models over the course of the training runs. The exception to this is that pnn4 and pnn3 have a 10–10 win-loss record against each other. In the competition, the more points a player earns, the higher their rank.

The Competitions

The test competitions haven't started yet at the time of writing. They start on September 5th, 2026 and run every three weeks. I'll update this blog as the results come in and continue to use Claude Desktop and Fable 5. I'm curious to see how well Fable 5 will do.

Conclusion

I've been impressed with the results so far. The initial Opus 5 was able to beat the provided players in just a few hours.

The code produced by Fable 5 has been submitted to the competition and I'm looking forward to the results. There will be more training runs from the Fable 5 codebase and we'll see how they do. There will likely be another submission to the competition.

After the final competition, I'll go through the technical details of the programs created by AI and describe the algorithms used. I'll also include links to the code.