Hexcells Infinite Solver

Hexcells Infinite is a puzzle game where you have to find all blue hexes on a hexagonal board. Orange hexes are unknown. Black hexes show the number of blue hexes touching them. Blue hexes with numbers indicate the number of blue hexes within two hexes around them. There may also be line constraints outside the grid.

A year or so ago, I was using Anthropic's AI to build a script that would solve the puzzles. That attempt was not successful. Today, a fresh start managed to create a script that solves puzzles.

Since the game doesn't have an API to interface with, the script needed to take screenshots, figure out what's inside, and then automate clicks on the screen.

Main Issues

There were three main issues: interacting with the game, detecting cells, and detecting the constraints.

Interacting with the Game

Letting AI run a computer was a bit nerve-wracking at first. It was clicking around in the menu bar and on various windows. That's not something that I would ever want.

I moved the work into a macOS virtual machine so that it would be isolated from my main macOS. Since it was in the virtual machine, Claude Code was able to operate without asking for permission.

Clicking on the correct window and the correct cell was a minor issue in the end. The script was updated to handle whether the game was on a secondary monitor or not.

Also, the size of the window was a minor factor that required the AI to make some updates so that the correct hexes were clicked.

Detecting the Cells

The cells were in a hexagonal grid and all the same size. However, the initial code from a year or so ago had issues with detecting the cells and their layout.

The cells were eventually detected, but didn't translate properly to the internal model. The model didn't manage the neighboring cells quite correctly and thought some of them were two cells away.

There was a bit of debugging to get cell detection correct in the first attempts, as seen below.

Detecting the Constraints

Since the application was taking screen captures, it needed to use optical character recognition (OCR) to read the numbers. This was challenging for a couple of reasons.

The constraints were numbers and a few symbols that were hard to decode. Most of them were horizontal with the default view, but some were marked outside the grid and were angled at ±60° from horizontal.

The size of the hexes and fonts changed slightly. This meant the script had to account for the changes.

The First Attempts

My first attempt at a script failed to work as expected. That was after a week or so of trying. I don't even remember the AI model used in the attempt.

While debugging the logic of the early attempts, I realized the issue was with the cell neighbors. Rather than attempting everything at once, it was better to add specifications and features incrementally.

The initial goals of the next iterations were to find each hex by color, then the numbers, then get into the logic for figuring out potential moves.

Eventually, I gave up on that approach and went with a different approach. That was to create a web page where I could manually enter the cells and numbers and get hints as to the next moves. It was a slower way to figure out possible moves, but it did work.

Second Attempt

After some time spent playing with various AI models and building other games and applications, I instructed Opus 5.5 to create a script that could solve Hexcells Infinite.

The initial prompt mentioned that the hexes were the same size within a puzzle, but could vary in size between puzzles.

Opus 5.5 was able to create the script and figured out most of it on its own. I didn't need to set any goals as I did with the first set of scripts. The hexes were detected easily.

The constraints were a bit tougher. Opus 5.5 wanted to look at the game files to determine the font used. It didn't have access to the game files and asked that I copy the files to a directory for it to read. It found the fonts, and the OCR detection was greatly improved.

I also had Opus 5.5 create a script to go through the training levels of the game as a test. A few minor issues came up on those levels, and Opus 5.5 fixed them.

The script worked after a few hours.

I ran the script that went through the training levels a few times to look at the output. There were a few uncertain but likely detections. In those cases, the solver would hold off on moves related to the uncertain detection.

One uncertain constraint detection was fixed, but others remained. I assumed other constraint detection issues would remain and go through the same cautionary check for moves.

After a few random puzzles, there was a cell with two line constraints that got interpreted as one. The line constraints fall outside of the hexes, and this specific one had two line constraints on one hex, but in different orientations. Opus 5.5 fixed the issue.

Thoughts

AI has come a long way in such a short amount of time. AI can now take a short, simple prompt without the need for multiple goals, and it requires less checking than before to solve a problem.

How long until we have a system that can handle all the issues encountered without requiring any checking? That said, we should always check.