Dec, 2020 - Jan, 2021
Gameboy Tetris System is a sort of AI system that can play Tetris (Gameboy) from just having access to the pixels (e.g. seeing the screen). At faster speeds, the system tends to fail to read the system, but due to the multi-threaded nature of the Gameboy emulator itself. I couldn't seem to make the system run faster than the emulator without issues. The paper and presentation are included on Github as this was a Graduation Work for the Digital Arts & Entertainment curriculum.
The very first step was research. It was important to know how the Gameboy emulator worked, what pixels I would need and how I would be able to recognize the different menus and the game itself. It was also important to already figure out what kind of AI was planned.
Before I could start on the main functionality / views, I needed to create functions and views that would show me the behaviour that my code would do. I needed to visualize what pixels I would be receiving.
Once I had my helper views and I could visualize what color the pixels manually, I could work on implementing it to visualize the current piece area, next piece area and the playfield.
I had the info necessary to play the game, but not the info of what menu I was in yet. Luckily, the screens had enough distinctive differences in very specific locations that I could make use of to guide the AI through them.
Now that everything was in place, I needed to work on having the AI / System be able to use all the information. In the mean-time I also created several more views that would allow me to manually see what the current best move would be and also calculating the score of a specific move. As I made those helper views, I was able to retrieve even more info for the AI such as the moveset it would have to do.
The system still had a major bug where it didn't want to do the correct move, but after a long time debugging and rewriting several times, I found the issue. You can see the buggy version in the gif.
The Pixel Buffer Inspector is one of the several Helper Views that allows me to debug and calculate the coordinates I need, to look at specific parts of the screen.
The Pixel Inspector is another helper view that allowed me to check individual pixels and their color. This represents one of the ways to gather additional data needed.
Block Prediction was a very helpful view later in development as it allowed me to check moves and their scores myself. That way I could compare what the system thinks is the best move and what the score actually is.
Best Move Prediction is the last helper view which would calculate the best moves based on given pieces. This allowed me to specifically see what the system would calculate as best move, it was a lot easier to debug this way.
These are the views that mattered and that the system used to play the game. You can see the playfield, the current piece and the next piece.
Last but not least, the play info. This showed you the current state of the game as well as the move the system would do when it is playing.
