Painting Tiles and Autotiling
I managed to get the autotile to roughly do what I want today, but it's far from perfect.
Early in the day, I figured out a few of the bugs that were causing weird autotiling to happen when painting. It turned out to be a mix of bad data in the tile connection stats, and adding a "priority" field to each tile, to control which tiles are preferred over others. With these fixes in place, I was able to paint just about any tile and get the surrounding tiles I expected.
However, this only handled the four orthogonal tiles around the painted tile. I.e. top, bottom, left, and right. The corners are ignored.
My next step was to expand the code to also try and figure out good fits for the corner pieces. This wasn't too hard to do, especially since the tile connection rules still only apply orthogonally. Basically, I don't have to check that diagonal tiles match sides, because the tile matching only matters in horizontal and vertical directions.
Once this was done, I was able to paint floor all over the screen, and see walls fill-in around it. I could even do winding, snaky corridors!
The bad news is that other tile types seem to screw the system up. Painting a solid wall, for example, seems to add empty space in the middle of floor if it gets too close. And the eraser tool seems to leave more garbage than it picks up.
I'm also starting to wonder if it may make more sense to approach the ship design differently. Instead of updating the walls instantly each time a floor tile is painted, maybe I should let the user just paint floor everywhere, and when they're ready, have the computer auto-tile walls around it? This way, the computer won't screw up player-painted tiles as they're working, and also doesn't waste as much processing time. The inverse could also be true: paint the walls and fill in gaps.
However, I suspect that even with this alternative painting style, players will still want to tweak the ship layout after the computer autotiles it for them, possibly in real-time.
Whatever the case, it definitely needs more testing and tweaking!