# MCTS (Monte Carlo Tree Search) Framework

This directory contains a game-agnostic Monte Carlo Tree Search implementation that can be used with any turn-based game. The framework separates the MCTS algorithm from game-specific logic through abstract interfaces.

## Core Abstract Classes

### `MCTSAction` (abstract/MCTSAction.hpp)
Abstract interface for representing game actions/moves.

**Key Methods:**
- `getIndex()` - Returns the action's unique identifier
- `getDescription()` - Human-readable description for debugging/logging
- `clone()` - Creates a deep copy of the action
- `equals()` - Compares actions for equality

### `MCTSGameState` (abstract/MCTSGameState.hpp)
Abstract interface for representing game states.

**Key Methods:**
- `hash()` - Returns a hash for transposition table lookups
- `score(playerId)` - Evaluates the state's value for a given player
- `currentPlayerId()` - Returns whose turn it is
- `isTerminal()` - Checks if the game has ended
- `getWinner()` - Returns the winning player (if terminal)
- `clone()` - Creates a deep copy of the state
- `equals()` - Compares states for equality

### `MCTSGameEngine` (abstract/MCTSGameEngine.hpp)
Abstract interface for game rule enforcement and state transitions. Many methods have efficient default implementations.

**Must Override (Pure Virtual):**
- `applyAction(state, action)` - Applies an action to create a new state
- `getLegalActions(state)` - Returns all valid moves from a state
- `isTerminal(state)` - Checks if a state is game-ending
- `evaluateState(state, playerId)` - Scores a state for a player

**Optional Overrides (Have Default Implementations):**
- `applyActionMutable(state, action)` - Apply action in-place for efficiency (default: calls applyAction)
- `filterActions(actions, state)` - Applies heuristic filtering (default: no filtering)
- `simulateRandomPlayout(state, playerId, maxDepth, policy)` - Runs simulation (default: efficient mutable implementation)
- `getActionScore(state, action, playerId)` - Scores an action (default: apply and evaluate)
- `shouldStopSearch(state, iterations, startTime)` - Early termination (default: no early stop)

**Performance Features:**
- The default `simulateRandomPlayout` clones the state once and mutates it throughout simulation for efficiency
- Games can override `applyActionMutable` to provide even more efficient in-place updates
- Games can override `simulateRandomPlayout` for custom optimizations (e.g., using internal engine state)

## MCTS Algorithm Implementation

### `AbstractMCTSAI` (abstract/AbstractMCTSAI.hpp)
The main MCTS algorithm implementation that works with any game implementing the abstract interfaces.

**Key Features:**
- **Selection**: Uses UCB1 (Upper Confidence Bound) for node selection
- **Expansion**: Adds new nodes to the search tree
- **Simulation**: Runs random playouts to estimate node values
- **Backpropagation**: Updates node statistics with simulation results
- **Multithreading**: Supports parallel MCTS with configurable thread count
- **Path Compression**: Optimizes move sequences for better performance

**Configuration Options:**
- `explorationConstant` - UCB1 exploration parameter (default: √2)
- `maxSimulationDepth` - Maximum depth for random playouts
- `maxTreeDepth` - Maximum tree depth to prevent stack overflow
- `useMultithreading` - Enable parallel search
- `numThreads` - Number of worker threads
- `simulationPolicy` - Strategy for action selection during simulation

### `MCTSNode` (abstract/MCTSNode.hpp)
Represents nodes in the MCTS search tree.

**Core Data:**
- `action` - The action that led to this node
- `actionIndex` - Index in the original actions array
- `gameState` - The game state at this node
- `visitCount` - Number of times this node was visited
- `totalReward` - Sum of simulation rewards
- `averageReward` - Average reward (totalReward / visitCount)
- `children` - Child nodes in the search tree
- `parent` - Parent node reference

**Key Methods:**
- `CanExpand()` - Checks if node has untried actions
- `GetBestChild(explorationConstant)` - UCB1-based child selection
- `GetBestFinalChild()` - Most-visited child (for final move selection)
- `CalculateUCB1(explorationConstant)` - Computes UCB1 value

## Simulation Policies

The framework supports multiple strategies for action selection during random playouts:

- **RANDOM** - Uniform random selection
- **FILTERED_RANDOM** - Random selection from filtered action set
- **BEST_IMMEDIATE** - Always choose the highest-scoring immediate action
- **WEIGHTED_BEST_IMMEDIATE** - Weighted random selection based on action scores

## Type Definitions

### `MCTSTypes` (abstract/MCTSTypes.hpp)
- `MCTSPlayerId` - Player identifier type (int)
- `MCTSSimulationPolicy` - Enumeration of simulation strategies
- `MCTSConfig` - Configuration structure for MCTS parameters

## Usage Pattern

To use this framework with your game:

1. **Implement the abstract interfaces** for your game:
   ```cpp
   class MyGameAction : public MCTSAction { /* ... */ };
   class MyGameState : public MCTSGameState { /* ... */ };
   class MyGameEngine : public MCTSGameEngine { /* ... */ };
   ```

2. **Create and configure the AI**:
   ```cpp
   MCTSConfig config;
   config.explorationConstant = 1.414;
   config.maxSimulationDepth = 100;
   AbstractMCTSAI ai(playerId, config);
   ```

3. **Run the search**:
   ```cpp
   auto actions = engine.getLegalActions(currentState);
   auto result = ai.Search(engine, currentState, actions, timeLimit);
   auto bestAction = actions[result.bestActionIndex];
   ```

## Testing

The framework includes comprehensive tests using a Tic-Tac-Toe implementation:
- `MockTicTacToe.hpp` - Example implementation of all abstract interfaces
- `AbstractMCTSAI_test.cpp` - Unit tests for the core algorithm
- `MCTSIntegration_test.cpp` - Integration tests with complete games
- `MCTSNode_test.cpp` - Tests for the node data structure

This demonstrates how to implement the interfaces and validates that the MCTS algorithm works correctly with any turn-based game.