# True Iterative Deepening Implementation

## Current Status

### Phase 1: Core Implementation ✅ COMPLETED
- ✅ Updated `IterativeDeepeningAI.hpp` with new data structures
- ✅ Implemented new `IterativeSearch` function with generalized depth loop
- ✅ Added `GetCommandsSortedByPreviousDepth` helper function
- ✅ Added `SelectBestResult` helper function
- ✅ Implemented 50% budget check to prevent incomplete deep searches
- ✅ Added SET_UP phase detection and depth limiting
- ✅ Ensured depth 1 always completes regardless of time budget
- ✅ Added END_TURN detection to prevent excessive depth exploration
- ✅ Implemented command change logging for debugging
- ✅ All tests passing

### Phase 2: Code Cleanup 🚧 PLANNED

#### Proposed Cleanup Tasks

1. **Replace Heuristic END_TURN Detection**
   - Current: Uses score comparison heuristic to detect when lookahead hits END_TURN
   - Proposed: Modify `AIScoreCalculator` to return explicit `performedLookahead` flag
   - Benefits: More reliable, cleaner architecture, explicit intent

2. **Refactor Return Structures**
   - Add `bool performedLookahead` to `CommandEvaluationResult`
   - Update `BasicLookaheadCalculator` to track and return lookahead status
   - Thread this information through the scoring pipeline

3. **Architecture Improvements**
   - Consider extracting iterative deepening statistics into a separate class
   - Improve separation of concerns between search algorithm and scoring

4. **Performance Optimizations**
   - Profile memory allocations in deep searches
   - Consider pre-allocating vectors for very deep searches
   - Investigate parallel evaluation opportunities at each depth

### Key Implementation Details

1. **Data Structure Changes**:
   - Replaced `reusableDepth1Results` with `scoresByDepth` (2D vector)
   - Added `highestDepthCompleted` to track the maximum depth achieved per command
   
2. **Algorithm Flow**:
   - Starts at depth 1, evaluates ALL commands regardless of time budget
   - For each subsequent depth, evaluates commands ordered by previous depth scores
   - Continues until time expires, all commands at max depth are evaluated, or 50% budget is used
   - SET_UP phase limits max depth to 2
   - **Important**: Depth 1 always completes even if time budget is exhausted

3. **Memory Efficiency**:
   - Reuses data structures across searches to minimize allocations
   - Dynamically resizes score vectors as needed

4. **Command Change Logging**:
   - Tracks the best command at each depth
   - Logs when a new depth results in a different best command selection
   - Provides detailed debug output showing old and new commands with scores

## Overview

This document tracks the implementation of true iterative deepening for the Shardok AI, upgrading from a hard-coded 2-depth limit to dynamic depth exploration based on available time budget. The implementation is complete and functional, with planned cleanup tasks for future improvement.

## Current Implementation

The current implementation:
- Evaluates ALL commands at depth 1
- Sorts commands by depth-1 scores
- Evaluates commands at depth 2 in sorted order until time expires
- Never proceeds beyond depth 2

## Proposed Implementation

### Core Algorithm

The new algorithm will:
1. **Depth 1**: Evaluate ALL commands (unchanged)
2. **Depth 2+**: For each depth, attempt to evaluate all commands ordered by their scores from the previous depth
3. **Completion check**: Only proceed to depth N+1 if all commands at depth N were evaluated
4. **50% budget check**: Only proceed to depth N+1 if less than 50% of total time budget has been used
5. **SET_UP phase limit**: Limit maximum depth to 2 during the SET_UP game phase

### Main Loop Pseudocode

```cpp
int currentDepth = 1;
bool isSetupPhase = (guessedState->status()->state() == GameStatus_::State_SET_UP);
int maxDepth = isSetupPhase ? 2 : std::numeric_limits<int>::max();

// Track initial budget for percentage calculations
const auto initialBudget = timeBudget.remainingBudget;
auto startTime = std::chrono::steady_clock::now();

// Track scores at each depth for each command
std::vector<std::vector<ScoreValue>> scoresByDepth(commands.size());
std::vector<int> highestDepthCompleted(commands.size(), 0);

while (!IsTimeExpired(timeBudget) && currentDepth <= maxDepth) {
    auto depthStartTime = std::chrono::steady_clock::now();
    
    // Get command indices sorted by best score from previous depth
    std::vector<size_t> sortedIndices = GetCommandsSortedByPreviousDepth(
        currentDepth, scoresByDepth, highestDepthCompleted);
    
    int evaluatedCount = 0;
    bool allEvaluated = true;
    
    // Try to evaluate all commands at this depth
    for (size_t cmdIndex : sortedIndices) {
        if (IsTimeExpired(timeBudget)) {
            allEvaluated = false;
            break;
        }
        
        auto result = SearchCommandAtDepthWithEngine(
            guessedEngine, settingsGetter, maxRepeatCount,
            commands, cmdIndex, currentDepth, currentUtility, timeBudget);
        
        scoresByDepth[cmdIndex][currentDepth] = result.bestScore;
        highestDepthCompleted[cmdIndex] = currentDepth;
        evaluatedCount++;
    }
    
    printf("ID AI: Depth %d - evaluated %d/%zu commands\n", 
           currentDepth, evaluatedCount, commands.size());
    
    // Only proceed to next depth if we completed all commands at current depth
    if (!allEvaluated) {
        printf("ID AI: Stopping - time expired during depth %d\n", currentDepth);
        break;
    }
    
    // Check if we've used more than 50% of total budget
    auto totalElapsed = std::chrono::steady_clock::now() - startTime;
    auto totalElapsedMs = std::chrono::duration_cast<std::chrono::milliseconds>(totalElapsed);
    double budgetUsedPercent = (double)totalElapsedMs.count() / initialBudget.count();
    
    if (budgetUsedPercent > 0.5) {
        printf("ID AI: Stopping after depth %d - used %.1f%% of time budget\n", 
               currentDepth, budgetUsedPercent * 100);
        break;
    }
    
    currentDepth++;
}

// Select best result from highest depth achieved for each command
SearchResult finalResult = SelectBestResult(scoresByDepth, highestDepthCompleted);
```

### Key Helper Functions

#### GetCommandsSortedByPreviousDepth

Sort commands by their scores at the previous depth:

```cpp
std::vector<size_t> GetCommandsSortedByPreviousDepth(
    int currentDepth,
    const std::vector<std::vector<ScoreValue>>& scoresByDepth,
    const std::vector<int>& highestDepthCompleted) {
    
    std::vector<size_t> indices(scoresByDepth.size());
    std::iota(indices.begin(), indices.end(), 0);
    
    if (currentDepth == 1) {
        // For depth 1, return natural order
        return indices;
    }
    
    // Sort by score at previous depth
    int prevDepth = currentDepth - 1;
    std::sort(indices.begin(), indices.end(), 
        [&](size_t a, size_t b) {
            // Only consider commands that were evaluated at previous depth
            if (highestDepthCompleted[a] >= prevDepth && 
                highestDepthCompleted[b] >= prevDepth) {
                return scoresByDepth[a][prevDepth] > scoresByDepth[b][prevDepth];
            }
            // Commands not evaluated at prev depth go to the end
            return highestDepthCompleted[a] >= prevDepth;
        });
    
    return indices;
}
```

#### SelectBestResult

Choose the best command considering the depth achieved:

```cpp
SearchResult SelectBestResult(
    const std::vector<std::vector<ScoreValue>>& scoresByDepth,
    const std::vector<int>& highestDepthCompleted) {
    
    SearchResult result;
    result.bestScore = -std::numeric_limits<ScoreValue>::infinity();
    
    // Find the command with best score at its highest evaluated depth
    for (size_t i = 0; i < scoresByDepth.size(); ++i) {
        if (highestDepthCompleted[i] > 0) {
            ScoreValue score = scoresByDepth[i][highestDepthCompleted[i]];
            if (score > result.bestScore) {
                result.bestScore = score;
                result.bestCommandIndex = i;
                result.depthAchieved = highestDepthCompleted[i];
            }
        }
    }
    
    return result;
}
```

### Data Structure Updates

Replace the current separate tracking with unified structures:

```cpp
class IterativeDeepeningAI {
    // ... existing members ...
    
    // New reusable storage to reduce allocations
    mutable std::vector<std::vector<ScoreValue>> scoresByDepth;
    mutable std::vector<int> highestDepthCompleted;
    mutable std::vector<size_t> reusableSortedIndices;
};
```

## Rationale for 50% Budget Check

The 50% time budget check is crucial because of the exponential nature of game tree search:
- If depth N takes time T, depth N+1 typically takes B×T (where B is the branching factor)
- If we've used >50% of budget at depth N, we likely can't complete even one command at depth N+1
- Better to have complete results at depth N than incomplete results at depth N+1

Example with branching factor ~40:
- Depth 1: 100ms (10% of 1000ms budget)
- Depth 2: 400ms (total 50%)
- Depth 3: Would take ~1600ms (total 210%) - don't attempt

## Benefits

1. **Adaptability**: Automatically adjusts search depth based on available time
2. **Completeness**: Ensures all commands are evaluated at each attempted depth
3. **Optimality**: Commands are always evaluated in order of promise from previous depth
4. **Scalability**: Can search arbitrarily deep when time permits
5. **Robustness**: 50% check prevents wasting time on incomplete deep searches

## Implementation Notes

- Maintain backward compatibility with existing time budget calculations
- Add comprehensive logging to track depth progression
- Consider memory allocation optimizations for deep searches
- Test thoroughly with various time budgets and game states

## Implementation Results

The true iterative deepening implementation has been successfully completed. The key changes include:

1. **Generalized Depth Loop**: The algorithm now supports arbitrary depths instead of being limited to depth 2
2. **50% Budget Check**: Prevents starting a new depth if more than half the time budget is consumed
3. **SET_UP Phase Handling**: Limits depth to 2 during game setup to avoid overthinking unit placement
4. **Efficient Sorting**: Commands are evaluated at each depth in order of their scores from the previous depth
5. **Memory Optimization**: Reuses data structures to minimize allocations during search

The implementation maintains backward compatibility while enabling deeper searches when time permits, leading to potentially better AI decisions in complex game situations.

### Critical Fixes Applied

#### 1. Depth 1 Always Completes
We ensured that depth 1 ALWAYS completes regardless of time budget by:
- Modifying the outer loop condition: `(currentDepth == 1 || !IsTimeExpired(timeBudget))`
- Modifying the inner loop condition: `if (currentDepth > 1 && IsTimeExpired(timeBudget))`

This guarantees the AI always has at least a depth-1 evaluation for every command, preventing the AI from making no decision due to time constraints.

#### 2. END_TURN Detection
Added logic to prevent excessive depth exploration when the game tree terminates:
- **Root-level check**: If all commands at the current game state are END_TURN_COMMAND, stop after depth 1
- **Lookahead termination check**: If scores don't change between depth N-1 and depth N for all commands, it indicates the lookahead hit END_TURN_COMMAND and stopped recursing

This prevents the AI from exploring to extreme depths (1000+) when there are no meaningful decisions to make, which can happen when there are very few commands available and the game tree quickly reaches states where only END_TURN_COMMAND is available.

#### 3. Command Change Logging
Added comprehensive logging to track when deeper search changes the AI's decision:
- After each depth, identifies the best command based on current evaluations
- Compares with the best command from the previous depth
- Logs detailed information when the best command changes, including:
  - Both commands' indices and scores
  - Full command debug strings for analysis
  
This helps understand when and why deeper search is beneficial, providing insights into the AI's decision-making process.