# Machine Learning Training Data Collection (Phase 1)

## Overview

This directory contains the initial implementation of Phase 1 from the ML Value Function plan: data collection infrastructure for training a neural network to predict win probability from game states.

## Components

### StateRecorder (`StateRecorder.hpp/cpp`)

Records game states during self-play for ML training. Key features:
- Records `GameStateView` (player's partial information view)
- Stores player ID and round number for each state
- Labels all states with final game outcome (win/loss/draw)
- Binary file format for efficient storage

**Status**: ✅ **Fully implemented and tested**

### Self-Play Generator (`self_play_generator_main.cpp`)

Command-line tool to generate training data by running AI vs AI games.

**Current Status**: ⚠️ **Partially implemented**

The tool currently:
- ✅ Runs complete AI vs AI battles using the existing `AiBattleSimulator`
- ✅ Collects game outcome statistics (attacker wins, defender wins, draws)
- ✅ Supports custom battle configurations
- ✅ Command-line interface with configurable options

What's missing:
- ❌ Per-action state recording (requires refactoring `AiBattleSimulator`)
- ❌ Writing training data files

### Next Steps for Full Phase 1 Implementation

To complete Phase 1 data collection, we need to:

1. **Refactor AiBattleSimulator** to support extensibility:
   ```cpp
   // Add callback parameter to RunBattlePhase
   using PostCommandCallback = std::function<void(
       const ShardokEngine& engine,
       PlayerId currentPlayer,
       const CommandProto& command,
       bool isEndTurn)>;

   BattleResult RunBattlePhase(
       ShardokEngine& engine,
       ShardokAIClient& attackerAI,
       ShardokAIClient& defenderAI,
       const PostCommandCallback& callback = nullptr);
   ```

2. **Integrate StateRecorder with game loop**:
   ```cpp
   StateRecorder recorder;

   auto recordState = [&](const ShardokEngine& engine, PlayerId player,
                          const CommandProto& cmd, bool isEndTurn) {
       if (!isEndTurn) {
           auto view = engine.GetGameStateView(player);
           recorder.RecordStateView(view, player);
       }
   };

   simulator.RunBattle(recordState);
   recorder.SetGameOutcome(result.winner);
   recorder.WriteToFile("game_0001.bin");
   ```

3. **Test full pipeline**: Generate 100 games and verify data files

## Usage

### Build

```bash
bazel build //src/main/cpp/net/eagle0/shardok/ai/ml:self_play_generator
```

### Run (Current Implementation)

```bash
# Generate 100 games with default settings
./bazel-bin/src/main/cpp/net/eagle0/shardok/ai/ml/self_play_generator --num_games=100

# Generate games with custom config
./bazel-bin/src/main/cpp/net/eagle0/shardok/ai/ml/self_play_generator \
    --config=my_battle.json \
    --num_games=1000 \
    --output_dir=/data/training \
    --verbose
```

### Expected Output (After Full Implementation)

```
/data/training/
├── game_000000.bin  # Binary file with StateRecords
├── game_000001.bin
├── game_000002.bin
...
└── game_099999.bin
```

Each `.bin` file contains:
- Multiple `StateRecord` entries (one per non-END_TURN action)
- Each record has: serialized GameStateView, player ID, round number, win/loss label

## File Format Specification

Binary format (little-endian):
```
[uint32_t num_records]
For each record:
  [uint32_t view_size]
  [uint8_t[] serialized_game_state_view]  # Protobuf bytes
  [int8_t current_player]                  # 0 or 1
  [int32_t round_number]
  [bool current_player_won]                # Label
```

## Integration Points

- **AiBattleSimulator**: Existing infrastructure for running complete games
- **ShardokEngine**: Provides `GetGameStateView(PlayerId)` for player perspectives
- **GameStateView**: Protobuf representing player's partial information
- **StateRecorder**: Records and serializes training examples

## Testing

```bash
# Build and run tests
bazel build //src/main/cpp/net/eagle0/shardok/ai/ml:state_recorder
bazel test //src/main/cpp/net/eagle0/shardok/ai/ml:...  # TODO: Add unit tests

# Quick integration test
./bazel-bin/src/main/cpp/net/eagle0/shardok/ai/ml/self_play_generator \
    --num_games=2 \
    --verbose
```

## Design Decisions

1. **Record player VIEW, not full state**: Ensures model trains on partial information that matches inference conditions

2. **Record AFTER each action**: Captures resulting positions the AI wants to evaluate (except END_TURN which is a transition)

3. **Current player perspective**: Makes model generalizable to both attacker and defender

4. **Binary format**: Compact storage for large datasets (expected 50K+ games)

## Performance Considerations

Expected data volumes:
- 50,000 games × ~50 states/game = 2.5M training examples
- Each state ~10-50KB serialized
- Total: ~50-250GB uncompressed
- Recommendation: Use gzip compression for storage

## References

- Full ML plan: `../ML_VALUE_FUNCTION_PLAN.md`
- AiBattleSimulator: `../../ai_battle_simulator/`
- GameStateView proto: `src/main/protobuf/net/eagle0/shardok/api/game_state_view.proto`
