Phase 4.5i: delete v15-v20 payload-parsing backfills (#6791)

After #6787 (upload-on-load) deployed and the admin-load pass pushed
every persisted game.db to v ≥ 21 in S3, the v15-v20 migration
backfills that re-derived their new columns/tables from
`action_results.payload` are unreachable on any healthy game. This PR
deletes that code so the proto dependency surface shrinks and the
"only used by backfill" file headers stop misleading future readers.

Removed:
- `backfill(Connection)` (and supporting `bindOpt*` helpers / `select`
  loops) from each of the 6 writers that populated the 4.5h.1–4
  scalars/vectors:
    - ActionResultEntityRefColumnsWriter (v15)
    - ActionResultEnumColumnsWriter (v16)
    - ActionResultNumericColumnsWriter (v17)
    - ActionResultDateBattleColumnsWriter (v18)
    - ActionResultStructuredBlobColumnsWriter (v19)
    - ActionResultVectorTablesWriter (v20)
- `applyV15`–`applyV20` collapse to a single line each — just
  `execEach(connection, ...schemaStatements)`. Doc note added that
  schema v < 15 is no longer supported (if `applyV15` ever runs on a
  non-empty `action_results` table, the new columns are left NULL and
  state diverges silently; the migration runner's `schema_version`
  gate is the only protection).
- 6 corresponding `"action_results vN backfill"` tests in
  `SqliteHistoryTest` (one per migration). The matching `vN live
  write` tests stay — those still exercise the live-write column
  binding.

BUILD dep cleanup falls out: each of those writers no longer needs
`internal:action_result_scala_proto` or
`proto_converters:action_result_proto_converter` (the
payload-parse + full-converter round-trip is gone). The remaining
narrow deps (`tutorial_phase_scala_proto`, `game_type_scala_proto`,
etc.) cover the proto-enum `.value` reads that `bindColumns` still
needs.

Verification: `sqlite_history_test` 90/90 green (96 - 6 backfill
tests deleted); full Scala suite 226/226 green.

The per-entity table backfills (v3-v14) still parse `payload` and are
out of scope here — they kept ChangedHero/Faction/Battalion games
alive across the .5c cutover, and removing them needs the same
"prove all games are at v ≥ 21 in S3" gate. Future cleanup.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-29 07:51:34 -07:00
committed by GitHub
co-authored by Claude Opus 4.7
parent d174c06ddb
commit 899162bbc5
9 changed files with 66 additions and 497 deletions
@@ -1,10 +1,8 @@
package net.eagle0.eagle.service
import java.sql.{Connection, PreparedStatement, Types}
import java.sql.{PreparedStatement, Types}
import net.eagle0.eagle.internal.action_result.ActionResult as ActionResultProto
import net.eagle0.eagle.model.action_result.ActionResultT
import net.eagle0.eagle.model.proto_converters.ActionResultProtoConverter
import net.eagle0.eagle.model.state.date.Date
/**
@@ -25,13 +23,10 @@ import net.eagle0.eagle.model.state.date.Date
* uniform across the table. A composite `(new_date_year, new_date_month)` index matches the v1 composite on the
* resulting-state date columns; `resolved_battle` is independently indexed.
*
* Live-write path is the existing `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these three columns at the appropriate positional offset. Backfill calls `bindColumns` too
* against the per-row UPDATE — single source of truth, no risk of drift between live + backfill (same pattern as .2a /
* .2b).
* Live-write path is the `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these three columns at the appropriate positional offset.
*
* Read path is **not** swapped — `replayFrom` still goes through the full converter on the `payload` blob; the columns
* are "shadow" until the final 4.5h sub-PR drops `payload` and switches the read path.
* The v18 backfill (and its `payload`-parsing) was deleted in 4.5i. Schema v < 18 is no longer supported.
*/
private[service] object ActionResultDateBattleColumnsWriter {
@@ -47,42 +42,9 @@ private[service] object ActionResultDateBattleColumnsWriter {
"CREATE INDEX idx_action_results_resolved_battle ON action_results(resolved_battle)"
)
/**
* Populate the three new columns for every existing `action_results` row by parsing its `payload` through the full
* `ActionResultProtoConverter.fromProto`, then delegating to `bindColumns`. Runs inside the migration transaction; a
* no-op on a fresh game. Payloads are read fully before any UPDATE so the SELECT `ResultSet` is closed first (xerial
* sqlite-jdbc dislikes interleaving writes with an open ResultSet).
*/
def backfill(connection: Connection): Unit = {
val rows = {
val select = connection.prepareStatement("SELECT action_seq, payload FROM action_results ORDER BY action_seq")
try {
val rs = select.executeQuery()
val builder = Vector.newBuilder[(Int, Array[Byte])]
while rs.next() do builder += ((rs.getInt("action_seq"), rs.getBytes("payload")))
builder.result()
} finally select.close()
}
if rows.isEmpty then return
val update = connection.prepareStatement(
"UPDATE action_results SET new_date_year = ?, new_date_month = ?, resolved_battle = ? WHERE action_seq = ?"
)
try {
rows.foreach {
case (seq, payload) =>
val ar = ActionResultProtoConverter.fromProto(ActionResultProto.parseFrom(payload))
bindColumns(update, startPosition = 1, ar)
update.setInt(4, seq)
update.addBatch()
}
update.executeBatch(): Unit
} finally update.close()
}
/**
* Bind the three columns to `ps` at positions `[startPosition (year), startPosition+1 (month), startPosition+2
* (resolved_battle)]`. Called by both the live-write `INSERT` in `SqliteHistory.withNewResults` and the migration's
* `UPDATE` in `backfill` — single source of truth.
* (resolved_battle)]`. Called by the live-write `INSERT` in `SqliteHistory.withNewResults`.
*/
def bindColumns(ps: PreparedStatement, startPosition: Int, ar: ActionResultT): Unit = {
bindOptDate(ps, yearIdx = startPosition, monthIdx = startPosition + 1, ar.newDate)
@@ -1,41 +1,28 @@
package net.eagle0.eagle.service
import java.sql.{Connection, PreparedStatement, Types}
import net.eagle0.eagle.internal.action_result.ActionResult as ActionResultProto
/**
* Schema + backfill for the first batch of top-level `ActionResultC` columns promoted onto the existing
* `action_results` table — Phase 4.5h.1, the first 4.5h chunk. Adds the four entity-ref `Option` fields as nullable
* INTEGER columns, indexed for the obvious query shapes ("show all actions by hero X"):
* Schema for the first batch of top-level `ActionResultC` columns promoted onto the existing `action_results` table —
* Phase 4.5h.1. Adds the four entity-ref `Option` fields as nullable INTEGER columns, indexed for the obvious query
* shapes ("show all actions by hero X"):
*
* - `acting_hero_id` (domain `actingHeroId` / proto `leader`)
* - `acting_faction_id` (domain `actingFactionId` / proto `player`)
* - `province_id` (domain `provinceId` / proto `province`)
* - `province_id_acted` (domain `provinceIdActed` / proto `province_acted`)
* - `acting_hero_id` (domain `actingHeroId`)
* - `acting_faction_id` (domain `actingFactionId`)
* - `province_id` (domain `provinceId`)
* - `province_id_acted` (domain `provinceIdActed`)
*
* Distinct from the per-entity table writers in this package: there's no new table here, and there's no `write(...)`
* helper either. The live-write path is the existing `INSERT INTO action_results (...)` statement in
* `SqliteHistory.withNewResults`, which is extended in the green commit to bind these four extra columns positionally
* (RED stub: the INSERT is unchanged — the new columns default to NULL on insert, which is why the live-write tests
* fail with `0 != expected`).
* The live-write path is the `INSERT INTO action_results (...)` statement in `SqliteHistory.withNewResults`, which
* binds these four columns positionally.
*
* Backfill reads each existing `action_results.payload`, parses it as `ActionResult` proto, and `UPDATE`s the row with
* the four optional values (or leaves them NULL when the proto field is unset). The proto field names don't match the
* domain (`leader`/`player`/`province`/`province_acted` vs
* `actingHeroId`/`actingFactionId`/`provinceId`/`provinceIdActed`) — the column names follow the domain, since that's
* what the read path will ultimately expose.
*
* The read path is not yet swapped to use these columns — `replayFrom` still parses the `payload` blob and goes through
* `ActionResultProtoConverter.fromProto`. The columns are "shadow" until the final 4.5h sub-PR drops `payload` and
* switches the read path.
* The v15 backfill (and its `payload`-parsing) was deleted in 4.5i. Schema v < 15 is no longer supported — applyV15 now
* only runs `schemaStatements`, so any pre-existing rows would land with NULL columns. Games at v < 21 should be
* brought forward via the upload-on-load path (#6787) before this code reaches them.
*/
private[service] object ActionResultEntityRefColumnsWriter {
/**
* v15: four `ALTER TABLE ... ADD COLUMN` + four `CREATE INDEX` statements. SQLite has no `IF NOT EXISTS` for
* `ALTER ... ADD COLUMN`; we rely on the migration runner's `schema_version` tracking to ensure this runs at most
* once per game.db, matching the existing migrations' assumption.
* once per game.db.
*/
val schemaStatements: Vector[String] = Vector(
"ALTER TABLE action_results ADD COLUMN acting_hero_id INTEGER",
@@ -47,46 +34,4 @@ private[service] object ActionResultEntityRefColumnsWriter {
"CREATE INDEX idx_action_results_province_id ON action_results(province_id)",
"CREATE INDEX idx_action_results_province_id_acted ON action_results(province_id_acted)"
)
/**
* Populate the four new columns for every existing `action_results` row by parsing its `payload`. Runs inside the
* migration transaction; a no-op on a fresh game. Payloads are read fully before any UPDATE so the SELECT `ResultSet`
* is closed first (xerial sqlite-jdbc dislikes interleaving writes with an open ResultSet). The UPDATE is issued
* unconditionally per row (binding NULL when the proto field is unset) — cheap on the migration path and keeps the
* logic uniform; rows whose four proto fields are all unset end up with the same `NULL × 4` they already had after
* `ALTER TABLE ADD COLUMN`.
*/
def backfill(connection: Connection): Unit = {
val rows = {
val select = connection.prepareStatement("SELECT action_seq, payload FROM action_results ORDER BY action_seq")
try {
val rs = select.executeQuery()
val builder = Vector.newBuilder[(Int, Array[Byte])]
while rs.next() do builder += ((rs.getInt("action_seq"), rs.getBytes("payload")))
builder.result()
} finally select.close()
}
if rows.isEmpty then return
val update = connection.prepareStatement(
"UPDATE action_results SET acting_hero_id = ?, acting_faction_id = ?, province_id = ?, " +
"province_id_acted = ? WHERE action_seq = ?"
)
try {
rows.foreach {
case (seq, payload) =>
val proto = ActionResultProto.parseFrom(payload)
bindOptInt(update, 1, proto.leader)
bindOptInt(update, 2, proto.player)
bindOptInt(update, 3, proto.province)
bindOptInt(update, 4, proto.provinceActed)
update.setInt(5, seq)
update.addBatch()
}
update.executeBatch(): Unit
} finally update.close()
}
/** Bind `Option[Int]` to a prepared-statement parameter: the value if `Some`, SQL NULL if `None`. */
private def bindOptInt(ps: PreparedStatement, idx: Int, opt: Option[Int]): Unit =
opt.fold(ps.setNull(idx, Types.INTEGER))(ps.setInt(idx, _))
}
@@ -1,16 +1,10 @@
package net.eagle0.eagle.service
import java.sql.{Connection, PreparedStatement, Types}
import java.sql.{PreparedStatement, Types}
import net.eagle0.eagle.common.tutorial_phase.TutorialPhase as TutorialPhaseProto
import net.eagle0.eagle.internal.action_result.ActionResult as ActionResultProto
import net.eagle0.eagle.model.action_result.ActionResultT
import net.eagle0.eagle.model.proto_converters.{
ActionResultProtoConverter,
CommandTypeConverter,
GameTypeConverter,
RoundPhaseConverter
}
import net.eagle0.eagle.model.proto_converters.{CommandTypeConverter, GameTypeConverter, RoundPhaseConverter}
/**
* Schema + backfill for the three enum-typed top-level `ActionResultC` fields promoted onto `action_results` — Phase
@@ -33,18 +27,10 @@ import net.eagle0.eagle.model.proto_converters.{
* `RoundPhase.value` happens to match the proto enum number, but going through the converter for all three keeps the
* pattern uniform (and forces the failure into one place — the converter — if the mappings ever drift).
*
* Backfill goes through the full `ActionResultProtoConverter.fromProto` rather than reading proto fields directly,
* because the domain `lastCommandTypeForActingProvince` has a back-compat fallback path to a deprecated
* `SelectedCommand` field. The converter is the single source of truth for "domain Option-value from this payload";
* reusing it eliminates drift risk for the cost of a one-time per-row conversion on the migration path.
* Live-write path is the `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these four columns at the appropriate positional offset.
*
* Live-write path is the existing `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these four columns at the appropriate positional offset. Backfill calls `bindColumns` too,
* against the per-row UPDATE — single source of truth for "what gets bound and from what", with no risk of drift
* between live + backfill.
*
* Read path is **not** swapped — `replayFrom` still goes through the full converter on the `payload` blob; the columns
* are "shadow" until the final 4.5h sub-PR drops `payload` and switches the read path.
* The v16 backfill (and its `payload`-parsing) was deleted in 4.5i. Schema v < 16 is no longer supported.
*/
private[service] object ActionResultEnumColumnsWriter {
@@ -63,43 +49,10 @@ private[service] object ActionResultEnumColumnsWriter {
"CREATE INDEX idx_action_results_new_tutorial_phase ON action_results(new_tutorial_phase)"
)
/**
* Populate the four new enum columns for every existing `action_results` row by parsing its `payload` through the
* full `ActionResultProtoConverter.fromProto`, then delegating to `bindColumns`. Runs inside the migration
* transaction; a no-op on a fresh game. Payloads are read fully before any UPDATE so the SELECT `ResultSet` is closed
* first (xerial sqlite-jdbc dislikes interleaving writes with an open ResultSet).
*/
def backfill(connection: Connection): Unit = {
val rows = {
val select = connection.prepareStatement("SELECT action_seq, payload FROM action_results ORDER BY action_seq")
try {
val rs = select.executeQuery()
val builder = Vector.newBuilder[(Int, Array[Byte])]
while rs.next() do builder += ((rs.getInt("action_seq"), rs.getBytes("payload")))
builder.result()
} finally select.close()
}
if rows.isEmpty then return
val update = connection.prepareStatement(
"UPDATE action_results SET new_round_phase = ?, last_command_type = ?, new_game_type = ?, " +
"new_tutorial_phase = ? WHERE action_seq = ?"
)
try {
rows.foreach {
case (seq, payload) =>
val ar = ActionResultProtoConverter.fromProto(ActionResultProto.parseFrom(payload))
bindColumns(update, startPosition = 1, ar)
update.setInt(5, seq)
update.addBatch()
}
update.executeBatch(): Unit
} finally update.close()
}
/**
* Bind the four enum columns to `ps` at positions `[startPosition, startPosition+1, startPosition+2,
* startPosition+3]` — `new_round_phase`, `last_command_type`, `new_game_type`, `new_tutorial_phase`. Called by both
* the live-write `INSERT` in `SqliteHistory.withNewResults` and the migration's `UPDATE` in `backfill`.
* startPosition+3]` — `new_round_phase`, `last_command_type`, `new_game_type`, `new_tutorial_phase`. Called by the
* live-write `INSERT` in `SqliteHistory.withNewResults`.
*
* Each integer stored is the **proto enum number** (`Converter.toProto(domain).value`). For `newGameType`, the
* `GameType.Tutorial(phase)` case populates both `new_game_type` (the kind) and `new_tutorial_phase` (the wrapped
@@ -1,10 +1,8 @@
package net.eagle0.eagle.service
import java.sql.{Connection, PreparedStatement, Types}
import java.sql.{PreparedStatement, Types}
import net.eagle0.eagle.internal.action_result.ActionResult as ActionResultProto
import net.eagle0.eagle.model.action_result.ActionResultT
import net.eagle0.eagle.model.proto_converters.ActionResultProtoConverter
/**
* Schema + backfill for the four numeric-primitive top-level `ActionResultC` fields promoted onto `action_results` —
@@ -23,12 +21,10 @@ import net.eagle0.eagle.model.proto_converters.ActionResultProtoConverter
* (always set), while this one is the action's *change-of-round* event (sparse, only set when an action advances the
* round). Both are indexed.
*
* Live-write path is the existing `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these four columns at the appropriate positional offset. Backfill calls `bindColumns` too
* against the per-row UPDATE — single source of truth, no risk of drift between live + backfill.
* Live-write path is the `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these four columns at the appropriate positional offset.
*
* Read path is **not** swapped — `replayFrom` still goes through the full converter on the `payload` blob; the columns
* are "shadow" until the final 4.5h sub-PR drops `payload` and switches the read path.
* The v17 backfill (and its `payload`-parsing) was deleted in 4.5i. Schema v < 17 is no longer supported.
*/
private[service] object ActionResultNumericColumnsWriter {
@@ -47,43 +43,10 @@ private[service] object ActionResultNumericColumnsWriter {
"CREATE INDEX idx_action_results_new_random_seed ON action_results(new_random_seed)"
)
/**
* Populate the four new numeric columns for every existing `action_results` row by parsing its `payload` through the
* full `ActionResultProtoConverter.fromProto`, then delegating to `bindColumns`. Runs inside the migration
* transaction; a no-op on a fresh game. Payloads are read fully before any UPDATE so the SELECT `ResultSet` is closed
* first (xerial sqlite-jdbc dislikes interleaving writes with an open ResultSet).
*/
def backfill(connection: Connection): Unit = {
val rows = {
val select = connection.prepareStatement("SELECT action_seq, payload FROM action_results ORDER BY action_seq")
try {
val rs = select.executeQuery()
val builder = Vector.newBuilder[(Int, Array[Byte])]
while rs.next() do builder += ((rs.getInt("action_seq"), rs.getBytes("payload")))
builder.result()
} finally select.close()
}
if rows.isEmpty then return
val update = connection.prepareStatement(
"UPDATE action_results SET new_round_id = ?, new_victor_faction_id = ?, game_ended = ?, " +
"new_random_seed = ? WHERE action_seq = ?"
)
try {
rows.foreach {
case (seq, payload) =>
val ar = ActionResultProtoConverter.fromProto(ActionResultProto.parseFrom(payload))
bindColumns(update, startPosition = 1, ar)
update.setInt(5, seq)
update.addBatch()
}
update.executeBatch(): Unit
} finally update.close()
}
/**
* Bind the four numeric columns to `ps` at positions `[startPosition .. startPosition+3]` — `new_round_id`,
* `new_victor_faction_id`, `game_ended`, `new_random_seed`. Called by both the live-write `INSERT` in
* `SqliteHistory.withNewResults` and the migration's `UPDATE` in `backfill` — single source of truth.
* `new_victor_faction_id`, `game_ended`, `new_random_seed`. Called by the live-write `INSERT` in
* `SqliteHistory.withNewResults`.
*
* `gameEnded` maps `Some(true)` → 1 and `Some(false)` → 0; `None` → NULL. `newRandomSeed` uses `setLong` (SQLite
* `INTEGER` holds int64); the other three use `setInt`.
@@ -1,12 +1,11 @@
package net.eagle0.eagle.service
import java.sql.{Connection, PreparedStatement, Types}
import java.sql.{PreparedStatement, Types}
import net.eagle0.eagle.internal.action_result.ActionResult as ActionResultProto
import net.eagle0.eagle.model.action_result.ActionResultT
import net.eagle0.eagle.model.proto_converters.{ActionResultProtoConverter, EagleMapInfoConverter}
import net.eagle0.eagle.model.proto_converters.chronicle_entry.ChronicleEntryConverter
import net.eagle0.eagle.model.proto_converters.shardok_battle.ShardokBattleConverter
import net.eagle0.eagle.model.proto_converters.EagleMapInfoConverter
/**
* Schema + backfill for the three structured-Option top-level `ActionResultC` fields promoted onto `action_results` —
@@ -22,16 +21,10 @@ import net.eagle0.eagle.model.proto_converters.shardok_battle.ShardokBattleConve
* actions started a battle / advanced chronicle / loaded a map" come from per-entity tables and scalar columns
* elsewhere on `action_results`.
*
* Live-write path is the existing `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these three columns at the appropriate positional offset. Backfill calls `bindColumns` too
* against the per-row UPDATE — single source of truth, no risk of drift between live + backfill (same pattern as
* 4.5h.2a / .2b / .2c).
* Live-write path is the `INSERT INTO action_results (...)` in `SqliteHistory.withNewResults`, which calls
* `bindColumns` to bind these three columns at the appropriate positional offset.
*
* Read path is **not** swapped — `replayFrom` still goes through the full converter on the `payload` blob; the columns
* are "shadow" until the final 4.5h sub-PR drops `payload` and switches the read path.
*
* After this lands, the remaining 4.5h work is the primitive-ID vectors + the small `newBattalionTypes` vector, and
* then the `payload` drop + read-path swap.
* The v19 backfill (and its `payload`-parsing) was deleted in 4.5i. Schema v < 19 is no longer supported.
*/
private[service] object ActionResultStructuredBlobColumnsWriter {
@@ -44,43 +37,10 @@ private[service] object ActionResultStructuredBlobColumnsWriter {
"ALTER TABLE action_results ADD COLUMN new_eagle_map_proto BLOB"
)
/**
* Populate the three new BLOB columns for every existing `action_results` row by parsing its `payload` through the
* full `ActionResultProtoConverter.fromProto`, then delegating to `bindColumns`. Runs inside the migration
* transaction; a no-op on a fresh game. Payloads are read fully before any UPDATE so the SELECT `ResultSet` is closed
* first (xerial sqlite-jdbc dislikes interleaving writes with an open ResultSet).
*/
def backfill(connection: Connection): Unit = {
val rows = {
val select = connection.prepareStatement("SELECT action_seq, payload FROM action_results ORDER BY action_seq")
try {
val rs = select.executeQuery()
val builder = Vector.newBuilder[(Int, Array[Byte])]
while rs.next() do builder += ((rs.getInt("action_seq"), rs.getBytes("payload")))
builder.result()
} finally select.close()
}
if rows.isEmpty then return
val update = connection.prepareStatement(
"UPDATE action_results SET new_battle_proto = ?, new_chronicle_entry_proto = ?, new_eagle_map_proto = ? " +
"WHERE action_seq = ?"
)
try {
rows.foreach {
case (seq, payload) =>
val ar = ActionResultProtoConverter.fromProto(ActionResultProto.parseFrom(payload))
bindColumns(update, startPosition = 1, ar)
update.setInt(4, seq)
update.addBatch()
}
update.executeBatch(): Unit
} finally update.close()
}
/**
* Bind the three BLOB columns to `ps` at positions `[startPosition .. startPosition+2]` — `new_battle_proto`,
* `new_chronicle_entry_proto`, `new_eagle_map_proto`. Called by both the live-write `INSERT` in
* `SqliteHistory.withNewResults` and the migration's `UPDATE` in `backfill` — single source of truth.
* `new_chronicle_entry_proto`, `new_eagle_map_proto`. Called by the live-write `INSERT` in
* `SqliteHistory.withNewResults`.
*
* Each BLOB is the per-entity proto's `toByteArray` (or SQL NULL when the domain Option is `None`); on the read path
* the bytes round-trip through the corresponding `<entity>Proto.parseFrom` + `Converter.fromProto`.
@@ -2,9 +2,8 @@ package net.eagle0.eagle.service
import java.sql.Connection
import net.eagle0.eagle.internal.action_result.ActionResult as ActionResultProto
import net.eagle0.eagle.model.action_result.ActionResultT
import net.eagle0.eagle.model.proto_converters.{ActionResultProtoConverter, BattalionTypeConverter}
import net.eagle0.eagle.model.proto_converters.BattalionTypeConverter
/**
* Schema + backfill for the five top-level `ActionResultC` vector fields not yet decomposed — Phase 4.5h.4. Unlike the
@@ -24,14 +23,10 @@ import net.eagle0.eagle.model.proto_converters.{ActionResultProtoConverter, Batt
* quest-hybrid shape (type_id column + whole `battalion_type_proto BLOB`) rather than fully columnizing — consistent
* with how flat-but-deep entities are handled elsewhere in this package.
*
* Live-write path: `SqliteHistory.withNewResults` calls `write(connection, seq, ar)` once per action; the facade
* iterates all five vectors internally. Backfill calls `write` too against the per-row parsed payload — single source
* of truth for "what gets inserted from a domain ActionResultT", no risk of drift.
* Live-write path: `SqliteHistory.withNewResults` calls `write(connection, seq, ar)` once per action (via the
* `PerEntityWriteDispatcher`); the facade iterates all five vectors internally.
*
* Read path is **not** swapped — `replayFrom` still goes through the full converter on the `payload` blob; the rows are
* "shadow" until the final 4.5h sub-PR drops `payload` and switches the read path.
*
* After this lands, the only remaining 4.5h work is the `payload` drop + read-path swap.
* The v20 backfill (and its `payload`-parsing) was deleted in 4.5i. Schema v < 20 is no longer supported.
*/
private[service] object ActionResultVectorTablesWriter {
@@ -84,32 +79,9 @@ private[service] object ActionResultVectorTablesWriter {
)
/**
* Decompose every existing `action_results.payload` into the five new vector tables. Runs inside the migration
* transaction; a no-op on a fresh game. Payloads are read fully before any INSERT so the SELECT `ResultSet` is closed
* first (xerial sqlite-jdbc dislikes interleaving writes with an open ResultSet). Each row goes through the full
* `ActionResultProtoConverter.fromProto` and delegates to `write` — same source of truth as the live path.
*/
def backfill(connection: Connection): Unit = {
val rows = {
val select = connection.prepareStatement("SELECT action_seq, payload FROM action_results ORDER BY action_seq")
try {
val rs = select.executeQuery()
val builder = Vector.newBuilder[(Int, Array[Byte])]
while rs.next() do builder += ((rs.getInt("action_seq"), rs.getBytes("payload")))
builder.result()
} finally select.close()
}
rows.foreach {
case (seq, payload) =>
val ar = ActionResultProtoConverter.fromProto(ActionResultProto.parseFrom(payload))
write(connection, seq, ar)
}
}
/**
* Insert all five vector tables' rows for the given `actionSeq` from a domain `ActionResultT`. Called by both the
* live-write loop in `SqliteHistory.withNewResults` (once per action) and the migration's `backfill` (once per
* existing payload, after `fromProto`). Each per-vector helper is a no-op when the source vector is empty.
* Insert all five vector tables' rows for the given `actionSeq` from a domain `ActionResultT`. Called by the
* live-write loop in `SqliteHistory.withNewResults` (once per action, via `PerEntityWriteDispatcher`). Each
* per-vector helper is a no-op when the source vector is empty.
*/
def write(connection: Connection, actionSeq: Int, ar: ActionResultT): Unit = {
insertIdVector(connection, "action_destroyed_battalion_ids", "battalion_id", actionSeq, ar.destroyedBattalionIds)
@@ -621,9 +621,6 @@ scala_library(
"//src/main/scala/net/eagle0/eagle/service:__pkg__",
"//src/test/scala/net/eagle0/eagle:__subpackages__",
],
deps = [
"//src/main/protobuf/net/eagle0/eagle/internal:action_result_scala_proto",
],
)
scala_library(
@@ -634,10 +631,8 @@ scala_library(
"//src/test/scala/net/eagle0/eagle:__subpackages__",
],
deps = [
"//src/main/protobuf/net/eagle0/eagle/internal:action_result_scala_proto",
"//src/main/scala/net/eagle0/eagle:eagle_pkg",
"//src/main/scala/net/eagle0/eagle/model/action_result:action_result_trait",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:action_result_proto_converter",
"//src/main/scala/net/eagle0/eagle/model/state/date",
],
)
@@ -706,10 +701,9 @@ scala_library(
"//src/test/scala/net/eagle0/eagle:__subpackages__",
],
deps = [
"//src/main/protobuf/net/eagle0/eagle/internal:action_result_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/common:battalion_type_scala_proto",
"//src/main/scala/net/eagle0/eagle:eagle_pkg",
"//src/main/scala/net/eagle0/eagle/model/action_result:action_result_trait",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:action_result_proto_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:battalion_type_converter",
],
)
@@ -722,10 +716,11 @@ scala_library(
"//src/test/scala/net/eagle0/eagle:__subpackages__",
],
deps = [
"//src/main/protobuf/net/eagle0/eagle/internal:action_result_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/common:chronicle_entry_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/common:eagle_map_info_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/internal:shardok_battle_scala_proto",
"//src/main/scala/net/eagle0/eagle:eagle_pkg",
"//src/main/scala/net/eagle0/eagle/model/action_result:action_result_trait",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:action_result_proto_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:eagle_map_info_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters/chronicle_entry:chronicle_entry_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters/shardok_battle:shardok_battle_converter",
@@ -740,10 +735,8 @@ scala_library(
"//src/test/scala/net/eagle0/eagle:__subpackages__",
],
deps = [
"//src/main/protobuf/net/eagle0/eagle/internal:action_result_scala_proto",
"//src/main/scala/net/eagle0/eagle:eagle_pkg",
"//src/main/scala/net/eagle0/eagle/model/action_result:action_result_trait",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:action_result_proto_converter",
],
)
@@ -755,11 +748,12 @@ scala_library(
"//src/test/scala/net/eagle0/eagle:__subpackages__",
],
deps = [
"//src/main/protobuf/net/eagle0/eagle/common:command_type_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/common:game_type_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/common:round_phase_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/common:tutorial_phase_scala_proto",
"//src/main/protobuf/net/eagle0/eagle/internal:action_result_scala_proto",
"//src/main/scala/net/eagle0/eagle:eagle_pkg",
"//src/main/scala/net/eagle0/eagle/model/action_result:action_result_trait",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:action_result_proto_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:command_type_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:game_type_converter",
"//src/main/scala/net/eagle0/eagle/model/proto_converters:round_phase_converter",
@@ -1125,72 +1125,32 @@ object SqliteHistory {
}
/**
* v15 for Phase 4.5h.1: the first batch of top-level `ActionResultC` columns promoted onto `action_results`. Adds
* four nullable INTEGER columns + indexes (`acting_hero_id`, `acting_faction_id`, `province_id`,
* `province_id_acted`), then back-fills them from every existing `action_results.payload`. The read path is **not**
* swapped here — `replayFrom` still parses `payload`; the columns are "shadow" until the final 4.5h sub-PR drops
* `payload` and switches the read path. See `ActionResultEntityRefColumnsWriter`.
* v15v20 (Phase 4.5h.14.5h.4) decompose top-level `ActionResultC` fields onto `action_results` columns plus five
* vector child tables. The original migrations also back-filled the new columns/tables from each existing
* `action_results.payload`; the backfills were deleted in 4.5i (after #6787 ensured every persisted game.db is at v
* ≥ 21 in S3 — see `loaded` and the post-mortem in `changed_entity_decomposition_pitfalls.md`).
*
* Schema v < 15 is no longer supported. If `applyV15` ever runs on a non-empty `action_results` table, the new
* columns are left NULL for every pre-existing row — silent state loss. The migration runner's `schema_version`
* gate is the only protection; do not weaken it.
*/
private def applyV15ActionResultEntityRefColumns(connection: Connection): Unit = {
private def applyV15ActionResultEntityRefColumns(connection: Connection): Unit =
execEach(connection, ActionResultEntityRefColumnsWriter.schemaStatements)
ActionResultEntityRefColumnsWriter.backfill(connection)
}
/**
* v16 for Phase 4.5h.2a: the three enum-typed top-level `ActionResultC` fields promoted onto `action_results`. Adds
* four nullable INTEGER columns (`new_round_phase`, `last_command_type`, `new_game_type`, plus a
* `new_tutorial_phase` for the TutorialPhase wrapped inside `GameType.Tutorial`) + indexes, then back-fills them
* from every existing `action_results.payload` via the full `ActionResultProtoConverter.fromProto`. The read path
* is **not** swapped here. See `ActionResultEnumColumnsWriter`.
*/
private def applyV16ActionResultEnumColumns(connection: Connection): Unit = {
private def applyV16ActionResultEnumColumns(connection: Connection): Unit =
execEach(connection, ActionResultEnumColumnsWriter.schemaStatements)
ActionResultEnumColumnsWriter.backfill(connection)
}
/**
* v17 for Phase 4.5h.2b: the four numeric-primitive top-level `ActionResultC` fields promoted onto
* `action_results`. Adds four nullable INTEGER columns (`new_round_id`, `new_victor_faction_id`, `game_ended`,
* `new_random_seed`) + indexes, then back-fills them from every existing `action_results.payload`. See
* `ActionResultNumericColumnsWriter`.
*/
private def applyV17ActionResultNumericColumns(connection: Connection): Unit = {
private def applyV17ActionResultNumericColumns(connection: Connection): Unit =
execEach(connection, ActionResultNumericColumnsWriter.schemaStatements)
ActionResultNumericColumnsWriter.backfill(connection)
}
/**
* v18 for Phase 4.5h.2c: the last scalar/Option top-level `ActionResultC` fields promoted onto `action_results` —
* the year/month pair for `newDate` plus a TEXT column for `resolvedBattle`. The year/month shape mirrors the v1
* `date_year`/`date_month` columns used for the resulting state's current date. See
* `ActionResultDateBattleColumnsWriter`. This closes the 4.5h.2 family.
*/
private def applyV18ActionResultDateBattleColumns(connection: Connection): Unit = {
private def applyV18ActionResultDateBattleColumns(connection: Connection): Unit =
execEach(connection, ActionResultDateBattleColumnsWriter.schemaStatements)
ActionResultDateBattleColumnsWriter.backfill(connection)
}
/**
* v19 for Phase 4.5h.3: the three structured-Option top-level `ActionResultC` fields promoted onto `action_results`
* as whole-entity BLOB columns (`new_battle_proto`, `new_chronicle_entry_proto`, `new_eagle_map_proto`). No
* indexes; the columns are stored opaquely. See `ActionResultStructuredBlobColumnsWriter`.
*/
private def applyV19ActionResultStructuredBlobColumns(connection: Connection): Unit = {
private def applyV19ActionResultStructuredBlobColumns(connection: Connection): Unit =
execEach(connection, ActionResultStructuredBlobColumnsWriter.schemaStatements)
ActionResultStructuredBlobColumnsWriter.backfill(connection)
}
/**
* v20 for Phase 4.5h.4: the five remaining top-level `ActionResultC` vector fields decomposed into child tables —
* four primitive-ID join tables (`action_destroyed_battalion_ids`, `action_removed_hero_ids`,
* `action_removed_faction_ids`, `action_affected_faction_ids`) plus one quest-hybrid table for `newBattalionTypes`
* (`action_new_battalion_types`, type_id queryable + whole `battalion_type_proto BLOB`). Back-filled from every
* existing `action_results.payload`. See `ActionResultVectorTablesWriter`.
*/
private def applyV20ActionResultVectorTables(connection: Connection): Unit = {
private def applyV20ActionResultVectorTables(connection: Connection): Unit =
execEach(connection, ActionResultVectorTablesWriter.schemaStatements)
ActionResultVectorTablesWriter.backfill(connection)
}
/**
* Final step of the 4.5x decomposition: drop the `payload` blob from `action_results`. By v21 every field on
@@ -2166,32 +2166,6 @@ class SqliteHistoryTest extends AnyFlatSpec with Matchers with BeforeAndAfterEac
}
}
"action_results v15 backfill" should "populate the four entity-ref columns from an existing payload" in {
// Backfill from a payload whose proto-side leader/player/province/provinceActed are all set. The proto field names
// don't match the domain — column names follow the domain. RED until backfill is implemented.
val payload = ActionResult(
leader = Some(9),
player = Some(3),
province = Some(5),
provinceActed = Some(11)
).toByteArray
seedV1WithPayloads(Vector(payload))
SqliteHistory.loaded(dbFile).close()
withRawConnection { conn =>
withRow(
conn,
"SELECT acting_hero_id, acting_faction_id, province_id, province_id_acted FROM action_results " +
"WHERE action_seq = 0"
) { rs =>
rs.getInt("acting_hero_id") shouldBe 9
rs.getInt("acting_faction_id") shouldBe 3
rs.getInt("province_id") shouldBe 5
rs.getInt("province_id_acted") shouldBe 11
}
}
}
// ---------------- action_results enum columns (v16 — RED until implemented) ----------------
"action_results v16 live write" should
@@ -2224,31 +2198,6 @@ class SqliteHistoryTest extends AnyFlatSpec with Matchers with BeforeAndAfterEac
}
}
"action_results v16 backfill" should "populate the four enum columns from an existing payload (Normal branch)" in {
// Normal branch — new_game_type set, new_tutorial_phase NULL. Backfill runs ActionResultProtoConverter.fromProto
// on the payload, so this exercises the shared converter logic end-to-end. RED until backfill is implemented.
val payload = ActionResult(
newRoundPhase = Some(NewRoundPhase(value = RoundPhaseProto.PLAYER_COMMANDS)),
lastCommandTypeForActingProvince = CommandTypeProto.COMMAND_TYPE_IMPROVE,
newGameType = GameTypeProto.NORMAL
).toByteArray
seedV1WithPayloads(Vector(payload))
SqliteHistory.loaded(dbFile).close()
withRawConnection { conn =>
withRow(
conn,
"SELECT new_round_phase, last_command_type, new_game_type, new_tutorial_phase FROM action_results " +
"WHERE action_seq = 0"
) { rs =>
rs.getInt("new_round_phase") shouldBe RoundPhaseProto.PLAYER_COMMANDS.value
rs.getInt("last_command_type") shouldBe CommandTypeProto.COMMAND_TYPE_IMPROVE.value
rs.getInt("new_game_type") shouldBe GameTypeProto.NORMAL.value
rs.getObject("new_tutorial_phase") shouldBe null
}
}
}
// ---------------- action_results numeric columns (v17 — RED until implemented) ----------------
"action_results v17 live write" should
@@ -2281,31 +2230,6 @@ class SqliteHistoryTest extends AnyFlatSpec with Matchers with BeforeAndAfterEac
}
}
"action_results v17 backfill" should "populate the four numeric columns from an existing payload" in {
// Backfill from a payload whose four proto fields are all set. RED until backfill is implemented.
val payload = ActionResult(
newRoundId = Some(7),
newVictor = Some(3),
gameEnded = Some(true),
newRandomSeed = Some(123456789L)
).toByteArray
seedV1WithPayloads(Vector(payload))
SqliteHistory.loaded(dbFile).close()
withRawConnection { conn =>
withRow(
conn,
"SELECT new_round_id, new_victor_faction_id, game_ended, new_random_seed FROM action_results " +
"WHERE action_seq = 0"
) { rs =>
rs.getInt("new_round_id") shouldBe 7
rs.getInt("new_victor_faction_id") shouldBe 3
rs.getInt("game_ended") shouldBe 1
rs.getLong("new_random_seed") shouldBe 123456789L
}
}
}
// ---------------- action_results date + resolvedBattle columns (v18 — RED until implemented) ----------------
"action_results v18 live write" should
@@ -2335,27 +2259,6 @@ class SqliteHistoryTest extends AnyFlatSpec with Matchers with BeforeAndAfterEac
}
}
"action_results v18 backfill" should "populate the date pair and resolved_battle from an existing payload" in {
// Backfill from a payload with proto newDate + resolvedBattle set. RED until backfill is implemented.
val payload = ActionResult(
newDate = Some(DateProto(year = 1234, month = 6)),
resolvedBattle = Some("shardok-game-xyz")
).toByteArray
seedV1WithPayloads(Vector(payload))
SqliteHistory.loaded(dbFile).close()
withRawConnection { conn =>
withRow(
conn,
"SELECT new_date_year, new_date_month, resolved_battle FROM action_results WHERE action_seq = 0"
) { rs =>
rs.getInt("new_date_year") shouldBe 1234
rs.getInt("new_date_month") shouldBe 6
rs.getString("resolved_battle") shouldBe "shardok-game-xyz"
}
}
}
// ---------------- action_results structured-Option blob columns (v19 — RED until implemented) ----------------
private def sampleBattle: ShardokBattle =
@@ -2412,35 +2315,6 @@ class SqliteHistoryTest extends AnyFlatSpec with Matchers with BeforeAndAfterEac
}
}
"action_results v19 backfill" should "populate the three blob columns from an existing payload" in {
// Backfill from a payload whose three proto fields are all set. RED until backfill is implemented.
val payload = ActionResult(
newBattle = Some(ShardokBattleConverter.toProto(sampleBattle)),
newChronicleEntry = Some(ChronicleEntryConverter.toProto(sampleChronicleEntry)),
newEagleMap = Some(EagleMapInfoConverter.toProto(sampleEagleMapInfo))
).toByteArray
seedV1WithPayloads(Vector(payload))
SqliteHistory.loaded(dbFile).close()
withRawConnection { conn =>
withRow(
conn,
"SELECT new_battle_proto, new_chronicle_entry_proto, new_eagle_map_proto FROM action_results " +
"WHERE action_seq = 0"
) { rs =>
ShardokBattleConverter.fromProto(
ShardokBattleProto.parseFrom(rs.getBytes("new_battle_proto"))
) shouldBe sampleBattle
ChronicleEntryConverter.fromProto(
ChronicleEntryProto.parseFrom(rs.getBytes("new_chronicle_entry_proto"))
) shouldBe sampleChronicleEntry
EagleMapInfoConverter.fromProto(
EagleMapInfoProto.parseFrom(rs.getBytes("new_eagle_map_proto"))
) shouldBe sampleEagleMapInfo
}
}
}
// ---------------- action_results vector tables (v20 — RED until implemented) ----------------
private def sampleBattalionType: BattalionType =
@@ -2510,20 +2384,6 @@ class SqliteHistoryTest extends AnyFlatSpec with Matchers with BeforeAndAfterEac
withRawConnection(conn => assertVectorTablesPopulated(conn))
}
"action_results v20 backfill" should "decompose all five vector tables from an existing payload" in {
val payload = ActionResult(
destroyedBattalionIds = Seq(17, 23),
removedHeroes = Seq(9),
removedFactionIds = Seq(3),
affectedPlayers = Seq(3, 7),
newBattalionTypes = Seq(BattalionTypeConverter.toProto(sampleBattalionType))
).toByteArray
seedV1WithPayloads(Vector(payload))
SqliteHistory.loaded(dbFile).close()
withRawConnection(conn => assertVectorTablesPopulated(conn))
}
"generated text requests v13 backfill" should "decompose generated text requests from an existing payload" in {
// The backfill-on-load path. The read path drops generated text requests, so the proto blob is their only home;
// backfill reads the repeated new_generated_text_requests field directly. Seed a payload, load (runs v13 backfill),