* Replace dotnet proto build with Bazel-hermetic C# proto generation
Use rules_proto_grpc_csharp to generate .cs source files via Bazel
instead of requiring a locally-installed dotnet SDK to build protos.dll.
This eliminates the dotnet dependency for CI runners and makes the
proto build fully hermetic.
- Add bazel_dep for rules_proto_grpc_csharp 5.8.0
- Bump grpc 1.74.0 -> 1.74.1 (required by rules_proto_grpc_csharp)
- Add csharp_proto_compile/csharp_grpc_compile targets across 8 proto
BUILD.bazel files (81 .cs files from 78 protos)
- Rewrite build_protos.sh to use bazel build + copy instead of dotnet
- Output goes to Assets/GeneratedProtos/ with package-based subdirs
to avoid filename collisions (e.g. ActionResultView.cs in both
shardok/api/ and eagle/views/)
- Add Assets/GeneratedProtos/ to .gitignore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove stale Eagle0Protos DLL references
The old build_protos.sh generated protos.dll via dotnet; the new one
generates .cs source files instead. Remove the tracked .meta and
.deps.json files that reference the no-longer-generated DLL, and
gitignore the directory to prevent re-adding.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The names.tsv file was being generated but only names.json is actually
used by BattalionNameGenerator. The TSV was just an intermediate format.
- Update downloadTsvs.sh to send checker output to /dev/null
- Remove .gitignore entry for names.tsv
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1. Speed up check_build_deps.sh:
- Cache expensive `bazel query deps(...)` results
- Reuse cached deps for all three checks instead of running
separate bazel query commands
- Use grep filtering on cached results instead of bazel intersect
2. Run lint job in parallel with test job:
- Split bazel_test.yml into separate lint and test jobs
- Jobs run concurrently, reducing total CI time
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Move SparklePlugin to separate Bazel workspace to resolve rules_swift conflict
The problem: grpc 1.76.0.bcr.1 requires rules_swift 3.x, but rules_apple 4.x
requires rules_swift 2.x. These have different compatibility levels, causing
bzlmod resolution to fail when both are in the same workspace.
The solution: Move SparklePlugin (the only thing using rules_apple) to a
separate Bazel workspace at `sparkle_workspace/`. This workspace has its own
MODULE.bazel with only rules_apple and Sparkle dependencies, completely
isolated from the grpc/flatbuffers dependency tree.
Changes:
- Create sparkle_workspace/ with isolated MODULE.bazel
- Move SparklePlugin source files to sparkle_workspace/
- Update build_sparkle_plugin.sh to build from subworkspace
- Remove rules_apple from main MODULE.bazel
- Add single_version_override for rules_swift 3.1.2 (for grpc/flatbuffers)
All 315 tests pass.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add WORKSPACE.bazel to sparkle_workspace to prevent parent workspace detection
Without this file, Bazel may walk up the directory tree and find the parent
workspace's MODULE.bazel, causing dependency conflicts.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Add CRUD handlers for managing what's-new entries stored in S3
- Add whats_new.html template with add/edit/delete forms
- Add navigation link to layout.html
- Add player-friendly summary generation to generate_changelog.sh
- Add seed JSON file for initial data
The admin console at /whats-new allows managing changelog entries
that will be displayed to players in the Unity client. The script
integration generates summaries and opens the admin console with
pre-populated content after sending weekly changelogs.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add eagle0.net as server name alias
Allows accessing the site via eagle0.net in addition to prod.eagle0.net.
Note: After deploying, run certbot to add eagle0.net to the SSL certificate:
certbot --nginx -d prod.eagle0.net -d eagle0.net
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add admin container image verification in deploy workflow
The admin container was running a stale image after the entrypoint was
changed from /app/admin_server to /app/admin_server_linux_amd64. This
happened because:
1. crane pull + docker load doesn't update the :latest tag locally
2. docker-compose fallback to :latest used the old cached image
3. --force-recreate recreated the container but with the old image
Fixes:
- Tag pulled admin image as :latest after docker load (ensures fallback
uses correct image)
- Add image digest verification after deploy (fails early if wrong
image is running)
- Add admin startup verification in deploy-blue-green.sh (catches
container crashes with helpful logs)
This follows the same pattern used in auth_build.yml for auth service
verification.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Remove --keychain flag from codesign commands
The --keychain flag was causing codesign to look for the private key in
the build keychain, but the matching cert+key is in login.keychain.
Since we now use the unambiguous SHA-1 hash, codesign will find the
correct certificate and key pair in whichever keychain contains them.
This fixes the intermittent errSecInternalComponent failures.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Use build keychain specifically for CI codesigning
In CI, the workflow imports the signing certificate into a temporary
build keychain. Previously, the script searched ALL keychains and
picked the first certificate found, which could be an old certificate
from login.keychain instead of the freshly imported one.
Now when KEYCHAIN_NAME is set (CI environment), the script looks for
certificates only in that specific keychain. For local dev (no
KEYCHAIN_NAME), it still searches all keychains.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix Mac codesign by isolating build keychain during signing
The --keychain flag alone doesn't prevent codesign from finding matching
identities in other keychains on the search list. Fix by temporarily
setting ONLY the build keychain as the search list during signing, then
restoring the original list on exit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Include System.keychain for Apple root certificates
The previous fix broke certificate chain verification because we removed
the login keychain but also lost access to Apple's root certificates.
Include System.keychain (which has Apple roots) but not login.keychain
(which has the duplicate signing identity).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Use certificate SHA-1 hash instead of name to avoid ambiguity
Instead of manipulating the keychain search list (which breaks certificate
chain verification), extract the SHA-1 hash of the specific certificate
from the build keychain and use that as the signing identity. SHA-1 hashes
are unambiguous and codesign will use the exact certificate specified.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix certificate hash extraction - get first valid identity
The previous grep for SIGNING_IDENTITY failed because GitHub Actions
masks the value. Instead, get the first valid codesigning identity from
the keychain (there should only be one since we just imported it).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Use full keychain path for find-identity
security find-identity requires the full path to the keychain file,
not just the keychain name. Resolve the full path from list-keychains
output before querying for identities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add debug output to certificate import and remove set-keychain-settings
Add debug output to see what identities are available after import.
Also remove the set-keychain-settings line which may be causing issues
(the -u flag locks keychain on sleep).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Search all keychains for identity hash
The build keychain import isn't creating a recognizable codesigning
identity, but the certificate exists in login.keychain. Search all
keychains and use the hash of the first valid identity - hashes are
unique and unambiguous regardless of which keychain contains them.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Add --keychain flag to all codesign commands to explicitly specify which
keychain to use. This fixes the "ambiguous" error when the same signing
identity exists in multiple keychains (build keychain and login keychain).
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
When multiple runners on the same machine try to sign simultaneously,
they were conflicting on the shared keychain names (build.keychain,
ios-build.keychain). This caused errSecInternalComponent errors.
Changes:
- mac_build.yml: Use build-${run_id}.keychain
- ios_testflight.yml: Use ios-build-${run_id}.keychain
- codesign_mac_app.sh: Support KEYCHAIN_NAME env var with fallback
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Usage:
eagle-logs # Tail logs (follow mode)
eagle-logs -n 100 # Show last 100 lines and follow
eagle-logs --no-follow -n 50 # Last 50 lines without following
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Remove sudo symlink creation (deploy user lacks passwordless sudo)
- Simplify eagle-exec to read from /opt/eagle0/.active-instance file
- Update deploy-blue-green.sh to write active instance to file
- Keep fallback to checking running containers if file doesn't exist
Users can add their own alias:
alias eagle-exec='/opt/eagle0/scripts/eagle-exec.sh'
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Adds a helper script that automatically runs docker exec against the
active Eagle instance (blue or green), determined by checking nginx
config.
Usage:
eagle-exec printenv GEMINI_API_KEY
eagle-exec jcmd 1 VM.flags
eagle-exec sh
The script is deployed to /opt/eagle0/scripts/ and symlinked to
/usr/local/bin/eagle-exec for easy access.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
The library/ code is now fully protoless (zero Scala proto dependencies).
Transitive deps through C++/Go build tools (map generation) are expected.
Changes:
- Delete docs/DEPROTO_PLAN.md - migration complete
- Delete scripts/build_deps_baseline.txt - no longer needed
- Update check_build_deps.sh to enforce zero Scala proto deps in library/
(C++/Go proto deps are allowed as they're build-time tools)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Verify code signature before attempting to staple (catches transfer corruption)
- Increase retry attempts from 5 to 10
- Increase wait between retries from 10s to 30s (total wait up to 5 minutes)
- Capture and display stapler error output for better debugging
- Show signature details if verification fails
This addresses intermittent stapling failures due to Apple CloudKit
propagation delays after notarization completes.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Remove stale proto exports from province:province
The province target was exporting province_event_scala_proto,
army_scala_proto, and unaffiliated_hero_scala_proto but no downstream
targets actually needed these through that path. Removing these stale
exports reduces library/ proto deps from 41 to 26.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix unused import and stale test dependency
Remove unused LlmRequestT import and stale unaffiliated_hero_scala_proto
dependency that were exposed by removing the province proto exports.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix test build failures from stale proto dependencies
- Remove unused ProtoMatchers from tests that don't use it
- Add missing unaffiliated_hero_quest_scala_proto dep to quest_fulfillment_utils_test
- Remove stale unaffiliated_hero_quest_scala_proto dep from check_for_fulfilled_quests_action_test
- Keep ProtoMatchers in ResolveBattleActionTest (uses equalProto matcher)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Remove unused eagle_unit deps from resolved_eagle_unit and
province_conquered_action. These files use ResolvedEagleUnit (which
uses Scala types), not EagleUnit (which uses proto types).
This breaks the transitive proto dependency chain, reducing library/
proto deps from 74 to 41.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Migrate ResolvedShardokPlayer to use native Scala EndGameCondition
instead of the proto type. This removes 1 transitive proto dependency
from library/ (75 → 74).
Changes:
- Add Scala EndGameCondition sealed trait with Victory, AllyVictory,
Draw, and Loss cases
- Add DrawType enum for draw conditions
- Add EndGameConditionConverter in ShardokBattleConverter
- Update ResolvedShardokPlayer to use Scala EndGameCondition
- Move proto conversion to ShardokInterfaceGrpcClient boundary
- Update ResolveBattleAction to use Scala EndGameCondition API
- Update ResolveBattleActionTest to construct Scala EndGameCondition
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Remove stale proto_converter deps from library/ BUILD files
Library code should not depend on proto_converters - those belong at
the service layer boundary. Removed 9 stale proto_converter deps that
were no longer used by any Scala code.
The only remaining proto_converter dep is shardok_battle_converter,
which is actually used by ResolveBattleAction.scala (to be fixed in
a follow-up PR).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Move ShardokBattle boundary conversion to ShardokInterfaceGrpcClient
Previously, BattleResolution contained a proto ShardokBattle, requiring
ResolveBattleAction in library/ to depend on ShardokBattleConverter.
This violated the boundary principle where proto conversions should
happen at the service layer, not in library code.
This change:
- Updates BattleResolution.battle to use Scala ShardokBattle
- Moves the proto-to-Scala conversion into ShardokInterfaceGrpcClient
- Removes ShardokBattleConverter dependency from library/ code
- Updates tests to use the scalaBattle() helper for conversion
This completely removes proto_converters dependencies from library/
(the last one was shardok_battle_converter).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add linter check for library/ proto_converters boundary
Adds Rule 3 to check_build_deps.sh that verifies library/ code does not
depend on proto_converters. Proto conversions should happen at service
boundaries (ShardokInterfaceGrpcClient, EagleServiceImpl, etc.), not in
library code.
Also updates baseline from 167 to 75 proto deps reflecting recent cleanup.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Change from com.Shardok-Games.eagle0 to net.eagle0.eagle0 to match
the project's domain (net.eagle0).
Updated in:
- Unity ProjectSettings (applicationIdentifier for Standalone)
- inject_sparkle.sh (CFBundleURLName for URL scheme)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Replace Hostility proto with Scala Hostility enum
This removes direct proto imports from library/ for the Hostility type:
- FactionUtils.hostilityStatus now returns Scala Hostility
- ArmyStats.hostility now uses Scala Hostility type
- AvailableAttackDecisionCommandFactory no longer imports proto
- AvailableFreeForAllDecisionCommandFactory no longer imports proto
- AttackDecisionCommandChooser uses Hostility.Enemy instead of proto
- BattleFilter no longer needs protoToScalaHostility converter
- AvailableCommandConverter uses HostilityConverter.toProto
This completes Phase 8b of the deproto migration, reducing direct
proto imports in library/ from 8 files to 4 (LLM prompt generators).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix AvailableCommandConverterTest to use Scala Hostility enum
Update test to use net.eagle0.eagle.model.state.Hostility.Self
instead of proto net.eagle0.common.hostility.Hostility.SELF_HOSTILITY.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add BUILD.bazel dependency linting
Adds tooling to enforce architectural boundaries in the codebase:
1. Shell script `scripts/check_build_deps.sh` with multiple modes:
- Default: runs all checks and shows dependency counts
- --ci: fails only on hard violations (src/main depends on src/test)
- --strict: also fails if proto deps in library/ increase from baseline
- --count: just show current dependency counts
- --update-baseline: update the baseline file for tracking
2. Bazel test `//:build_deps_test` for integration with test suite
3. Baseline file `scripts/build_deps_baseline.txt` tracking proto dep count
Current enforced rules:
- src/main must not depend on src/test (enforced now)
- library/ proto dependencies tracked (167 currently, deproto in progress)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Update DEPROTO_PLAN.md with accurate proto import inventory
The previous status was inaccurate. This update:
- Corrects summary table: Availability, Command Choice Helpers,
and other utilities still have Hostility proto imports
- Adds Phase 8b documenting Hostility proto migration (4 files)
- Updates detailed inventory showing actual 13 files with direct
proto imports (vs 167 transitive deps from bazel query)
- Corrects success criteria checkboxes to reflect actual status
- Documents the grep command used to verify imports
The 167 proto deps from bazel query are transitive dependencies.
Only 13 files have direct proto imports:
- 5 boundary files (expected to keep proto)
- 4 files using Hostility proto (Phase 8b)
- 4 LLM prompt generator files (Phase 8)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Add --no-deps to nginx and admin docker compose commands to prevent
them from cascading to auth. The auth container has secrets (like
TWITCH_CLIENT_ID) that are only available in auth_build.yml, not in
docker_build.yml. Without --no-deps, docker compose would recreate
auth with blank env vars when it detected config changes.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Change version scheme from git describe (which picked up unrelated tags
like busybox-1.35.0) to commit-count based versions like 1.0.9548.
This gives automatic, always-incrementing version numbers that Sparkle
can properly compare for updates.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Use ditto instead of cp -R to copy Sparkle.framework (preserves bundle structure)
- Skip individual signing of Sparkle's internal XPC services, apps, and executables
- Use --deep flag when signing Sparkle.framework to handle its internal components
- Verify cached Sparkle.framework has proper symlink structure, re-download if corrupted
Fixes "bundle format unrecognized" error during codesigning.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add native Sparkle plugin to enable Mac auto-updates
The Sparkle framework was being injected into the app bundle, but
nothing was initializing it. This adds:
- Native Objective-C plugin (SparklePlugin.m) that initializes
SPUStandardUpdaterController at runtime
- C# wrapper (SparkleUpdater.cs) for Unity to call the native plugin
- SparkleInitializer.cs uses RuntimeInitializeOnLoadMethod to
automatically initialize Sparkle at app startup
- Build script to compile the plugin as a universal binary
The plugin is weak-linked against Sparkle.framework, which is injected
separately by inject_sparkle.sh during the build process.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Convert Sparkle plugin build from clang to Bazel
- Add Sparkle framework as http_archive dependency in MODULE.bazel
- Add BUILD.sparkle to import the framework
- Add BUILD.bazel for SparklePlugin using macos_bundle rule
- Update build_sparkle_plugin.sh to use Bazel instead of direct clang
- Register Apple CC toolchain extension
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix symbol exports for SparklePlugin native library
The C functions need to be exported with visibility("default") and
explicit linker flags for Unity P/Invoke to find them. Without this,
the bundle binary had no exported symbols.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Re-enable Sparkle auto-update integration for Mac builds
Restores Sparkle integration that was temporarily removed in #5317:
- Restore inject_sparkle.sh script
- Add Sparkle injection step to mac_build.yml
- Re-enable Sparkle signing and appcast updates in deploy step
Combined with native SparklePlugin that initializes Sparkle at runtime.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Build SparklePlugin.bundle before Unity build
The SparklePlugin.bundle.meta file tells Unity to include the plugin,
but the actual bundle needs to be built by Bazel first.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Convert SparklePlugin Info.plist to XML format
Unity's build system requires Info.plist files in XML format,
but Bazel outputs them in binary plist format.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Remove Sparkle auto-update integration from Mac builds
Temporarily removing Sparkle integration to get Mac builds working:
- Remove inject_sparkle.sh script and workflow step
- Make mac_build_handler's Sparkle private key optional
- Skip Sparkle signing and appcast updates when no key provided
This allows Mac builds to complete without Sparkle. Auto-updates can
be re-enabled later once the basic build pipeline is stable.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Use clean checkout to remove stale SparklePlugin.bundle
The runner had a leftover SparklePlugin.bundle from previous builds
which was causing Unity to fail when trying to process it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Eagle server already persists to S3 on every game save via CompoundPersister
when S3Credentials.isEnabled. The deploy script's s3cmd backup was redundant
and caused warnings when s3cmd wasn't installed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Add dedicated self-hosted runners for parallel Unity builds:
- unity-mac: Mac Unity builds and deployment
- unity-windows: Windows Unity builds (cross-compiled on Mac)
- notarize: Notarization waiting (lightweight, doesn't block builds)
Split mac_build.yml into 3 jobs:
1. build-and-sign (unity-mac): Build, sign, submit to Apple
2. wait-notarization (notarize): Wait for Apple, staple ticket
3. deploy (unity-mac): Deploy notarized app
This allows:
- Mac and Windows Unity builds to run in parallel
- Notarization waiting doesn't block other builds
- All runners share the same Mac Mini hardware
New scripts:
- notarize_submit.sh: Submit without waiting, output submission ID
- notarize_wait.sh: Wait for submission ID, staple ticket
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Use rm -f instead of rm when cleaning up the zip file after notarization.
The zip may already be deleted if a previous step failed and was retried,
causing the script to fail even when notarization actually succeeded.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Auto-invalidate game cache when flush marker is updated
During warmup, the staging server may cache stale game data that was
loaded before the active server flushed. Instead of exposing an RPC
for cache invalidation (which leaks internal state), the server now
automatically detects when the flush marker is updated and invalidates
any cached games.
Changes:
- GamesManager: Added `invalidateCacheIfFlushMarkerUpdated()` that
checks the flush marker's modification time and clears the cache
if it's been updated since the last check
- Called before checking if a game is in cache, so stale data is
cleared before any attempt to use it
- Removed the InvalidateGameCache RPC (no longer needed)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix user wait window metric to measure actual user impact
The max user wait window was measuring from when nginx switching
started (before recreation) instead of when it completed. This
inflated the metric by ~12s because nginx recreation time was included.
Users can only experience a wait AFTER nginx starts routing to the
staging server, so we now measure from nginx_switch_end (after
recreation completes) to flush_end.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
During warmup, the staging server may cache stale game data that was
loaded before the active server flushed. Instead of exposing an RPC
for cache invalidation (which leaks internal state), the server now
automatically detects when the flush marker is updated and invalidates
any cached games.
Changes:
- GamesManager: Added `invalidateCacheIfFlushMarkerUpdated()` that
checks the flush marker's modification time and clears the cache
if it's been updated since the last check
- Called before checking if a game is in cache, so stale data is
cleared before any attempt to use it
- Removed the InvalidateGameCache RPC (no longer needed)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
The saves directory is owned by root (created by Docker) but the deploy
script runs as the deploy user. Use docker exec to create marker files
from inside a running container that has the saves directory mounted.
- create_deployment_marker: uses active container (running before staging starts)
- create_flush_marker: uses staging container (running after active stops)
- cleanup_markers_on_failure: uses active container (still running on failure)
- remove_stale_deployment_marker: finds any running eagle container
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix nginx not picking up config changes during blue-green deploy
Root cause: `docker compose restart nginx` doesn't refresh bind-mounted
volume files. The nginx container keeps using its cached copy of
nginx.conf even after we update the host file with sed.
This caused nginx to keep trying to connect to the old (now deleted)
eagle instance, resulting in 502 errors after deployment.
Fix:
- Use `docker compose up -d --force-recreate nginx` instead of `restart`
This recreates the container, forcing it to read the updated config
- Add verification that nginx picked up the correct backend
- Remove the useless pre-validation (it validated old config in old container)
The progression of failed fixes:
1. `nginx -s reload` - doesn't re-resolve Docker DNS
2. `docker compose restart` - doesn't refresh bind-mounted files
3. `docker compose up -d --force-recreate` - THIS WORKS
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Reorder deploy: switch nginx BEFORE stopping blue to avoid 502s
Previous order (caused 502 errors):
1. Stop blue → nginx still points to blue → 502!
2. Update nginx config
3. Recreate nginx → traffic finally works
New order (eliminates 502 window):
1. Update nginx config
2. Recreate nginx → traffic goes to green (blue still running)
3. Stop blue → flushes state to disk
4. 3-second pause for flush to complete
The stale data race condition is minimized by stopping blue immediately
after the nginx switch. Users reconnecting to green will lazy-load
fresh game data from disk (after blue has flushed).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add flush marker coordination for zero-downtime blue-green deploys
This ensures green never serves stale game data during deployments:
1. Deploy script creates .deployment_in_progress marker at start
2. Green's lazy-load waits for flush marker if deployment in progress
3. nginx switches to green BEFORE stopping blue (zero 502 downtime)
4. Blue stops, flushes state to disk
5. Deploy script creates .flush_complete marker
6. Green's waiting lazy-loads proceed with fresh disk data
Key changes:
- GamesManager.scala: Add waitForFlushMarker() that blocks lazy-load
during deployment until flush marker appears (30s timeout)
- GamesManager.scala: Auto-clean stale markers >5 minutes old
- GamesManager.scala: Add deployment ID correlation in logs [DEPLOY:xxx]
- GamesManager.scala: Report flush marker timeouts to Sentry
- deploy-blue-green.sh: Reorder to switch nginx BEFORE stopping blue
- deploy-blue-green.sh: Add marker file coordination with deployment ID
- deploy-blue-green.sh: Add timing metrics (flush duration, user wait window)
- nginx.conf: Keep variable-based routing (Docker DNS only resolves
running containers, so upstream+backup doesn't work)
Monitoring and observability:
- All deployment-related logs tagged with [DEPLOY:timestamp] for correlation
- Wait duration logged for each lazy-load during deployment
- Flush marker timeouts reported to Sentry for alerting
- Deploy script logs total duration, flush duration, max user wait window
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
nginx reload doesn't always force DNS re-resolution in Docker. During
blue-green deployment, after updating nginx.conf to point to the new
instance (e.g., eagle-green:40032), nginx -s reload would sometimes
keep trying to connect to the old (now removed) container, causing
502 errors.
A full restart ensures nginx picks up the new upstream correctly.
The ~1-2 second restart time is acceptable for deployment reliability.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
The warmup tool was timing out during CreateGame because the per-step
timeout was only 30 seconds. On a cold JVM (freshly started Eagle
instance during blue-green deployment), CreateGame can take longer
than 30 seconds due to:
- JIT compilation not yet warmed up
- First-time class loading
- Game initialization including Shardok communication
Changes:
- Increase per-operation timeout from 30s to 90s
- Increase overall warmup timeout from 60s to 300s (5 minutes)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1. Add missing jfr and jvm-tmp volumes to eagle-green
- Without these, JFR sidecar can't attach to the JVM when green is active
- JFR recordings wouldn't work either
2. Remove redundant sync_config_files from deploy script
- CI already copies config files via scp before running deploy
- Fetching from GitHub main could cause version mismatches
- Eliminates unnecessary network calls during deployment
3. Skip image pull if already present locally
- CI already pulls images before running deploy script
- Saves time during CI deployments
- Manual deployments still pull if needed
4. Add nginx config validation before reload
- Run nginx -t before nginx -s reload
- Rollback to backup config if validation fails
- Prevents broken config from taking down nginx
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Problems fixed:
1. CI was recreating admin BEFORE blue-green, causing stale .env
2. CI was restarting nginx AFTER blue-green (double restart)
3. Deploy script used slow nginx restart instead of reload
4. Cleanup was blocking the critical path
Changes:
- CI: Only restart shardok before blue-green, let script handle rest
- CI: Remove duplicate nginx restart and fallback path
- Deploy: Use nginx reload (faster) with restart fallback
- Deploy: Update .env before restarting services
- Deploy: Run container cleanup in background
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix: Move S3 backup to BEFORE stopping old server
Critical fix: The old server may wipe games.e0es on shutdown if it
doesn't have the save() merge fix. The backup must happen BEFORE
stopping to capture valid data.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Admin console shows all games, auto-install s3cmd, restart nginx properly
1. Admin console now shows all games from games.e0es, not just loaded ones:
- Added getAllRunningGamesSummary() to GamesManager
- Updated getRunningGames() to include unloaded games with "[Not loaded]" status
- Clicking into a game triggers lazy loading via getGameHistory()
2. Deploy script improvements:
- Auto-install s3cmd if not present (via pip or apt)
- Auto-configure s3cmd for DigitalOcean Spaces from .env
- Use nginx restart instead of reload to ensure all workers pick up new config
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix critical bug: save() now merges with existing games.e0es
CRITICAL BUG FIX: With lazy loading, gameControllerInfos only contains
games that have been loaded into memory. The old save() would overwrite
games.e0es with only the loaded games, losing all unloaded games.
This caused complete data loss of user-to-game mappings when:
1. New server started (empty gameControllerInfos)
2. Any operation triggered save()
3. games.e0es was overwritten with empty data
Fix: save() now reads existing games.e0es first and merges:
- Unloaded games are preserved from disk
- Loaded games use fresh in-memory state
Also adds logging to diagnose games.e0es read failures.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add S3 backup of games.e0es during blue-green deployment
After stopping the old container (which flushes state to storage),
create a timestamped backup in S3 before switching nginx traffic.
This provides a recovery point if something goes wrong during deployment.
Backups are stored at: s3://eagle0/eagle/save/backups/games.e0es.<timestamp>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix blue-green deployment dependencies and auth routing
1. docker-compose.prod.yml:
- Remove eagle-blue from nginx depends_on (use EAGLE_ADDR variable instead)
- Remove eagle-blue from admin depends_on (use EAGLE_ADDR variable instead)
- Add TODO note about jfr-sidecar limitation during green deployments
2. nginx/nginx.conf:
- Fix auth.Auth location on port 443 to route to auth:40033 instead of
eagle_backend. This was causing auth failures when clients connected
via the main HTTPS port.
These changes allow blue-green deployments to work without hard
dependencies that cause docker-compose to recreate stopped containers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add jfr-sidecar-green for blue-green JFR profiling support
- Add jfr-sidecar-green service that shares PID namespace with eagle-green
- Make admin's jfr-sidecar address configurable via JFR_SIDECAR_ADDR env var
- Update deploy script to:
- Start appropriate jfr-sidecar with each eagle instance
- Update .env with EAGLE_ADDR and JFR_SIDECAR_ADDR after switching
- Restart admin service to pick up new addresses
- Clean up old jfr-sidecar when removing old eagle instance
This ensures the JFR button in admin console works regardless of
whether blue or green is the active instance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Sync config files from GitHub at start of deployment
Downloads docker-compose.prod.yml and nginx.conf from the main branch
before starting deployment. This ensures new services (like jfr-sidecar-green)
are available when the deploy script runs.
- Preserves the current active instance in nginx.conf
- Creates .bak backups before overwriting
- Continues with existing files if GitHub fetch fails
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix: Move S3 backup to BEFORE stopping old server
Critical fix: The old server may wipe games.e0es on shutdown if it
doesn't have the save() merge fix. The backup must happen BEFORE
stopping to capture valid data.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix critical bug: save() now merges with existing games.e0es
CRITICAL BUG FIX: With lazy loading, gameControllerInfos only contains
games that have been loaded into memory. The old save() would overwrite
games.e0es with only the loaded games, losing all unloaded games.
This caused complete data loss of user-to-game mappings when:
1. New server started (empty gameControllerInfos)
2. Any operation triggered save()
3. games.e0es was overwritten with empty data
Fix: save() now reads existing games.e0es first and merges:
- Unloaded games are preserved from disk
- Loaded games use fresh in-memory state
Also adds logging to diagnose games.e0es read failures.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add S3 backup of games.e0es during blue-green deployment
After stopping the old container (which flushes state to storage),
create a timestamped backup in S3 before switching nginx traffic.
This provides a recovery point if something goes wrong during deployment.
Backups are stored at: s3://eagle0/eagle/save/backups/games.e0es.<timestamp>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix blue-green deployment dependencies and auth routing
1. docker-compose.prod.yml:
- Remove eagle-blue from nginx depends_on (use EAGLE_ADDR variable instead)
- Remove eagle-blue from admin depends_on (use EAGLE_ADDR variable instead)
- Add TODO note about jfr-sidecar limitation during green deployments
2. nginx/nginx.conf:
- Fix auth.Auth location on port 443 to route to auth:40033 instead of
eagle_backend. This was causing auth failures when clients connected
via the main HTTPS port.
These changes allow blue-green deployments to work without hard
dependencies that cause docker-compose to recreate stopped containers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add jfr-sidecar-green for blue-green JFR profiling support
- Add jfr-sidecar-green service that shares PID namespace with eagle-green
- Make admin's jfr-sidecar address configurable via JFR_SIDECAR_ADDR env var
- Update deploy script to:
- Start appropriate jfr-sidecar with each eagle instance
- Update .env with EAGLE_ADDR and JFR_SIDECAR_ADDR after switching
- Restart admin service to pick up new addresses
- Clean up old jfr-sidecar when removing old eagle instance
This ensures the JFR button in admin console works regardless of
whether blue or green is the active instance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Sync config files from GitHub at start of deployment
Downloads docker-compose.prod.yml and nginx.conf from the main branch
before starting deployment. This ensures new services (like jfr-sidecar-green)
are available when the deploy script runs.
- Preserves the current active instance in nginx.conf
- Creates .bak backups before overwriting
- Continues with existing files if GitHub fetch fails
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
With lazy game loading (merged in #5223), games are loaded on-demand
when users reconnect after nginx switches traffic. No explicit reload
call is needed - the new server reads fresh state from storage.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Add optional Ed25519 signing to the manifest_manager. When a signing key
is provided, the manifest is signed and the signature is prepended as
a header comment that clients can verify.
Changes:
- manifest_manager: Accept optional private key file as 3rd argument
- manifest_manager: Sign manifest content and prepend signature line
- installer_build.yml: Pass MANIFEST_SIGNING_KEY to manifest_manager
- unity_build.yml: Pass MANIFEST_SIGNING_KEY to manifest_manager
- Add generate_manifest_keys.go script to create key pairs
The signature line format is: # signature=<base64-encoded-ed25519-signature>
To enable signing:
1. Run: go run scripts/generate_manifest_keys.go
2. Add the private key as GitHub secret MANIFEST_SIGNING_KEY
3. Embed the public key in the installer for verification (PR 4)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Apple's CloudKit can have a brief delay after notarization completes
before the ticket is available for stapling. This adds a retry loop
with 10-second delays, up to 5 attempts.
Error was:
CloudKit query for eagle0.app failed due to "Record not found".
The staple and validate action failed! Error 65.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Improves Sparkle sign_update error reporting by capturing stderr, and allows full signing/notarization/deploy pipeline on feature branches via workflow_dispatch.
1. Fix crane installation - don't try to mv crane to itself
(tar extracts to cwd which is already /opt/eagle0)
2. Keep crane binary after deploy for blue-green script to use
3. Update blue-green deploy to use crane instead of docker pull
(fixes OCI/Docker digest mismatch issue)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix codesign script to sign all Sparkle framework components
Sign XPC services, nested apps (Updater.app), and standalone
executables (Autoupdate) before signing the framework itself.
Apple notarization requires all nested binaries to be signed
with Developer ID certificate and secure timestamp.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add missing path triggers for pull_request in Mac build workflow
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Re-add rules_swift for Mac GoDice plugin build
The DarwinGodiceBundle requires rules_swift to build.
This was inadvertently removed in #5194.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Handles intermittent Docker registry digest mismatch errors like:
"failed commit on ref: unexpected commit digest"
This is a known Docker/containerd issue that can occur due to:
- Registry caching
- Network/proxy issues
- Race conditions during push
Now retries up to 3 times with 5 second delays.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Unity Mac build scripts (build_mac.sh, build_unity_mac.sh)
- Code signing with Developer ID certificate
- Apple notarization for Gatekeeper compliance
- Sparkle framework injection for delta auto-updates
- Go build handler for S3 upload and appcast.xml generation
- Update InvitationCodeManager.cs for Mac platform paths
- GitHub Actions workflow triggered on main branch pushes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add blue-green deployment infrastructure for Eagle server
- Add ReloadGames RPC to eagle.proto for reloading game state from disk
- Implement reloadAllGames() and flushToDisk() in GamesManager.scala
- Add reloadGames() handler to EagleServiceImpl.scala
- Update docker-compose.prod.yml with eagle-blue/green services
- Update nginx.conf for switchable upstream
- Create scripts/deploy-blue-green.sh for zero-downtime deployment
- Create Go warmup tool (src/main/go/net/eagle0/warmup) that:
- Uses bidirectional streaming to create test games
- Posts Improve command and verifies ActionResults
- Cleans up test game after warmup
- Create scripts/warmup-eagle.sh wrapper that uses Go tool or falls back to grpcurl
- Update docker_build.yml to build and deploy warmup binary
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add X-Warmup-User header support with localhost restriction
- AuthorizationInterceptor: Accept X-Warmup-User header for warmup authentication
- Only allow X-Warmup-User from localhost connections (security)
- Go warmup tool: Send x-warmup-user metadata header
- Add grpc/metadata dependency to warmup BUILD
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
