CRDT State¶
Two daemons, one infrastructure. CRDTs are how s.a.u avoids split-brain without a central coordinator.
Important
When multiple daemon instances or concurrent goroutines mutate remediation state, CloudSlash needs a way to converge without a central lock. This page covers the Last-Writer-Wins Map CRDT implemented in pkg/engine/sau/crdt/: how it merges, rejects stale updates, and guarantees eventual consistency.
Network Partitions¶
and CAP
Running CloudSlash across multiple CI/CD runner nodes creates unavoidable network partitions. If Runner A mutates node 1 while Runner B mutates node 2, routing both mutations through a centralized SQL store risks transaction lock failure: the fundamental CAP Theorem constraint.
The answer is to drop centralized locking entirely and track internal state with a Last-Writer-Wins (LWW) Map CRDT.
Mathematical Formalization¶
A CRDT guarantees convergence mathematically. The merge(X, Y) operation must satisfy three properties:
- Commutativity: \(X \cup Y = Y \cup X\). Order doesn't matter.
- Associativity: \((X \cup Y) \cup Z = X \cup (Y \cup Z)\). Grouping doesn't matter.
- Idempotency: \(X \cup X = X\). Applying the same update twice has no effect.
Go Implementation¶
Each mutating event is indexed against local physical clocks.
// pkg/engine/sau/crdt.go
type LWWRegister struct {
Value interface{}
Timestamp int64
}
type LWWMap struct {
Data map[string]LWWRegister
Mu sync.RWMutex
}
func (m *LWWMap) Merge(remoteMap map[string]LWWRegister) {
m.Mu.Lock()
defer m.Mu.Unlock()
for key, remoteReg := range remoteMap {
localReg, exists := m.Data[key]
if !exists || remoteReg.Timestamp > localReg.Timestamp {
m.Data[key] = remoteReg
}
// In exact timestamp collisions, arbitrarily prefer remote to satisfy commutativity
if exists && remoteReg.Timestamp == localReg.Timestamp {
// Compare bytes deterministically if utilizing vectors
m.Data[key] = remoteReg
}
}
}
Saga Resumption¶
If a daemon crashes with exit status 137 (OOM kill) mid-saga, the LWW Map guarantees exact state continuity on restart.
The restarted daemon reads its persisted CRDT payload. Because steps n=1 and n=2 were logged with monotonic timestamps, the map mathematically reflects exactly what happened before the kernel fault. The daemon picks up at n=3 or executes the required compensation sequence.