Clustering

​

Modes

• Standalone: What it gives you: Single-process actor system; all messaging is in-process. Setup: NewActorSystem + Start, no extra options.
• Clustered: What it gives you: Multi-node membership: location transparency, distribution, relocation, cluster singletons. Setup: WithRemote + WithCluster.

​

Standalone mode

• No remoting: actors cannot span multiple processes.
• No clustering: no discovery, no relocation, no cluster singletons, no grain distribution.
• Placement options on SpawnOn and GrainOf are ignored; everything runs locally.
• Single point of failure: a process crash stops everything.

​

Clustered mode

• Discovery: a pluggable provider (Consul, etcd, Kubernetes, NATS, mDNS, static) that tells the cluster how to find peers. See Service Discovery.
• Cluster registry: a distributed, partitioned key/value store (built on Olric) that records where every actor and grain lives. See Partition Hashing.
• Remoting: the TCP transport that carries messages between nodes. See Remoting.

​

Enabling cluster mode

1. Remoting: clustering needs remoting; Start fails with clustering needs remoting to be enabled when WithRemote is missing.
2. A discovery provider, set with WithDiscovery.
3. A discovery port and a peers port, set with WithDiscoveryPort and WithPeersPort.
4. At least one registered actor kind or grain kind, set with WithKinds or WithGrains, so remote spawns, activations, and relocations can instantiate workloads on this node.

remoteConfig := remote.NewConfig("0.0.0.0", 4041)

clusterConfig := actor.NewClusterConfig().
    WithDiscovery(provider).
    WithDiscoveryPort(4042).
    WithPeersPort(4043).
    WithKinds(new(CartActor), new(ProjectionActor)).
    WithGrains(new(UserGrain)).
    WithRoles("entity")

system, err := actor.NewActorSystem("ecommerce",
    actor.WithRemote(remoteConfig),
    actor.WithCluster(clusterConfig),
)
if err != nil {
    return err
}

if err := system.Start(ctx); err != nil {
    return err
}

• Remoting port: Set with: remote.NewConfig. Carries: Actor and grain messages between nodes.
• Discovery port: Set with: WithDiscoveryPort. Carries: Gossip and membership traffic.
• Peers port: Set with: WithPeersPort. Carries: Cluster registry replication.

​

Configuration reference

​

Workload and topology

• WithKinds: Registers actor kinds this node can host (remote spawn, relocation). Default: none.
• WithGrains: Registers grain kinds this node can host (remote activation, recreation, relocation). Default: none.
• WithRoles: Advertises node roles for role-constrained placement. Duplicates are removed; order is not meaningful. Default: none.
• WithGrainActivationBarrier: Delays grain activations until the minimum peers quorum is reached, or until d elapses. 0 waits indefinitely. Reduces churn during startup and rolling deployments. Default: disabled.
• WithDataCenter: Enables the multi-DC control plane. See Multi Datacenter. Default: disabled.
• WithCRDT: Enables CRDT replication through a Replicator system actor. Without it there is zero CRDT overhead. See Distributed Data. Default: disabled.

​

Registry replication and storage

• WithPartitionCount: Number of registry partitions. Should be a prime number. See Partition Hashing. Default: 271.
• WithPartitionHasher: Hash mapping names and identities to partitions. Must be identical, deterministic, and stable on every node. Default: xxh3.
• WithReplicaCount: Copies kept of each partition. The default survives one node loss. Use 3 to tolerate two concurrent losses. Default: 2.
• WithWriteQuorum: Replicas that must acknowledge a registry write. Must be >= 1 and <= replicaCount. Default: 1.
• WithReadQuorum: Replicas consulted per registry read. Must be >= 1 and <= replicaCount. Default: 1.
• WithMinimumPeersQuorum: Members a node must see in its own membership view before the registry serves it: at startup, when the registry map is created, before every registry request received from another node, and before the leader recomputes the routing table. Also the quorum the grain activation barrier waits for. GoAkt has no split-brain resolver, so this setting is the only partition safeguard. With the default of 1, each side of a network split keeps serving, takes its own oldest member as leader, sees the other side’s nodes as departed and relocates their actors, so an actor or a singleton can run on both sides until the split heals. A value above half the cluster size keeps the minority side from recomputing its routing table and from serving registry requests to its peers, so it cannot relocate; it does not stop the minority’s already-running actors. Keep it <= replicaCount. Default: 1.
• WithTableSize: In-memory size of the registry key/value store on this node. Default: 4 MB.
• WithStoreDir: Directory for the node’s cluster store, the BoltDB file that holds peer state while the node runs. The file is created on start and deleted on stop, so an ephemeral writable volume (for example an emptyDir) is enough. Set it when the home directory is not writable, such as a pod with a read-only root filesystem or a sandbox without $HOME. Default: ~/.goakt/cluster.

​

Timing

• WithWriteTimeout: Maximum duration of a registry write before it fails. Default: 1s.
• WithReadTimeout: Maximum duration of a registry read before it fails. Default: 1s.
• WithBootstrapTimeout: Maximum wait for cluster formation during Start; the system fails to start when it elapses. Default: 10s.
• WithShutdownTimeout: Maximum duration of a graceful cluster shutdown. Default: 3m.
• WithClusterStateSyncInterval: How often nodes synchronize routing tables. Keep it greater than the write timeout. Default: 1m.
• WithClusterBalancerInterval: How often the balancer evaluates placement and acknowledges rebalance epochs. Keep it shorter than the state sync interval so epochs do not stack. Default: 1s.
• WithNetworkProfile: Describes the network the cluster nodes share and tunes failure detection for it. NetworkProfileLAN (default) is for nodes in one data center or availability zone. NetworkProfileLocal is for nodes on one host or a low-latency test cluster; it confirms failures fastest and is the most likely to mistake a long pause for a crash. NetworkProfileWAN is for nodes spread across regions or reached over the public internet; it tolerates latency and loss and confirms failures slowly.
• WithConvergenceTimeout: Bounds how long the cluster waits, after it confirmed that a node joined or left, for every node to route on the same membership before it publishes NodeJoined or NodeLeft anyway. Default: 10s. This is not the failure confirmation time, which the network profile sets and which ends before this wait starts; it bounds the convergence that follows, normally under a second and up to a few seconds when the departed node was the oldest member. A shorter bound does not make failures visible sooner and can publish the event while a node still routes to the departed one; to detect failures sooner, pick a faster network profile.

​

Failure detection

​

Registry consistency

​

Placement

​

Relocation

​

Leadership

• ActorSystem.IsLeader: Reports whether the local node is the current cluster coordinator.
• ActorSystem.Leader: Returns the *remote.Peer describing the current cluster leader.

isLeader, err := system.IsLeader(ctx)
if err != nil {
    return err
}
if isLeader {
    // this node is the cluster coordinator
}

for msg := range subscriber.Iterator() {
    if leaderChanged, ok := msg.Payload().(*actor.LeaderChanged); ok {
        log.Printf("cluster leader is now %s", leaderChanged.Address())
    }
}

​

See also

• Placement: where actors and grains run
• Service Discovery: discovery provider options
• Partition Hashing: how names map to registry partitions
• Multi Datacenter: spanning clusters across data centers
• Relocation: node departure and workload redistribution
• Remoting: transport configuration and TLS