Configuration Reference

Marmot v2 uses a TOML configuration file (default: config.toml). All settings have sensible defaults.

Core Configuration

node_id = 0  # 0 = auto-generate
data_dir = "./marmot-data"

Transaction Manager

[transaction]
heartbeat_timeout_seconds = 10  # Transaction timeout without heartbeat
conflict_window_seconds = 10    # Conflict resolution window
lock_wait_timeout_seconds = 50  # Lock wait timeout (MySQL: innodb_lock_wait_timeout)

Note: Transaction log garbage collection is managed by the replication configuration to coordinate with anti-entropy. See replication.gc_min_retention_hours and replication.gc_max_retention_hours.

Connection Pool

[connection_pool]
pool_size = 4              # Number of SQLite connections
max_idle_time_seconds = 10 # Max idle time before closing
max_lifetime_seconds = 300 # Max connection lifetime (0 = unlimited)

gRPC Client

[grpc_client]
keepalive_time_seconds = 10    # Keepalive ping interval
keepalive_timeout_seconds = 3  # Keepalive ping timeout
max_retries = 3                # Max retry attempts
retry_backoff_ms = 100         # Retry backoff duration
compression_level = 1          # zstd compression (0=disabled, 1-4)

Compression Levels:

Level	Speed	Ratio	Use Case
0	-	-	Disabled
1	~318 MB/s	2.88x	Default - best for most deployments
2	~134 MB/s	3.0x	Balanced speed/compression
3	~67 MB/s	3.2x	Better compression, slower
4	~12 MB/s	3.5x	Bandwidth-constrained networks

Decompression is always fast (~1600 MB/s) regardless of compression level. Uses zstd (opens in a new tab) via klauspost/compress (opens in a new tab).

Coordinator

[coordinator]
prepare_timeout_ms = 2000 # Prepare phase timeout
commit_timeout_ms = 2000  # Commit phase timeout
abort_timeout_ms = 2000   # Abort phase timeout

Cluster

[cluster]
grpc_bind_address = "0.0.0.0"
grpc_port = 8080
seed_nodes = []                # List of seed node addresses
cluster_secret = ""            # PSK for cluster authentication (see Security section)
gossip_interval_ms = 1000      # Gossip interval
gossip_fanout = 3              # Number of peers to gossip to
suspect_timeout_ms = 5000      # Suspect timeout
dead_timeout_ms = 10000        # Dead timeout

Replication

[replication]
default_write_consistency = "QUORUM"      # Write consistency level: ONE, QUORUM, ALL
default_read_consistency = "LOCAL_ONE"    # Read consistency level
write_timeout_ms = 5000                   # Write operation timeout
read_timeout_ms = 2000                    # Read operation timeout
 
# Anti-Entropy: Background healing for eventual consistency
# - Detects and repairs divergence between replicas
# - Uses delta sync for small lags, snapshot for large lags
# - Includes gap detection to prevent incomplete data after GC
enable_anti_entropy = true                 # Enable automatic catch-up for lagging nodes
anti_entropy_interval_seconds = 30         # How often to check for lag (default: 30s)
gc_interval_seconds = 60                   # GC interval (MUST be >= anti_entropy_interval)
delta_sync_threshold_transactions = 10000  # Delta sync if lag < 10K txns
delta_sync_threshold_seconds = 3600        # Snapshot if lag > 1 hour
 
# Garbage Collection: Reclaim disk space by deleting old transaction records
# - gc_interval must be >= anti_entropy_interval (validated at startup)
# - gc_min must be >= delta_sync_threshold (validated at startup)
# - gc_max should be >= 2x delta_sync_threshold (recommended)
# - Set gc_max = 0 for unlimited retention
gc_min_retention_hours = 2   # Keep at least 2 hours (>= 1 hour delta threshold)
gc_max_retention_hours = 24  # Force delete after 24 hours

Anti-Entropy Tuning:

Small clusters (2-3 nodes): Use default settings (30s AE, 60s GC)
Large clusters (5+ nodes): Consider increasing AE interval to 60-120s and GC to 2x that value
High write throughput: Increase delta_sync_threshold_transactions to 50000+
Long-running clusters: Keep gc_max_retention_hours at 24+ to handle extended outages

GC Configuration Rules (Validated at Startup):

gc_min_retention_hours must be >= delta_sync_threshold_seconds (in hours)
gc_max_retention_hours should be >= 2x delta_sync_threshold_seconds
Violating these rules will cause startup failure with helpful error messages

Query Pipeline

[query_pipeline]
transpiler_cache_size = 10000  # LRU cache for MySQL→SQLite transpilation

Batch Commit

Controls SQLite write batching for improved throughput. Batches multiple transactions into single SQLite commits, reducing fsync overhead.

[batch_commit]
enabled = true                       # Enable batch committing
max_batch_size = 128                 # Max transactions per batch
max_wait_ms = 10                     # Max wait before flush (ms)
 
# WAL Checkpoint: Manages WAL file size by checkpointing after batches
checkpoint_enabled = true            # Enable automatic checkpointing
checkpoint_passive_thresh_mb = 4.0   # PASSIVE checkpoint when WAL > 4MB
checkpoint_restart_thresh_mb = 16.0  # RESTART checkpoint when WAL > 16MB
 
# Incremental Vacuum: Reclaims freelist pages in time-limited background task
# Requires sqlite_vacuum_incr build tag (included in default build)
incremental_vacuum_enabled = true    # Enable incremental vacuum after checkpoint
incremental_vacuum_pages = 100       # Pages to vacuum per iteration
incremental_vacuum_time_limit_ms = 10 # Max time budget for vacuum (ms)

How Incremental Vacuum Works:

After successful WAL checkpoint, vacuum is triggered in background goroutine
Runs PRAGMA incremental_vacuum(N) in loop until time limit or freelist empty
Non-blocking: doesn't delay commit responses to clients
Prevents goroutine pileup with atomic flag (only one vacuum runs at a time)

Tuning Tips:

High write workload: Decrease incremental_vacuum_time_limit_ms to 5-10ms
Large deletes: Increase incremental_vacuum_pages to 500+ for faster reclamation
Disable: Set incremental_vacuum_enabled = false if using external vacuum scheduling

MySQL Protocol Server

[mysql]
enabled = true
bind_address = "0.0.0.0"
port = 3306
max_connections = 1000
auto_id_mode = "compact"  # "compact" (53-bit, default) or "extended" (64-bit)

Auto ID Mode:

"compact" (default): Generates 53-bit IDs that are safe for all clients including JavaScript (Number.MAX_SAFE_INTEGER). Format: 41 bits timestamp + 6 bits node + 6 bits sequence. Supports ~69 years from epoch (Jan 2, 2025) with 64K IDs/sec per node.
"extended": Generates full 64-bit HLC-based IDs for maximum uniqueness and ordering guarantees. Use when all clients support 64-bit integers natively.

Logging

[logging]
verbose = false          # Enable verbose logging (debug level)
format = "json"          # Log format: "console" or "json" (json is 21% faster)
file = ""                # Log file path (empty = stdout only)
max_size_mb = 100        # Max size in MB before rotation (default: 100)
max_backups = 5          # Number of old log files to keep (default: 5)
compress = true          # Compress rotated files with gzip (default: true)

File Logging with Rotation:

When file is set, logs are written to both stdout and the specified file. The file is automatically rotated when it reaches max_size_mb, keeping the last max_backups files.

[logging]
file = "/var/log/marmot/marmot.log"
max_size_mb = 100
max_backups = 5
compress = true

CLI Flags

Marmot supports the following command-line flags that override config file settings:

Flag	Description
`-config`	Path to configuration file (default: `config.toml`)
`-data-dir`	Data directory (overrides `data_dir` in config)
`-node-id`	Node ID (overrides `node_id` in config, 0=auto)
`-grpc-port`	gRPC port (overrides `cluster.grpc_port` in config)
`-mysql-port`	MySQL port (overrides `mysql.port` in config)
`-follow-addresses`	Comma-separated addresses for replica mode
`-daemon`	Run as background daemon (detached from terminal)
`-pid-file`	PID file path (used with `-daemon`)

Example:

./marmot -config=/etc/marmot/config.toml -data-dir=/var/lib/marmot -grpc-port=8081 -mysql-port=3307

Running as Daemon:

# Start as daemon with PID file
./marmot -config=/etc/marmot/config.toml -daemon -pid-file=/var/run/marmot.pid
 
# Check if running
cat /var/run/marmot.pid
 
# Stop daemon
kill $(cat /var/run/marmot.pid)

Note: When running as daemon, configure logging.file in config to capture logs since stdout/stderr are detached.

Prometheus Metrics Demo