Warp is a data orchestration and synchronization layer for distributed Go backends.
It sits between your application and your primary storage (Database, API), providing:
- L1 Caching (In-Memory)
- Distributed Invalidation (via NATS, Kafka, etc.)
- Consistency Modes (Strong Local, Eventual Distributed, Strong Distributed)
- Distributed Locking & Leases
- Conflict Resolution (Merge Engine)
- Request Coalescing: Built-in
singleflightprotection against Thundering Herd. - Adaptive Batching: Automatically groups and compresses invalidation events under high load.
- Zero-Copy Serialization: Optional
ByteCodecfor maximum throughput. - Polyglot Sidecar: Connect standard Redis clients (Node.js, Python, etc.) via
warp-proxy.Note: The Proxy is optimized for Consistency and Interoperability, providing access to Warp's unified state from any language. For raw millisecond-level throughput in Go, use the Core library directly.
Most caching libraries are just... caches. You Set a key, and it sits there until it expires. In a distributed system, this leads to stale data.
Warp is different. It treats data as a living entity that needs to be synchronized across your cluster.
Results from cmd/bench/ suite running on a standard developer machine (Docker/Podman):
| System | Mode | Ops/sec | Avg Latency | P99 Latency |
|---|---|---|---|---|
| Warp | StrongLocal | ~2,700,000 | 370 ns | 0.6 ms |
| Warp | Eventual (Mesh) | ~3,000,000 | 330 ns | 0.6 ms |
| Warp | Eventual (L2=Redis) | ~2,400,000 | 400 ns | 0.8 ms |
| Ristretto | Local Cache | ~12,000,000 | 80 ns | 0.004 ms |
| Redis | Remote | ~55,000 | 18 µs | 1.6 ms |
| DragonFly | Remote | ~118,000 | 8.5 µs | 1.1 ms |
| Warp Proxy | Sidecar (Node.js) | ~170,000 | - | Pipelined (Batch=50) |
| Warp Proxy | Sidecar (Python) | ~80,000 | - | Pipelined (Batch=50) |
Takeaway: Warp allows you to access data ~10-20x faster than standard Redis by serving reads from memory (L1) while maintaining eventual consistency across the cluster.
- Need speed? Use
ModeStrongLocal. - Need consistency? Use
ModeEventualDistributedand Warp will automatically notify other nodes when you update data. - Need coordination? Use
ModeStrongDistributedto ensure writes are acknowledged by a quorum of nodes. - Need to group data? Use Leases to invalidate a user's entire session cache with one call.
- Getting Started: Build a User Session Store in 5 minutes.
- Presets: Production-ready configurations (
NewRedisEventual, etc.). - Core Concepts: Understand Consistency Modes and Architecture.
- Best Practices: Recommended patterns for common scenarios.
- Modules:
- Cache: Adaptive TTL, Pluggable Backends.
- Sync Bus: NATS, Kafka, Warp Mesh (UDP/Gossip), Custom Adapters.
- Lock: Distributed Locking.
- Leases: Group Invalidation.
- Merge: Conflict Resolution.
- Validator: Background Consistency Checks.
- Federation: Multi-region synchronization.
- Sidecar: Polyglot proxy (RESP protocol).
- Glossary: Terminology definition.
- Warp Store (JS): A high-performance, dual-layer (Memory + IndexedDB) caching store for frontend applications. Supports offline mode, session restoration, and is designed to work seamlessly with Warp's backend concepts.
- Warp Sidecar: A polyglot proxy that allows you to access Warp's unified state from any language.
go get github.com/mirkobrombin/go-warp/v1package main
import (
"context"
"fmt"
"time"
"github.com/mirkobrombin/go-warp/v1/core"
"github.com/mirkobrombin/go-warp/v1/presets"
)
type User struct {
Name string
}
func main() {
// 1. Setup with Preset (Production-ready)
// Automatically wires In-Memory L1 + Redis L2 + Redis Bus
w := presets.NewRedisEventual[User](presets.RedisOptions{
Addr: "localhost:6379",
})
// 2. Register Key
w.Register("user:*", core.ModeEventualDistributed, 10*time.Minute)
// 3. Use
ctx := context.Background()
w.Set(ctx, "user:123", User{Name: "Alice"})
val, _ := w.Get(ctx, "user:123")
fmt.Println(val.Name)
}Warp Mesh allows you to cluster your Go backends without Redis, NATS, or Kafka. It uses UDP Multicast for local discovery and a Unicast Gossip protocol for cloud environments.
w := presets.NewMeshEventual[User](mesh.MeshOptions{
Port: 7946,
Peers: []string{"10.0.0.1:7946"}, // Static seeds for cloud
})MIT License. See LICENSE for details.
I am using Gemini, to help me write the documentation and debug issues, since the cocebase is really large. I want to make sure I am not missing anything important as this library aims to be used in critical production systems.
I am just honest and transparent. Peace and love.