OmniBuf
for Go.
Reference runtime and code generator for Go: serialize plain Go structs, drop in on existing .pb.go types, or generate single-pass codecs and lazy Opaque views.
$ go get github.com/sudorandom/omnibufThree ways to use OmniBuf in Go.
Use plain Go structs with struct tags, pass existing .pb.go types with no build changes, or generate single-pass methods and lazy Opaque views with the compiler plugin.
Existing .pb.go structs
Already generating Go code with protoc-gen-go? Pass your existing .pb.go structs directly to omnibuf.Marshal and omnibuf.Unmarshal with no extra plugin step.
OmniBuf reads the existing protobuf:"..." struct tags once per type, builds a cached pointer-offset table, and runs ~2.5x to 3x faster than standard Go Protobuf (and 4x to 11x faster than JSON and MessagePack) with zero heap allocations on reused structs.
// Encode any existing *pb.Peer (.pb.go) struct with zero extra codegen:
pkt, err := omnibuf.Marshal(msg)
// Decode back into a reused or new *pb.Peer struct:
var out pb.Peer
err = omnibuf.Unmarshal(pkt, &out)Generated code with protoc-gen-omnibuf
Add protoc-gen-omnibuf to your buf.gen.yaml (supports both proto3 and Editions 2023/2024 schemas). The plugin attaches single-pass MarshalOmniBufAppend and UnmarshalOmniBuf methods plus lazy Opaque views directly onto your .pb.go structs.
Generated codecs run ~6x faster on encode and ~8x faster on decode than standard Go Protobuf, and beat vtprotobuf by over 50% while generating less than half the code.
version: v2
plugins:
- local: protoc-gen-go
out: gen
opt: paths=source_relative
- local: protoc-gen-omnibuf
out: gen
opt: paths=source_relative$ go install github.com/sudorandom/omnibuf/cmd/protoc-gen-omnibuf@latest
$ buf generate// Allocate a new slice, or append to a pooled buffer (buf[:0]):
pkt := msg.MarshalOmniBufAppend(nil)
// Decode back into *pb.Peer (pass true to alias strings zero-copy):
var out pb.Peer
err := out.UnmarshalOmniBuf(pkt, false)Go questions.
When should I use Opaque lazy views instead of Unmarshal?
Use Unmarshal when your handler reads most fields of a message. Use generated Opaque views in routers, proxies, and ingestion pipelines that only inspect a header field or update a single value before forwarding the packet.
Getters hop directly to the target field with zero heap allocations, and setters splice updated fields onto the raw byte slice without decoding the rest of the message.
view := pb.PeerOpaque(pkt)
router, ok := view.Router() // Zero-alloc header hop
// Update field #3 (Active) in-place without decoding the message:
updated := view.SetActive(false)How do I enable Field #0 inline field names or decode without a schema?
Pass omnibuf.MarshalOptions{SelfDescribing: true} when marshaling any struct or .pb.go message to prepend Field #0 with the root and nested field-name tables.
To inspect a packet without a target struct or schema file, call omnibuf.DecodeBlind(pkt). It returns a map keyed by field numbers, or by field names automatically when Field #0 is present.
// Embed Field #0 (SchemaMeta) inline with the packet:
pkt, err := (omnibuf.MarshalOptions{SelfDescribing: true}).Marshal(&peer)
// Decode any OmniBuf packet without a schema (uses Field #0 names if present):
fields, err := omnibuf.DecodeBlind(pkt)What does the zeroCopy flag do, and how do I pool buffers?
The second argument to UnmarshalOmniBuf(pkt, zeroCopy) controls string allocation. Passing false copies strings onto the heap so the packet buffer can be reused right away; passing true points string headers directly into the packet buffer for zero-allocation reads.
On the encode path, pass a pooled slice via MarshalOmniBufAppend(buf[:0]) to serialize messages without allocating a new buffer.
// Reuse an existing byte slice capacity across encodes (0 allocs):
buf = msg.MarshalOmniBufAppend(buf[:0])
// Pass zeroCopy=true when pkt outlives out to alias strings directly:
err := out.UnmarshalOmniBuf(buf, true)How does protoc-gen-omnibuf compare to vtprotobuf?
vtprotobuf speeds up Go Protobuf by generating unrolled varint loops and a separate size-calculation pass before writing backwards into the buffer, because Protobuf requires knowing every sub-message length up front.
OmniBuf reserves a 2-byte length slot and fills it in after writing, and reads integers in one step instead of a byte-by-byte loop. As a result, protoc-gen-omnibuf runs over 50% faster on Marshal and over 2x faster on Read-Modify-Write while generating less than half the code.