Keep your .proto files.
Upgrade the wire format.
Protobuf got schemas right, mostly, but the wire format from 2001 made two costly mistakes: it requires the schema just to decode values, and its byte-by-byte LEB128 varints are slow as hell with real hardware.
OmniBuf gives you all the benefits of Protobuf (your existing .proto files, compact binary payloads, schema evolution, and raw byte concatenation) in a wire format that is 6x to 8x faster and inspectable without a schema.
Faster, inspectable, and spliceable.
You shouldn't have to give up .proto files or switch to pointer-heavy zero-copy formats just to get fast serialization and readable packets.
No byte-by-byte varint loops. No two-pass encoding
Protobuf's LEB128 varints check a continuation bit on every byte, and its length-prefixed sub-messages force encoders to walk the struct tree twice (once to compute sizes, once to write bytes). OmniBuf puts the integer byte width in the first byte (TZCNT) so the CPU reads the whole integer at once without looping over each byte, and reserves 2-byte length slots so messages encode in a single pass.
Inspect packets without a schema & use normal structs
Protobuf lumps strings, byte slices, sub-messages, maps, and packed arrays into a single wire type (Wire Type 2). Without the .proto file, a decoder can only guess what a payload contains. OmniBuf uses a 4-bit wire type (~6% larger on the wire) so every type is explicit. Any CLI or log viewer can decode packets without a schema, and you still work with normal structs instead of FlatBuffers builders.
Read, drop, or replace individual fields on raw bytes
Because every integer and container carries its exact byte width in its header, OmniBuf can jump over unrelated fields in a single addition without parsing their contents. Generated Opaque views let routers and storage engines read a single nested field or splice updated scalar values directly into a serialized packet over 11x faster than Protobuf with zero heap allocations.
LocateField • SpliceFields • 0 allocsKeep your .proto files, schema evolution, and compact payloads
OmniBuf works directly with your existing .proto files and generated structs. You keep field numbers, forward/backward compatibility, compact payloads (40% smaller than FlatBuffers), and raw byte concatenation (append(msgA, msgB...)) for merging messages without decoding them.
All the benefits of Protobuf.
Measured on an 11-field message with scalars, strings, packed arrays, and nested messages.
| Format | Uses .proto | Wire Size | 1-Pass Encode | Schema-Less Types | append(a, b) Merge |
|---|---|---|---|---|---|
| OmniBuf | Yes | Baseline (1x) | Yes (6x faster) | Yes (+ opt Field #0) | Yes |
| Protobuf | Yes | 6% smaller | No (2-pass Size) | No (Opaque Type 2) | Yes |
| FlatBuffers | No (.fbs) | +66% larger | Bottom-up only | No | No |
| Cap'n Proto | No (.capnp) | +55% larger | Arena pointers | No | No |
| MessagePack | Optional | +81% larger | Yes (Slow) | Yes (Repeated Keys) | No |
| JSON | Optional (ProtoJSON) | +122% larger | Yes (Slow) | Yes (Repeated Keys) | No |
Frequently asked questions.
Why is it named OmniBuf?
The name reflects a more complete Protobuf that goes further than the original wire encoding. Standard Protobuf requires a schema just to make sense of raw bytes, and it will never compromise on carrying field metadata on the wire.
OmniBuf payloads can always be parsed into explicit types without a schema, and can optionally embed field names (Field #0) when you want full named JSON output straight from a binary packet.
Why a new format?
The main goal is simple: we want a better Protobuf, and showing a concrete re-imagining of the wire format can help push the ecosystem further.
Protobuf is generally fine, and JSON is generally fine, but both have rough edges. Protobuf's 2001 wire format is slow to decode and opaque in logs; JSON is bulky and breaks on 64-bit integers and binary data. At the same time, Protobuf's IDL methodology is genuinely great: schema-first development with explicit field numbers lets systems adapt and evolve at a much faster pace. OmniBuf keeps the .proto schema workflow intact and shows what becomes possible when you modernize the wire encoding underneath it.
Do I have to rewrite my .proto files?
No. OmniBuf uses standard proto3 and Editions (2023, 2024) field numbers, scalar types, nested messages, repeated lists, and maps without requiring a custom IDL or builder API.
What languages are supported?
Right now, Go is implemented (supporting plain Go structs with omnibuf:"..." struct tags, existing .pb.go structs, and protoc-gen-omnibuf code generation). The wire specification is language-agnostic and built around standard little-endian loads and hardware bit-scans, with additional language implementations planned as the specification stabilizes.