A binary format to replace
Protobuf and JSON.
Most systems have to choose between Protobuf (compact binary payloads, but unreadable without a schema and slowed down by byte-by-byte LEB128 varints) and JSON (works on plain structs and is inspectable anywhere, but is over 2x larger and 10x to 30x slower).
OmniBuf replaces both. Use it with .proto files or directly on plain structs and objects: it runs 6x to 8x faster than Protobuf (and 10x to 30x faster than JSON) while remaining inspectable without a schema.
Faster, inspectable, and spliceable.
You shouldn't have to give up .proto files or switch to pointer-heavy zero-copy formats just to get fast serialization and readable packets.
No byte-by-byte varint loops. No two-pass encoding
Protobuf's LEB128 varints check a continuation bit on every byte, and its length-prefixed sub-messages force encoders to walk the struct tree twice (once to compute sizes, once to write bytes). OmniBuf puts the integer byte width in the first byte (TZCNT) so the CPU reads the whole integer at once without looping over each byte, and reserves 2-byte length slots so messages encode in a single pass.
Inspect packets without a schema & use normal structs
Protobuf lumps strings, byte slices, sub-messages, maps, and packed arrays into a single wire type (Wire Type 2). Without the .proto file, a decoder can only guess what a payload contains. OmniBuf uses a 4-bit wire type (~6% larger on the wire) so every type is explicit. Any CLI or log viewer can decode packets without a schema, and you still work with normal structs instead of FlatBuffers builders.
Read, drop, or replace individual fields on raw bytes
Because every integer and container carries its exact byte width in its header, OmniBuf can jump over unrelated fields in a single addition without parsing their contents. Generated Opaque views let routers and storage engines read a single nested field or splice updated scalar values directly into a serialized packet over 11x faster than Protobuf with zero heap allocations.
LocateField • SpliceFields • 0 allocsKeep your .proto files, schema evolution, and compact payloads
OmniBuf works directly with your existing .proto files and generated structs. You keep field numbers, forward/backward compatibility, compact payloads (40% smaller than FlatBuffers), and raw byte concatenation (append(msgA, msgB...)) for merging messages without decoding them.
All the benefits of Protobuf.
Measured on an 11-field message with scalars, strings, packed arrays, and nested messages.
| Format | Uses .proto | Wire Size | 1-Pass Encode | Schema-Less Types | append(a, b) Merge |
|---|---|---|---|---|---|
| OmniBuf | Yes | Baseline (1x) | Yes (6x faster) | Yes (+ opt Field #0) | Yes |
| Protobuf | Yes | 6% smaller | No (2-pass Size) | No (Opaque Type 2) | Yes |
| FlatBuffers | No (.fbs) | +66% larger | Bottom-up only | No | No |
| Cap'n Proto | No (.capnp) | +55% larger | Arena pointers | No | No |
| MessagePack | Optional | +81% larger | Yes (Slow) | Yes (Repeated Keys) | No |
| JSON | Optional (ProtoJSON) | +122% larger | Yes (Slow) | Yes (Repeated Keys) | No |
Frequently asked questions.
Why is it named OmniBuf?
The name reflects a more complete Protobuf that goes further than the original wire encoding. Standard Protobuf requires a schema just to make sense of raw bytes, and it will never compromise on carrying field metadata on the wire.
OmniBuf payloads can always be parsed into explicit types without a schema, and can optionally embed field names (Field #0) when you want full named JSON output straight from a binary packet.
Why a new format?
The main goal is simple: we want a better Protobuf, and showing a concrete re-imagining of the wire format can help push the ecosystem further.
Protobuf is generally fine, and JSON is generally fine, but both have rough edges. Protobuf's 2001 wire format is slow to decode and opaque in logs; JSON is bulky and breaks on 64-bit integers and binary data. At the same time, Protobuf's IDL methodology is genuinely great: schema-first development with explicit field numbers lets systems adapt and evolve at a much faster pace. OmniBuf keeps the .proto schema workflow intact and shows what becomes possible when you modernize the wire encoding underneath it.
Do I have to rewrite my .proto files?
No. OmniBuf uses standard proto3 and Editions (2023, 2024) field numbers, scalar types, nested messages, repeated lists, and maps without requiring a custom IDL or builder API.
What languages are supported?
Go and TypeScript / JavaScript are currently supported:
Go supports plain Go structs with omnibuf:"..." struct tags, existing .pb.go structs, and protoc-gen-omnibuf code generation.
TypeScript / JavaScript (@omnibuf/core, @omnibuf/web, @omnibuf/node) supports declarative defineMessage schemas, existing @bufbuild/protobuf descriptors, and protoc-gen-omnibuf-es code generation.
The wire specification is language-agnostic, with additional language implementations planned as the specification stabilizes.