OmniBuf
for Rust.
Supports plain Rust structs via derive macros, existing buffa and prost Protobuf messages, and Buf code generation. Borrowed packet views read nested fields with zero heap allocation and splice updates in place. All Protobuf crate dependencies are optional.
$ cargo add omnibuf# Default: derive + buffa + prost
omnibuf = "0.1"
# buffa + #[derive(OmniBuf)] only
omnibuf = { version = "0.1", default-features = false, features = ["derive", "buffa"] }
# Pure wire, opaque splicing & blind decode
omnibuf = { version = "0.1", default-features = false }Ways to use OmniBuf in Rust.
Every integration path produces identical wire bytes.
Procedural #[derive(OmniBuf)] structs
#[derive(OmniBuf)] expands straight into single-pass Prefix-Varint writes and unrolled tag-switch loops at compile time. Zero .proto build scripts are needed. Field attributes declare wire IDs, ZigZag or fixed-width overrides (wire = "sint", "fixed64"), oneof variant tags, and Field #0 names.
use omnibuf::{Message, OmniBuf};
#[derive(Debug, Default, PartialEq, OmniBuf)]
#[omnibuf(name = "PeerUpdate")]
pub struct PeerUpdate {
#[omnibuf(id = 1, name = "asn")]
pub asn: u32,
#[omnibuf(id = 2, name = "router_id")]
pub router_id: String,
#[omnibuf(id = 3, name = "active")]
pub active: bool,
#[omnibuf(id = 5, name = "rssi_dbm", wire = "sint")]
pub rssi_dbm: i32,
#[omnibuf(id = 7, name = "tags")]
pub tags: Vec<String>,
}
let pkt = peer.encode_omnibuf()?;
let out = PeerUpdate::from_omnibuf(&pkt)?;Protobuf messages with buffa
First-class support for anthropics/buffa (features = ["buffa"], active by default) covers both static co-derivation and runtime reflection. Attaching #[derive(::omnibuf::OmniBuf)] via buffa_build::Config::message_attribute and oneof_attribute parses /// Field N: doc comments, unwraps buffa::MessageField<T>, buffa::EnumValue<E>, and Option<...> oneofs, and emits an unrolled zero-reflection codec.
Pre-compiled buffa crates keep their build.rs untouched through omnibuf::buffa_codec. encode_buffa_reflect walks Reflectable vtables directly in memory with zero intermediate Protobuf buffers, while encode_buffa and decode_buffa transcode against any Arc<buffa_descriptor::DescriptorPool>.
buffa_build::Config::new()
.files(&["omnibuf/bench/peer.proto"])
.includes(&["proto"])
.generate_views(true)
.generate_reflection(true)
.message_attribute(".", "#[derive(::omnibuf::OmniBuf)]")
.oneof_attribute(".", "#[derive(::omnibuf::OmniBuf)]")
.compile()?;use omnibuf::{EncodeOptions, Message};
use omnibuf::buffa_codec::{decode_buffa, encode_buffa, encode_buffa_reflect};
// Co-derived #[derive(OmniBuf)] on buffa structs
let pkt = buffa_peer.encode_omnibuf()?;
let direct = buffa_pb::PeerUpdate::from_omnibuf(&pkt)?;
// Pre-compiled buffa crates via Reflectable vtables or DescriptorPool
let pkt = encode_buffa_reflect(&buffa_peer, EncodeOptions::default())?;
let pkt_pool = encode_buffa(&buffa_peer, &pool, EncodeOptions::default())?;
let reflected: buffa_pb::PeerUpdate = decode_buffa(&pkt, &pool)?;Protobuf messages with prost
Workspaces built around prost and tonic pair static co-derivation with runtime descriptors (features = ["prost"], active by default). Passing #[derive(::omnibuf::OmniBuf)] to prost_build::Config::type_attribute compiles #[prost(...)] field tags into monomorphized Message and Oneof impls. Upstream crates implementing prost_reflect::ReflectMessage or DynamicMessage pass directly into encode_prost and decode_prost.
use omnibuf::{EncodeOptions, Message};
use omnibuf::prost_codec::{decode_prost, encode_prost};
// Co-derived #[derive(prost::Message, OmniBuf)] via prost_build::Config::type_attribute
let pkt = prost_peer.encode_omnibuf()?;
let direct = pb::PeerUpdate::from_omnibuf(&pkt)?;
// Pre-compiled prost + prost-reflect crates
let pkt = encode_prost(&prost_peer, EncodeOptions::default())?;
let mut reflected = pb::PeerUpdate::default();
decode_prost(&pkt, &mut reflected)?;Generated views with protoc-gen-omnibuf-rs
Pointing buf.gen.yaml at protoc-gen-omnibuf-rs generates owned Rust structs paired with lifetime-bound Opaque<'a> views. Getters hop across preceding field headers by byte length and borrow &str and &[u8] straight from the packet slice. try_new and try_marshal validate untrusted buffers up front, while splice_fields rewrites dirty fields in place and copies clean spans verbatim.
version: v2
plugins:
- local: protoc-gen-omnibuf-rs
out: src/gen
strategy: all$ cargo install protoc-gen-omnibuf-rs
$ buf generateuse crate::gen::omnibuf::bench::{OpaquePeerUpdate, PeerUpdate};
use omnibuf::Message;
let pkt = peer.encode_omnibuf()?;
let mut view = OpaquePeerUpdate::try_new(&pkt)?;
if view.get_active() {
let city: &str = view.get_location().get_city();
view.set_sequence(view.get_sequence() + 1);
let updated: Vec<u8> = view.try_marshal()?;
}Schema-less inspection with decode_blind
Every OmniBuf tag carries an explicit 4-bit wire type. decode_blind and decode_blind_json inspect arbitrary wire slices with zero compile-time schemas, emitting synthetic field_1, field_2, ... keys with exact integer signs, booleans, IEEE-754 -0.0, UTF-8 strings, and Base64 byte slices. Passing EncodeOptions::self_describing() prepends Field #0 (SchemaMeta) so keys resolve to declared field names.
use std::collections::BTreeMap;
use omnibuf::{BlindValue, EncodeOptions, Message, decode_blind, decode_blind_json};
let pkt = peer.encode_omnibuf_with_options(EncodeOptions::self_describing())?;
let fields: BTreeMap<String, BlindValue> = decode_blind(&pkt)?;
let json: String = decode_blind_json(&pkt)?;Rust benchmarks.
11-field PeerUpdate struct on Apple M1 Pro (rustc 1.94.0) across omnibuf, buffa, prost, rmp-serde, and serde_json. Shorter bars = Faster.
Serializing PeerUpdate into a freshly allocated Vec<u8> across every library.
Decoding &[u8] into a fresh owned PeerUpdate struct and reading all 11 fields.
Borrowing a nested &str directly from &[u8] via OpaquePeerUpdate vs buffa PeerUpdateView vs full decode.
Checking a nested city string and splicing an incremented 64-bit sequence counter into &[u8] vs decode and re-encode.
Serializing a 39 KiB packet into Vec::with_capacity(128). Two-pass encoded_len() pre-sizing in buffa and prost avoids single-pass 2B-to-4B header shifts.
Growing a nested string field past 16 KiB ahead of 32 trailing Neighbor entries via Opaque splicing vs full decode and re-encode.
Benchmark Environment
Compiled with rustc 1.94.0 in release mode on an Apple M1 Pro Mac. Encode benchmarks start from identical Vec::with_capacity(128) buffers, and decode benchmarks read all 11 fields into a black_box checksum sink.
Benchmark source lives in crates/omnibuf-bench/src/main.rs, with recorded output in testdata/benchmarks.json.
Rust questions.
How do buffa *View<'a> types compare with OmniBuf Opaque<'a> views?
buffa generates PeerUpdateView<'a> structs that parse a Protobuf buffer in one upfront pass, populating borrowed &'a str and &'a [u8] slices across every field. OpaquePeerUpdate<'a> wraps the raw OmniBuf slice as a Cow<'a, [u8]> and defers decoding until individual getter calls, stepping over earlier length-prefixed containers in O(1) time per field.
Pipelines that read most fields of an incoming Protobuf message benefit from buffa's single-pass *View<'a> borrow. Routing planes and telemetry proxies that inspect one nested header or overwrite a single sequence counter skip full-struct materialization with Opaque<'a> and splice_fields.
What differs under the hood between buffa and prost integration?
prost encodes schema metadata inside #[prost(...)] field attributes, boxes recursive messages in Option<Box<T>>, and stores enums as bare i32 values. buffa emits /// Field N: `name` doc attributes, wraps singular sub-messages in buffa::MessageField<T>, preserves unknown enum discriminants via buffa::EnumValue<E>, and exposes zero-allocation Reflectable vtables.
#[derive(OmniBuf)] recognizes both field layouts natively at macro expansion time. At runtime, buffa_codec::encode_buffa_reflect reads buffa structs in place through their Reflectable vtable, whereas prost_codec::encode_prost converts through prost_reflect::DynamicMessage.
How are untrusted network payloads bounded before allocation?
DecodeOptions caps packet byte length (max_message_bytes), cumulative array and map element counts (max_elements), total heap reservation (max_alloc_bytes), and recursion depth (max_depth, capped at 32). All four checks run ahead of owned value allocation in from_slice_with_options, decode_buffa_with_options, decode_prost_with_options, and decode_blind_with_options, verified across 758/758 conformance cases.
Can crates depend on omnibuf with zero Protobuf dependencies?
Setting default-features = false disables derive, buffa, and prost. Only the core Prefix-Varint wire encoder, zero-copy decoder, Field #0 schema reader, opaque::splice_fields, and decode_blind compile, leaving itoa and ryu as the sole dependencies. Adding features = ["derive"] re-enables #[derive(OmniBuf)] for standalone Rust structs.