Repository navigation
Releases: kpy3/thrift-erl
Release list
v0.7.0
All applications
- The runtime application is renamed to
thrift_runtimeahead of the
first hex.pm release:thrift_erlis already taken on the registry.
Module names change with it (thrift_erl_client→
thrift_runtime_clientand so on); the wire format, public APIs and
behaviour are unchanged. - All three packages are wired for hexdocs.pm: rebar3_ex_doc runs as
the hex doc provider from each package directory (rebar3 ex_doc
is the local dry-run), every package carries its own README, and
scripts/hex-prepublishcopies the project CHANGELOG/LICENSE into
the package directories beforerebar3 hex build/publish. Doc
glitches the docs pipeline surfaced are fixed along the way: broken
t:type references, old-style`X'quoting that renders as
unclosed backquotes (runtime sources and the codec index doc
templates the generator emits), and a%% @moduledoccomment block
inthrift_runtime_jsonthat never reached the compiled docs chunk.
v0.6.0
A consolidation release. The server's specific handshake_timeout
option now actually reaches the accept loop it was documented for; the
generic JSON engine validates unions exactly like binary, compact and
the specialized codecs; README catches up with the generator and runtime
as they are; and the three protocol modules share their cold-path
plumbing instead of carrying three copies. The full gate is green on
both OTP 28 and 29.
Runtime (thrift_runtime)
-
The acceptor's connection-options whitelist silently dropped
handshake_timeout: a server setting the specific key (without the
timeoutumbrella) ran TLS handshakes at the 5000 ms default. The key
now passes the whitelist, and a regression test asserts end to end
that it reaches the accept loop. -
The generic JSON engine's union validation now matches binary, compact
and the specialized codecs: a map with one declared field plus extra
undeclared keys encodes (undeclared keys are ignored, as every
struct-kind always did) instead of answeringunion_multiple, and a
map whose only key matches no declared field answersunion_empty
instead of encoding an empty union object that would not survive a
re-encode. -
The codec probe (
binary_codec/compact_codec/json_codecdispatch
to the specialized per-struct codecs) and the union/required
field-presence checks live inthrift_runtime_protocolonce instead of
three near-identical copies (~200 lines). Thepersistent_termcache
key still names the protocol module and the probe still takes its funs
fresh from the module on every call.
Documentation
- README catches up: the specialized per-struct codecs exist on every
protocol (the Performance section still described them as binary-only),
the four invocation option axes (--repr,--union-type,--set,
--typedef-type) are documented for both the CLI and the rebar3
plugin, theserviceoption (plusmax_queue_depthandname) joins
the start_link option list, the codegen-annotation catalogue grows
erl.union_type = "sum"anderl.struct_type = "native"and documents
the meaningfulkey_type = "binary", and the representations section
says everything once instead of twice.
Generator (thrift_generator)
- Dead code out — unused exports (
thrift_runtime_binary:type_id/1,
thrift_runtime_options:backlog/1,record_mode/1), an inert-docon
the unexportedsources/2, the write-onlyerlfields of four model
records — followed by behavior-preserving simplifications: one
mode/1dispatch instead of four three-boolean products, a shared
namespace-default helper, direct map lookups, and a comments pass
(history narration out, displaced blocks re-attached,-docgaps
filled).
Testing and tooling
- The ssl suite's plain-client case no longer flakes the teardown the
same way the server-events test did: the failed call stops the client
(its design), so the after-block's bare stop races the client's death
and raisesexit:noprocon loaded runners; it now goes through the
suite's quiet-stop helper like every other case with a client
expected dead.
v0.5.0
A name of its own and sharper edges: the project is thrift-erl,
generated service modules carry the file's prefix (no more cross-file
name collisions), declared exceptions surface as typed answers on the
client, IDL errors point at the exact column with a rendered source
excerpt, and the plugin can sweep generated files on project clean.
The CI gate is clean on both OTP 28 and 29.
Breaking changes
- Generated service modules carry the file's identifier prefix and the
snake-cased service name:Pinginnamespace erl ping.apinow
generatesping_api_ping_client/ping_api_ping_server/
ping_api_ping_handlerinstead of'Ping_client'/'Ping_server'/
'Ping_handler'— same-named services of different IDL files no
longer collide. Regenerate and update-behaviourandservice
references. - Declared exceptions answer
{error, {exception, Value}}on the
generated client, and handlers signal them the same way: the value's
representation — record tag,__struct__map, native record or
sum-union tuple — identifies the exception. The explicit-name form
{error, {exception, Name, Value}}stays valid for hand-written
descriptors and map-representation exceptions (plain maps carry no
identity) and remains the rawthrift_runtime_client:call/4wire-level
shape.
Project
- Renamed the project thrift-erl, after the flagship runtime
librarythrift_runtime: it has been a full Thrift stack for Erlang
(IDL compiler, runtime, rebar3 plugin) for a while, not just a
generator. Application and module names are unchanged —thrift_runtime
is baked into all generated code and stays the public contract. The
README now opens with a "Why over the official Apache Thrift
runtime" summary.
rebar3 plugin (rebar3_thrift)
- Generated sources can now be removed on project cleanup:
rebar3 thrift cleandeletes every.erl/.hrlin the configured
out dir carrying the generator'sAutogenerated by Thrift Generator
banner (so orphans of removed services or deleted IDL files go too,
and hand-written files stay), and
{provider_hooks, [{post, [{clean, {thrift, clean}}]}]}in the
consumer'srebar.confighooks the same cleanup intorebar3 clean.
A missing out dir is a no-op, not an error.
Generator (thrift_generator)
- Located errors (lexer/parser and semantic findings) render as
rebar3/elp-style source excerpts by default — the offending line
under a gutter, a╰──marker naming the problem. Positions are
exact: the scanner threads{Line, Column}pairs through tokens,
AST nodes and the semantic model, so the marker sits on the exact
column (codepoint-counted, tab stops expanded like the scanner's)
and the compact form readspath:line:column: message. Unreadable
sources and unlocated errors fall back to the compact
path:line[:column]: messageline, which is also what the new
--compact-errorsCLI switch forces; the rebar3 plugin renders
through the same formatter. - The generated client and handler module docs word the boundary per
the representations the service actually passes —records,maps,
native records,tagged tuplesfor sum unions, or a neutral
clause on a mix — instead of always claiming records under
--repr map/native. Services over no struct-typed types carry no
boundary sentence at all. - Generated modules put an empty line between top-level definitions
(-typeblocks,struct_info/enum_info/function_info, codecs) —
-spec/-callbackcontracts and-doccomments stay flush with
their definition.-doc/-moduledocstrings are wrapped at word
boundaries within 80 columns instead of erl_prettypr's mid-word
cuts, so`start_link/1`-style references stay whole. Regenerate
to pick the new layout up (check/3flags the drift). - IDL keywords are accepted as field names, matching Apache Thrift
0.18+ (THRIFT-5626):1: i32 onewaynames the fieldoneway.
Function, const, enum and type names keep the plain-identifier rule,
as upstream. requiredin athrowsclause is rejected by the semantic pass
(throw field X cannot be required), matching Apache Thrift 0.18+
(THRIFT-5669);optionaland the default requiredness stay valid.
v0.4.0
Specialized codecs for every protocol, an observability surface, and
the operational niceties pools ask for: graceful drains, per-worker
statistics and event callbacks. The bench now measures what generated
code actually runs.
Generator (thrift_generator)
- Specialized compact wire codecs: per struct
encode_<X>_compact/1/
decode_<X>_compact/1with the generic engine's byte-for-byte error
semantics (declaration-order missing-required on encode,
ascending-id on decode,varintreturning[Byte | Rest]so the
term shape — and with it the map-key sort order — stays identical),
plus acompact_codec/1index keyed by the wire struct name.
Equivalence is pinned by PropEr (roundtrip and prefix properties on
both protocols) and a hand-computed negative battery. - Specialized JSON wire codecs: the same contract with the
_json
suffix and ajson_codec/1index. The envelope's closing]folds
only at top-level entry points: the pair's decode side is the
non-folding struct body,thrift_runtime_json:decode/2applies the
fold, and the newthrift_runtime_json:decode_nested/2serves generated
cross-module references fold-free (nested same-module references
are local_bodycalls). Map keys outside the JSON object model
(uuid, struct, list, set, map) answer constanttype_not_supported
errors even when the map is empty; uuid values under a supported
key fail at the pair level. - Plugin:
rebar3 thrift --check(and the CLI--checkflag)
verifies that the generated sources are fresh — the same pipeline
without writing, answering{error, {stale, Paths}}on drift.
write_allskips unchanged files, so regeneration no longer bumps
mtimes of identical outputs (no spurious recompiles).
Runtime (thrift_runtime)
- Observability callbacks: the
on_eventoption (absent, a 1-arity
fun, or{M, F, A}with the event appended to the arguments) on the
client, the reconnecting wrapper (passed down to its child) and the
server (whitelisted through the acceptor into every connection).
Dispatch is synchronous in the emitting process and crash-isolated;
the default absent-handler path costs one pattern match and builds
no event map. Events:call_ok/call_error— exactly one per
caller-facing answer, counters matchingget_stats;connected/
connect_failed/disconnectedon the wrapper's connection axis;
connection_open/connection_closed/handler_error(handler
crashes and unsupported results only — declared exceptions are
protocol answers, not errors) on the server side. The option is
validated at start, not at the first emit. - Graceful server stop:
stop(Server, #{drain => Timeout})closes
the listen socket, tells every live connection to finish its
in-flight request and exit, and answersokonce no connections
remain or the deadline passes. The deadline never fails a call; a
connection stuck past it is reaped by the supervised stop. - Client statistics:
get_stats/1andget_and_reset_stats/1report
#{calls_ok, calls_failed}per worker — the same accounting the
reconnecting wrapper aggregates, without the connection axis. A
pending call closed by a connection failure counts as failed (the
funnel covers terminate and queue-failure paths too). - The reconnecting wrapper's and the server's
start_linkshield the
caller from the failed-init exit race (the same trapping dance the
plain client already did): a bad option answers{error, Reason}
instead of killing the caller with the child's exit signal after
the answer arrived. - Server stop works on OTP 28 again:
supervisor:stop/1only exists
from OTP 29 on, sothrift_runtime_server:stop/1(and the drain path of
stop/2) stops the supervisor as the gen_server it is —
gen_server:stop(Server, normal, infinity), waiting so a busy
connection answers its in-flight request before its own shutdown.
Testing and tooling
- The eunit output stays readable on negative tests: the suites that
start intentionally-bad clients, wrappers and servers quiet the
primary logger with levelnone(SASL crash reports log at level
error, so the previouserrorlevel passed them straight through
to the CI log), and the bad-option observability test silences too. - Both toolchains analyze the native axis cleanly. Elvis parses raw
sources without the preprocessor, so on an OTP-28 toolchain it
cannot resolve the OTP-29 builtinrecord()in exported specs —
thrift_runtime_nativenow exports the spec-level alias
native_record()(opaque;record()on 29,term()on 28)
instead. Dialyzer on 28 keeps the facade's native arms alive the
same way: the release stubs answer through the unfoldable
supported()rather than literals, andnative_to_wire/3is
preprocessor-split per release with the stub keeping the signature
and raise contract. - The elp lint gate fails on findings: the wrapper filters
--severity warningout of the pinned release's JSON (elp itself
always exits 0) and runs the deterministic--no-connectmode, the
same one CI uses. - CI publishes a coverage report; the README gained a
differences-from-Apache section. - PropEr properties for compact and JSON join the binary ones:
roundtrip, prefix (always-morefor binary and compact,
never-completing for JSON) and representation conversion, plus deep
batteries that run all three protocols. - The codec benchmark measures the specialized path for all three
protocols (ref descriptors through the codec probes) against the
official runtime, and adds a generic-vs-specialized block per case —
inline descriptor (the generic interpreter with its per-process
schema memo) against the ref descriptor — with a byte-equality gate
between the two paths before any timing.
v0.3.0
Generated code that reads like it was written: typed records and
IDL-named aliases, section comments, snake_case identifiers, and a
banner that carries the generator version. Unions can now be sums, the
target representation is a generator-level option, and IDL text is
UTF-8 end to end.
Generator (thrift_generator)
- Typed generated code: every named definition exports its original IDL
name as a type alias (-type 'Invoice'(),-type 'Color'());
records carry field types, with every non-required fieldType | undefined(optional and default) and required fields bare; map-mode
structs alias their shape with exact (:=) and associative (=>)
fields; typedefs alias their target (-nominalunder
erl.type = "nominal"); enum members alias the atom union. Type
definitions moved from the.hrlinto<mod>_types.erl; the include
keeps typed record declarations and constant macros only. - Banner on every generated
.erland.hrl:
%% Autogenerated by Thrift Generator (<version>)…DO NOT EDIT,
with the version taken from the running application and blank lines
separating each section comment from the code. - Section comments: every struct, union, exception and enum block — the
record/type forms in both files and the codec function groups —
opens with a divider naming the IDL construct (%% Struct Invoice). - Readable identifiers: derived names go through a snake_case pass
(InvoiceBankAccountgeneratesdomain_invoice_bank_accountand
encode_domain_invoice_bank_account), constant functions snake-case
too and macros uppercase the whole prefixed name (DEMO_KIND_VALUE).
With nonamespace erlthe file base name becomes the prefix, so
same-named structs of namespace-less files cannot collide. - Sum unions:
erl.union_type = "sum"(per union or as a
namespace erl *default) drops the record — the value is
{FieldAtom, Value}, so the exactly-one-field invariant holds by
construction and is dialyzer-visible. The wire format is unchanged,
and the sum representation is repr-invariant: all four views keep
the tagged tuple and convert only the payload. reproption:#{repr => record | map | native}(CLI--repr,
defaultrecord) materializes the chosen representation onto every
struct and exception without an expliciterl.struct_type; an
explicit annotation still wins.nativegenerates no record
declarations at all —record_fields/1exports{native, Tag}
keyed by the snake-cased wire name, the same identity the runtime
mints native records under.- UTF-8 end to end: the IDL file is decoded once as UTF-8 — doc
comments and string constants no longer double-encode, and invalid
input answers{error, {invalid_utf8, Path}}instead of a cryptic
lexer failure. Printable const binaries emit UTF-8 string segments
instead of byte lists. - The redundant
-compile([nowarn_unused_record])is gone from the
types modules: every declared record is referenced by its exported
alias, so the modules compile clean underwarnings_as_errors
without the suppression.
Runtime (thrift_runtime)
- Sum-union support:
record_fields/1understands the
{sum, [FieldAtoms]}mode andtag_indexkeys by field atoms.
Decoding an empty or multi-field union raises{invalid_union, Map},
encoding a value that matches no field raises
{value_does_not_match_type, V}. thrift_runtime_namesmirrors the generator's snake_case pass, so native
record tags,tag_indexkeys and map-mode__struct__tags agree
with the generated identities.- The specialized-codec cache stores only the fact of an export (not
the funs), so eunit module reloads no longer serve purged funs.
v0.2.0: thrift-generator 0.2.0
Specialized wire codecs, a file-wide set default, and the parser fixes
that make real-world official IDL documents parse: # comments in
every position, merged doc blocks, and cross-module typedef
references.
Generator (thrift_generator)
- Specialized binary wire codecs emitted into
<mod>_types.erl:
per struct,encode_<X>_wire/1(wire map to iodata) and
decode_<X>_wire/1(binary to{ok, WireMap, Rest} | more | {error, _}) with byte-for-byte the generic engine's error
semantics, plus abinary_codec/1index keyed by the wire struct
name. Required knowledge is decided at emit time — only required
fields pay the keydelete and missing-scan clauses — so small structs
are the win (Xtruct encode 18 vs 33 reductions/op, decode 16 vs 24)
while deep payloads like Insanity stay at parity, dispatch
amortizing over the walk. - Parser: the official IDL's comment forms in every position —
#
line comments (still text inside string literals, the string rule
wins on length), doc comments beforeinclude/namespace, a
dangling doc at end of input, and consecutive doc blocks merged
into one doc token, as the official compiler does. - Emitter: cross-module references to typedefs resolve. A qualified
name landing on an included file'stypedefused to die with
{unresolved_type, Mod, Name}at descriptor build time; the alias
now resolves in its defining module's context, keeping refs inside
the typedef body home while the result lands in the referencing
module's descriptor. - CLI: arguments parsed with the stdlib
argparsemodule (shipped
inside stdlib since OTP 28, so no new dependency), and the CLI gains
-h/--help. - Sets: a
namespace erl(ornamespace *) header may carry
erl.set = "sets" | "ordset"as the file-wide default, injected
into every set type of the document (theerlscope wins over the
wildcard); record defaults of set-typed fields are wrapped in the
impl constructor. The global default stayslist— parity with
Apache is on the wire. Writingerl.setafter the name is now a
semantic error instead of a silently ignored annotation.
Runtime (thrift_runtime)
- Specialized-codec dispatch: named struct references probe the
generatedbinary_codec/1export (hits and misses cached) and fall
back to the generic engine when it is absent;skip_by_wire/2is
exported for the generated skip clauses. - Set implementations: the record engine covers
list,ordsetand
setsend to end in both directions, including nested sets.
v0.1.0: thrift-generator 0.1.0
Initial release. An alternative Apache Thrift IDL generator and the
thrift_runtime runtime library for Erlang/OTP 28+. Generated code is
binary-compatible with the stock Apache Thrift implementation (verified
both ways by golden interop tests against the official 0.25.0 Erlang
runtime) and depends only on thrift_runtime — kernel and stdlib, no
external dependencies. The generator itself is never a runtime
dependency.
Generator (thrift_generator)
- Thrift IDL pipeline: leex/yecc lexer and parser → semantic analysis →
emitter. Doc comments (/** ... */,///) are carried into the
generated code as-moduledoc/-doc. - Semantic validation: unresolved type and constant references, name
collisions (record tags, enum value prefixes, constants against
generated functions) anderl.*annotation values
(erl.struct_type,erl.struct_type.map.key_type,erl.set). - Output per IDL file:
<mod>_types.erlplus a<mod>_types.hrl
include; per service:<Svc>_client.erl,<Svc>_server.erland the
<Svc>_handler.erlbehaviour. - Per-struct codecs (
encode_x,decode_x,x_from_map,
tag_index/0,convert/2), per-enum int↔atom helpers, constants as
zero-argument functions plus-definemacros for scalars. - Apache-compatible introspection:
struct_info/1,enum_info/1,
function_info/2; plusrecord_fields/1,service_info/0,
tag_index/0,convert/2andsources/1. namespace erl/namespace *: names the generated modules and
prefixes every generated identifier (record tags, type aliases,
function names, macros), so same-named structs of different files
coexist in one compilation. Wire names (struct_info(<<"Point">>))
and enum atoms stay untouched.- Codegen annotations:
erl.struct_type = "map"(with
erl.struct_type.map.key_type),erl.set = "sets" | "ordset"and
erl.type = "nominal"for typedefs.erl.*annotations are consumed
by the generator; all other annotations are preserved. - Distribution: a CLI escript (
thrift_generator generate my_api.thrift -o src) and therebar3 thriftplugin provider (files/out_dir,
typically wired as a pre-compile hook).
Runtime (thrift_runtime)
- Three wire protocols built in —
binary,compact(zigzag varints,
delta-encoded field ids, nibble-packed type tags) andjson(the
Apache JSON wire format) — resolved through thethrift_runtime_protocol
behaviour, so any module can plug into everyprotocolslot. - Four in-process representations behind one facade
(thrift_runtime:encode/decode/convert):record(generated records),
map(atom-key maps),struct(tagged maps) andnative(EEP-79
native records, OTP 29+; older releases answer
{error, {unsupported_repr, native}}). Encode auto-detects the
input; decode is strict — trailing bytes answer{error, trailing},
truncation{error, truncated}. - Transports:
buffered,framedandhttp— one Thrift exchange per
HTTP/1.1 request-response pair with keep-alive; a oneway gets an
empty 200 response; Content-Length framing with body size caps on
both ends. - TLS on both ends:
ssl_optsswitches the socket to
ssl:listen/ssl:connect; the handshake runs in the connect path
under the configured timeouts, and a failed handshake closes the
connection. - Server: supervisor → acceptor → connection processes, event-driven
{active, N}I/O (no passive recv loops), an idle timeout instead of
recv timeouts, andmax_connections/max_frame_sizecaps. - Client: one connection per process with no automatic reconnection —
any failure fails the outstanding calls and stops the process, which
makes it a pool worker out of the box (unlinkedstart/1). Calls are
serialized; the queue is bounded with enqueue-time deadlines;
stop/1answers outstanding calls with{error, closed}. - Reconnecting client wrapper: exponential backoff (
reconnect_min
doubling up toreconnect_max), fail-fast{error, disconnected}
during an outage, and call statistics (get_stats,
get_and_reset_stats) for pool health checks. Server-reported
exceptions are protocol replies and never trigger a reconnect. - Multiplexed services: several services on one port;
multiplex => <<"Service">>prefixes outgoing message names as
Service:function(the Apache multiplexed convention). Dispatch is
strict in multiplexed mode and liberal on a single-service server.
Testing and tooling
- EUnit unit suites, Common Test end-to-end scenarios and PropEr codec
properties. - Golden interop suite against the vendored official Apache Thrift
0.25.0 Erlang runtime, in both directions, over the buffered, framed
and HTTP transports, in the binary, compact and JSON protocols,
including the official multiplexed client. - Codec benchmark (
scripts/bench, erlperf):thrift_runtimeagainst the
official runtime on the same ThriftTest payloads — binary, compact
and JSON, encode and decode (numbers in the README). - Quality gates in CI (OTP 28 and 29 matrix): compile with
warnings_as_errors, elvis, hank, xref, dialyzer, EUnit, Common Test
and elp lint.