Skip to content

@sbinet sbinet released this Jan 15, 2019 · 69 commits to master since this release


Release v0.17.1 is fresh-ish from the oven.

This new release continues the refactoring and consolidation work of the groot package (meant to replace rootio.)


  • brio/cmd/brio-gen gained some documentation


  • groot/rdict has seen a bunch of work to create user-type StreamerInfos,
  • groot/cmd/root-gen-streamer: new command to automatically generate a StreamerInfo and its StreamerElements given a ROOT or user type,
  • groot/rdict now properly handles streamers with arrays of builtins, and properly visits std::vector<T> fields (where T is a builtin)
  • groot/cmd/root-dump: now properly handle root.List values
  • groot dropped the use of gofrs/uuid and replaced it with hashicorp/go-uuid. gofrs/uuid dropped support for Go modules and broke the build of Go-HEP.

Here is usage example of the new root-gen-streamer command:

$> root-gen-streamer -help
Usage: root-gen-streamer [options]

 $> root-gen-streamer -p image -t Point -o streamers_gen.go
 $> root-gen-streamer -p -t Dist0D,Dist1D,Dist2D -o foo_streamer_gen.go

  -o string
    	output file name
  -p string
    	package import path
  -t string
    	comma-separated list of type names
  -v	enable verbose mode


The xrootd/client and xrootd/server packages have been merged into a single package, xrootd.
This simplified the import structure as well as reduced the amount of boilerplate code that was duplicated between the two packages.

For users of xrootd/client:

diff --git a/xrootd/cmd/xrd-ls/main.go b/xrootd/cmd/xrd-ls/main.go
index 784a5c7..9961802 100644
--- a/xrootd/cmd/xrd-ls/main.go
+++ b/xrootd/cmd/xrd-ls/main.go
@@ -31,7 +31,7 @@ import (
-       xrdclient ""
+       ""
@@ -93,7 +93,7 @@ func xrdls(name string, long, recursive bool) error {
        ctx := context.Background()
-       c, err := xrdclient.NewClient(ctx, url.Addr, url.User)
+       c, err := xrootd.NewClient(ctx, url.Addr, url.User)
        if err != nil {
                return errors.Errorf("could not create client: %v", err)

For users of xrootd/server:

diff --git a/xrootd/cmd/xrd-srv/main.go b/xrootd/cmd/xrd-srv/main.go
index 2a09dbd..230bf31 100644
--- a/xrootd/cmd/xrd-srv/main.go
+++ b/xrootd/cmd/xrd-srv/main.go
@@ -14,7 +14,7 @@ import (
-       ""
+       ""
 func init() {
@@ -56,7 +56,7 @@ func main() {
                log.Fatalf("could not listen on %q: %v", *addr, err)
-       srv := server.New(server.NewFSHandler(baseDir), func(err error) {
+       srv := xrootd.NewServer(xrootd.NewFSHandler(baseDir), func(err error) {
                log.Printf("an error occured: %v", err)


Support for writing TTrees didn't make it under the X-mas tree.
This has been converted to a NYE resolution, though.

Before tackling this big item, support for reading TClonesArray is on the way (and tracked in the sbinet/hep#issue-419 branch.)

A proposal for data frames is in the works in the gonum/exp repository.
Feel free to comment on the associated pull request (#19) or on the gonum-dev forum.

Finally, this is GSoC proposal season.
I (@sbinet) will probably send a GDML oriented proposal.
Feel free to send or discuss yours on the go-hep mailing list.

Stay tuned! (and, as always, any kind of help (reviews, patches, nice emails, constructive criticism) deeply appreciated.)

Assets 2
Dec 22, 2018
groot/cmd/root-dump: handle root.List

@sbinet sbinet released this Dec 3, 2018 · 124 commits to master since this release


Release v0.16.0 is fresh from the oven.

This new release introduces a new package, groot, that replaces rootio.

groot & rootio


groot is a new package that is meant to replace rootio.
For backward compatibility, rootio will remain in the repository for a couple of releases (presumably until v1.0.0.)

groot is the result of refactoring rootio in a couple of more focused packages with clear API boundaries:

  • groot/rbase: definitions of basic ROOT classes (Object, Named, ObjString, ...)
  • groot/rbytes: definitions of types useful for serializing and deserializing ROOT data buffers, interfaces to interact with ROOT's metadata classes such as StreamerInfo and StreamerElements.
  • groot/rcont: definitions of ROOT container types (TList, THashList, TObjArray, TArrayX)
  • groot/rdict: definitions of ROOT streamers (TStreamerArtificial, TStreamerLoop, ...)
  • groot/rhist: definitions of ROOT types related to histograms and graphs (TH1x, TH2x, TGraph, TGraphErrors, TGraphAsymmErrors)
  • groot/riofs: low-level types and functions to deal with opening and creating ROOT files; users should prefer using the groot package to open and create ROOT files
  • groot/root: ROOT core interfaces (Object, Named, ObjArray, ...)
  • groot/rsrv: exposes HTTP end-point to manipulate ROOT files, plot and create histograms and graphs from files or trees
  • groot/rtree: interface to decode, read, concatenate and iterate over ROOT Trees
  • groot/rtypes: rtypes contains the means to register types (ROOT ones and user defined ones) with the ROOT type factory system
  • groot/rvers: rvers contains the ROOT version and the classes' versions groot is supporting and currently reading.

Interacting with ROOT files should be performed with the groot package:

import ""

func F() {
	f1, err := groot.Open("some/file.root")
	f2, err := groot.Create("some/other.root")

groot has the needed bootstrap code that allows to automatically import the registration code for all the ROOT types groot knows how to handle.

Here is a quick and dirty Rosetta code for migrating to groot:

  • rootio.H1groot/rhist.H1
  • rootio.H2groot/rhist.H2
  • rootio.Graphgroot/rhist.Graph
  • rootio.Treegroot/rtree.Tree
  • rootio.ChainOfgroot/rtree.ChainOf
  • rootio.Scannergroot/rtree.Scanner
  • rootio.Filegroot/riofs.File
  • rootio.Directorygroot/riofs.Directory
  • rootio.Opengroot.Open
  • rootio.Creategroot.Create

groot/rsrv & root-srv

root-srv has been refactored to extract the pure plot/file interaction machinery from the GUI part.
The plot creation and the ROOT file interaction parts have been refactored into a new package that contains a couple of HTTP end-points that can be reused in third-party packages or applications.

rsrv exposes a REST API that expects JSON requests (OpenFileRequest, PlotH1Request, PlotTreeRequest, ...) and returns JSON responses.
The HTTP end-points are attached to the rsrv.Server type.

root-srv can now open ROOT files served over xrootd.


hbook now exposes a Bin method on H{1,2}D to retrieve a bin by its (x,y) coordinates.


Following an issue raised on gonum/plot, hplot now creates histograms with transparent background by default.


Improved test coverage (by adding some more tests.)


The low-level bits for the following requests have been implemented:

  • kXR_query
  • kXR_prepare
  • kXR_endsess
  • kXR_locate
  • kXR_decrypt
  • kXR_admin

The first steps to support the "host" security provider have also been implemented.

We are still missing the implementation for the GSI authentification protocol: still waiting on xrootd to provide specifications for this protocol (progress is tracked here: issue-757.)

Improved test coverage.


We will try to have preliminary support for writing TTrees in the next release.
That should be fun.

Interoperability with Apache Arrow Arrays is in the works.
It might even prove to be easier to support Apache Arrow first and then implement TTrees writing support on top of that.
We will see...

Stay tuned! (and, as always, any kind of help (reviews, patches, nice emails, constructive criticism) deeply appreciated.)

Assets 2

@sbinet sbinet released this Sep 6, 2018 · 192 commits to master since this release


Release v0.15.0 is fresh from the oven.

This new release dropped explicit support for Go-1.8.x but, in turn, gained support for Go-1.11.x and "Go Modules".
In a nutshell, Go modules allow to explicitly declare what are the dependencies a given module needs for the go build tool to successfully build it.
And, more importantly, Go modules allow to explicitly declare what are the needed versions of these dependencies, essentially making a build completely reproducible.

You can find more informations about Go modules over there:

Currently, modules are only tested in Travis-CI, on the Go master branch.
But as Go-1.12.x will get closer and modules get more ubiquitous, we'll gradually switch to "Go modules" being the mainstream way to build Go-HEP.

Do not hesitate to report any issues you encounter when building with GO111MODULE=on enabled.


Another big news for the v0.15.0 release is the support for writing ROOT files:

  • writing TObjStrings, TH1x, TH2x, TGraph, TGraph{,Assymm}Errors is in,
  • support for writing compressed ROOT files as well (including lz4, lzma and zlib)
  • 2 new ROOT-related commands:
    • cmd/yoda2root: a command to convert YODA files into ROOT ones (so: histograms and scatters)
    • rootio/cmd/root-cp: a command to extract objects from a ROOT file into a new ROOT file

To support writing TH1x, TH2x and TGraphs, hbook types have been modified to export most of their fields -- so one can create a rootio.H1D from a hbook.H1D.
This enabled hbook/rootcnv to gain 3 new functions:

  • rootcnv.FromH1D: a function that converts an hbook.H1D into a rootio.H1D, loosing a bit of informations along the way (ROOT isn't as precise as hbook or YODA are)
  • rootcnv.FromH2D: a function that converts hbook.H2Ds into rootio.H2Ds,
  • rootcnv.FromS2D: a function that converts hbook.S2D into rootio.TGraphAsymmErrors.

rootio & xrootd

Finally, we have received 2 patches from Paul Seyfert (a.k.a pseyfert), our first "CERNois" committer :).
Paul enhanced the UI of root-ls and xrd-ls to better deal with nested directories (and how they are displayed) in both of these commands.
Thanks Paul!


Without further ado, here is how you would create a ROOT file, with lz4 compression, containing a TObjString:

func main() {
	w, err := rootio.Create("out.root", rootio.WithLZ4(flate.BestCompression))
	if err != nil {
	defer w.Close()

	var (
		k = "my-objstring"
		v = rootio.NewObjString("Hello World from Go-HEP!")

	err = w.Put(k, v)
	if err != nil {

	fmt.Printf("wkeys: %d\n", len(w.Keys()))

	err = w.Close()
	if err != nil {
		log.Fatalf("could not close file: %v", err)

and here is how you would use root-cp:

$> root-cp -h
Usage: root-cp [options] file1.root [file2.root [...]] out.root

 $> root-cp f.root out.root
 $> root-cp f1.root f2.root f3.root out.root
 $> root-cp f1.root:hist* f2.root:h2 out.root


$> root-cp ./testdata/graphs.root:g* out.root
$> root-cp root://* out.root

More ROOT files writing examples can be found here:


We will try to have preliminary support for writing TTrees in the next release.
That should be fun.

Interoperability with Apache Arrow Arrays is still on the table.
It might even prove to be easier to support Apache Arrow first and then implement TTrees writing support on top of that.
We will see...

Stay tuned! (and, as always, any kind of help (reviews, patches, nice emails, constructive criticism) deeply appreciated.)

Assets 2

@sbinet sbinet released this Aug 23, 2018 · 226 commits to master since this release


Release v0.14.0 is fresh from the oven.

This release is the result of some massive work in the xrootd package thanks to Mikhail Ivchenko (a.k.a @EgorMatirov), our Google Summer of Code 2018 student.

While GSoC-2018 is now over, it's time to reflect on what Mikhail wrote:

  • an almost complete xrootd client, compatible with the C++ implementation;
  • xrd-fuse, a command to mount the contents of a remote XRootD server, locally;
  • the beginnings of an xrootd server.

Most notably, the client package allowed to:

  • to create the xrd-cp and xrd-ls commands that copy and list the contents of a remote XRootD server,
  • to seamlessly read ROOT files over XRootD.

The client package handles authentication with the unix and kerberos protocols.
Unfortunately, authentication via GSI couldn't be implemented because there were no publicly available specifications for that protocol, see xrootd/xrootd#757 for more details.

Here is the final report of this year's GSoC:

Thanks a lot Mikhail, hope we'll see you around :)

Another big new feature is the ability to write ROOT files, directly with go-hep/rootio.
This is still very much a work in progress, though, as only writing "empty" ROOT files or writing ROOT files with TObjStrings have been explicitly tested.
Next release should see explicit support for writing histograms and graphs.

Lastly, improvements on the build and continuous integration procedure have been applied during this release cycle:

Code coverage improvements

The following packages have been updated, with additional tests, to improve their code coverage:

  • brio, csvutil, csvutil/csvdriver,
  • fit, fmom,
  • heppdt, hepmc, hepevt,
  • hbook/ntup, hplot,
  • lcio, lhef,
  • rootio, sio,
  • xrootd.

Still some more work is needed to bring code coverage to a saner level (from ~55% to ~70-80%.)
Help more than welcome: it's "just" a matter of creating examples and tests.


  • AsciiEncoder was renamed into simply Encoder
  • AsciiDecoder was renamed into simply Decoder


As noted above, it is now possible to create ROOT files.
The rootio package has a couple of examples:

Here is how you would create a ROOT file with one TObjString in it:

package main

import (


func main() {
	w, err := rootio.Create(fname)
	if err != nil {
	defer w.Close()

	var (
	    k   = "my-objstring"
	    v   = rootio.NewObjString("Hello World from Go-HEP!")

	err = w.Put(k, v)
	if err != nil {

	fmt.Printf("wkeys: %d\n", len(w.Keys()))
	err = w.Close()
	if err != nil {
	    log.Fatalf("could not close file: %v", err)

It is quite possible the API may change as we gain experience with what it ought to look like in a Go world.
e.g. it is possible that rootio.Directory.Put would get clever enough to automatically translate a Go builtin, like string, in the ROOT equivalent, say TObjString, on the fly.
Still pondering on that...

Again, still a lot of work to do on the writing side of things:

  • support for TH1x,
  • support for TH2x,
  • support for TGraph, TGraphErrors, TGraphAsymErrors
  • support for TTree (this will take some time)
  • support for user-provided types.


The xrootd/server package ships with support for the following XRootD requests:

  • open, read, close,
  • write, stat, truncate, sync, rename.

There is no authentication support (yet) on the server: DO NOT RUN THIS ON PUBLICLY ACCESSIBLE MACHINES :)

The xrootd package gained a new sub-command:

$> xrd-srv -h
xrd-srv serves data from a local filesystem over the XRootD protocol. 


 $> xrd-srv [OPTIONS] <base-dir>


 $> xrd-srv /tmp
 $> xrd-srv -addr= /tmp

  -addr string
    	listen to the provided address (default "")


Support for Go-1.6 and Go-1.7 has been dropped.
Please upgrade to the latest and finest Go version (1.11 is around the corner, with support for Go modules.)

Another interesting possible development avenue: exposing ROOT TTrees as Apache Arrow Arrays.
This would allow for a better interoperability with that ecosystem and the tools it provides for data science and data analysis in general.
Help wanted!

Assets 2

@sbinet sbinet released this Jun 19, 2018 · 390 commits to master since this release


Release v0.13.0 is fresh from the oven.

This release ships with major improvements in the xrootd implementation and a few fixes in rootio.


  • leveraging the work that happened in xrootd, rootio is now able to read files over [x]root:
import (

func foo() {
    f, err := rootio.Open("root://")
    if err != nil { ... }
    defer f.Close()
  • all the root-xyz commands can now also leverage xrootd:
$> root-ls -t root://
=== [root://] ===
version: 60806
TTree          tree                 my tree title (entries=100)
│ Int32        "Int32/I"            TBranch
│ Int64        "Int64/L"            TBranch
│ UInt32       "UInt32/i"           TBranch
│ UInt64       "UInt64/l"           TBranch
│ Float32      "Float32/F"          TBranch
│ Float64      "Float64/D"          TBranch
│ Str          "Str/C"              TBranch
│ ArrayInt32   "ArrayInt32[10]/I"   TBranch
│ ArrayInt64   "ArrayInt64[10]/L"   TBranch
│ ArrayUInt32  "ArrayInt32[10]/i"   TBranch
│ ArrayUInt64  "ArrayInt64[10]/l"   TBranch
│ ArrayFloat32 "ArrayFloat32[10]/F" TBranch
│ ArrayFloat64 "ArrayFloat64[10]/D" TBranch
│ N            "N/I"                TBranch
│ SliceInt32   "SliceInt32[N]/I"    TBranch
│ SliceInt64   "SliceInt64[N]/L"    TBranch
│ SliceUInt32  "SliceInt32[N]/i"    TBranch
│ SliceUInt64  "SliceInt64[N]/l"    TBranch
│ SliceFloat32 "SliceFloat32[N]/F"  TBranch
│ SliceFloat64 "SliceFloat64[N]/D"  TBranch
  • support for seeking (i.e.: event random access) has been added to scanners connected to chains of rootio.Trees,

  • rootio can now automatically generate streamers for std::vector<T> when a streamer for T exists,

  • rootio has been updated to v2 of pierrec/lz4 library to decode LZ4 compressed ROOT files.


  • support for ping and protocol requests
  • support for dirlist, open, close and sync requests
  • support for read and write requests
  • support for rm, rmdir and truncate requests
  • support for stat, vstat, statx, mkdir, mv and chmod requests
  • support for signing requests
  • support for auth+unix request
  • introduction of the xrd-cp command to copy files from a remote xrootd server:
$> go doc
Command xrd-cp copies files and directories from a remote xrootd server to
local storage.


    $> xrd-cp [OPTIONS] <src-1> [<src-2> [...]] <dst>


    $> xrd-cp root:// .
    $> xrd-cp root:// .
    $> xrd-cp root:// foo.txt
    $> xrd-cp root:// - > foo.txt
    $> xrd-cp -r root:// .
    $> xrd-cp -r root:// outdir


    -r	copy directories recursively
    -v	enable verbose mode
  • introduction of the xrd-ls command to list the contents of directories on a remote xrootd server:
$> go doc
Command xrd-ls lists directory contents on a remote xrootd server.


    $> xrd-ls [OPTIONS] <dir-1> [<dir-2> [...]]


    $> xrd-ls root://
    $> xrd-ls -l root://
    $> xrd-ls -R root://
    $> xrd-ls -l -R root://


    -R	list subdirectories recursively
    -l	use a long listing format
  • a convenience xrootd/xrdfs.FileSystem interface has been introduced to model interacting with the remote xrootd server's filesystem, following the os package API
  • a convenience xrootd/xrdfs.File interface has been introduced to model interacting with the remote xrootd file, following the os.File API
  • a convenience xrootd/xrdio.File type, implementing various io.Xyz interfaces has been introduced as well.
Assets 2

@sbinet sbinet released this Jun 1, 2018 · 457 commits to master since this release


Release v0.12.0 is fresh from the oven.

This release is first one to introduce preliminary support for vgo, the official Go way to handle versioning.
vgo is still in flux: the first Go version with experimental opt-in support should be Go 1.11 (to be released in August 2018.)
Obviously, on the Go-HEP side, adjustments will probably still be required as the user story solidifies and experience is accumulated.

Nonetheless, it is still an interesting new development!


This release adds preliminary support for parsing Geometry Description Markup Language (GDML) files, a de facto standard for describing (detector) geometries.
The documentation for this new package is here: geo/gdml.

Help wanted and (gladly) accepted to get this package in a shape where it could be used for detailed detector studies!
This is tracked here:


hplot was slightly updated to cope with an interesting development percolating from upstream gonum/plot, namely: the migration to a new PDF backend that allows to embed fonts inside the output PDF file.
No more PDFs that display weirdly on foreign computer. Yay!

This obviously means the resulting PDF files may be quite larger than with previous versions.
(You can't have your cake and eat it.)
You can use vgpdf.Canvas.EmbedFonts to get the old behaviour.


This release adds preliminary support for chaining multiple rootio.Trees into a logical view: the famed rootio.Chain.

Mohamed Amine El Gnaoui (a.k.a @maloft), our new summer student @LPC-Clermont, provided the initial implementation: thanks!
More tests and benchmarks improvements yet to come :)

Another noteworthy change: rootio/cmd/root-srv dropped its dependency against in favor of
The latter exposes a more stable API.


This release adds yet another new package: xrootd.
This package will provide (eventually) a pure-Go implementation of an XRootD client as well as a server.

Mikhail Ivchenko (a.k.a @EgorMatirov), our Google Summer of Code 2018 student has been already hard at work, providing support for:

  • the initial xrootd client,
  • the handshake with an XRootD-compliant server (C++ or otherwise), and
  • the protocol and login requests/responses.

The dirlist request/response is already in the pipe.

Assets 2

@sbinet sbinet released this Apr 5, 2018 · 490 commits to master since this release


Release v0.11 is fresh from the oven.

This release drops official support for Go-1.7 and earlier, adds support for Go-1.10.

This allowed to migrate the fwk package away from
fwk now directly uses the context.Context type from the standard library.


This release brings quite a few improvements in the rootio area:

  • add support for empty ROOT files,
  • add support for reading remote ROOT files over HTTP.
    This is implemented in a quite naive way, using net/http.Get to
    download the whole file under a temporary directory.
    See go-hep/hep#142 for ideas on
    how to improve this,
  • add support for TH1-v6,
  • add support for streamer-less TDirectoryFile,
  • add support for TBranchElements in cmd/root-dump,
  • add support for displaying TH1, TH2 and TGraph{,Error}s in cmd/root-dump,
  • add support for branches and leaves with names only differing by case in
    Now, the struct type being generated contains fields whose names start
    with ROOT_.
    The original branch or leaf name is then appended after ROOT_, unmodified.
  • add support for TKeys with a large binary payload,
  • add preliminary support for creating new Go types at runtime directly from
    a StreamerInfo value,
  • add more documentation about ROOT binary file format (TFile, TKey and a bit about
    TStreamerInfo so far.)


  • fixed a bug in the YODA ASCII file parsing code that would choke on
    analysis objects (histograms, scatters, ...) that contain whitespace.


LPC-Clermont has funded a 5 months internship
student, starting now.
Mohamed Amine El Gnaoui (@maloft) will:

  • implement reading all ROOT files created with ROOT-6,
  • implement writing ROOT files, in a ROOT-6 compatible way,
  • improve the read/write performances of Go-HEP to be on par with that of ROOT/C++ (using the builtin performance tools of the Go toolchain: pprof, execution tracer, ... and/or linux perf)
  • extend the test suite of Go-HEP for reading and writing ROOT files,
  • add benchmark tests for reading and writing ROOT files,
  • document the ROOT file format as understood by Go-HEP.

Welcome to Go-HEP, Amine!

Assets 2
Apr 5, 2018
rootio: doc cosmetics to better convey record data set
Feb 15, 2018
rootio: add support for TBranch-v10, TBranchElement-v8 and TH1-v5
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.