Zed offers a new approach to data that makes it easier to manipulate and manage your data.
With Zed's new super-structured data model, messy JSON data can easily be given the fully-typed precision of relational tables without giving up JSON's uncanny ability to represent eclectic data.
For a non-technical user, Zed is as easy to use as web search while for a technical user, Zed exposes its technical underpinnings in a gradual slope, providing as much detail as desired, packaged up in the easy-to-understand ZSON data format and Zed language.
We think data is hard and it should be much, much easier.
While schemas are a great way to model and organize your data, they often get in the way when you are just trying to store or transmit your semi-structured data.
Also, why should you have to set up one system for search and another completely different system for historical analytics? And the same unified search/analytics system that works at cloud scale should run easily as a lightweight command-line tool on your laptop.
And rather than having to set up complex ETL pipelines with brittle
transformation logic, managing your data lake should be as easy as
Finally, we believe a lightweight data store that provides easy search and analytics would be a great place to store data sets for data science and data engineering experiments running in Python and providing easy integration with your favorite Python libraries.
Zed solves all these problems with a new foundational data format called ZSON, which is a superset of JSON and the relational models. ZSON is syntax-compatible with JSON but it has a comprehensive type system that you can use as little or as much as you like. Zed types can be used as schemas.
Zed also has a cloud-based object design that was modeled after
git design pattern. Commits to the lake are transactional
and consistent. Search index updates are also transactionally
consistent with any ingested data, and searches can run with or
Check out the installation page for a quick and easy install.
Detailed documentation for the entire Zed system and language is available on the Zed docs site.
The Zui app is an Electron-based desktop app to explore, query, and shape data in your Zed lake.
We originally developed Zui for security-oriented use cases (having tight integration with Zeek, Suricata, and Wireshark), but we are actively extending Zui with UX for handling generic data sets to support data science, data engineering, and ETL use cases.
See the contributing guide on how you can help improve Zed!
Join the Community
Join our public Slack workspace for announcements, Q&A, and to trade tips!
We modeled this README after
Philip O'Toole's brilliantly succinct