Neblic provides application observability through data. Application data is monitored continuously, and it’s done at many points of the application as a way to have an understanding of how each component behaves and how that changes across time.
Some of the core challenges of making data usable in a systematic way are: accessing the data, past and future, understanding the data including the business domain and its constraints, and knowing what to prioritize rather than getting overwhelmed by the sheer volume and complexity and all the possible dimensions that the data itself can take.
To make the data practical and useful for application troubleshooting Neblic is built on analyzing the data based on three principles: Data Value Statistics, Data Structure and Business Logic and Data Validation.
Generate statistics about the data at field level that can be tailored to what is relevant for each field within a specific application. For instance, min-max-avg to understand numerical distributions across time, cardinality to understand distinct events, and nulls and zero-values as a way to find if data is missing unexpectedly.
Get field level structural digests that lets you understand the schema, field types and field presence for every period at every point of your application. This lets you visualize the actual schema seen, know the field type for each field and figure out if it has changed over time, and monitor field presence relative to each event.
Validate incoming data to get proactively alerted when something goes wrong. For instance, validate that the timestamps you’re receiving are current as a way to detect late events. Or correlate two events from two different fields or samplers to make sure they’re working correctly.
We sometimes refer to the combination of these three elements, Value Stats, Structure Analysis and, Logic/Data Validation as Telemetry, or Data Telemetry.
Neblic is born out of our experience managing complex software applications that combine data streaming pipelines, 3rd party APIs and microservices at global scale.
In developing for and supporting these applications, we often found that while existing Application Performance Monitoring services were critical in helping us maintain the services, they often weren’t enough to really answer What happened, where and why it happened.
While answering these questions will in most cases require a person with domain knowledge to provide the final touch, Neblic’s vision is to provide the insights and the collaboration space for individuals and teams working on solving some of the most difficult questions in software development and software management.
How we are solving these challenges is by creating an infrastructure to sample application-data from any application in real-time, providing the tooling to manage the data sampling infrastructure, and providing tools to filter through and see this data.
