BigConnect provides an open and extensible platform built on Big Data technologies to understand and work with unstructured data lakes.
It’s an information-agnostic system where all data that flows into the system is transformed, mapped, enriched and then stored in a logical way using a semantic model of concepts, attributes and relationships.
It provides an extensible, unified visual interface with tools for data discovery and analysis, collaboration, alerting and information management.
- Dynamic Data Model The core foundation of BigConnectT is the dynamic, semantic data model. It represents the way you store, correlate and query all information and it’s a conceptual data model that expresses information in a factual way: objects, relations and properties
- Data ingestion Any kind of information can be ingested: databases, office documents, text files, HTML files, images, audio and video files.
- Enterprise Search Built on top of ElasticSearch, it features enterprise-grade search features
- Enrichment Distributed enrichment pipeline based on pluggable Data Workers allow information to be altered anytime, using rules, algorithms and external systems
- Collaboration Work inside spaces, share invite users to join and share information with others
- Security Security is enforced both at the UI and datastore level
- Scalability Featuring a pluggable storage layer, RocksDB and Apache Accumulo implementations are available to achieve massive scalability
- SQL support Query data using SQL queries
- Cypher support Run Cypher queries on top of the property graph data model
Check our YouTube channel for demos
A beautiful and extensible user interface built with FlightJS and React to help you get the most of BigConnect. Everything in BigConnect UI happens in the context of a Space. When a user is first logged in, a Default space is created for him, using his username. The dashboard and the analysis tools are specific to a certain Space.
The visual console includes a dashboard where users are being taken upon login. Various widgets can be added or removed from the dashboard to bring to focus any relevant information. New widgets can be developed using the available extension points on docs.bigconnect.io
The UI provides a Google-like search experience, enriched with filters, sorting, refinements and spatial queries. The search field supports the Lucene query syntax so you can use special things like wildcard (*), question mark (?), keyword matching, phrase matching, grouping, boolean queries (AND, OR), fuzzy queries (~), proximity matching (~4), range searches (amount:[500 TO 800]) and boosts (^).
Searches can be saved and shared with everybody and can be later used on the space dashboard or inside analysis tools. Sometimes BigConnect refers to a saved search as a "dataset".
Data loading (in development)
Data ingestion is a visual process. Choose a datasource, provide connection details, preview the data and map it to the ontology in a visual way and you are ready to go !
Data ingestion can be done using the UI or using scalable ETL and streaming tools. We currently provide plugins for Pentaho Data Integration and Apache NiFi (as part of the Enterprise offering)
Data can be analyzed using the various tools available in the platform. BigConnect provides an extensible tools mechanism, where developers can create their own tools. This process is desribed on docs.bigconnect.io We do provide some tools out of the box to get you started, as described below.
This tool allows you to chart a dataset (or saved search) using the Plotly charting library. Charts can be saved for later use, and can be placed on the space dashboard.
This tool allows you to view and manipulate data and it currently has a lot of features:
- Create entities and relations
- Add datasets to quickly visualize them
- Find hidden paths between two entities
- Add related items for an entity
- Layout the graph using various algorithms: Cola, Euler, Cose etc.
- Adjust vertex size based on Page Rank, Degree Centrality, Closeness Centrality, Betweenness etc.
This tool allows you to view entities that have geospatial information associated, and provides things like clustering, heatmaps and adding new map layers.
This tool currently allows you use Jupyter notebooks and H2O to apply machine learning algoritms on a saved dataset. BigConnect includes its own Python runtime and special APIs to load BigConnect data from Python or H2O.
The admin console allows to customize the data model (ontology), users, roles, UI extensions and various configuration options extraction. The console is extensible, so new items can be easily added as plugins using the provided extension points.