Data Process and Advanced Graph Analytics Update

[ZhongFuze]

Data Process Project

A GitHub repository dedicated to fetching and processing blockchain data. It facilitates both online and offline data access essential for services in need of data views or intricate analytics.

Data Fetching and Processing

Metadata is systematically procured from a variety of upstream data sources daily. Post the preprocessing phase of the project, we achieve the data file format indispensable for the graph database. This data is then efficiently batched into the database.

Using RelationService, we can seamlessly acquire fresh and more inclusive data from the database. This positions us advantageously to build numerous graph-specific dashboards and algorithms atop these datasets.

Upstream Data Sources:

These contain the methods for obtaining each upstream data and frequency limit requirements.

• Chainbase
• Polygon/Ethereum RPC
• Lens
• RSS3
• Crossbell

Running Tasks:

• Ethereum Transactions Fetcher
• Lens Social Data Fetcher / Lens Handle Transfer Fetcher
• Crossbell Social Data Fetcher

Algorithms:

The repository incorporates various algorithms spanning across different domains:

Centrality

• Betweenness
• PageRank

Classification

• K Nearest Neighbors (KNN)

Clustering:

• DBSCAN
• KMeans
• RCC

Embeddings

• fastRP(Fast Random Projection)

With the presented structure, viewers and contributors can effortlessly navigate through various segments of the project and understand its functionality. Personalize the board by combining content based on specific needs.