Neard

ETL Data Pipeline Tool for Near Protocol

Overview

Neard is a powerful data engineering tool designed to enhance the data infrastructure for developers working with the Near Protocol. This tool simplifies the extraction, transformation, and loading (ETL) process for Near Protocol data, providing developers with an efficient and streamlined way to interact with and utilize blockchain data. Developers have the option of using the Python script directly or using the user interface to easily configure data pipelines for Near protocol data. Also, for each successful contribution that a Neard user makes to the project, he/she is automatically rewarded through the smart contract.

Demo Image

ETL Backend Code Snippet

Neard Rewards Near Protocol Smart Contract

Automatically rewards Neard uses for sucessful contributions to Neard

Tools Used

• Programming Languages:
  • Python
  • Rust
  • JavaScript
• Web Frameworks:
  • Flask (for backend)
• Data Source:
  • Pagoda Near API
• Data Processing:
  • Apache Airflow
• Frontend Framework:
  • Next.js (React)
• Version Control:
  • Git
• Smart Contract Development:
  • Rust
  • Near SDK
• Documentation:
  • Markdown

Why Use Neard?

• Simplified Data Processing: The tool simplifies complex data processing tasks by providing easy-to-use functions for compression, decompression, storage, and retrieval of data records.

• Data Compression: Efficient data compression using the deflate algorithm helps optimize storage and transmission of data on the blockchain.

• Flexible Storage: Developers can store and retrieve compressed data records seamlessly, facilitating the creation of scalable and efficient applications on the Near Protocol.

• Balances Tracking: The tool includes a mechanism for tracking account balances, making it easy for developers to manage and monitor data usage.

Key Features

1. Data Compression and Decompression: Utilize the deflate algorithm to compress and decompress data efficiently.

2. Flexible Data Storage: Store and retrieve data records in a structured and organized manner.

3. Balances Tracking: Keep track of account balances for effective management of data usage.

4. Integration with Near Protocol: Seamlessly integrates with the Near Protocol, enhancing the overall development experience for Near developers.

5. User Friendly: With option of using the Python script directly or the user interface, this allows developers with diverse data experience to easily use Neard.

Getting Started

Follow these steps to get started with the Neard:

1. Installation: Clone the repository and install the required dependencies.

   git clone https://github.com/TechieTeee/Neard.git
   cd Neard

2. Usage: Explore the provided functionalities and integrate the tool into your Near Protocol projects. To run Neard ETL script, use following.

    python near_etl_script.py

3. Contributing: If you'd like to contribute to the development of the tool, see the Contributing Guidelines.

4. License: This project is licensed under the MIT License.

Community and Support

• Future community (If the repo gets enough stars I might start a Discord or Telegram channel )
• For bug reports and feature requests, please open an issue.

License

This project is licensed under the MIT License - see the LICENSE file for details.