Skip to content

Fenxvs/PySpark-Task

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

4 Commits
Β 
Β 
Β 
Β 

Repository files navigation

PySpark RDD Operations – Practice Notebook

This repository contains a Jupyter Notebook that demonstrates basic operations using Apache Spark RDDs (Resilient Distributed Datasets).
It covers the creation of RDDs, basic actions, and transformations to help understand Spark’s distributed data processing model.


πŸ“‘ Contents

πŸ”Ή Spark Initialization

  • Creating a SparkSession and SparkContext.

πŸ”Ή RDD Creation

  • Generating an RDD from a NumPy array of numbers (1–49).

πŸ”Ή Basic RDD Actions

  • Sum
  • Average
  • Count
  • Minimum
  • Maximum

πŸ”Ή RDD Transformations

  • Filtering even numbers.
  • Mapping values (e.g., doubling numbers).
  • Collecting results for inspection.

🎯 Learning Goals

  • Understand how Spark RDDs are created and manipulated.
  • Practice fundamental RDD operations (actions vs transformations).
  • Build intuition for distributed data processing.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published