Skip to content
/ BDM Public

Course covers big data fundamentals, processes, technologies, platform ecosystem, and management for practical application development.

Notifications You must be signed in to change notification settings

drshahizan/BDM

Repository files navigation

Stars Badge Forks Badge Pull Requests Badge Issues Badge GitHub contributors Visitors

Big Data Management

Course Synopsis

This course provides a basic fundamental of big data architecture and management. Students will learn the big data processes and the current big data technologies that are available. Further, students will be exposed to the big data platform ecosystem for big data manipulation. The big data management will be explored for the best practice in managing and manipulating large amount of data. At the end of the course, students should be able to understand the architecture and management of big data and also can develop simple application of big data handling using particular platform in assignment.

Course Learning Outcomes

  1. Understand the technology for managing, processing and manipulating large amount of data.
  2. Design big data platform demonstrating the implementation of big data applications.
  3. Discuss current technology that support for sustainability of the big data platform ecosystem.

🔥 Important things

  1. How to Become a Data Engineer in 2024
  2. Essential Preparations for a Successful Start in BDM Class
  3. Student Information
  4. Course Information
  5. Assignment
  6. Exercise

Notes

1. Data

No. Content File
1. Charting Your Path in Data and Machine Learning
2. Navigating the Data Science Landscape
3. Database Types
3a. NoSQL Database
4. Navigating the Database Landscape
5. Data Management
6. The Data Journey: From Raw to Refined
7. The Data to MLOps Journey: An End-to-End Process
8. Data Platforms Architecture: Governance and Operations
9. Creating Data Products to Monetize Data
10. Revolutionizing Data and Machine Learning with DataOps and MLOps
11. Data Science for Beginners - A Curriculum

2. Case Study

No. Content File
1. Unveiling Instagram's Engagement Magic through Machine Learning
2. Unlocking Spotify's Musical Enchantment with Machine Learning
3. Netflix
4. Big data in healthcare: management, analysis and future prospects
5. Introduction to Big Data Computing for Geospatial Applications
6. 40 Stats and Real-Life Examples of How Companies Use Big Data

3. Big Data Management

No. Section Content
1 Introduction to Big Data Management A. What is Big Data?
B. The Importance of Managing Big Data
C. Why Big Data Management is Important?
D. The History of Big Data
2 Understanding Big Data A. Defining Big Data
B. Characteristics of Big Data
C. Sources of Big Data
D. Challenges in Dealing with Big Data
E. The workflow of Big Data Management
3 The Role of Big Data Management A. Managing big data
B. Benefits of Effective Big Data Management
C. Risks of Ignoring Big Data Management
D. Industries Benefiting from Big Data Management
4 Data Collection and Storage A. Data Collection Methods
   1. Traditional Data Sources
   2. Emerging Data Sources
B. Data Storage and Warehousing
C. Data Security and Privacy Concerns
5 Data Processing and Analysis A. Data Preprocessing
B. Data Analysis Tools and Techniques
C. Real-time Data Processing
D. Machine Learning in Big Data Analysis
6 Data Integration and Governance A. Data Integration Strategies
B. Data Governance Best Practices
C. Ensuring Data Quality
7 Data Visualization A. Importance of Data Visualization
B. Data Visualization Tools
C. Creating Effective Data Visualizations
8 Case Studies A. Successful Big Data Management Implementations
B. Lessons Learned from Failed Big Data Projects
C. Real-world Examples
9 Challenges in Big Data Management A. Scalability Issues
B. Security Concerns
C. Compliance and Regulatory Challenges
10 Future Trends in Big Data Management A. The Evolving Landscape of Big Data
B. Predictions for the Future
C. Preparing for the Next Generation of Big Data
11 Best Practices in Big Data Management A. Key Takeaways
B. Actionable Strategies
C. Tips for Effective Big Data Management
12 Other References

Weekly Schedule

Week Topic
1 Introduction to Big Data and Big Data Analytics
- Fundamentals and concepts of big data
2 Big Data Processing and Technology
- Batch, real-time, and streaming processing.
- Scalability, storage, sourcing challenges.
3-4 - ACID, BASE, and CAP theorem
- Distributed File Processing & Map Reduce Processing
5 - Lambda Architecture
6-7 Relational Database (RDBMS)
- Relational Data Modelling
- Database design phases
8 Relational Database (RDBMS)
- SQL programming (DDL, DML, CRUD Operation)
9 Relational Database (RDBMS)
- SQL programming (Subqueries, Join Tables, Aggregate)
10 No SQL Database
- Introduction to No SQL database
- Semi-structured data Modelling (Key Value, Column Family, Document, and Graph)
11-12 No SQL database (Document-based Database)
- Document-based data modelling
- MongoDB query language
13-14 Cloud Technology
- Introduction to Cloud
- AWS Cloud (via AWS Learning Management System)
15 Project Presentation

Contribution 🛠️

Please create an Issue for any improvements, suggestions or errors in the content.

You can also contact me using Linkedin for any other queries or feedback.

Visitors

About

Course covers big data fundamentals, processes, technologies, platform ecosystem, and management for practical application development.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published