Skip to content
View otacilio-psf's full-sized avatar
21 followers · 7 following

Achievements

Achievement: Pull Sharkx2Achievement: Starstruck

Achievements

Achievement: Pull Sharkx2Achievement: Starstruck

Block or report otacilio-psf

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
otacilio-psf/README.md

Hi! I'm a Brazilian Data Engineer working around the world!

Experience with data ingestion and pipelines in Big Data ecosystem on-premises and on-cloud, data lake and lakehouse storage, relational and noSQL databases, development with Python and Spark in object-oriented and functional projects, SQL and Shell, DataBricks and Azure platform.

I am graduated in Production Engineering, passionate about technology and programming and I have a data driven and goal oriented culture.

Skills

Azure Databricks Python Spark Delta Lake Docker Kubernetes

Certifications

Databricks Certified Data Engineer Professional Databricks Certified Data Engineer Associate Databricks Certified Associate Developer for Apache Spark 3.0 Databricks Certified Data Analyst Associate Partner Training - Solutions Architect Essentials Academy Accreditation - Databricks Lakehouse Fundamentals

Azure Data Engineer Associate Azure Data Fundamentals Azure Fundamentals Azure AI Fundamentals Microsoft Security, Compliance, and Identity Fundamentals

Pinned Loading

  1. spark-dev-env-docker spark-dev-env-docker Public

    Spark development environment for kubernetes, spark-submit and jupyter notebook

    Python 19 6

  2. azure-cloud-handler azure-cloud-handler Public

    Python library to interact with some resources on Azure as AKS (Azure kubernetes service) and Data Lake Storage Gen2

    Python 6 1

  3. azure-functions-ingestion azure-functions-ingestion Public

    Example of how to use Azure Functions to Ingest an API data to datalake

    HCL 1

  4. azure-spark-on-kubernetes azure-spark-on-kubernetes Public

    Spark on Kubernetes with Azure resources: Azure Kubernetes Service (AKS), Azure Data Lake Storage Gen2 and Azure Synapse

    Python 5 2

  5. ingestion-on-postgres ingestion-on-postgres Public

    Ingestion of NY Taxy data on Postgres Database with Python or Spark

    Python

  6. lineage-keeper lineage-keeper Public

    A lightweight lineage tool based on Spark and Delta Lake

    Python