Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Spark Application and Session
In this video session, we will dive deep into the Spark application. Let's create a Spark program and learn more about Spark sessions. This clip is from the chapter "Spark Execution Concepts" of the series "Apache Spark 3 for Data...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Exposing Bad Records
In this session, we will remove bad records to ensure that we have good quality data. This clip is from the chapter "Introduction to Spark SQL and Databricks" of the series "Apache Spark 3 for Data Engineering and Analytics with...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - PySpark Installation
In this session, we will go ahead and install PySpark (Windows). This clip is from the chapter "Introduction to Spark and Installation" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section will give...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Reading CSV Files into DataFrame
In this session, we will load our CSV files from our previous project into the DataFrame. This clip is from the chapter "Introduction to Spark SQL and Databricks" of the series "Apache Spark 3 for Data Engineering and Analytics with...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Aggregations - Min Max Sum SumDistinct AVG
In this session, we will work with the aggregation functions: min, max, sum, sumdistinct, and avg. This clip is from the chapter "Structured API - Spark DataFrame" of the series "Apache Spark 3 for Data Engineering and Analytics with...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Distinct and Filter Transformations
In this session, we will explore the distinct() and filter() transformations. This clip is from the chapter "RDD Crash Course" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section primarily focuses...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Install Jupyter Notebooks
We will be installing the Jupyter notebook in this session for macOS. This clip is from the chapter "Introduction to Spark and Installation" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section will...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - PySpark DataFrame, Schema, and DataTypes
In this session, let's learn about the DataFrame, schema, and data types. This clip is from the chapter "Structured API - Spark DataFrame" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Challenge - Convert Fahrenheit to Centigrade
In this video, we will look at a challenge problem and find the solution. This clip is from the chapter "RDD Crash Course" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section primarily focuses on...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Spark Transformations and Actions Part 1
In this video, we will see how Spark executes its application. This clip is from the chapter "Spark Execution Concepts" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section completely focuses on the...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Spark SQL Introduction
In this session, we will discuss about SQL and Spark SQL. This clip is from the chapter "Introduction to Spark SQL and Databricks" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section primarily...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - MacOS - PySpark Installation
We will be installing PySpark in this session (macOS). This clip is from the chapter "Introduction to Spark and Installation" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section will give you a...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Summary
Congratulations! You have successfully completed the course. Let's look at a short summary of the things you have learned so far before we wrap up. This clip is from the chapter "Introduction to Spark SQL and Databricks" of the series...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Data Preparation
In this session, we're going to unpack RDD transformations and actions, but first, let's prepare the session and data. This clip is from the chapter "RDD Crash Course" of the series "Apache Spark 3 for Data Engineering and Analytics with...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Challenge Part 1 - Data Preparation
Let's tackle the first task that was discussed in our previous lesson. You can compare your solution with the solution provided in this video. This clip is from the chapter "Structured API - Spark DataFrame" of the series "Apache Spark 3...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Aggregations
In this session, we will understand the concept of aggregations. This clip is from the chapter "Structured API - Spark DataFrame" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section primarily...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Reading a JSON File
In this session, we will learn to read the JSON file. This clip is from the chapter "Structured API - Spark DataFrame" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section primarily focuses on the...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Challenge Part 3 - Prepare 2019 Data
In the session, we will address the first task of the challenge that was discussed in the previous video. We will be preparing 2019 data and modularizing our programs. This clip is from the chapter "Structured API - Spark DataFrame" of...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Spark Transformations and Actions Part 2
In this session, we will learn about narrow and wide transformations and Spark actions. This clip is from the chapter "Spark Execution Concepts" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Section Summary
Let's recap what we have learned so far in this section. This clip is from the chapter "Introduction to Spark and Installation" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section will give you a...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Creating a Database and Table
In this session, we will create a database and a table. This clip is from the chapter "Introduction to Spark SQL and Databricks" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section primarily focuses...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Hadoop Installation
We will be installing and setting up Hadoop in this session (Windows). This clip is from the chapter "Introduction to Spark and Installation" of the series "Apache Spark 3 for Data Engineering and Analytics with Python".This section will...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Creating our First 2 Databricks Notebooks
We are now ready with our newly created cluster. In this session, we will go ahead and create our first two notebooks. This clip is from the chapter "Introduction to Spark SQL and Databricks" of the series "Apache Spark 3 for Data...
Curated Video
Apache Spark 3 for Data Engineering and Analytics with Python - Challenge - XYZ Research Part 1
In this session, we will address part one of the challenge that we discussed in our previous lesson. Let's look at how many research projects were initiated in the three-year period. This clip is from the chapter "RDD Crash Course" of...