Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Rectifying the Error
In this session, we will rectify the errors we faced while creating DataFrames from RDD. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - RDD Filter
In this lesson, you will learn about Filter() function. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on PySpark RDDs.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Quiz (Sort, OrderBy)
It's time for a short quiz on sorting. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark DataFrames in depth.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Quiz (Average)
It's time for a short quiz on average. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on PySpark RDDs.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Project (Filtering)
In this lesson, we will print the names of employees working in NY state under finance department whose bonuses are greater than the average bonuses of employees in NY state. This clip is from the chapter "Spark DFs" of the series...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Glue Job (Full Load)
This video covers how to use the right glue job (full load). This clip is from the chapter "Project - Change Data Capture / Replication Ongoing" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section covers...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - ETL Pipeline Flow
In this video, we will discuss the pipeline we will be working on. This clip is from the chapter "ETL Pipeline" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section covers Extract Transform Load (ETL)...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Total Enrollments per Course
In this lesson, we will work on the 4th step, which is to show the total number of students enrolled per course. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark Streaming Reading Data
In this session, you will read data from file and get it in the Spark streaming context. This clip is from the chapter "Spark Streaming" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".The section primarily focuses...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark Streaming Display
In this lesson, we will work on Spark streaming display for better data visualization. This clip is from the chapter "Spark Streaming" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".The section primarily focuses on...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark Streaming Context
In this video, you will learn how to read data and specify the directory in the Spark streaming context with RDD. This clip is from the chapter "Spark Streaming" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".The...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark RDDs
Learn about spark RDDs (Resilient Distributed Dataset) in this session. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on PySpark RDDs.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark DF with Column Renamed and Alias
In this video, we will learn how we can rename the column in Spark DataFrames. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark DataFrames in...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark DF (Sort, OrderBy)
In this lesson, we will explore sorting in Spark DataFrame using sort and orderBy. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark DataFrames...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Spark DF (Group By - Filtering)
Let's explore filtering for group by in this video. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark DataFrames in depth.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Solution (Average)
Let's discuss a solution for the quiz on average. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on PySpark RDDs.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Select DF Columns
Learn how to select multiple columns from the DataFrames. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark DataFrames in depth.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - RDD Distinct
In this lesson, you will learn about the Distinct() RDD function. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on PySpark RDDs.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - RDD (saveAsTextFile)
In this session, you will learn about how to save RDD to a text file using the saveAsTextFile() function. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - RDD (Partition)
In this lesson, we will explore the concepts of partition. This clip is from the chapter "Spark RDDs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on PySpark RDDs.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Quiz (UDFs)
It's time for a short quiz on UDFs. This clip is from the chapter "Spark DFs" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section focuses on Spark DataFrames in depth.
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Change Data Capture Pipeline
In this lesson, we will perform the last step of the project, which is the Change Data Capture (CDC) pipeline. Let's wind up. This clip is from the chapter "Project - Change Data Capture / Replication Ongoing" of the series "PySpark and...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - Running Spark
In this session, we will run Spark and confirm if it was configured successfully. This clip is from the chapter "Introduction to Hadoop, Spark Ecosystems and Architectures" of the series "PySpark and AWS: Master Big Data with PySpark and...
Curated Video
PySpark and AWS: Master Big Data with PySpark and AWS - RDS Networking
Now that our database is ready, we will go ahead and work on RDS networking. This clip is from the chapter "ETL Pipeline" of the series "PySpark and AWS: Master Big Data with PySpark and AWS".This section covers Extract Transform Load...