Spark And Python For Big Data With Pyspark

Learn how to use Spark with Python, including Spark Streaming, Machine Learning, Spark 2.0 DataFrames and more!

What you'll learn
Use Python and Spark together to analyze Big Data
Learn how to use the new Spark 2.0 DataFrame Syntax
Work on Consulting Projects that mimic real world situations!
Classify Customer Churn with Logisitic Regression
Use Spark with Random Forests for Classification
Learn how to use Spark's Gradient Boosted Trees
Use Spark's MLlib to create Powerful Machine Learning Models
Learn about the DataBricks Platform!
Get set up on Amazon Web Services EC2 for Big Data Analysis
Learn how to use AWS Elastic MapReduce Service!
Learn how to leverage the power of Linux with a Spark Environment!
Create a Spam filter using Spark and Natural Language Processing!
Use Spark Streaming to Analyze Tweets in Real Time!
Requirements
General Programming Skills in any Language (Preferrably Python)
20 GB of free space on your local computer (or alternatively a strong internet connection for AWS)
Description
Learn the latest Big Data Technology - Spark! And learn to use it with one of the most popular programming languages, Python!
One of the most valuable technology skills is the ability to analyze huge data sets, and this course is specifically designed to bring you up to speed on one of the best technologies for this task, Apache Spark! The top technology companies like Google, Facebook, Netflix, Airbnb, Amazon, NASA, and more are all using Spark to solve their big data problems!
Spark can perform up to 100x faster than Hadoop MapReduce, which has caused an explosion in demand for this skill! Because the Spark 2.0 DataFrame framework is so new, you now have the ability to quickly become one of the most knowledgeable people in the job market!
This course will teach the basics with a crash course in Python, continuing on to learning how to use Spark DataFrames with the latest Spark 2.0 syntax! Once we've done that we'll go through how to use the MLlib Machine Library with the DataFrame syntax and Spark. All along the way you'll have exercises and Mock Consulting Projects that put you right into a real world situation where you need to use your new skills to solve a real problem!
We also cover the latest Spark Technologies, like Spark SQL, Spark Streaming, and advanced models like Gradient Boosted Trees! After you complete this course you will feel comfortable putting Spark and PySpark on your resume! This course also has a full 30 day money back guarantee and comes with a LinkedIn Certificate of Completion!
If you're ready to jump into the world of Python, Spark, and Big Data, this is the course for you!

Overview

Section 1: Introduction to Course

Lecture 1 Introduction

Lecture 2 Course Overview

Lecture 3 Frequently Asked Questions

Lecture 4 What is Spark? Why Python?

Section 2: Setting up Python with Spark

Lecture 5 Set-up Overview

Lecture 6 Note on Installation Sections

Section 3: Databricks Setup

Lecture 7 Recommended Setup

Lecture 8 Databricks Setup

Section 4: Local VirtualBox Set-up

Lecture 9 Local Installation VirtualBox Part 1

Lecture 10 Local Installation VirtualBox Part 2

Lecture 11 Setting up PySpark

Section 5: AWS EC2 PySpark Set-up

Lecture 12 AWS EC2 Set-up Guide

Lecture 13 Creating the EC2 Instance

Lecture 14 SSH with Mac or Linux

Lecture 15 Installations on EC2

Section 6: AWS EMR Cluster Setup

Lecture 16 AWS EMR Setup

Section 7: Python Crash Course

Lecture 17 Introduction to Python Crash Course

Lecture 18 Jupyter Notebook Overview

Lecture 19 Python Crash Course Part One

Lecture 20 Python Crash Course Part Two

Lecture 21 Python Crash Course Part Three

Lecture 22 Python Crash Course Exercises

Lecture 23 Python Crash Course Exercise Solutions

Section 8: Spark DataFrame Basics

Lecture 24 Introduction to Spark DataFrames

Lecture 25 Spark DataFrame Basics

Lecture 26 Spark DataFrame Basics Part Two

Lecture 27 Spark DataFrame Basic Operations

Lecture 28 Groupby and Aggregate Operations

Lecture 29 Missing Data

Lecture 30 Dates and Timestamps

Section 9: Spark DataFrame Project Exercise

Lecture 31 DataFrame Project Exercise

Lecture 32 DataFrame Project Exercise Solutions

Section 10: Introduction to Machine Learning with MLlib

Lecture 33 Introduction to Machine Learning and ISLR

Lecture 34 Machine Learning with Spark and Python with MLlib

Section 11: Linear Regression

Lecture 35 Linear Regression Theory and Reading

Lecture 36 Linear Regression Documentation Example

Lecture 37 Regression Evaluation

Lecture 38 Linear Regression Example Code Along

Lecture 39 Linear Regression Consulting Project

Lecture 40 Linear Regression Consulting Project Solutions

Section 12: Logistic Regression

Lecture 41 Logistic Regression Theory and Reading

Lecture 42 Logistic Regression Example Code Along

Lecture 43 Logistic Regression Code Along

Lecture 44 Logistic Regression Consulting Project

Lecture 45 Logistic Regression Consulting Project Solutions

Section 13: Decision Trees and Random Forests

Lecture 46 Tree Methods Theory and Reading

Lecture 47 Tree Methods Documentation Examples

Lecture 48 Decision Tress and Random Forest Code Along Examples

Lecture 49 Random Forest - Classification Consulting Project

Lecture 50 Random Forest Classification Consulting Project Solutions

Section 14: K-means Clustering

Lecture 51 K-means Clustering Theory and Reading

Lecture 52 KMeans Clustering Documentation Example

Lecture 53 Clustering Example Code Along

Lecture 54 Clustering Consulting Project

Lecture 55 Clustering Consulting Project Solutions

Section 15: Collaborative Filtering for Recommender Systems

Lecture 56 Introduction to Recommender Systems

Lecture 57 Recommender System - Code Along Project

Section 16: Natural Language Processing

Lecture 58 Introduction to Natural Language Processing

Lecture 59 NLP Tools Part One

Lecture 60 NLP Tools Part Two

Lecture 61 Natural Language Processing Code Along Project

Section 17: Spark Streaming with Python

Lecture 62 Introduction to Streaming with Spark!

Lecture 63 Spark Streaming Documentation Example

Lecture 64 Spark Streaming Twitter Project - Part

Lecture 65 Spark Streaming Twitter Project - Part Two

Lecture 66 Spark Streaming Twitter Project - Part Three

Section 18: Bonus

Lecture 67 Bonus Lecture:

Someone who knows Python and would like to learn how to use it for Big Data,Someone who is very familiar with another programming language and needs to learn Spark