{"page":"\n\n
\n
\n
\n
\n
\n\nVideo\n
\n

\nSplit Data for Machine Learning\n

\n
\n
\nGet Free Access for 10 Days!\n
\n
\n
\n
\n
\n
×
\n\n\n
\n\nPublisher:\n\nAPMonitor\n\n\n
\n
\nSplitting data ensures that there are independent sets for training, testing, and validation. Data can be divided into sequential blocks where the order is preserved (e.g. time series) or with random selection (shuffle). Cross-validation...\nSplitting data ensures that there are independent sets for training, testing, and validation. Data can be divided into sequential blocks where the order is preserved (e.g. time series) or with random selection (shuffle). Cross-validation demonstrates the effect of choosing alternating test sets.

0:00 Train, Validate, Test
2:04 Split DataFrame
3:05 Split by Index
4:30 Split Numpy Array
7:49 Cross Validation
15:50 Overview
18:27 Overfit Detection

The test set is to evaluate the model fit independently of the training and to improve the hyper-parameters without overfitting on the training. Scikit-learn has a train / test split function with a test_size that is the fraction to reserve for testing.\n
\n
\n\n
\n
\n
\n
\n

Resource Details

\n
\n
\n
\n
Curator Rating
\n
\n
\n
\n
\n
\n
Educator Rating
Not yet Rated
\n
\n
\n
\n
\n
Media Length
\n
19:20
\n
\n
\n
\n
\n
Grade
10th - Higher Ed
\n
\n
\n
\n
\n
Subjects
Social Studies & History
5 more...
\n
\n
\n
\n
\n
Media Type
Instructional Videos
\n
\n
\n
\n
\n
Source:
\n
\nAPMonitor.com\n\n
\n
\n
\n
\n
\n
Date
\n
2021
\n
\n
\n
\n
\n\n
Language
English
\n
\n
\n
\n
\n
Audiences
For Teacher Use
2 more...
\n
\n
\n
\n
\n
Usage Permissions
Fine Print: Educational Use
\n
\n
\n
\n
\n
\n

Additional Materials

\n
\n\nSign in or Join Now
\" data-title=\"Get Full Access\" data-container=\"body\" rel=\"popover\" tabindex=\"0\" href=\"/subscription/new\">Video Transcript\n
\n
\n\nSign in or Join Now
\" data-title=\"Get Full Access\" data-container=\"body\" rel=\"popover\" tabindex=\"0\" href=\"/subscription/new\">Video Preview\n
\n
\n
\n

Concepts

\n
\n
index, accuracy, data, classification
\n
\n\n\n
\n
\n
\n
\n

Additional Tags

\n
\n
apmonitor, data engineering, divide, separate, train_test_split, sklearn, scikit, python, sk learn, randomly select, neural network, confusion matrix, loss function, test set, test size, data frame, validation set, accuracy score, test index, splitting data, train index, fold split, function, false, numpy, import, splitting, fold, validation, make, split, scores
\n
\n\n\n
\n
\n
\n
\n

Classroom Considerations

\n
Best For: Explaining a topic
Video is ad-free
\n
\n
\n

Educator Ratings

\n
\n
\n
\n
\n
\n\nShow resource details\n
\n
\n\" data-title=\"Get Full Access\" data-container=\"body\" rel=\"popover\" tabindex=\"0\" aria-label=\"Play video: Split Data for Machine Learning\" href=\"/subscription/new\">\"Split\n\n
\n
\n
\n\n\n\n"}