Nettoyage des fichiers obsolètes
- Suppression de .instructions/INSTRUCTIONS.mdoc (remplacé par docs/INSTRUCTIONS.md) - Suppression de .vscode/settings.json (déjà supprimé localement) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This commit is contained in:
spham
parent
b1c3606102
commit
b497b363c9
@ -1,210 +0,0 @@
|
||||
{% steps %}
|
||||
{% step title="Introduction to ML Model Generalization" %}
|
||||
|
||||
|
||||
### Introduction
|
||||
|
||||
|
||||
Welcome to the "Introduction to ML Model Generalization" lab!
|
||||
This lab is designed to give you a foundational understanding of generalization in machine learning models, and how to prevent
|
||||
over or under fitting in models.
|
||||
|
||||
|
||||
### Learning Objectives
|
||||
|
||||
|
||||
- Review generalization and the importance of not over or under fitting models.
|
||||
- Practice implementing early learning cutoff and learning rate decay
|
||||
|
||||
|
||||
### Prerequisites
|
||||
|
||||
|
||||
Familiarity with basic ML principals and key concepts around, learning rates, and model structure.
|
||||
|
||||
|
||||
{% /step %}
|
||||
|
||||
|
||||
{% step title="Synthetic Data Generation" %}
|
||||
|
||||
|
||||
### Synthetic Data Generation
|
||||
Provided below is a basic function to create some synthetic data for classification. This data will have 2000 samples, each with 20
|
||||
features, where 5 features do not affect the outcome of the classification and 15 are directly correlated to the classification. The
|
||||
options for correct classification will only be between two options. Most importantly, the random state is defined to allow repeatability
|
||||
of the model generation.
|
||||
```python
|
||||
X, y = make_classification(n_samples=2000,
|
||||
n_features=20,
|
||||
n_classes=2,
|
||||
n_informative=15,
|
||||
n_redundant=5,
|
||||
random_state=42)
|
||||
scaler = StandardScaler()
|
||||
X = scaler.fit_transform(X)
|
||||
```
|
||||
|
||||
|
||||
### Train/Validation/Test Splits
|
||||
For splitting the data you will first split into the test and training/validation sets. From there you will split out training/validation
|
||||
into their separate sets of training and validation, resulting in a data distribution of train (64%), val (16%), test (20%).
|
||||
|
||||
|
||||
```python
|
||||
X_trainval, X_test, y_trainval, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
|
||||
X_train, X_val, y_train, y_val = train_test_split(X_trainval, y_trainval, test_size=0.2, random_state=42)
|
||||
```
|
||||
|
||||
|
||||
In practice it is best to define your test dataset before model creation begins and to keep it out of production environments entirely
|
||||
to ensure there is no data leakage or over fitting of a model.
|
||||
|
||||
|
||||
|
||||
|
||||
{% /step %}
|
||||
|
||||
|
||||
{% step title="Model and Generalization Feature Setup" %}
|
||||
|
||||
### Introduction
|
||||
For this lab you will use a basic feed forward neural network because neural networks allow you to implement additional features
|
||||
such as learning rate schedulers, and early stopping, that more traditional models such as linear regression do not have.
|
||||
|
||||
### Model Setup
|
||||
For this
|
||||
model you will use two dense layers of Relu activation functions, allowing for
|
||||
more complex patterns to be learned, and ending the model with a sigmoid.
|
||||
When setting up the model you could also include additional generalization techniques such as drop out, which
|
||||
selectively turns off a certain percentage of neurons to ensure no single neuron within the neural net
|
||||
learns to perform a single aspect of prediction.
|
||||
|
||||
**Note:** RELU is used to introduce non linearity into a neural networks learning, and sigmoid is used as a classification function
|
||||
|
||||
|
||||
```python
|
||||
model = tf.keras.Sequential([
|
||||
tf.keras.layers.Dense(64, activation='relu', input_shape=(20,)),
|
||||
tf.keras.layers.Dense(32, activation='relu'),
|
||||
tf.keras.layers.Dense(1, activation='sigmoid')
|
||||
])
|
||||
```
|
||||
|
||||
|
||||
### Loss function and Model Initialization Parameters
|
||||
For this lab you will be using the Adam optimizer as Adam is a good starting optimizer for most problems.
|
||||
Adam takes one parameter which is the starting learning rate, Most
|
||||
models begin with a learning rate under 0.3 and usually closer to 0.1 at most. Here you also define the loss function
|
||||
as ```binary_crossentropy``` which is a simple loss function that just compares is the predicted values match
|
||||
the actual values.
|
||||
```python
|
||||
model.compile(
|
||||
optimizer=tf.keras.optimizers.Adam(learning_rate=0.05),
|
||||
loss='binary_crossentropy',
|
||||
metrics=['accuracy']
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
#### Learning Rate Scheduler
|
||||
For your learning rate scheduler in this lab you will be using "ReduceLROnPlateau" from the keras library which sets the learning rate
|
||||
decay to plateau at a certain amount, ensuring the model never slows to a halt during training. Below the function parameters are defined:
|
||||
- ```val_loss``` is the loss applied on the validation training set
|
||||
- ```factor``` is the factor as which the learning rate will be cut
|
||||
- ```patience``` is the number of epochs between learning rate reductions
|
||||
- ```min_lr``` defines the lowest possible value of learning rate
|
||||
- ``` verbose``` has 3 possible values, 0 which returns nothing, 1 which returns a progress bar,
|
||||
and 2 which displays the values for each epoch individually as its own line.
|
||||
|
||||
```python
|
||||
lr_scheduler = tf.keras.callbacks.ReduceLROnPlateau(
|
||||
monitor='val_loss', # metric to monitor
|
||||
factor=0.5, # reduce by a factor
|
||||
patience=2, # wait 2 epochs before reducing LR
|
||||
min_lr=1e-5, # don't reduce below this
|
||||
verbose=1
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
#### Early Stopping
|
||||
|
||||
|
||||
Next in the code you will implement the early stopping aspect of the model. This function will prevent the model from over fitting by
|
||||
returning to a previous model's parameter in the training if the monitored value does not increase by a specific increment. In your case
|
||||
The monitored value is the validation loss again. A patience of 3 is defined, state the model has 3 times of not incrementing by a
|
||||
great enough change before the previous model that did, is selected as the final model. ```min_delta``` defines how much the value
|
||||
needs to change to be considered a large enough value difference to keep the model. Finally the ```restore_best_weights``` set to true
|
||||
allows the model to restore to the last model that performed the best, in cases where the models ```min_delta``` was not met. This
|
||||
functionality is important to ensure the model does not overfit to the training data and keeps some aspect of generalization.
|
||||
|
||||
|
||||
```python
|
||||
early_stop = tf.keras.callbacks.EarlyStopping(
|
||||
monitor='val_loss',
|
||||
patience=3,
|
||||
min_delta=0.01, # minimum change to be considered an improvement
|
||||
restore_best_weights=True,
|
||||
verbose=1
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
{% /step %}
|
||||
|
||||
|
||||
{% step title="Model training" %}
|
||||
|
||||
|
||||
### Model Training
|
||||
|
||||
|
||||
Finally onto the model training, you will use the basic fit method and set the validation set in the hyper parameters, this lets the
|
||||
```val_loss``` correctly be used by the learning rate schedule and the early stopping mechanism. For this case the epochs defaulted
|
||||
to 100 and the verbose is set as 2, this will ensure you have plenty of epochs to end early and the line by line model training information
|
||||
can help you better understand the values of ```val_loss``` and how they are changing per epoch. As you run the model pay close attention
|
||||
to the change in ```val_loss``` and how it correlates to when the model initiates early stopping and rolling back to previous models.
|
||||
```python
|
||||
model.fit(
|
||||
X_train, y_train,
|
||||
validation_data=(X_val, y_val),
|
||||
epochs=100,
|
||||
callbacks=[early_stop, lr_scheduler], # your custom early stopping + LR scheduler
|
||||
verbose=2
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
{% /step %}
|
||||
|
||||
|
||||
{% step title="Evaluating Model Results" %}
|
||||
|
||||
|
||||
### Evaluating Model Results
|
||||
|
||||
|
||||
The following code provides a basic metric test of your neural network. Depending on the domain of the model different levels of accuracy
|
||||
are acceptable. It's more important to see a considerable increase in accuracy in predictions compared to existing methods, than it is
|
||||
to hit a particular threshold of accuracy. Accuracy above 99.5% for validation can be a bit concerning as it may be a sign of over fitting,
|
||||
and an accuracy below previous methods may be a sign of under fitting.
|
||||
|
||||
|
||||
```python
|
||||
y_pred_probs = model.predict(X_test).flatten()
|
||||
y_pred = (y_pred_probs >= 0.5).astype(int)
|
||||
|
||||
|
||||
print("\n Test Set Evaluation:")
|
||||
print(classification_report(y_test, y_pred))
|
||||
print("Confusion Matrix:")
|
||||
print(confusion_matrix(y_test, y_pred))
|
||||
```
|
||||
|
||||
|
||||
{% /step %}
|
||||
{% /steps %}
|
||||
|
||||
22
.vscode/settings.json
vendored
22
.vscode/settings.json
vendored
@ -1,22 +0,0 @@
|
||||
{
|
||||
"terminal.integrated.fontSize": 15,
|
||||
"editor.fontSize": 15,
|
||||
"terminal.integrated.defaultProfile.linux": "bash",
|
||||
"workbench.colorTheme": "Default Dark Modern",
|
||||
"workbench.startupEditor": "none",
|
||||
"files.associations": {
|
||||
"*.md": "markdoc"
|
||||
},
|
||||
"workspace": {
|
||||
"view": "readme",
|
||||
"terminals": [
|
||||
{
|
||||
"name": "Terminal",
|
||||
"active": false
|
||||
}
|
||||
],
|
||||
"files": [
|
||||
"./lab.ipynb"
|
||||
],
|
||||
}
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user