mlops_learn/ML-Model-Generalization
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Nettoyage des fichiers obsolètes

Browse Source 
- Suppression de .instructions/INSTRUCTIONS.mdoc (remplacé par docs/INSTRUCTIONS.md)
- Suppression de .vscode/settings.json (déjà supprimé localement)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
This commit is contained in:
spham 2025-11-12 18:28:12 +01:00
parent b1c3606102
commit b497b363c9
2 changed files with 0 additions and 232 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

210
.instructions/INSTRUCTIONS.mdoc
View File

@ -1,210 +0,0 @@
{% steps %}
{% step title="Introduction to ML Model Generalization" %}
### Introduction
Welcome to the "Introduction to ML Model Generalization" lab!
This lab is designed to give you a foundational understanding of generalization in machine learning models, and how to prevent
over or under fitting in models.
### Learning Objectives
- Review generalization and the importance of not over or under fitting models.
- Practice implementing early learning cutoff and learning rate decay
### Prerequisites
Familiarity with basic ML principals and key concepts around, learning rates, and model structure.
{% /step %}
{% step title="Synthetic Data Generation" %}
### Synthetic Data Generation
Provided below is a basic function to create some synthetic data for classification. This data will have 2000 samples, each with 20
features, where 5 features do not affect the outcome of the classification and 15 are directly correlated to the classification. The
options for correct classification will only be between two options. Most importantly, the random state is defined to allow repeatability
of the model generation.
```python
X, y = make_classification(n_samples=2000,
n_features=20,
n_classes=2,
n_informative=15,
n_redundant=5,
random_state=42)
scaler = StandardScaler()
X = scaler.fit_transform(X)
```
### Train/Validation/Test Splits
For splitting the data you will first split into the test and training/validation sets. From there you will split out training/validation
into their separate sets of training and validation, resulting in a data distribution of train (64%), val (16%), test (20%).
```python
X_trainval, X_test, y_trainval, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X_trainval, y_trainval, test_size=0.2, random_state=42)
```
In practice it is best to define your test dataset before model creation begins and to keep it out of production environments entirely
to ensure there is no data leakage or over fitting of a model.
{% /step %}
{% step title="Model and Generalization Feature Setup" %}
### Introduction
For this lab you will use a basic feed forward neural network because neural networks allow you to implement additional features
such as learning rate schedulers, and early stopping, that more traditional models such as linear regression do not have.
### Model Setup
For this
model you will use two dense layers of Relu activation functions, allowing for
more complex patterns to be learned, and ending the model with a sigmoid.
When setting up the model you could also include additional generalization techniques such as drop out, which
selectively turns off a certain percentage of neurons to ensure no single neuron within the neural net
learns to perform a single aspect of prediction.
**Note:** RELU is used to introduce non linearity into a neural networks learning, and sigmoid is used as a classification function
```python
model = tf.keras.Sequential([
tf.keras.layers.Dense(64, activation='relu', input_shape=(20,)),
tf.keras.layers.Dense(32, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
```
### Loss function and Model Initialization Parameters
For this lab you will be using the Adam optimizer as Adam is a good starting optimizer for most problems.
Adam takes one parameter which is the starting learning rate, Most
models begin with a learning rate under 0.3 and usually closer to 0.1 at most. Here you also define the loss function
as ```binary_crossentropy``` which is a simple loss function that just compares is the predicted values match
the actual values.
```python
model.compile(
optimizer=tf.keras.optimizers.Adam(learning_rate=0.05),
loss='binary_crossentropy',
metrics=['accuracy']
)
```
#### Learning Rate Scheduler
For your learning rate scheduler in this lab you will be using "ReduceLROnPlateau" from the keras library which sets the learning rate
decay to plateau at a certain amount, ensuring the model never slows to a halt during training. Below the function parameters are defined:
- ```val_loss``` is the loss applied on the validation training set
- ```factor``` is the factor as which the learning rate will be cut
- ```patience``` is the number of epochs between learning rate reductions
- ```min_lr``` defines the lowest possible value of learning rate
- ``` verbose``` has 3 possible values, 0 which returns nothing, 1 which returns a progress bar,
and 2 which displays the values for each epoch individually as its own line.
```python
lr_scheduler = tf.keras.callbacks.ReduceLROnPlateau(
monitor='val_loss', # metric to monitor
factor=0.5, # reduce by a factor
patience=2, # wait 2 epochs before reducing LR
min_lr=1e-5, # don't reduce below this
verbose=1
)
```
#### Early Stopping
Next in the code you will implement the early stopping aspect of the model. This function will prevent the model from over fitting by
returning to a previous model's parameter in the training if the monitored value does not increase by a specific increment. In your case
The monitored value is the validation loss again. A patience of 3 is defined, state the model has 3 times of not incrementing by a
great enough change before the previous model that did, is selected as the final model. ```min_delta``` defines how much the value
needs to change to be considered a large enough value difference to keep the model. Finally the ```restore_best_weights``` set to true
allows the model to restore to the last model that performed the best, in cases where the models ```min_delta``` was not met. This
functionality is important to ensure the model does not overfit to the training data and keeps some aspect of generalization.
```python
early_stop = tf.keras.callbacks.EarlyStopping(
monitor='val_loss',
patience=3,
min_delta=0.01, # minimum change to be considered an improvement
restore_best_weights=True,
verbose=1
)
```
{% /step %}
{% step title="Model training" %}
### Model Training
Finally onto the model training, you will use the basic fit method and set the validation set in the hyper parameters, this lets the
```val_loss``` correctly be used by the learning rate schedule and the early stopping mechanism. For this case the epochs defaulted
to 100 and the verbose is set as 2, this will ensure you have plenty of epochs to end early and the line by line model training information
can help you better understand the values of ```val_loss``` and how they are changing per epoch. As you run the model pay close attention
to the change in ```val_loss``` and how it correlates to when the model initiates early stopping and rolling back to previous models.
```python
model.fit(
X_train, y_train,
validation_data=(X_val, y_val),
epochs=100,
callbacks=[early_stop, lr_scheduler], # your custom early stopping + LR scheduler
verbose=2
)
```
{% /step %}
{% step title="Evaluating Model Results" %}
### Evaluating Model Results
The following code provides a basic metric test of your neural network. Depending on the domain of the model different levels of accuracy
are acceptable. It's more important to see a considerable increase in accuracy in predictions compared to existing methods, than it is
to hit a particular threshold of accuracy. Accuracy above 99.5% for validation can be a bit concerning as it may be a sign of over fitting,
and an accuracy below previous methods may be a sign of under fitting.
```python
y_pred_probs = model.predict(X_test).flatten()
y_pred = (y_pred_probs >= 0.5).astype(int)
print("\n Test Set Evaluation:")
print(classification_report(y_test, y_pred))
print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))
```
{% /step %}
{% /steps %}

22
.vscode/settings.json vendored
View File

@ -1,22 +0,0 @@
{
"terminal.integrated.fontSize": 15,
"editor.fontSize": 15,
"terminal.integrated.defaultProfile.linux": "bash",
"workbench.colorTheme": "Default Dark Modern",
"workbench.startupEditor": "none",
"files.associations": {
"*.md": "markdoc"
},
"workspace": {
"view": "readme",
"terminals": [
{
"name": "Terminal",
"active": false
}
],
"files": [
"./lab.ipynb"
],
}
}