Get each epoch's validation scores from GridSearchCV models-CodePudding

I am using GridSearchCV with keras and I want to plot and analyze the training vs validation history. However, I've check the documentation and really searched around SO but I cannot find a way to obtain the validation history (i.e. scores for each epoch) when the models are fitted using GridSearchCV. I am able to get the training history in a callback, but not the validation one. The thing is that some models overfit a lot and I want to be able to see how tunning the paramenters affects overfitting.

I am using GridSearchCV like this:

class MyCallback(keras_callbacks.Callback):
   def on_train_end(self, logs=None):
      # here I can get the model history from self.model.history.history

def create_model(...):
   ...
   model = Model(...)
   model.compile(optimizer=optimizer, loss="binary_crossentropy", metrics=['acc'])
   return model

callbacks = [MyCallback()]
model = KerasClassifier(build_fn=create_model, verbose=3)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=4, cv=3, verbose=10, return_train_score=True)
grid_result = grid.fit(X_train["padded"], y_train["binary"], epochs=30, batch_size=16, callbacks=callbacks)

CodePudding user response：

You are looking to track the validation performances like using validation_data or validation_split when you fit your Keras model (see here for a reference).

However GridSearchCV (from sklearn) is not so clever to understand that the validation set (created during CV splitting) must be used with KerasClassifier as validation_data in order to track scores/losses for each epoch.

In other words, you can't track the performances of each validation set (created during CV splitting) using GridSearchCV

This is a possible solution.