Transfer Learning MobileNet with dataset that contains Black&White images-CodePudding

I have a problem. I would like to train a network using MobileNevV2, but from what I know you can only pass color images to it.

My dataset contains ONLY Black and White images. If I try to pass the shape of the image as (224, 224, 1) obviously the error returns me:

Traceback (most recent call last):
  File "/home/andrea/Scrivania/COMPUTER-VISION/MobileNet_train.py", line 40, in <module>
    mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights='imagenet')
  File "/home/andrea/.local/lib/python3.9/site-packages/keras/applications/mobilenet_v2.py", line 279, in MobileNetV2
    input_shape = imagenet_utils.obtain_input_shape(
  File "/home/andrea/.local/lib/python3.9/site-packages/keras/applications/imagenet_utils.py", line 372, in obtain_input_shape
    raise ValueError('The input must have 3 channels; Received '
ValueError: The input must have 3 channels; Received `input_shape=(224, 224, 1)`

How can I train the black and white image model? The following is the code that I've created:

import matplotlib.pyplot as plt
import tensorflow
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator

IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE   (1,)
DATASET_DIR = "/home/andrea/Scrivania/COMPUTER-VISION/DATASET/KAGGLE/new_train"
BATCH_SIZE = 32
EPOCHS = 5

datagen = ImageDataGenerator(
    validation_split=0.2,
    rescale=1. / 255,  # per processare più velocemente i dati
    brightness_range=[1, 2]
)

train_generator = datagen.flow_from_directory(
    DATASET_DIR,
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="training"
)

test_generator = datagen.flow_from_directory(
    DATASET_DIR,
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="validation"
)

mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights='imagenet')

for layer in mobielNetV2.layers:
    layer.trainable = False

x = Flatten()(mobielNetV2.output)
prediction = Dense(6, activation='softmax')(x)

model = Model(inputs=mobielNetV2.input, outputs=prediction)
# tell the model what cost and optimization method to use
model.summary()

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

r = model.fit(train_generator, validation_data=test_generator, epochs=EPOCHS, steps_per_epoch=len(train_generator),
              validation_steps=len(test_generator))
model.save("MobileNet_Hand.h5")

CodePudding user response：

Let's assume you already have greyscale images and you want to convert them to rgb images to train your model:

import tensorflow as tf
import matplotlib.pyplot as plt

flowers = tf.keras.utils.get_file(
    'flower_photos',
    'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
    untar=True)

img_gen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)

ds = tf.data.Dataset.from_generator(
    lambda: img_gen.flow_from_directory(flowers, batch_size=32, shuffle=False),
    output_types=(tf.float32, tf.float32))

ds = ds.map(lambda x, y: (tf.image.rgb_to_grayscale(x), y))
images, _ = next(iter(ds.take(1)))

image = images[0]
print(image.shape)
plt.imshow(image.numpy()[:,:,0], cmap='gray')

(256, 256, 1)

Greyscale:

RGB:

ds = ds.map(lambda x, y: (tf.image.grayscale_to_rgb(x), y))
images, _ = next(iter(ds.take(1)))

image = images[0]
print(image.shape)
plt.imshow(image.numpy())

(256, 256, 3)

So, just use tf.image.grayscale_to_rgb combined with dataset.map and you should be good to go.

Update 1:

datagen = ImageDataGenerator(
    validation_split=0.2,
    rescale=1. / 255,  # per processare più velocemente i dati
    brightness_range=[1, 2]
)

train_generator = datagen.flow_from_directory(
    DATASET_DIR,
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="training"
)
ds = tf.data.Dataset.from_generator(
    lambda: train_generator,
    output_types=(tf.float32, tf.float32))

CodePudding user response：

The error that you've received it's likely because you're passing a grayscale input but your dataloader expects color_model = rgb by default. Look at the possible options for color mode here - (ISSUE 1). And also you're passing a grayscale input (h, w, 1) to a model with ImageNet weights which trained on RGB input (h, w, 3) - (ISSUE 2).

IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE   (1,)

.. .flow_from_directory(
    DATASET_DIR,
    color_mode='rgb', # < ------ ISSUE 1 
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="training"
)

mobielNetV2 = ..applications.MobileNetV2(
                    input_shape=IMG_SHAPE, # < ------ ISSUE 2 
                    include_top=False, 
                    weights='imagenet')
...
...
ValueError: The input must have 3 channels; Received `input_shape=(224, 224, 1)`

Solutions 1

If you don't care about the pretrained weight of ImageNet, then you need to change as follows in your setup:

.. .flow_from_directory(
        DATASET_DIR,
        color_mode='grayscale',  # < ------ SOVLED ISSUE 1 
        ...
    )

mobielNetV2 = ...applications.MobileNetV2(
                 input_shape=IMG_SHAPE, 
                 include_top=False, 
                 weights=None) # < ------ SOVLED ISSUE 2

Solutions 2

If you want ImageNet weight and also you want your grayscale as an input at the same time, then there are two options to choose from.

Option a: (recommended)

Mapping the grayscale input to RGB with trainable layer followed by ImageNet model.

.. .flow_from_directory(
      DATASET_DIR,
      color_mode='grayscale', # < ------ SOVLED ISSUE 1 
       ...
   )
    
# rgb-imagenet-model
mobilenet = keras.applications.MobileNetV2(include_top=False, 
                                           weights='imagenet', 
                                           input_shape=(100, 100, 3))

# mapping grayscale to RGB 
input_tensor = keras.Input(shape=(100, 100, 1))
x = keras.layers.Conv2D(3, (3, 3), padding='same')(input_tensor)

# followed by image-net model 
x = mobilenet(x)

y = keras.layers.GlobalAveragePooling2D()(x)
y = ....
model = keras.Model(input_tensor, y)

Option b:

Mapping the grayscale input to RGB with non-trainable tensorflow operation followed by ImageNet model.

 .. .flow_from_directory(
           DATASET_DIR,
           color_mode='grayscale', # < ------ SOVLED ISSUE 1 
           ....
         )

# rgb-imagenet-model
mobilenet = keras.applications.MobileNetV2(include_top=False, 
                                           weights='imagenet', 
                                           input_shape=(100, 100, 3))

# mapping grayscale to RGB 
input_tensor = keras.Input(shape=(100, 100, 1))
gray_to_rgb = keras.layers.Lambda(lambda x:tf.image.grayscale_to_rgb(x),
                                  output_shape=lambda x:x)
x = gray_to_rgb(input_tensor)

# followed by image-net model 
x = mobilenet(x)

y = keras.layers.GlobalAveragePooling2D()(x)
model = keras.Model(input_tensor, y)
``