I am new to deep learning and I am utterly confused about the terminology.

In the Tensorflow documentation,

for [RNN layer] https://www.tensorflow.org/api_docs/python/tf/keras/layers/RNN#input_shape

N-D tensor with shape [batch_size, timesteps, ...] 

for [LSTM layer] https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM

inputs: A 3D tensor with shape [batch, timesteps, feature].

1. I understand for the input_shape, we don't have to specify the batch/batch size. But still I would like to know the difference between batch & batch size.

2. What is time-steps vs features?

Is the 1st Dimension always the batch? The 2nd-D = Time-steps, and 3rd-D = Features?

Example 1

data = array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
data = data.reshape((1, 5, 2))
print(data.shape) --> (1, 5, 2)

print(data)

[[[ 1  2]
  [ 3  4]
  [ 5  6]
  [ 7  8]
  [ 9 10]]]


model = Sequential()
model.add(LSTM(32, input_shape=(5, 2)))

Example 2

data1 = array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10,11])
n_features = 1
data1 = data1.reshape((len(data1), n_features))

print(data1)
# define generator
n_input = 2
generator = TimeseriesGenerator(data1, data1, length=n_input, stride=2, batch_size=10)

# number of batch
print('Batches: %d' % len(generator))
# OUT --> Batches: 1

# print each batch
for i in range(len(generator)):
   x, y = generator[i]
   print('%s => %s' % (x, y))
x, y = generator[0]
print(x.shape)

[[[ 1]
  [ 2]]

 [[ 3]
  [ 4]]

 [[ 5]
  [ 6]]

 [[ 7]
  [ 8]]

 [[ 9]
  [10]]] => [[ 3]
 [ 5]
 [ 7]
 [ 9]
 [11]]
(5, 2, 1)

# define model
model = Sequential()
model.add(LSTM(100, activation='relu', input_shape=(n_input, n_features)))

CodePudding user response：

Difference between batch_size v. batch

In the documentation you quoted, batch means batch_size.

Meaning of timesteps and feature

Taking a glance at https://www.tensorflow.org/tutorials/structured_data/time_series (weather forecast example with real-world data!) will help you understand more about time-series data.

feature is what you want the model to make predictions from; in the above forecast example, it is an vector (array) of pressure, temperature, etc...

RNN/ LSTM are designed to handle time-series. This is why you need to feed timesteps, along with feature, to your model. timesteps represents when the data is recorded; again, in the example above, data is sampled every hour, so timesteps == 0 is the data taken at the first hour, timesteps == 1 the second hour, ...

Order of dimensions of the input/ output data

In TensorFlow, the first dimension of data often represents a batch.

What comes after the batch axis, depends on the problem field. In general, global features (like batch size) precedes element-specific features (like image size).

Examples:

• time-series data are in (batch_size, timesteps, feature) format.
• Image data are often represented in NHWC format: (batch_size, image_height, image_width, channels).

From https://www.tensorflow.org/guide/tensor#about_shapes :

While axes are often referred to by their indices, you should always keep track of the meaning of each. Often axes are ordered from global to local: The batch axis first, followed by spatial dimensions, and features for each location last. This way feature vectors are contiguous regions of memory.