python – Tensorflow: An error of incompatible between output of Dense layer and the shape of the label: logits and labels must have the same first dimension

I have a question about the output shape of Dense layer.

In my case, the ground truth of labels is a 155 long list. The data part is converted from image. The dataset was assembled:

dataset = tf.data.Dataset.from_tensor_slices((fetched_image_list, labels))
dataset = dataset.batch(1)

In order to know the shape of dataset, a iteration is conducted:

import pprint
print(5)
pp = pprint.PrettyPrinter()
pp.pprint(f"---------Test----------")
iterator = iter(dataset)
data, label = iterator.get_next()
pp.pprint(data)
pp.pprint(data.shape)
print(6)
pp.pprint(label)
print(7)
pp.pprint(label.shape)
pred = model(data)
pp.pprint(pred)

The output goes like this:

5
'---------Test----------'
<tf.Tensor: shape=(1, 128, 128, 3), dtype=float64, numpy=
array([[[[0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         ...,
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628]],

        [[0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         ...,
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628]],

        [[0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         ...,
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628],
         [0.        , 0.        , 0.01568628]],

        ...,

        [[0.94117653, 0.37254903, 0.36862746],
         [0.9571079 , 0.52573532, 0.44705883],
         [0.99215692, 0.86274517, 0.61960787],
         ...,
         [0.99607849, 0.80000007, 0.56078434],
         [0.99607849, 0.79460794, 0.55539221],
         [0.99607849, 0.79215693, 0.5529412 ]],

        [[0.94117653, 0.37254903, 0.36862746],
         [0.9571079 , 0.52573532, 0.44705883],
         [0.99215692, 0.86274517, 0.61960787],
         ...,
         [0.99607849, 0.80000007, 0.56078434],
         [0.99607849, 0.79460794, 0.55539221],
         [0.99607849, 0.79215693, 0.5529412 ]],

        [[0.94117653, 0.37254903, 0.36862746],
         [0.9571079 , 0.52573532, 0.44705883],
         [0.99215692, 0.86274517, 0.61960787],
         ...,
         [0.99607849, 0.80000007, 0.56078434],
         [0.99607849, 0.79460794, 0.55539221],
         [0.99607849, 0.79215693, 0.5529412 ]]]])>
TensorShape([1, 128, 128, 3])
6
<tf.Tensor: shape=(1, 155), dtype=float64, numpy=
array([[8.91835868e-01, 1.61147356e-01, 9.44936812e-01, 4.79378939e-01,
        9.26279604e-01, 7.08247900e-01, 4.03371662e-01, 2.42161050e-01,
        7.58832633e-01, 2.45140955e-01, 5.95810533e-01, 5.55686355e-01,
        4.10343736e-01, 1.01989068e-01, 9.37476814e-01, 2.99018085e-01,
        7.69650578e-01, 1.40782073e-01, 9.09654021e-01, 1.92012936e-01,
        7.34486938e-01, 8.55885744e-01, 1.76614538e-01, 7.64757156e-01,
        2.10310340e-01, 4.56104606e-01, 4.36684415e-02, 5.69215000e-01,
        9.66832399e-01, 6.58945441e-02, 4.06597972e-01, 7.80399263e-01,
        4.80346024e-01, 4.50175732e-01, 4.19802338e-01, 1.73634008e-01,
        6.62129939e-01, 1.90741390e-01, 5.19230068e-01, 3.40338022e-01,
        9.93254662e-01, 5.85740149e-01, 1.23961419e-01, 4.23315704e-01,
        4.46642399e-01, 1.70831442e-01, 4.26856011e-01, 6.79394126e-01,
        8.49008381e-01, 3.71807963e-01, 5.95002830e-01, 6.48025036e-01,
        8.52742434e-01, 5.85844040e-01, 2.63090760e-01, 8.08216035e-01,
        1.77170098e-01, 5.54630041e-01, 8.71276438e-01, 1.51532432e-02,
        5.42835772e-01, 4.50450569e-01, 8.67140651e-01, 7.46993721e-01,
        2.93535799e-01, 6.16679311e-01, 8.18386495e-01, 8.89258325e-01,
        2.44287089e-01, 9.78610814e-02, 7.41236091e-01, 2.83756137e-01,
        9.55440521e-01, 1.29756078e-01, 7.89062083e-02, 9.80626345e-01,
        3.48210335e-01, 1.80298656e-01, 2.09250078e-01, 5.02212763e-01,
        6.83446884e-01, 3.77888530e-01, 7.92944014e-01, 7.17646062e-01,
        5.21967947e-01, 1.64039448e-01, 2.38602638e-01, 6.41449213e-01,
        8.03010702e-01, 6.57776058e-01, 9.93400097e-01, 5.37553847e-01,
        7.76276410e-01, 1.67151600e-01, 6.57575801e-02, 8.22488546e-01,
        9.04812932e-01, 6.87725365e-01, 6.25198781e-01, 5.46188891e-01,
        7.28083611e-01, 2.48713419e-01, 8.49577785e-03, 3.45114410e-01,
        4.72436696e-01, 5.18904009e-04, 7.11714387e-01, 5.68020903e-02,
        4.46687073e-01, 5.86559355e-01, 4.97622102e-01, 8.79296482e-01,
        3.48446429e-01, 9.93079960e-01, 8.52040827e-01, 2.86172181e-01,
        4.79072779e-02, 1.99929491e-01, 6.68457508e-01, 2.99482167e-01,
        7.45040834e-01, 6.69219851e-01, 2.88914382e-01, 6.74020052e-01,
        3.49843800e-01, 6.04531705e-01, 8.62082243e-01, 5.37057579e-01,
        9.24359977e-01, 6.03809692e-02, 5.95386386e-01, 7.45377243e-01,
        7.66371191e-01, 9.77223754e-01, 6.99952602e-01, 9.01550591e-01,
        9.87700880e-01, 8.23613644e-01, 6.59727752e-02, 5.01880348e-02,
        9.88116086e-01, 6.87006056e-01, 5.53662062e-01, 2.83738524e-01,
        1.63693763e-02, 6.71897829e-01, 3.24583441e-01, 8.60079885e-01,
        5.15744328e-01, 7.96320975e-01, 3.06785226e-01, 2.29816660e-01,
        1.45110682e-01, 3.73360485e-01, 4.97248143e-01]])>
7
TensorShape([1, 155])
<tf.Tensor: shape=(1, 155), dtype=float32, numpy=
array([[-0.00493571, -0.0149157 ,  0.01081896, -0.03671832, -0.01307277,
         0.0052427 , -0.0004999 ,  0.01772174,  0.01839588,  0.00302138,
        -0.01457157, -0.04406402, -0.020754  , -0.00404932, -0.01874996,
         0.00491968, -0.03810474,  0.00173163,  0.0038096 , -0.02179225,
        -0.01307295,  0.0308277 ,  0.0057005 ,  0.01481715,  0.01454856,
        -0.01919779, -0.00421686,  0.00507459, -0.01877508,  0.02818902,
         0.00067322, -0.01517171, -0.01724914,  0.02005699,  0.03113937,
         0.03459163,  0.00675361,  0.03252977, -0.01206714,  0.0349409 ,
         0.00211568, -0.00094821,  0.01893008, -0.004606  ,  0.01851429,
         0.01533854,  0.00420215, -0.01205096,  0.00688495,  0.00549618,
        -0.00780923,  0.02259055, -0.01044716,  0.02286388, -0.01468294,
         0.02589482,  0.02846106, -0.01865975,  0.00017489,  0.0032438 ,
        -0.01974735, -0.01395764,  0.0002151 , -0.01726824, -0.03011391,
         0.01531181, -0.01357993, -0.01517356, -0.00039392,  0.00736391,
        -0.00109728,  0.00426431,  0.00932758, -0.01441178,  0.00582732,
        -0.0145042 ,  0.00178996, -0.00641309,  0.01330794, -0.00941933,
         0.0197188 ,  0.02219081,  0.00228922,  0.01103073, -0.00485348,
         0.01004216, -0.00408934,  0.01896594, -0.01999603, -0.02391465,
        -0.00473422,  0.01725628, -0.00505494,  0.01361683,  0.00677673,
         0.01822549, -0.01269314,  0.0133673 , -0.00343115, -0.00155426,
        -0.0192154 ,  0.00130817,  0.00326256,  0.01699893, -0.01849774,
        -0.0042346 ,  0.04019818, -0.01278104,  0.01476647, -0.02140133,
        -0.00120644,  0.0124856 , -0.00885894,  0.02092881, -0.02376898,
        -0.00658559,  0.0193585 ,  0.01865558,  0.01625854, -0.02255119,
         0.0061716 ,  0.01248062, -0.00062311, -0.02172874,  0.01459885,
         0.00093102, -0.02506954, -0.01486003,  0.00519851,  0.02153389,
        -0.03172886,  0.01537657, -0.00440038,  0.0373816 , -0.00942443,
        -0.02697089,  0.01962075,  0.00209401, -0.01224589, -0.00643955,
         0.01197981,  0.02161493, -0.0373175 ,  0.00137456, -0.00574227,
         0.00947102,  0.00560775,  0.00576195, -0.00067998, -0.02010158,
        -0.01511136, -0.01706636,  0.00370965, -0.00531666, -0.01198446]],
      dtype=float32)>

It’s worth mentioning the output of the last Dense layer is a tensor with [1, 155]. And, the CNN model is provided as:

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, (3, 3), strides=(2,2), padding="same", dilation_rate=(1,1), input_shape=(128, 128, 3), activation='relu'),
    tf.keras.layers.Conv2D(71, (3, 3), strides=(2,2), padding="same", dilation_rate=(1,1), activation='relu'),
    tf.keras.layers.Conv2D(128, (3, 4), strides=(2,3), padding="same", dilation_rate=(1,1),activation='relu'),
    tf.keras.layers.Conv2D(128, (3, 3), strides=(2,2), padding="same", dilation_rate=(1,1),activation='relu'),
    tf.keras.layers.Conv2D(128, (3, 4), strides=(2, 3), padding="same", dilation_rate=(1, 1), activation='relu'),
    tf.keras.layers.Conv2D(128, (3, 3), strides=(2, 2), padding="same", dilation_rate=(1, 1), activation='relu'),
    tf.keras.layers.Dropout(0.20),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(155)
])
model.summary()

The summary of the model is:

 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 71)        20519     
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 11, 128)       109184    
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 6, 128)         147584    
                                                                 
 conv2d_4 (Conv2D)           (None, 4, 2, 128)         196736    
                                                                 
 conv2d_5 (Conv2D)           (None, 2, 1, 128)         147584    
                                                                 
 dropout (Dropout)           (None, 2, 1, 128)         0         
                                                                 
 flatten (Flatten)           (None, 256)               0         
                                                                 
 dense (Dense)               (None, 155)               39835     
                                                             

So far so good, I think I did the right things. But when I ran the code, it popped out an error which I think would not take place:

Node: 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits'
logits and labels must have the same first dimension, got logits shape [1,155] and labels shape [155]
     [[{{node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits}}]] [Op:__inference_train_function_1047]

Then, I checked out compile method and fit method:

model.compile(
  optimizer="adam",
  loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
  metrics=['accuracy']),

#Training
model.fit(
  training_dataset,
  validation_data=validating_dataset,
  epochs=3
)

I think it should work. So,

  • Is there anything wrong with the raw data?
  • How can I reshape the labels to fit this model?

Leave a Comment