Background

Recorded historical documents give us a peek into the past. We are able to glimpse the world before our time; and see its culture, norms, and values to reflect on our own. Japan has very unique historical pathway. Historically, Japan and its culture was relatively isolated from the West, until the Meiji restoration in 1868 where Japanese leaders reformed its education system to modernize its culture. This caused drastic changes in the Japanese language, writing and printing systems. Due to the modernization of Japanese language in this era, cursive Kuzushiji (くずし字) script is no longer taught in the official school curriculum. Even though Kuzushiji had been used for over 1000 years, most Japanese natives today cannot read books written or published over 150 years ago.

The result is that there are hundreds of thousands of Kuzushiji texts that have been digitised but have never been transcribed, and can only currently be read by a few experts. We've built Kuzushiji-MNIST and sister datasets by taking handwritten characters from these texts and preprocessing them in a format similar to the MNIST dataset, to create easy to use benchmark datasets that are more modern and difficult to classify than the original MNIST dataset.

By releasing these datasets, we're also hoping to bring together the fields of Japanese literature and ML 😊

The datasets

📚 Read the paper to learn more about Kuzushiji, the datasets and their motivations for making them!
Download the dataset at Kuzushi-MNIST.

Structure of the dataset:

Kuzushiji-MNIST
    ├── kmnist-test-imgs.npz
    ├── kmnist-test-labels.npz
    ├── kmnist-train-imgs.npz
    ├── kmnist-train-labels.npz
    ├── ...

kmnist-[train/test]-[images/labels].npz: These files contain the Kuzushiji-MNIST as compressed numpy arrays.

import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, accuracy_score

import tensorflow as tf

DATA_DIR = '/kaggle/input/kuzushiji'

X_path = os.path.join(DATA_DIR, 'kmnist-train-imgs.npz')
y_path = os.path.join(DATA_DIR, 'kmnist-train-labels.npz')

X_test_path = os.path.join(DATA_DIR, 'kmnist-test-imgs.npz')
y_test_path = os.path.join(DATA_DIR, 'kmnist-test-labels.npz')

X = np.load(X_path)['arr_0']
y = np.load(y_path)['arr_0']

print('X shape:', X.shape)
print('y shape:', y.shape)

X shape: (60000, 28, 28)
y shape: (60000,)

# Plot some samples

plt.figure(figsize=(10, 10))
for i in range(25):
    plt.subplot(5, 5, i + 1)
    plt.imshow(X[i], cmap='viridis')
    plt.axis('off')
    plt.grid(False)
plt.show()

Prepare dataset

X = X.reshape(-1, 28, 28, 1)
X = X / 255.0

X_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)

Build the model

0. Configuration

NUM_CLASSES = 10
IMG_SIZE = 28
BATCH_SIZE = 128
EPOCHS = 100

CALLBACKS = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=20)]

1. Baseline model

baseline_model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(IMG_SIZE, IMG_SIZE)),
    tf.keras.layers.Dense(NUM_CLASSES, activation='softmax')
])

baseline_model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
    metrics=['accuracy']
)

history = baseline_model.fit(
    X_train, y_train,
    validation_data=(X_valid, y_valid),
    epochs=EPOCHS, batch_size=BATCH_SIZE,
    callbacks=CALLBACKS
)

Epoch 1/100
375/375 [==============================] - 6s 4ms/step - loss: 0.9198 - accuracy: 0.7348 - val_loss: 0.7045 - val_accuracy: 0.7952
Epoch 2/100
375/375 [==============================] - 2s 6ms/step - loss: 0.6541 - accuracy: 0.8073 - val_loss: 0.6505 - val_accuracy: 0.8113
Epoch 3/100
375/375 [==============================] - 2s 4ms/step - loss: 0.6182 - accuracy: 0.8164 - val_loss: 0.6364 - val_accuracy: 0.8138
Epoch 4/100
375/375 [==============================] - 1s 4ms/step - loss: 0.6013 - accuracy: 0.8211 - val_loss: 0.6295 - val_accuracy: 0.8161
Epoch 5/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5916 - accuracy: 0.8239 - val_loss: 0.6226 - val_accuracy: 0.8173
Epoch 6/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5836 - accuracy: 0.8268 - val_loss: 0.6208 - val_accuracy: 0.8192
Epoch 7/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5783 - accuracy: 0.8293 - val_loss: 0.6174 - val_accuracy: 0.8181
Epoch 8/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5732 - accuracy: 0.8308 - val_loss: 0.6138 - val_accuracy: 0.8212
Epoch 9/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5696 - accuracy: 0.8328 - val_loss: 0.6132 - val_accuracy: 0.8214
Epoch 10/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5653 - accuracy: 0.8330 - val_loss: 0.6140 - val_accuracy: 0.8213
Epoch 11/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5630 - accuracy: 0.8329 - val_loss: 0.6133 - val_accuracy: 0.8212
Epoch 12/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5596 - accuracy: 0.8354 - val_loss: 0.6132 - val_accuracy: 0.8223
Epoch 13/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5573 - accuracy: 0.8354 - val_loss: 0.6132 - val_accuracy: 0.8217
Epoch 14/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5552 - accuracy: 0.8368 - val_loss: 0.6109 - val_accuracy: 0.8206
Epoch 15/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5537 - accuracy: 0.8389 - val_loss: 0.6106 - val_accuracy: 0.8213
Epoch 16/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5525 - accuracy: 0.8370 - val_loss: 0.6102 - val_accuracy: 0.8230
Epoch 17/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5503 - accuracy: 0.8376 - val_loss: 0.6135 - val_accuracy: 0.8217
Epoch 18/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5484 - accuracy: 0.8385 - val_loss: 0.6120 - val_accuracy: 0.8209
Epoch 19/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5473 - accuracy: 0.8382 - val_loss: 0.6108 - val_accuracy: 0.8205
Epoch 20/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5457 - accuracy: 0.8388 - val_loss: 0.6149 - val_accuracy: 0.8203
Epoch 21/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5446 - accuracy: 0.8393 - val_loss: 0.6122 - val_accuracy: 0.8200
Epoch 22/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5436 - accuracy: 0.8388 - val_loss: 0.6108 - val_accuracy: 0.8213
Epoch 23/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5421 - accuracy: 0.8404 - val_loss: 0.6127 - val_accuracy: 0.8219
Epoch 24/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5416 - accuracy: 0.8397 - val_loss: 0.6126 - val_accuracy: 0.8194
Epoch 25/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5412 - accuracy: 0.8395 - val_loss: 0.6157 - val_accuracy: 0.8179
Epoch 26/100
375/375 [==============================] - 2s 5ms/step - loss: 0.5392 - accuracy: 0.8412 - val_loss: 0.6122 - val_accuracy: 0.8203
Epoch 27/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5388 - accuracy: 0.8396 - val_loss: 0.6158 - val_accuracy: 0.8207
Epoch 28/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5380 - accuracy: 0.8408 - val_loss: 0.6147 - val_accuracy: 0.8198
Epoch 29/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5365 - accuracy: 0.8413 - val_loss: 0.6169 - val_accuracy: 0.8202
Epoch 30/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5364 - accuracy: 0.8416 - val_loss: 0.6161 - val_accuracy: 0.8184
Epoch 31/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5356 - accuracy: 0.8416 - val_loss: 0.6148 - val_accuracy: 0.8194
Epoch 32/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5346 - accuracy: 0.8411 - val_loss: 0.6159 - val_accuracy: 0.8190
Epoch 33/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5346 - accuracy: 0.8420 - val_loss: 0.6170 - val_accuracy: 0.8194
Epoch 34/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5333 - accuracy: 0.8417 - val_loss: 0.6157 - val_accuracy: 0.8198
Epoch 35/100
375/375 [==============================] - 1s 4ms/step - loss: 0.5330 - accuracy: 0.8417 - val_loss: 0.6179 - val_accuracy: 0.8173
Epoch 36/100
375/375 [==============================] - 1s 3ms/step - loss: 0.5321 - accuracy: 0.8426 - val_loss: 0.6185 - val_accuracy: 0.8186

loss_baseline, accuracy_baseline = baseline_model.evaluate(X_valid, y_valid, verbose=2)

375/375 - 1s - loss: 0.6185 - accuracy: 0.8186 - 763ms/epoch - 2ms/step

plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
plt.plot(history.history['loss'], label='loss')
plt.plot(history.history['val_loss'], label='val_loss')
plt.legend()
plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'], label='accuracy')
plt.plot(history.history['val_accuracy'], label='val_accuracy')
plt.legend()
plt.show()

2. CNN model

# Learning rate schedule func

def scheduler(epoch, lr):
    if epoch < 5:
        return lr
    else:
        return lr * tf.math.exp(-0.1)

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(4, 3, activation='relu', input_shape=(IMG_SIZE, IMG_SIZE, 1)),
    tf.keras.layers.Conv2D(8, 3, activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.BatchNormalization(),
    
    tf.keras.layers.Conv2D(16, 3, activation='relu'),
    tf.keras.layers.Conv2D(16, 3, activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.BatchNormalization(),
    
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dropout(0.5),
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dropout(0.5),
    tf.keras.layers.Dense(NUM_CLASSES, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
    metrics=['accuracy']
)

history = model.fit(
    X_train, y_train,
    validation_data=(X_valid, y_valid),
    epochs=EPOCHS, batch_size=BATCH_SIZE,
    callbacks=CALLBACKS + [tf.keras.callbacks.LearningRateScheduler(scheduler)]
)

Epoch 1/100
375/375 [==============================] - 12s 10ms/step - loss: 0.7432 - accuracy: 0.7606 - val_loss: 0.8401 - val_accuracy: 0.7383 - lr: 0.0010
Epoch 2/100
375/375 [==============================] - 3s 9ms/step - loss: 0.2788 - accuracy: 0.9144 - val_loss: 0.1521 - val_accuracy: 0.9526 - lr: 0.0010
Epoch 3/100
375/375 [==============================] - 2s 6ms/step - loss: 0.1971 - accuracy: 0.9401 - val_loss: 0.1220 - val_accuracy: 0.9639 - lr: 0.0010
Epoch 4/100
375/375 [==============================] - 3s 7ms/step - loss: 0.1514 - accuracy: 0.9525 - val_loss: 0.1311 - val_accuracy: 0.9597 - lr: 0.0010
Epoch 5/100
375/375 [==============================] - 3s 9ms/step - loss: 0.1292 - accuracy: 0.9604 - val_loss: 0.1005 - val_accuracy: 0.9703 - lr: 0.0010
Epoch 6/100
375/375 [==============================] - 2s 6ms/step - loss: 0.1078 - accuracy: 0.9671 - val_loss: 0.1031 - val_accuracy: 0.9688 - lr: 9.0484e-04
Epoch 7/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0929 - accuracy: 0.9716 - val_loss: 0.0766 - val_accuracy: 0.9778 - lr: 8.1873e-04
Epoch 8/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0826 - accuracy: 0.9748 - val_loss: 0.0759 - val_accuracy: 0.9787 - lr: 7.4082e-04
Epoch 9/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0774 - accuracy: 0.9768 - val_loss: 0.0723 - val_accuracy: 0.9803 - lr: 6.7032e-04
Epoch 10/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0630 - accuracy: 0.9805 - val_loss: 0.0709 - val_accuracy: 0.9810 - lr: 6.0653e-04
Epoch 11/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0610 - accuracy: 0.9809 - val_loss: 0.0637 - val_accuracy: 0.9820 - lr: 5.4881e-04
Epoch 12/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0540 - accuracy: 0.9833 - val_loss: 0.0621 - val_accuracy: 0.9832 - lr: 4.9659e-04
Epoch 13/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0524 - accuracy: 0.9837 - val_loss: 0.0620 - val_accuracy: 0.9837 - lr: 4.4933e-04
Epoch 14/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0448 - accuracy: 0.9858 - val_loss: 0.0606 - val_accuracy: 0.9836 - lr: 4.0657e-04
Epoch 15/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0442 - accuracy: 0.9861 - val_loss: 0.0629 - val_accuracy: 0.9819 - lr: 3.6788e-04
Epoch 16/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0394 - accuracy: 0.9876 - val_loss: 0.0612 - val_accuracy: 0.9841 - lr: 3.3287e-04
Epoch 17/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0370 - accuracy: 0.9884 - val_loss: 0.0609 - val_accuracy: 0.9847 - lr: 3.0119e-04
Epoch 18/100
375/375 [==============================] - 3s 8ms/step - loss: 0.0385 - accuracy: 0.9879 - val_loss: 0.0600 - val_accuracy: 0.9843 - lr: 2.7253e-04
Epoch 19/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0313 - accuracy: 0.9902 - val_loss: 0.0633 - val_accuracy: 0.9840 - lr: 2.4660e-04
Epoch 20/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0281 - accuracy: 0.9910 - val_loss: 0.0646 - val_accuracy: 0.9833 - lr: 2.2313e-04
Epoch 21/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0286 - accuracy: 0.9912 - val_loss: 0.0629 - val_accuracy: 0.9842 - lr: 2.0190e-04
Epoch 22/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0282 - accuracy: 0.9910 - val_loss: 0.0585 - val_accuracy: 0.9862 - lr: 1.8268e-04
Epoch 23/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0255 - accuracy: 0.9920 - val_loss: 0.0631 - val_accuracy: 0.9849 - lr: 1.6530e-04
Epoch 24/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0255 - accuracy: 0.9918 - val_loss: 0.0666 - val_accuracy: 0.9849 - lr: 1.4957e-04
Epoch 25/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0243 - accuracy: 0.9919 - val_loss: 0.0621 - val_accuracy: 0.9857 - lr: 1.3534e-04
Epoch 26/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0227 - accuracy: 0.9926 - val_loss: 0.0634 - val_accuracy: 0.9852 - lr: 1.2246e-04
Epoch 27/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0227 - accuracy: 0.9926 - val_loss: 0.0620 - val_accuracy: 0.9857 - lr: 1.1080e-04
Epoch 28/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0217 - accuracy: 0.9930 - val_loss: 0.0621 - val_accuracy: 0.9862 - lr: 1.0026e-04
Epoch 29/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0212 - accuracy: 0.9934 - val_loss: 0.0630 - val_accuracy: 0.9856 - lr: 9.0718e-05
Epoch 30/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0201 - accuracy: 0.9934 - val_loss: 0.0632 - val_accuracy: 0.9853 - lr: 8.2085e-05
Epoch 31/100
375/375 [==============================] - 3s 7ms/step - loss: 0.0190 - accuracy: 0.9941 - val_loss: 0.0653 - val_accuracy: 0.9860 - lr: 7.4273e-05
Epoch 32/100
375/375 [==============================] - 2s 7ms/step - loss: 0.0203 - accuracy: 0.9934 - val_loss: 0.0635 - val_accuracy: 0.9858 - lr: 6.7205e-05
Epoch 33/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0192 - accuracy: 0.9940 - val_loss: 0.0649 - val_accuracy: 0.9862 - lr: 6.0810e-05
Epoch 34/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0184 - accuracy: 0.9940 - val_loss: 0.0652 - val_accuracy: 0.9855 - lr: 5.5023e-05
Epoch 35/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0185 - accuracy: 0.9938 - val_loss: 0.0657 - val_accuracy: 0.9857 - lr: 4.9787e-05
Epoch 36/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0188 - accuracy: 0.9939 - val_loss: 0.0649 - val_accuracy: 0.9862 - lr: 4.5049e-05
Epoch 37/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0178 - accuracy: 0.9939 - val_loss: 0.0633 - val_accuracy: 0.9863 - lr: 4.0762e-05
Epoch 38/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0180 - accuracy: 0.9943 - val_loss: 0.0645 - val_accuracy: 0.9865 - lr: 3.6883e-05
Epoch 39/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0164 - accuracy: 0.9946 - val_loss: 0.0653 - val_accuracy: 0.9859 - lr: 3.3373e-05
Epoch 40/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0171 - accuracy: 0.9946 - val_loss: 0.0658 - val_accuracy: 0.9858 - lr: 3.0197e-05
Epoch 41/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0152 - accuracy: 0.9952 - val_loss: 0.0655 - val_accuracy: 0.9859 - lr: 2.7324e-05
Epoch 42/100
375/375 [==============================] - 2s 6ms/step - loss: 0.0158 - accuracy: 0.9950 - val_loss: 0.0660 - val_accuracy: 0.9862 - lr: 2.4723e-05

loss_cnn, accuracy_cnn = model.evaluate(X_valid, y_valid, verbose=2)

375/375 - 1s - loss: 0.0660 - accuracy: 0.9862 - 998ms/epoch - 3ms/step

plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
plt.plot(history.history['loss'], label='loss')
plt.plot(history.history['val_loss'], label='val_loss')
plt.legend()
plt.subplot(1, 2, 2)
plt.plot(history.history['accuracy'], label='accuracy')
plt.plot(history.history['val_accuracy'], label='val_accuracy')
plt.legend()
plt.show()

Summary

pd.DataFrame(
    {'Model': ['Baseline model', 'CNN model'],
     'Loss': [loss_baseline, loss_cnn],
     'Accuracy score': [accuracy_baseline, accuracy_cnn]}
)

	Model	Loss	Accuracy score
0	Baseline model	0.618453	0.818583
1	CNN model	0.065979	0.986167

Validate on testset

Prepare testset

X_test = np.load(X_test_path)['arr_0']
y_test = np.load(y_test_path)['arr_0']

X_test = X_test.reshape(-1, 28, 28, 1)
X_test = X_test / 255.0

Predict

y_pred = model.predict(X_test, verbose=0)
y_pred = np.argmax(y_pred, axis=1)

print('Accuracy score on testset:', accuracy_score(y_test, y_pred))

Accuracy score on testset: 0.9553

plt.figure(figsize=(12, 10))
sns.heatmap(confusion_matrix(y_test, y_pred, normalize='pred'), cmap='viridis', annot=True)
plt.show()

Plot some error predicts

To display japanese characters

!pip install japanize-matplotlib

import japanize_matplotlib

kmnist_classmap = pd.read_csv('/kaggle/input/kuzushiji/kmnist_classmap.csv')
kmnist_classmap.head()

	index	codepoint	char
0	0	U+304A	お
1	1	U+304D	き
2	2	U+3059	す
3	3	U+3064	つ
4	4	U+306A	な

idx_error = []
for i in range(len(y_test)):
    if y_pred[i] != y_test[i]:
        idx_error.append(i)

plt.figure(figsize=(10, 10))
for i in range(25):
    idx = idx_error[i]
    pred = kmnist_classmap.loc[y_pred[idx], 'char']
    actual = kmnist_classmap.loc[y_test[idx], 'char']
    plt.subplot(5, 5, i + 1)
    plt.imshow(X_test[idx], cmap='viridis')
    plt.title(f'pred: {pred} actual: {actual}')
    plt.axis('off')
    plt.grid(False)
plt.show()

You can see that the error in the predictions lies in hard cases, leading to confusion between similar characters. My CNN model is customized from the VGG16 architecture (you can call it by VGG4), which is a very popular architecture for image classification. Additionally, I have used Batch Normalization to speed up the training process and Dropout to prevent overfitting. Furthermore, I implemented a Learning Rate Scheduler to reduce the learning rate after a few epochs, which helps the model converge faster

Overall, the model is doing a good job and serves as a solid baseline for further improvements.

Thank you for reading my notebook. I hope you enjoy it.