from d2l import torch as d2l
import torch
from torch import nn
from torch.nn import functional as F18.5 Natural Language Inference: Using Attention
We introduced the natural language inference task and the SNLI dataset in Section 18.4. In view of many models that are based on complex and deep architectures, Parikh et al. (2016) proposed to address natural language inference with attention mechanisms and called it a “decomposable attention model”. This results in a model without recurrent or convolutional layers, achieving the best result at the time on the SNLI dataset with much fewer parameters. In this section, we will describe and implement this attention-based method (with MLPs) for natural language inference, as depicted in Figure 18.5.1.
18.5.1 The Model
Rather than preserving the order of tokens in premises and hypotheses, we can simply align tokens in one text sequence to every token in the other, and vice versa, then compare and aggregate such information to predict the logical relationships between premises and hypotheses. Similar to alignment of tokens between source and target sentences in machine translation, the alignment of tokens between premises and hypotheses can be neatly accomplished by attention mechanisms.
Figure 18.5.2 depicts the natural language inference method using attention mechanisms. At a high level, it consists of three jointly trained steps: attending, comparing, and aggregating. We will illustrate them step by step in the following.
from d2l import tensorflow as d2l
import tensorflow as tf
import kerasfrom d2l import jax as d2l
import jax
from jax import numpy as jnp
from flax import nnx
import optax
import numpy as npfrom d2l import mxnet as d2l
from mxnet import gluon, init, np, npx
from mxnet.gluon import nn
npx.set_np()18.5.1.1 Attending
The first step is to align tokens in one text sequence to each token in the other sequence. Suppose that the premise is “i do need sleep” and the hypothesis is “i am tired”. Due to semantical similarity, we may wish to align “i” in the hypothesis with “i” in the premise, and align “tired” in the hypothesis with “sleep” in the premise. Likewise, we may wish to align “i” in the premise with “i” in the hypothesis, and align “need” and “sleep” in the premise with “tired” in the hypothesis. Note that such alignment is soft using weighted average, where ideally large weights are associated with the tokens to be aligned. For ease of demonstration, Figure 18.5.2 shows such alignment in a hard way.
Now we describe the soft alignment using attention mechanisms in more detail. Denote by \(\mathbf{A} = (\mathbf{a}_1, \ldots, \mathbf{a}_m)\) and \(\mathbf{B} = (\mathbf{b}_1, \ldots, \mathbf{b}_n)\) the premise and hypothesis, whose number of tokens are \(m\) and \(n\), respectively, where \(\mathbf{a}_i, \mathbf{b}_j \in \mathbb{R}^{d}\) (\(i = 1, \ldots, m, j = 1, \ldots, n\)) is a \(d\)-dimensional word vector. For soft alignment, we compute the attention weights \(e_{ij} \in \mathbb{R}\) as
\[e_{ij} = f(\mathbf{a}_i)^\top f(\mathbf{b}_j), \tag{18.5.1}\]
where the function \(f\) is an MLP defined in the following mlp function. The output dimension of \(f\) is specified by the num_hiddens argument of mlp.
def mlp(num_inputs, num_hiddens, flatten):
net = []
net.append(nn.Dropout(0.2))
net.append(nn.Linear(num_inputs, num_hiddens))
net.append(nn.ReLU())
if flatten:
net.append(nn.Flatten(start_dim=1))
net.append(nn.Dropout(0.2))
net.append(nn.Linear(num_hiddens, num_hiddens))
net.append(nn.ReLU())
if flatten:
net.append(nn.Flatten(start_dim=1))
return nn.Sequential(*net)def mlp(num_hiddens, flatten):
net = keras.Sequential()
net.add(keras.layers.Dropout(0.2))
net.add(keras.layers.Dense(num_hiddens, activation='relu'))
if flatten:
net.add(keras.layers.Flatten())
net.add(keras.layers.Dropout(0.2))
net.add(keras.layers.Dense(num_hiddens, activation='relu'))
if flatten:
net.add(keras.layers.Flatten())
return netclass MLP(nnx.Module):
def __init__(self, num_inputs, num_hiddens, flatten, rngs=None):
rngs = nnx.Rngs(params=0, dropout=1) if rngs is None else rngs
self.dropout1 = nnx.Dropout(0.2, rngs=rngs)
self.dense1 = nnx.Linear(num_inputs, num_hiddens, rngs=rngs)
self.dropout2 = nnx.Dropout(0.2, rngs=rngs)
self.dense2 = nnx.Linear(num_hiddens, num_hiddens, rngs=rngs)
self.flatten = flatten
def __call__(self, x):
x = nnx.relu(self.dense1(self.dropout1(x)))
if self.flatten:
x = x.reshape((x.shape[0], -1))
x = nnx.relu(self.dense2(self.dropout2(x)))
if self.flatten:
x = x.reshape((x.shape[0], -1))
return xdef mlp(num_hiddens, flatten):
net = nn.Sequential()
net.add(nn.Dropout(0.2))
net.add(nn.Dense(num_hiddens, activation='relu', flatten=flatten))
net.add(nn.Dropout(0.2))
net.add(nn.Dense(num_hiddens, activation='relu', flatten=flatten))
return netIt should be highlighted that, in Equation 18.5.1 \(f\) takes inputs \(\mathbf{a}_i\) and \(\mathbf{b}_j\) separately rather than taking a pair of them together as input. This decomposition trick leads to only \(m + n\) applications (linear complexity) of \(f\) rather than \(mn\) applications (quadratic complexity).
Normalizing the attention weights in Equation 18.5.1, we compute the weighted average of all the token vectors in the hypothesis to obtain representation of the hypothesis that is softly aligned with the token indexed by \(i\) in the premise:
\[ \boldsymbol{\beta}_i = \sum_{j=1}^{n}\frac{\exp(e_{ij})}{ \sum_{k=1}^{n} \exp(e_{ik})} \mathbf{b}_j. \]
Likewise, we compute soft alignment of premise tokens for each token indexed by \(j\) in the hypothesis:
\[ \boldsymbol{\alpha}_j = \sum_{i=1}^{m}\frac{\exp(e_{ij})}{ \sum_{k=1}^{m} \exp(e_{kj})} \mathbf{a}_i. \]
Below we define the Attend class to compute the soft alignment of hypotheses (beta) with input premises A and soft alignment of premises (alpha) with input hypotheses B.
class Attend(nn.Module):
def __init__(self, num_inputs, num_hiddens, **kwargs):
super(Attend, self).__init__(**kwargs)
self.f = mlp(num_inputs, num_hiddens, flatten=False)
def forward(self, A, B):
# Shape of `A`/`B`: (`batch_size`, no. of tokens in sequence A/B,
# `embed_size`)
# Shape of `f_A`/`f_B`: (`batch_size`, no. of tokens in sequence A/B,
# `num_hiddens`)
f_A = self.f(A)
f_B = self.f(B)
# Shape of `e`: (`batch_size`, no. of tokens in sequence A,
# no. of tokens in sequence B)
e = torch.bmm(f_A, f_B.permute(0, 2, 1))
# Shape of `beta`: (`batch_size`, no. of tokens in sequence A,
# `embed_size`), where sequence B is softly aligned with each token
# (axis 1 of `beta`) in sequence A
beta = torch.bmm(F.softmax(e, dim=-1), B)
# Shape of `alpha`: (`batch_size`, no. of tokens in sequence B,
# `embed_size`), where sequence A is softly aligned with each token
# (axis 1 of `alpha`) in sequence B
alpha = torch.bmm(F.softmax(e.permute(0, 2, 1), dim=-1), A)
return beta, alphaclass Attend(keras.layers.Layer):
def __init__(self, num_hiddens, **kwargs):
super(Attend, self).__init__(**kwargs)
self.f = mlp(num_hiddens=num_hiddens, flatten=False)
def call(self, A, B):
# Shape of `A`/`B`: (`batch_size`, no. of tokens in sequence A/B,
# `embed_size`)
# Shape of `f_A`/`f_B`: (`batch_size`, no. of tokens in sequence A/B,
# `num_hiddens`)
f_A = self.f(A)
f_B = self.f(B)
# Shape of `e`: (`batch_size`, no. of tokens in sequence A,
# no. of tokens in sequence B)
e = tf.matmul(f_A, tf.transpose(f_B, perm=[0, 2, 1]))
# Shape of `beta`: (`batch_size`, no. of tokens in sequence A,
# `embed_size`), where sequence B is softly aligned with each token
# (axis 1 of `beta`) in sequence A
beta = tf.matmul(tf.nn.softmax(e, axis=-1), B)
# Shape of `alpha`: (`batch_size`, no. of tokens in sequence B,
# `embed_size`), where sequence A is softly aligned with each token
# (axis 1 of `alpha`) in sequence B
alpha = tf.matmul(
tf.nn.softmax(tf.transpose(e, perm=[0, 2, 1]), axis=-1), A)
return beta, alphaclass Attend(nnx.Module):
def __init__(self, embed_size, num_hiddens, rngs=None):
self.f = MLP(embed_size, num_hiddens, flatten=False, rngs=rngs)
def __call__(self, A, B):
# Shape of `A`/`B`: (`batch_size`, no. of tokens in sequence A/B,
# `embed_size`)
# Shape of `f_A`/`f_B`: (`batch_size`, no. of tokens in sequence A/B,
# `num_hiddens`)
f_A = self.f(A)
f_B = self.f(B)
# Shape of `e`: (`batch_size`, no. of tokens in sequence A,
# no. of tokens in sequence B)
e = jnp.matmul(f_A, f_B.transpose(0, 2, 1))
# Shape of `beta`: (`batch_size`, no. of tokens in sequence A,
# `embed_size`), where sequence B is softly aligned with each token
# (axis 1 of `beta`) in sequence A
beta = jnp.matmul(jax.nn.softmax(e, axis=-1), B)
# Shape of `alpha`: (`batch_size`, no. of tokens in sequence B,
# `embed_size`), where sequence A is softly aligned with each token
# (axis 1 of `alpha`) in sequence B
alpha = jnp.matmul(jax.nn.softmax(e.transpose(0, 2, 1), axis=-1), A)
return beta, alphaclass Attend(nn.Block):
def __init__(self, num_hiddens):
super().__init__()
self.f = mlp(num_hiddens=num_hiddens, flatten=False)
def forward(self, A, B):
# Shape of `A`/`B`: (`batch_size`, no. of tokens in sequence A/B,
# `embed_size`)
# Shape of `f_A`/`f_B`: (`batch_size`, no. of tokens in sequence A/B,
# `num_hiddens`)
f_A = self.f(A)
f_B = self.f(B)
# Shape of `e`: (`batch_size`, no. of tokens in sequence A,
# no. of tokens in sequence B)
e = npx.batch_dot(f_A, f_B, transpose_b=True)
# Shape of `beta`: (`batch_size`, no. of tokens in sequence A,
# `embed_size`), where sequence B is softly aligned with each token
# (axis 1 of `beta`) in sequence A
beta = npx.batch_dot(npx.softmax(e), B)
# Shape of `alpha`: (`batch_size`, no. of tokens in sequence B,
# `embed_size`), where sequence A is softly aligned with each token
# (axis 1 of `alpha`) in sequence B
alpha = npx.batch_dot(npx.softmax(e.transpose(0, 2, 1)), A)
return beta, alpha18.5.1.2 Comparing
In the next step, we compare a token in one sequence with the other sequence that is softly aligned with that token. Note that in soft alignment, all the tokens from one sequence, though with probably different attention weights, will be compared with a token in the other sequence. For ease of demonstration, Figure 18.5.2 pairs tokens with aligned tokens in a hard way. For example, suppose that the attending step determines that “need” and “sleep” in the premise are both aligned with “tired” in the hypothesis, the pair “tired–need sleep” will be compared.
In the comparing step, we feed the concatenation (operator \([\cdot, \cdot]\)) of tokens from one sequence and aligned tokens from the other sequence into a function \(g\) (an MLP):
\[\mathbf{v}_{A,i} = g([\mathbf{a}_i, \boldsymbol{\beta}_i]), i = 1, \ldots, m\\ \mathbf{v}_{B,j} = g([\mathbf{b}_j, \boldsymbol{\alpha}_j]), j = 1, \ldots, n. \tag{18.5.2}\]
In Equation 18.5.2, \(\mathbf{v}_{A,i}\) is the comparison between token \(i\) in the premise and all the hypothesis tokens that are softly aligned with token \(i\); while \(\mathbf{v}_{B,j}\) is the comparison between token \(j\) in the hypothesis and all the premise tokens that are softly aligned with token \(j\). The following Compare class defines such a comparing step.
class Compare(nn.Module):
def __init__(self, num_inputs, num_hiddens, **kwargs):
super(Compare, self).__init__(**kwargs)
self.g = mlp(num_inputs, num_hiddens, flatten=False)
def forward(self, A, B, beta, alpha):
V_A = self.g(torch.cat([A, beta], dim=2))
V_B = self.g(torch.cat([B, alpha], dim=2))
return V_A, V_Bclass Compare(keras.layers.Layer):
def __init__(self, num_hiddens, **kwargs):
super(Compare, self).__init__(**kwargs)
self.g = mlp(num_hiddens=num_hiddens, flatten=False)
def call(self, A, B, beta, alpha):
V_A = self.g(tf.concat([A, beta], axis=2))
V_B = self.g(tf.concat([B, alpha], axis=2))
return V_A, V_Bclass Compare(nnx.Module):
def __init__(self, embed_size, num_hiddens, rngs=None):
self.g = MLP(2 * embed_size, num_hiddens, flatten=False, rngs=rngs)
def __call__(self, A, B, beta, alpha):
V_A = self.g(jnp.concatenate([A, beta], axis=2))
V_B = self.g(jnp.concatenate([B, alpha], axis=2))
return V_A, V_Bclass Compare(nn.Block):
def __init__(self, num_hiddens):
super().__init__()
self.g = mlp(num_hiddens=num_hiddens, flatten=False)
def forward(self, A, B, beta, alpha):
V_A = self.g(np.concatenate([A, beta], axis=2))
V_B = self.g(np.concatenate([B, alpha], axis=2))
return V_A, V_B18.5.1.3 Aggregating
With two sets of comparison vectors \(\mathbf{v}_{A,i}\) (\(i = 1, \ldots, m\)) and \(\mathbf{v}_{B,j}\) (\(j = 1, \ldots, n\)) on hand, in the last step we will aggregate such information to infer the logical relationship. We begin by summing up both sets:
\[ \mathbf{v}_A = \sum_{i=1}^{m} \mathbf{v}_{A,i}, \quad \mathbf{v}_B = \sum_{j=1}^{n}\mathbf{v}_{B,j}. \]
Next we feed the concatenation of both summarization results into function \(h\) (an MLP) to obtain the classification result of the logical relationship:
\[ \hat{\mathbf{y}} = h([\mathbf{v}_A, \mathbf{v}_B]). \]
The aggregation step is defined in the following Aggregate class.
class Aggregate(nn.Module):
def __init__(self, num_inputs, num_hiddens, num_outputs, **kwargs):
super(Aggregate, self).__init__(**kwargs)
self.h = mlp(num_inputs, num_hiddens, flatten=True)
self.linear = nn.Linear(num_hiddens, num_outputs)
def forward(self, V_A, V_B):
# Sum up both sets of comparison vectors
V_A = V_A.sum(dim=1)
V_B = V_B.sum(dim=1)
# Feed the concatenation of both summarization results into an MLP
Y_hat = self.linear(self.h(torch.cat([V_A, V_B], dim=1)))
return Y_hatclass Aggregate(keras.layers.Layer):
def __init__(self, num_hiddens, num_outputs, **kwargs):
super(Aggregate, self).__init__(**kwargs)
self.h = mlp(num_hiddens=num_hiddens, flatten=True)
self.linear = keras.layers.Dense(num_outputs)
def call(self, V_A, V_B):
# Sum up both sets of comparison vectors
V_A = tf.reduce_sum(V_A, axis=1)
V_B = tf.reduce_sum(V_B, axis=1)
# Feed the concatenation of both summarization results into an MLP
Y_hat = self.linear(self.h(tf.concat([V_A, V_B], axis=1)))
return Y_hatclass Aggregate(nnx.Module):
def __init__(self, num_hiddens, num_outputs, rngs=None):
rngs = nnx.Rngs(params=0, dropout=1) if rngs is None else rngs
self.h = MLP(2 * num_hiddens, num_hiddens, flatten=True, rngs=rngs)
self.output = nnx.Linear(num_hiddens, num_outputs, rngs=rngs)
def __call__(self, V_A, V_B):
# Sum up both sets of comparison vectors
V_A = V_A.sum(axis=1)
V_B = V_B.sum(axis=1)
# Feed the concatenation of both summarization results into an MLP
Y_hat = self.output(self.h(jnp.concatenate([V_A, V_B], axis=1)))
return Y_hatclass Aggregate(nn.Block):
def __init__(self, num_hiddens, num_outputs):
super().__init__()
self.h = mlp(num_hiddens=num_hiddens, flatten=True)
self.h.add(nn.Dense(num_outputs))
def forward(self, V_A, V_B):
# Sum up both sets of comparison vectors
V_A = V_A.sum(axis=1)
V_B = V_B.sum(axis=1)
# Feed the concatenation of both summarization results into an MLP
Y_hat = self.h(np.concatenate([V_A, V_B], axis=1))
return Y_hat18.5.1.4 Putting It All Together
By putting the attending, comparing, and aggregating steps together, we define the decomposable attention model to jointly train these three steps.
class DecomposableAttention(nn.Module):
def __init__(self, vocab, embed_size, num_hiddens, num_inputs_attend=100,
num_inputs_compare=200, num_inputs_agg=400, **kwargs):
super(DecomposableAttention, self).__init__(**kwargs)
self.embedding = nn.Embedding(len(vocab), embed_size)
self.attend = Attend(num_inputs_attend, num_hiddens)
self.compare = Compare(num_inputs_compare, num_hiddens)
# There are 3 possible outputs: entailment, contradiction, and neutral
self.aggregate = Aggregate(num_inputs_agg, num_hiddens, num_outputs=3)
def forward(self, X):
premises, hypotheses = X
A = self.embedding(premises)
B = self.embedding(hypotheses)
beta, alpha = self.attend(A, B)
V_A, V_B = self.compare(A, B, beta, alpha)
Y_hat = self.aggregate(V_A, V_B)
return Y_hatclass DecomposableAttention(keras.Model):
def __init__(self, vocab, embed_size, num_hiddens, **kwargs):
super(DecomposableAttention, self).__init__(**kwargs)
self.embedding = keras.layers.Embedding(len(vocab), embed_size)
self.attend = Attend(num_hiddens)
self.compare = Compare(num_hiddens)
# There are 3 possible outputs: entailment, contradiction, and neutral
self.aggregate = Aggregate(num_hiddens, 3)
def call(self, inputs, training=False, **kwargs):
premises, hypotheses = inputs
A = self.embedding(premises)
B = self.embedding(hypotheses)
beta, alpha = self.attend(A, B)
V_A, V_B = self.compare(A, B, beta, alpha)
Y_hat = self.aggregate(V_A, V_B)
return Y_hatclass DecomposableAttention(nnx.Module):
def __init__(self, vocab_size, embed_size, num_hiddens, rngs=None):
rngs = nnx.Rngs(params=0, dropout=1) if rngs is None else rngs
self.embedding = nnx.Embed(vocab_size, embed_size, rngs=rngs)
self.attend = Attend(embed_size, num_hiddens, rngs=rngs)
self.compare = Compare(embed_size, num_hiddens, rngs=rngs)
self.aggregate = Aggregate(num_hiddens, 3, rngs=rngs)
def __call__(self, premises, hypotheses):
A = self.embedding(premises)
B = self.embedding(hypotheses)
beta, alpha = self.attend(A, B)
V_A, V_B = self.compare(A, B, beta, alpha)
# There are 3 possible outputs: entailment, contradiction, and neutral
Y_hat = self.aggregate(V_A, V_B)
return Y_hatclass DecomposableAttention(nn.Block):
def __init__(self, vocab, embed_size, num_hiddens):
super().__init__()
self.embedding = nn.Embedding(len(vocab), embed_size)
self.attend = Attend(num_hiddens)
self.compare = Compare(num_hiddens)
# There are 3 possible outputs: entailment, contradiction, and neutral
self.aggregate = Aggregate(num_hiddens, 3)
def forward(self, X):
premises, hypotheses = X
A = self.embedding(premises)
B = self.embedding(hypotheses)
beta, alpha = self.attend(A, B)
V_A, V_B = self.compare(A, B, beta, alpha)
Y_hat = self.aggregate(V_A, V_B)
return Y_hat18.5.2 Training and Evaluating the Model
Now we will train and evaluate the defined decomposable attention model on the SNLI dataset. We begin by reading the dataset.
18.5.2.1 Reading the dataset
We download and read the SNLI dataset using the function defined in Section 18.4. Most implementations use minibatches of \(256\) examples padded or truncated to \(50\) tokens. The JAX implementation uses minibatches of \(512\) and length \(32\) to avoid spending most of its time on padding. With the tokenization performed by read_snli, about 1.2% of the labeled training premises and 0.02% of the hypotheses exceed 32 tokens. This substantially reduces computation while introducing little additional truncation.
batch_size, num_steps = 256, 50
train_iter, test_iter, vocab = d2l.load_data_snli(batch_size, num_steps)read 549367 examplesread 9824 examplesbatch_size, num_steps = 256, 50
train_iter, test_iter, vocab = d2l.load_data_snli(batch_size, num_steps)read 549367 examples
read 9824 examplesbatch_size, num_steps = 512, 32
train_iter, test_iter, vocab = d2l.load_data_snli(batch_size, num_steps)read 549367 examples
read 9824 examplesbatch_size, num_steps = 256, 50
train_iter, test_iter, vocab = d2l.load_data_snli(batch_size, num_steps)read 549367 examplesread 9824 examples18.5.2.2 Creating the Model
We use the pretrained 100-dimensional GloVe embedding to represent the input tokens. Thus, we predefine the dimension of vectors \(\mathbf{a}_i\) and \(\mathbf{b}_j\) in Equation 18.5.1 as 100. The output dimension of functions \(f\) in Equation 18.5.1 and \(g\) in Equation 18.5.2 is set to 200. Then we create a model instance, initialize its parameters, and load the GloVe embedding to initialize vectors of input tokens.
embed_size, num_hiddens, devices = 100, 200, d2l.try_all_gpus()
net = DecomposableAttention(vocab, embed_size, num_hiddens)
glove_embedding = d2l.TokenEmbedding('glove.6b.100d')
embeds = glove_embedding[vocab.idx_to_token]
net.embedding.weight.data.copy_(embeds);embed_size, num_hiddens, devices = 100, 200, d2l.try_all_gpus()
net = DecomposableAttention(vocab, embed_size, num_hiddens)
glove_embedding = d2l.TokenEmbedding('glove.6b.100d')
embeds = glove_embedding[vocab.idx_to_token]
# Build the embedding layer before assigning pretrained weights.
net.embedding.build((None,))
net.embedding.set_weights([embeds])embed_size, num_hiddens, devices = 100, 200, d2l.try_all_gpus()
net = DecomposableAttention(len(vocab), embed_size, num_hiddens)
glove_embedding = d2l.TokenEmbedding('glove.6b.100d')
embeds = glove_embedding[vocab.idx_to_token]embed_size, num_hiddens, devices = 100, 200, d2l.try_all_gpus()
net = DecomposableAttention(vocab, embed_size, num_hiddens)
net.initialize(init.Xavier(), ctx=devices)
glove_embedding = d2l.TokenEmbedding('glove.6b.100d')
embeds = glove_embedding[vocab.idx_to_token]
net.embedding.weight.set_data(embeds)18.5.2.3 Training and Evaluating the Model
In contrast to the split_batch function in Section 13.5 that takes single inputs such as text sequences (or images), we define a split_batch_multi_inputs function to take multiple inputs such as premises and hypotheses in minibatches.
def split_batch_multi_inputs(X, y, devices):
"""Split multi-input `X` and `y` into multiple devices."""
X = list(zip(*[gluon.utils.split_and_load(
feature, devices, even_split=False) for feature in X]))
return (X, gluon.utils.split_and_load(y, devices, even_split=False))Now we can train and evaluate the model on the SNLI dataset.
lr, num_epochs = 0.001, 4
trainer = torch.optim.Adam(net.parameters(), lr=lr)
loss = nn.CrossEntropyLoss(reduction="none")
d2l.train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices)loss 0.495, train acc 0.805, test acc 0.825
30948.6 examples/sec on [device(type='cuda', index=0)]
lr, num_epochs = 0.001, 4
# Wrap tf.data batches: each yields (premises, hypotheses, labels);
# Keras model expects (X, y) where X = (premises, hypotheses)
def reformat(premises, hypotheses, labels):
return (premises, hypotheses), labels
train_iter_tf = train_iter.map(reformat)
test_iter_tf = test_iter.map(reformat)
net.compile(optimizer=keras.optimizers.Adam(lr),
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
net.fit(train_iter_tf, validation_data=test_iter_tf, epochs=num_epochs)Epoch 1/4 1/2146 ━━━━━━━━━━━━━━━━━━━━ 14:38:54 25s/step - accuracy: 0.3359 - loss: 2.9821 12/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.3354 - loss: 3.4116 23/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.3386 - loss: 2.7819 34/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.3393 - loss: 2.4373 45/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.3410 - loss: 2.2193 56/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.3434 - loss: 2.0677 68/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3457 - loss: 1.9468 81/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3482 - loss: 1.8484 92/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3503 - loss: 1.7826 105/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3528 - loss: 1.7193 117/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3549 - loss: 1.6711 128/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3567 - loss: 1.6334 137/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3581 - loss: 1.6063 147/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3596 - loss: 1.5795 157/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3610 - loss: 1.5554 170/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3628 - loss: 1.5276 182/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3643 - loss: 1.5049 194/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.3658 - loss: 1.4846 206/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3672 - loss: 1.4662 217/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3685 - loss: 1.4508 231/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3702 - loss: 1.4330 242/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3715 - loss: 1.4201 253/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3728 - loss: 1.4082 263/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3740 - loss: 1.3980 274/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3752 - loss: 1.3876 284/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3763 - loss: 1.3786 296/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3777 - loss: 1.3684 307/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3790 - loss: 1.3597 318/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3803 - loss: 1.3514 329/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3815 - loss: 1.3435 341/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3830 - loss: 1.3353 353/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3844 - loss: 1.3276 364/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3857 - loss: 1.3208 374/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3869 - loss: 1.3148 384/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3881 - loss: 1.3091 394/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3894 - loss: 1.3036 404/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3906 - loss: 1.2983 414/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3919 - loss: 1.2931 425/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3932 - loss: 1.2876 435/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3945 - loss: 1.2828 447/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.3960 - loss: 1.2772 459/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.3975 - loss: 1.2717 471/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.3989 - loss: 1.2665 483/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4004 - loss: 1.2614 495/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4019 - loss: 1.2565 507/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4034 - loss: 1.2517 518/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4047 - loss: 1.2475 528/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4060 - loss: 1.2437 538/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4072 - loss: 1.2400 548/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4084 - loss: 1.2364 558/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4096 - loss: 1.2328 569/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4109 - loss: 1.2290 580/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4122 - loss: 1.2253 591/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4135 - loss: 1.2217 602/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4148 - loss: 1.2182 613/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4161 - loss: 1.2147 624/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4173 - loss: 1.2113 635/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4186 - loss: 1.2080 647/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4199 - loss: 1.2045 659/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.4213 - loss: 1.2010 670/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4225 - loss: 1.1979 681/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4237 - loss: 1.1949 693/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4250 - loss: 1.1916 703/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4261 - loss: 1.1889 714/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4272 - loss: 1.1861 724/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4283 - loss: 1.1835 734/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4294 - loss: 1.1810 745/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4305 - loss: 1.1782 756/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4316 - loss: 1.1755 767/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4328 - loss: 1.1729 778/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4339 - loss: 1.1703 788/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4349 - loss: 1.1680 800/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4361 - loss: 1.1652 812/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4373 - loss: 1.1625 823/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4384 - loss: 1.1601 834/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4395 - loss: 1.1577 845/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4405 - loss: 1.1553 856/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4416 - loss: 1.1530 866/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4425 - loss: 1.1509 877/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.4436 - loss: 1.1487 891/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4449 - loss: 1.1458 902/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4459 - loss: 1.1437 912/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4468 - loss: 1.1417 922/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4477 - loss: 1.1398 932/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4486 - loss: 1.1378 943/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4496 - loss: 1.1358 954/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4506 - loss: 1.1337 966/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4516 - loss: 1.1315 978/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4527 - loss: 1.1294 989/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4536 - loss: 1.12741000/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4546 - loss: 1.12551012/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4556 - loss: 1.12341021/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4563 - loss: 1.12181031/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4572 - loss: 1.12011041/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4580 - loss: 1.11851052/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4589 - loss: 1.11661063/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4598 - loss: 1.11481075/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4608 - loss: 1.11291087/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.4617 - loss: 1.11101099/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4627 - loss: 1.10911111/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4636 - loss: 1.10721124/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4646 - loss: 1.10521137/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4656 - loss: 1.10331148/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4665 - loss: 1.10161159/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4673 - loss: 1.10001170/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4681 - loss: 1.09841181/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4689 - loss: 1.09681192/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4697 - loss: 1.09521203/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4706 - loss: 1.09371215/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4714 - loss: 1.09201225/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4721 - loss: 1.09061238/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4731 - loss: 1.08881250/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4739 - loss: 1.08721261/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4747 - loss: 1.08571271/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4754 - loss: 1.08441280/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4760 - loss: 1.08321291/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.4768 - loss: 1.08181301/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4774 - loss: 1.08051311/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4781 - loss: 1.07921322/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4789 - loss: 1.07781333/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4796 - loss: 1.07641344/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4803 - loss: 1.07511354/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4810 - loss: 1.07381365/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4817 - loss: 1.07251376/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4824 - loss: 1.07121386/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4830 - loss: 1.07001396/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4837 - loss: 1.06881406/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4843 - loss: 1.06761418/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4851 - loss: 1.06621430/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4858 - loss: 1.06481441/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4865 - loss: 1.06351452/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4872 - loss: 1.06231463/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4878 - loss: 1.06111473/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4884 - loss: 1.05991483/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4890 - loss: 1.05881493/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4896 - loss: 1.05771503/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4902 - loss: 1.05661514/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.4909 - loss: 1.05541525/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4915 - loss: 1.05431536/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4922 - loss: 1.05311547/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4928 - loss: 1.05191559/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4935 - loss: 1.05071571/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4942 - loss: 1.04941582/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4948 - loss: 1.04831595/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4955 - loss: 1.04691606/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4962 - loss: 1.04581617/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4968 - loss: 1.04471630/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4975 - loss: 1.04341640/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4980 - loss: 1.04241650/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4986 - loss: 1.04141661/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4992 - loss: 1.04031672/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.4998 - loss: 1.03931683/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.5004 - loss: 1.03821694/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.5010 - loss: 1.03711705/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.5015 - loss: 1.03611715/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.5021 - loss: 1.03511724/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.5025 - loss: 1.03431735/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5031 - loss: 1.03331746/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5037 - loss: 1.03221756/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5042 - loss: 1.03131765/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5047 - loss: 1.03051775/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5052 - loss: 1.02961786/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5057 - loss: 1.02861797/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5063 - loss: 1.02761807/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5068 - loss: 1.02671816/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5072 - loss: 1.02591826/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5077 - loss: 1.02501837/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5083 - loss: 1.02401849/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5089 - loss: 1.02301860/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5094 - loss: 1.02201871/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5100 - loss: 1.02111881/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5104 - loss: 1.02021892/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5110 - loss: 1.01931902/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5115 - loss: 1.01841911/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5119 - loss: 1.01771920/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5123 - loss: 1.01691929/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.5128 - loss: 1.01611939/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5132 - loss: 1.01531949/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5137 - loss: 1.01451960/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5142 - loss: 1.01361971/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5147 - loss: 1.01271982/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5152 - loss: 1.01181992/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5157 - loss: 1.01102002/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5162 - loss: 1.01012012/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5166 - loss: 1.00932021/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5170 - loss: 1.00862030/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5174 - loss: 1.00792039/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5179 - loss: 1.00722048/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5183 - loss: 1.00652058/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5187 - loss: 1.00572067/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5191 - loss: 1.00502078/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5196 - loss: 1.00412089/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5201 - loss: 1.00332100/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5206 - loss: 1.00242113/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5211 - loss: 1.00142124/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5216 - loss: 1.00062134/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5221 - loss: 0.99992144/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.5225 - loss: 0.99912146/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.5226 - loss: 0.99892146/2146 ━━━━━━━━━━━━━━━━━━━━ 64s 18ms/step - accuracy: 0.6153 - loss: 0.8376 - val_accuracy: 0.7529 - val_loss: 0.6175Epoch 2/4 1/2146 ━━━━━━━━━━━━━━━━━━━━ 43:18 1s/step - accuracy: 0.7344 - loss: 0.6440 12/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7191 - loss: 0.6687 23/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7224 - loss: 0.6668 33/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7234 - loss: 0.6661 43/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7237 - loss: 0.6649 53/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7239 - loss: 0.6639 64/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7244 - loss: 0.6623 75/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7248 - loss: 0.6610 87/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7254 - loss: 0.6596 100/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7261 - loss: 0.6583 111/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7264 - loss: 0.6574 122/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7267 - loss: 0.6568 133/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7270 - loss: 0.6563 144/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7272 - loss: 0.6559 155/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7275 - loss: 0.6554 166/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7277 - loss: 0.6549 177/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7279 - loss: 0.6545 188/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7281 - loss: 0.6541 199/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7282 - loss: 0.6537 210/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7283 - loss: 0.6534 221/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7284 - loss: 0.6532 232/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7285 - loss: 0.6530 243/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7286 - loss: 0.6528 253/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7286 - loss: 0.6526 264/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7287 - loss: 0.6524 275/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7287 - loss: 0.6523 286/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7288 - loss: 0.6522 298/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7289 - loss: 0.6520 309/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7289 - loss: 0.6519 321/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7290 - loss: 0.6517 333/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7291 - loss: 0.6516 345/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7292 - loss: 0.6514 356/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7293 - loss: 0.6513 368/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7294 - loss: 0.6511 380/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7295 - loss: 0.6509 393/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7296 - loss: 0.6506 405/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7297 - loss: 0.6504 416/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7298 - loss: 0.6502 427/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7299 - loss: 0.6501 438/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7300 - loss: 0.6499 449/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7301 - loss: 0.6497 460/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7302 - loss: 0.6496 471/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7303 - loss: 0.6494 482/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7304 - loss: 0.6493 493/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7304 - loss: 0.6491 504/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7305 - loss: 0.6490 515/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7306 - loss: 0.6488 526/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7306 - loss: 0.6487 537/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7307 - loss: 0.6486 548/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7308 - loss: 0.6484 559/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7308 - loss: 0.6483 570/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7309 - loss: 0.6482 581/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7310 - loss: 0.6480 592/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7310 - loss: 0.6479 603/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7311 - loss: 0.6478 613/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7312 - loss: 0.6477 623/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7312 - loss: 0.6475 634/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7313 - loss: 0.6474 645/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7314 - loss: 0.6473 655/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7314 - loss: 0.6472 667/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7315 - loss: 0.6470 680/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7315 - loss: 0.6469 694/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7316 - loss: 0.6467 707/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7317 - loss: 0.6466 719/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7318 - loss: 0.6465 730/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7318 - loss: 0.6464 741/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7319 - loss: 0.6462 752/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7319 - loss: 0.6461 760/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7320 - loss: 0.6460 771/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7320 - loss: 0.6459 782/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7321 - loss: 0.6458 792/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7322 - loss: 0.6457 803/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7322 - loss: 0.6456 814/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7323 - loss: 0.6455 825/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7323 - loss: 0.6454 836/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7324 - loss: 0.6452 847/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7325 - loss: 0.6451 858/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7325 - loss: 0.6450 869/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7326 - loss: 0.6449 880/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7326 - loss: 0.6448 891/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7327 - loss: 0.6447 902/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7328 - loss: 0.6445 913/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7328 - loss: 0.6444 925/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7329 - loss: 0.6443 937/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7330 - loss: 0.6442 948/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7330 - loss: 0.6441 960/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7331 - loss: 0.6440 971/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7331 - loss: 0.6438 982/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7332 - loss: 0.6437 993/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7333 - loss: 0.64361005/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7333 - loss: 0.64351016/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7334 - loss: 0.64341027/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7335 - loss: 0.64331038/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7335 - loss: 0.64321050/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7336 - loss: 0.64301062/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7337 - loss: 0.64291073/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7337 - loss: 0.64281084/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7338 - loss: 0.64271095/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7338 - loss: 0.64261106/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7339 - loss: 0.64251117/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7340 - loss: 0.64241128/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7340 - loss: 0.64221139/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7341 - loss: 0.64211150/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7342 - loss: 0.64201161/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7342 - loss: 0.64191172/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7343 - loss: 0.64181183/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7344 - loss: 0.64171194/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7344 - loss: 0.64151205/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7345 - loss: 0.64141216/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7346 - loss: 0.64131227/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7346 - loss: 0.64121238/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7347 - loss: 0.64111248/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7347 - loss: 0.64101259/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7348 - loss: 0.64091270/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7349 - loss: 0.64071281/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7349 - loss: 0.64061292/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7350 - loss: 0.64051303/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7351 - loss: 0.64041314/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7351 - loss: 0.64031324/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7352 - loss: 0.64021336/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7353 - loss: 0.64001347/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7353 - loss: 0.63991358/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7354 - loss: 0.63981369/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7354 - loss: 0.63971380/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7355 - loss: 0.63961391/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7356 - loss: 0.63951402/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7356 - loss: 0.63941413/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7357 - loss: 0.63931424/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7357 - loss: 0.63911435/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7358 - loss: 0.63901446/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7359 - loss: 0.63891457/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7359 - loss: 0.63881468/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7360 - loss: 0.63871479/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7360 - loss: 0.63861491/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7361 - loss: 0.63851502/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7362 - loss: 0.63841512/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7362 - loss: 0.63831522/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7363 - loss: 0.63821533/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7363 - loss: 0.63811544/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7364 - loss: 0.63801556/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7365 - loss: 0.63781568/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7365 - loss: 0.63771580/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7366 - loss: 0.63761592/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7367 - loss: 0.63751603/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7367 - loss: 0.63741614/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7368 - loss: 0.63731625/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7368 - loss: 0.63721636/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7369 - loss: 0.63711647/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7369 - loss: 0.63691658/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7370 - loss: 0.63681669/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7371 - loss: 0.63671680/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7371 - loss: 0.63661691/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7372 - loss: 0.63651703/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7372 - loss: 0.63641714/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7373 - loss: 0.63631726/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7373 - loss: 0.63621739/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7374 - loss: 0.63611750/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7375 - loss: 0.63601760/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7375 - loss: 0.63591771/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7376 - loss: 0.63581782/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7376 - loss: 0.63571793/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7377 - loss: 0.63561803/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7377 - loss: 0.63551813/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7378 - loss: 0.63541823/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7378 - loss: 0.63531833/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7379 - loss: 0.63521843/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7379 - loss: 0.63511853/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7380 - loss: 0.63501863/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7380 - loss: 0.63501873/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7381 - loss: 0.63491883/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7381 - loss: 0.63481893/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7382 - loss: 0.63471903/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7382 - loss: 0.63461914/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7383 - loss: 0.63451926/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7383 - loss: 0.63441936/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7384 - loss: 0.63431947/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7384 - loss: 0.63421957/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7385 - loss: 0.63411967/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7385 - loss: 0.63401979/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7386 - loss: 0.63391992/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7386 - loss: 0.63382003/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7387 - loss: 0.63372014/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7387 - loss: 0.63362026/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7388 - loss: 0.63352038/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7389 - loss: 0.63342049/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7389 - loss: 0.63332060/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7390 - loss: 0.63322071/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7390 - loss: 0.63312082/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7391 - loss: 0.63302093/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7391 - loss: 0.63292104/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7392 - loss: 0.63282115/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7392 - loss: 0.63272125/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7393 - loss: 0.63262135/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7393 - loss: 0.63252145/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7394 - loss: 0.63242146/2146 ━━━━━━━━━━━━━━━━━━━━ 12s 5ms/step - accuracy: 0.7493 - loss: 0.6135 - val_accuracy: 0.7974 - val_loss: 0.5246Epoch 3/4 1/2146 ━━━━━━━━━━━━━━━━━━━━ 29:18 820ms/step - accuracy: 0.8125 - loss: 0.4944 13/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 4ms/step - accuracy: 0.7745 - loss: 0.5733 24/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7735 - loss: 0.5730 35/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7730 - loss: 0.5729 46/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7730 - loss: 0.5723 57/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7732 - loss: 0.5714 68/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7731 - loss: 0.5709 79/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7728 - loss: 0.5709 90/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7724 - loss: 0.5709 101/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7722 - loss: 0.5708 112/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7721 - loss: 0.5704 123/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7720 - loss: 0.5701 134/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7719 - loss: 0.5698 144/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7719 - loss: 0.5695 154/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7719 - loss: 0.5693 164/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7720 - loss: 0.5692 175/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7720 - loss: 0.5690 188/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7720 - loss: 0.5689 200/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7719 - loss: 0.5689 211/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7718 - loss: 0.5690 222/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7718 - loss: 0.5690 233/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7717 - loss: 0.5690 244/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7716 - loss: 0.5691 255/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7716 - loss: 0.5690 266/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7716 - loss: 0.5690 278/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7715 - loss: 0.5690 289/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7715 - loss: 0.5689 300/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7715 - loss: 0.5688 310/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7715 - loss: 0.5688 321/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7714 - loss: 0.5687 331/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7714 - loss: 0.5686 341/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7714 - loss: 0.5686 351/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7714 - loss: 0.5686 361/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7713 - loss: 0.5686 371/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7713 - loss: 0.5686 381/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7712 - loss: 0.5686 391/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7712 - loss: 0.5686 402/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7712 - loss: 0.5685 413/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7712 - loss: 0.5685 423/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7711 - loss: 0.5685 434/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7711 - loss: 0.5684 444/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7711 - loss: 0.5684 455/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7711 - loss: 0.5683 466/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7711 - loss: 0.5683 477/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7711 - loss: 0.5683 487/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5682 497/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5682 507/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5682 517/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5681 527/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5681 538/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5681 548/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5680 558/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5680 568/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5679 578/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5679 588/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5679 598/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7711 - loss: 0.5678 607/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5678 617/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5677 626/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5677 635/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5676 644/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5676 653/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5675 662/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5675 671/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7712 - loss: 0.5674 680/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7713 - loss: 0.5673 689/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7713 - loss: 0.5673 698/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7713 - loss: 0.5672 708/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7713 - loss: 0.5672 718/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7713 - loss: 0.5671 728/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7713 - loss: 0.5670 738/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7714 - loss: 0.5670 748/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7714 - loss: 0.5669 757/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7714 - loss: 0.5669 766/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7714 - loss: 0.5668 775/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7714 - loss: 0.5668 785/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7714 - loss: 0.5667 794/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7715 - loss: 0.5667 803/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7715 - loss: 0.5666 812/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7715 - loss: 0.5666 822/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7715 - loss: 0.5665 831/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7715 - loss: 0.5665 840/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7716 - loss: 0.5664 850/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7716 - loss: 0.5664 859/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7716 - loss: 0.5663 868/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7716 - loss: 0.5663 878/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7717 - loss: 0.5662 888/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7717 - loss: 0.5661 898/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7717 - loss: 0.5661 908/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7717 - loss: 0.5660 918/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7718 - loss: 0.5660 927/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7718 - loss: 0.5659 936/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7718 - loss: 0.5659 946/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7718 - loss: 0.5658 956/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7718 - loss: 0.5658 967/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7719 - loss: 0.5657 979/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7719 - loss: 0.5656 990/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7719 - loss: 0.56561001/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7719 - loss: 0.56551012/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7720 - loss: 0.56551023/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7720 - loss: 0.56541034/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7720 - loss: 0.56541044/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7720 - loss: 0.56531054/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7720 - loss: 0.56531064/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7721 - loss: 0.56521074/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7721 - loss: 0.56521085/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7721 - loss: 0.56511096/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7721 - loss: 0.56511106/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7722 - loss: 0.56501116/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7722 - loss: 0.56501126/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7722 - loss: 0.56501136/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7722 - loss: 0.56491146/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7722 - loss: 0.56491156/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7723 - loss: 0.56481166/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7723 - loss: 0.56481176/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7723 - loss: 0.56471185/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7723 - loss: 0.56471195/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7723 - loss: 0.56461205/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7724 - loss: 0.56461215/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7724 - loss: 0.56451224/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7724 - loss: 0.56451235/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7724 - loss: 0.56441246/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7725 - loss: 0.56441258/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7725 - loss: 0.56431271/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7725 - loss: 0.56431282/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7725 - loss: 0.56421293/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7726 - loss: 0.56421303/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7726 - loss: 0.56411313/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7726 - loss: 0.56411323/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7726 - loss: 0.56401333/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7727 - loss: 0.56401343/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7727 - loss: 0.56391354/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7727 - loss: 0.56391365/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7727 - loss: 0.56381375/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7728 - loss: 0.56381386/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7728 - loss: 0.56371397/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7728 - loss: 0.56371407/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7728 - loss: 0.56361418/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7729 - loss: 0.56361430/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7729 - loss: 0.56351441/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7729 - loss: 0.56351451/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7729 - loss: 0.56341461/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7730 - loss: 0.56341472/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7730 - loss: 0.56331483/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7730 - loss: 0.56331494/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7730 - loss: 0.56321505/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7730 - loss: 0.56321515/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7731 - loss: 0.56311525/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7731 - loss: 0.56311536/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7731 - loss: 0.56301548/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7731 - loss: 0.56301558/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7732 - loss: 0.56291569/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7732 - loss: 0.56291579/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7732 - loss: 0.56291590/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7732 - loss: 0.56281601/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7733 - loss: 0.56281612/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7733 - loss: 0.56271622/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7733 - loss: 0.56271633/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7733 - loss: 0.56261644/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7734 - loss: 0.56261654/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7734 - loss: 0.56251664/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7734 - loss: 0.56251675/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7734 - loss: 0.56241686/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7735 - loss: 0.56241697/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7735 - loss: 0.56231709/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7735 - loss: 0.56231721/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7735 - loss: 0.56221733/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7736 - loss: 0.56221745/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7736 - loss: 0.56211757/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7736 - loss: 0.56211767/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7736 - loss: 0.56201777/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7736 - loss: 0.56201788/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7737 - loss: 0.56191798/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7737 - loss: 0.56191809/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7737 - loss: 0.56181820/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7737 - loss: 0.56181830/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7738 - loss: 0.56181840/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7738 - loss: 0.56171850/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7738 - loss: 0.56171860/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7738 - loss: 0.56161870/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7738 - loss: 0.56161879/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7739 - loss: 0.56151889/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7739 - loss: 0.56151900/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7739 - loss: 0.56151910/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7739 - loss: 0.56141920/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7739 - loss: 0.56141931/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7740 - loss: 0.56131941/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7740 - loss: 0.56131951/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7740 - loss: 0.56131961/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7740 - loss: 0.56121972/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7740 - loss: 0.56121982/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7741 - loss: 0.56111992/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7741 - loss: 0.56112002/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7741 - loss: 0.56112012/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7741 - loss: 0.56102022/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7741 - loss: 0.56102033/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7742 - loss: 0.56092043/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7742 - loss: 0.56092053/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7742 - loss: 0.56092062/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7742 - loss: 0.56082072/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7742 - loss: 0.56082082/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7743 - loss: 0.56082093/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7743 - loss: 0.56072104/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7743 - loss: 0.56072115/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7743 - loss: 0.56062125/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7743 - loss: 0.56062135/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7744 - loss: 0.56062146/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7744 - loss: 0.56052146/2146 ━━━━━━━━━━━━━━━━━━━━ 12s 5ms/step - accuracy: 0.7781 - loss: 0.5529 - val_accuracy: 0.8103 - val_loss: 0.4891Epoch 4/4 1/2146 ━━━━━━━━━━━━━━━━━━━━ 28:29 797ms/step - accuracy: 0.7695 - loss: 0.5231 12/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7959 - loss: 0.5072 23/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7990 - loss: 0.5011 35/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.8005 - loss: 0.5001 46/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.8003 - loss: 0.5012 58/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.8001 - loss: 0.5020 70/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7999 - loss: 0.5032 80/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7998 - loss: 0.5038 90/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7998 - loss: 0.5044 100/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7996 - loss: 0.5051 110/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7993 - loss: 0.5060 120/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7991 - loss: 0.5069 130/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7988 - loss: 0.5076 140/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7986 - loss: 0.5083 150/2146 ━━━━━━━━━━━━━━━━━━━━ 10s 5ms/step - accuracy: 0.7983 - loss: 0.5089 160/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7981 - loss: 0.5095 170/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7979 - loss: 0.5099 180/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7977 - loss: 0.5103 190/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7975 - loss: 0.5107 201/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7973 - loss: 0.5111 212/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7971 - loss: 0.5115 224/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7970 - loss: 0.5118 236/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7969 - loss: 0.5121 248/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7968 - loss: 0.5124 260/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7968 - loss: 0.5126 272/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7967 - loss: 0.5128 284/2146 ━━━━━━━━━━━━━━━━━━━━ 9s 5ms/step - accuracy: 0.7966 - loss: 0.5130 296/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7965 - loss: 0.5132 309/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7965 - loss: 0.5134 322/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7964 - loss: 0.5135 335/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7963 - loss: 0.5137 348/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7963 - loss: 0.5139 360/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7962 - loss: 0.5141 373/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7961 - loss: 0.5142 386/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7961 - loss: 0.5144 399/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7960 - loss: 0.5145 412/2146 ━━━━━━━━━━━━━━━━━━━━ 8s 5ms/step - accuracy: 0.7960 - loss: 0.5146 425/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7959 - loss: 0.5148 438/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7958 - loss: 0.5149 451/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7958 - loss: 0.5151 464/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7957 - loss: 0.5152 477/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7956 - loss: 0.5154 490/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7956 - loss: 0.5156 503/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7955 - loss: 0.5157 515/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7954 - loss: 0.5159 526/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7954 - loss: 0.5160 537/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7953 - loss: 0.5161 549/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7953 - loss: 0.5162 561/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7952 - loss: 0.5164 573/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7952 - loss: 0.5165 584/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7951 - loss: 0.5166 595/2146 ━━━━━━━━━━━━━━━━━━━━ 7s 5ms/step - accuracy: 0.7951 - loss: 0.5166 605/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7950 - loss: 0.5167 615/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7950 - loss: 0.5168 625/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7950 - loss: 0.5169 635/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7949 - loss: 0.5170 646/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7949 - loss: 0.5171 657/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7949 - loss: 0.5171 668/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7948 - loss: 0.5172 678/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7948 - loss: 0.5173 690/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7948 - loss: 0.5174 701/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7947 - loss: 0.5175 711/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7947 - loss: 0.5175 722/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7947 - loss: 0.5176 732/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7947 - loss: 0.5176 742/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7947 - loss: 0.5177 753/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5177 764/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5178 774/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5178 785/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5179 796/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5179 806/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5180 816/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7946 - loss: 0.5180 826/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7945 - loss: 0.5180 836/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7945 - loss: 0.5181 846/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7945 - loss: 0.5181 856/2146 ━━━━━━━━━━━━━━━━━━━━ 6s 5ms/step - accuracy: 0.7945 - loss: 0.5181 866/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7945 - loss: 0.5182 876/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7945 - loss: 0.5182 887/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7945 - loss: 0.5182 898/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7945 - loss: 0.5183 909/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5183 921/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5184 931/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5184 941/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5184 952/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5184 963/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5185 974/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5185 984/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.5185 994/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.51861005/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7944 - loss: 0.51861016/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51861026/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51861036/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51871046/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51871056/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51871067/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51871077/2146 ━━━━━━━━━━━━━━━━━━━━ 5s 5ms/step - accuracy: 0.7943 - loss: 0.51881088/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7943 - loss: 0.51881099/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7943 - loss: 0.51881109/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51881120/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51891131/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51891141/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51891152/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51891162/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51891173/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51891185/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51901197/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51901209/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7942 - loss: 0.51901220/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51901231/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51901243/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51911254/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51911265/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51911276/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51911287/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51911298/2146 ━━━━━━━━━━━━━━━━━━━━ 4s 5ms/step - accuracy: 0.7941 - loss: 0.51921310/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7941 - loss: 0.51921321/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7941 - loss: 0.51921332/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51921343/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51921354/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51921365/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51921375/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51931385/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51931395/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51931405/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51931415/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51931425/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7940 - loss: 0.51931435/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51931444/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941454/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941463/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941472/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941481/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941490/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941500/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941509/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51941519/2146 ━━━━━━━━━━━━━━━━━━━━ 3s 5ms/step - accuracy: 0.7939 - loss: 0.51951530/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7939 - loss: 0.51951540/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7939 - loss: 0.51951550/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51951560/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51951570/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51951580/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51951590/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51951601/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51961611/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51961623/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51961634/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51961646/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7938 - loss: 0.51961658/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51961669/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51961680/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51971691/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51971701/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51971711/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51971721/2146 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7937 - loss: 0.51971731/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7937 - loss: 0.51971741/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7937 - loss: 0.51971752/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7937 - loss: 0.51981762/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7937 - loss: 0.51981772/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7937 - loss: 0.51981783/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51981795/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51981806/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51981817/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51981828/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51981838/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991848/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991859/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991869/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991879/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991890/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991901/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991911/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991923/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.51991935/2146 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7936 - loss: 0.52001945/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52001954/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52001964/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52001974/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52001984/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52001994/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52002004/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52002014/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52002024/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52002034/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52002044/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52002054/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012066/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012079/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012091/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012103/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012114/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012125/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012135/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012145/2146 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.7935 - loss: 0.52012146/2146 ━━━━━━━━━━━━━━━━━━━━ 11s 5ms/step - accuracy: 0.7929 - loss: 0.5209 - val_accuracy: 0.8184 - val_loss: 0.4732<keras.src.callbacks.history.History at 0x7f804badf350>lr, num_epochs = 0.001, 4
net.embedding.embedding[...] = jnp.array(embeds)
optimizer = nnx.Optimizer(net, optax.adam(lr), wrt=nnx.Param)
train_net = nnx.view(net, deterministic=False)
eval_net = nnx.view(net, deterministic=True)
@nnx.jit
def train_step(net, optimizer, premises, hypotheses, labels):
def loss_fn(model):
logits = model(premises, hypotheses)
return optax.softmax_cross_entropy_with_integer_labels(
logits, labels).mean()
loss, grads = nnx.value_and_grad(loss_fn)(net)
optimizer.update(net, grads)
return loss
@nnx.jit
def eval_step(net, premises, hypotheses, labels):
logits = net(premises, hypotheses)
return (logits.argmax(axis=-1) == labels).sum()
timer = d2l.Timer()
for epoch in range(num_epochs):
batch_losses, n_train = [], 0
for batch in train_iter:
premises, hypotheses, labels = batch[0], batch[1], batch[2]
batch_losses.append(train_step(
train_net, optimizer, premises, hypotheses, labels))
n_train += len(labels)
# Synchronize once per epoch, rather than once per minibatch.
train_loss = float(jnp.stack(batch_losses).mean())
# Evaluate on test set
batch_correct, n_test = [], 0
for batch in test_iter:
premises, hypotheses, labels = batch[0], batch[1], batch[2]
batch_correct.append(eval_step(
eval_net, premises, hypotheses, labels))
n_test += len(labels)
n_correct = int(jnp.stack(batch_correct).sum())
print(f'epoch {epoch + 1}, loss {train_loss:.4f}, '
f'test acc {n_correct / n_test:.4f}')
print(f'{num_epochs * n_train / timer.stop():.1f} examples/sec')epoch 1, loss 0.8543, test acc 0.7414epoch 2, loss 0.6290, test acc 0.7910epoch 3, loss 0.5628, test acc 0.8029epoch 4, loss 0.5267, test acc 0.8184
937.5 examples/seclr, num_epochs = 0.001, 4
trainer = gluon.Trainer(net.collect_params(), 'adam', {'learning_rate': lr})
loss = gluon.loss.SoftmaxCrossEntropyLoss()
d2l.train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices,
split_batch_multi_inputs)loss 0.513, train acc 0.796, test acc 0.821
2130.2 examples/sec on [gpu(0)]
18.5.2.4 Using the Model
Finally, define the prediction function to output the logical relationship between a pair of premise and hypothesis.
def predict_snli(net, vocab, premise, hypothesis):
"""Predict the logical relationship between the premise and hypothesis."""
net.eval()
premise = torch.tensor(vocab[premise], device=d2l.try_gpu())
hypothesis = torch.tensor(vocab[hypothesis], device=d2l.try_gpu())
label = torch.argmax(net([premise.reshape((1, -1)),
hypothesis.reshape((1, -1))]), dim=1)
return 'entailment' if label == 0 else 'contradiction' if label == 1 \
else 'neutral'
def predict_snli(net, vocab, premise, hypothesis):
"""Predict the logical relationship between the premise and hypothesis."""
premise = tf.constant([vocab[premise]], dtype=tf.int32)
hypothesis = tf.constant([vocab[hypothesis]], dtype=tf.int32)
label = tf.argmax(net((premise, hypothesis), training=False), axis=1)
return 'entailment' if label == 0 else 'contradiction' if label == 1 \
else 'neutral'
def predict_snli(net, vocab, premise, hypothesis):
"""Predict the logical relationship between the premise and hypothesis."""
premise = jnp.array(vocab[premise]).reshape((1, -1))
hypothesis = jnp.array(vocab[hypothesis]).reshape((1, -1))
label = jnp.argmax(nnx.view(net, deterministic=True)(premise, hypothesis),
axis=1)
return 'entailment' if label == 0 else 'contradiction' if label == 1 \
else 'neutral'
def predict_snli(net, vocab, premise, hypothesis):
"""Predict the logical relationship between the premise and hypothesis."""
premise = np.array(vocab[premise], ctx=d2l.try_gpu())
hypothesis = np.array(vocab[hypothesis], ctx=d2l.try_gpu())
label = np.argmax(net([premise.reshape((1, -1)),
hypothesis.reshape((1, -1))]), axis=1)
return 'entailment' if label == 0 else 'contradiction' if label == 1 \
else 'neutral'We can use the trained model to obtain the natural language inference result for a sample pair of sentences.
predict_snli(net, vocab, ['he', 'is', 'good', '.'], ['he', 'is', 'bad', '.'])'contradiction'predict_snli(net, vocab, ['he', 'is', 'good', '.'], ['he', 'is', 'bad', '.'])'contradiction'predict_snli(net, vocab, ['he', 'is', 'good', '.'],
['he', 'is', 'bad', '.'])'contradiction'predict_snli(net, vocab, ['he', 'is', 'good', '.'], ['he', 'is', 'bad', '.'])'contradiction'18.5.3 Summary
- The decomposable attention model consists of three steps for predicting the logical relationships between premises and hypotheses: attending, comparing, and aggregating.
- With attention mechanisms, we can align tokens in one text sequence to every token in the other, and vice versa. Such alignment is soft using weighted average, where ideally large weights are associated with the tokens to be aligned.
- The decomposition trick leads to a more desirable linear complexity than quadratic complexity when computing attention weights.
- We can use pretrained word vectors as the input representation for downstream natural language processing tasks such as natural language inference.
18.5.4 Exercises
- Train the model with other combinations of hyperparameters. Can you get better accuracy on the test set?
- What are major drawbacks of the decomposable attention model for natural language inference?
- Suppose that we want to get the level of semantical similarity (e.g., a continuous value between 0 and 1) for any pair of sentences. How shall we collect and label the dataset? Can you design a model with attention mechanisms?