Semantic Image Segmentation - Part 2¶
U-Net¶
U-net is built on top of the fully convolutional network from above. It was built for medical purposes to find tumors in the lungs or brain. It also consists of an encoder that downsamples the input image to a feature map and the decoder that samples the feature map to the size of the input image using learned deconvolution layers.
The main contribution of the U-Net architecture is the shortcut connections. We saw above in FCN that since we downsample an image as part of the encoder, we lose a lot of information that cannot be easily recovered at the encoder part. FCN tries to solve this by getting information from the pooling layers before the final feature layer.
U-Net proposes a new approach to solve this information loss problem. It proposes to send information to each upsampling layer in the decoder from the corresponding downsampling layer in the encoder, as can be seen in the figure above, thus capturing finer information while keeping the computation low. Since the layers at the beginning of the encoder would have more information, they would reinforce the decoder’s upsampling operation, providing fine details corresponding to the input images, greatly improving the results. The paper also suggested the use of a new loss function that we will discuss below.
It can be split into an encoder-decoder path or an expansion-contraction path equivalently.
Encoder (left side): Consists of repeatedly applying two 3x3 convolutions. Each conv is followed by a ReLU and batch normalization. Then, a 2x2 max pooling operation is applied to reduce the spatial dimensions. Again, at each downsampling step, we double the number of feature channels while halving the spatial dimensions.
Decoder path (right side): Each step in the expansion path consists of an upsampling of the feature map followed by a 2x2 transposition convolution, which halves the number of feature channels. We also have a concatenation with the corresponding feature map from the contracting path and usually a 3x3 convolution (each followed by a ReLU). In the final layer, a 1x1 convolution is used to map the channels to the desired number of classes.
In this U-Net example, we will use the following dataset: Mapping floodwater from radar images:
The dataset consists of SAR images, with Sentinel 1 VV and VH bands, and their respective masks representing flooded areas.
Let's then import the necessary libraries, functions and data:
In [ ]:
!pip install rasterio
Collecting rasterio
Downloading rasterio-1.3.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (21.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.3/21.3 MB 47.5 MB/s eta 0:00:00
Collecting affine (from rasterio)
Downloading affine-2.4.0-py3-none-any.whl (15 kB)
Requirement already satisfied: attrs in /usr/local/lib/python3.10/dist-packages (from rasterio) (23.1.0)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from rasterio) (2023.7.22)
Requirement already satisfied: click>=4.0 in /usr/local/lib/python3.10/dist-packages (from rasterio) (8.1.6)
Requirement already satisfied: cligj>=0.5 in /usr/local/lib/python3.10/dist-packages (from rasterio) (0.7.2)
Requirement already satisfied: numpy>=1.18 in /usr/local/lib/python3.10/dist-packages (from rasterio) (1.22.4)
Collecting snuggs>=1.4.1 (from rasterio)
Downloading snuggs-1.4.7-py3-none-any.whl (5.4 kB)
Requirement already satisfied: click-plugins in /usr/local/lib/python3.10/dist-packages (from rasterio) (1.1.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from rasterio) (67.7.2)
Requirement already satisfied: pyparsing>=2.1.6 in /usr/local/lib/python3.10/dist-packages (from snuggs>=1.4.1->rasterio) (3.1.0)
Installing collected packages: snuggs, affine, rasterio
Successfully installed affine-2.4.0 rasterio-1.3.8 snuggs-1.4.7
In [ ]:
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
In [ ]:
import rasterio
import pandas as pd
import glob
import os
import cv2
import numpy as np
import matplotlib.pyplot as plt
In [ ]:
path_csv = 'drive/My Drive/Datasets/Flood_challenge/flood-training-metadata.csv'
In [ ]:
metadata = pd.read_csv(path_csv)
In [ ]:
metadata.head()
Out[ ]:
| image_id | chip_id | flood_id | polarization | location | scene_start | |
|---|---|---|---|---|---|---|
| 0 | awc00_vh | awc00 | awc | vh | Bolivia | 2018-02-15 |
| 1 | awc00_vv | awc00 | awc | vv | Bolivia | 2018-02-15 |
| 2 | awc01_vh | awc01 | awc | vh | Bolivia | 2018-02-15 |
| 3 | awc01_vv | awc01 | awc | vv | Bolivia | 2018-02-15 |
| 4 | awc02_vh | awc02 | awc | vh | Bolivia | 2018-02-15 |
In [ ]:
len(pd.unique(metadata.chip_id))
Out[ ]:
542
In [ ]:
path_images = 'drive/My Drive/Datasets/Flood_challenge/train_features'
path_labels = 'drive/My Drive/Datasets/Flood_challenge/train_labels'
We can use Lee Filter to smooth and remove noise from the S1 bands:
In [ ]:
from scipy.ndimage.filters import uniform_filter
from scipy.ndimage.measurements import variance
<ipython-input-9-85178c643845>:1: DeprecationWarning: Please use `uniform_filter` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated. from scipy.ndimage.filters import uniform_filter <ipython-input-9-85178c643845>:2: DeprecationWarning: Please use `variance` from the `scipy.ndimage` namespace, the `scipy.ndimage.measurements` namespace is deprecated. from scipy.ndimage.measurements import variance
In [ ]:
def lee_filter(img, size):
img_mean = uniform_filter(img, (size, size))
img_sqr_mean = uniform_filter(img**2, (size, size))
img_variance = img_sqr_mean - img_mean**2
overall_variance = variance(img)
img_weights = img_variance**2 / (img_variance**2 + overall_variance**2)
img_output = img_mean + img_weights * (img - img_mean)
return img_output
Let's import input data and masks:
In [ ]:
X = []
for pref in pd.unique(metadata.chip_id):
import_vv = os.path.join(path_images,pref + '_vv.tif')
import_vh = os.path.join(path_images,pref + '_vh.tif')
with rasterio.open(import_vv) as src:
vv = src.read()
vv = vv.transpose([1,2,0])
vv = lee_filter(vv[:,:,0],5)
with rasterio.open(import_vh) as src:
vh = src.read()
vh = vh.transpose([1,2,0])
vh = lee_filter(vh[:,:,0],5)
im = np.dstack((vv,vh))
print(im.shape)
X.append(im)
X = np.array(X)
print(X.shape)
(512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (512, 512, 2) (542, 512, 512, 2)
In [ ]:
Y = []
for pref in pd.unique(metadata.chip_id):
import_lb = os.path.join(path_labels,pref + '.tif')
with rasterio.open(import_lb) as src:
lb = src.read()
lb = lb.transpose([1,2,0])
print(lb.shape)
Y.append(lb)
Y = np.array(Y)
print(Y.shape)
(512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (512, 512, 1) (542, 512, 512, 1)
After converting to numpy array, let's normalize the data to the range 0 to 1:
In [ ]:
X_int = cv2.normalize(X, None, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX)
In [ ]:
Y_int = np.where(Y==255,0,Y)
We can then plot the two bands and the mask:
In [ ]:
plt.figure(figsize=(20,8))
plt.subplot(1,3,1),
plt.imshow(X_int[10,:,:,0], cmap='Greys_r')
plt.subplot(1,3,2),
plt.imshow(X_int[10,:,:,1], cmap='Greys_r')
plt.subplot(1,3,3),
plt.imshow(Y_int[10,:,:,0], cmap='Greys_r')
Out[ ]:
<matplotlib.image.AxesImage at 0x7d1b03b49d20>
Let's split the data into training and testing:
In [ ]:
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
In [ ]:
x_train, x_test, y_train, y_test = train_test_split(X_int, Y_int, test_size=0.25, random_state=10)
In [ ]:
y_train = y_train.astype('float')
y_test = y_test.astype('float')
In [ ]:
del X,Y, X_int, Y_int
In [ ]:
from keras.models import Model
from keras.layers import Input, Conv2D, MaxPooling2D, UpSampling2D, concatenate, Concatenate, add, Conv2DTranspose, BatchNormalization, Dropout, Activation, Add, AveragePooling2D, Lambda, SeparableConv2D, GlobalAveragePooling2D, DepthwiseConv2D, ZeroPadding2D, LeakyReLU
#from tensorflow.keras.optimizers import Adam
from keras.optimizers import Adam
from keras.activations import relu
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from keras.losses import binary_crossentropy
from keras import backend as K
import tensorflow as tf
from keras.models import load_model
from keras.models import model_from_json
from tensorflow.keras.losses import Dice
In this example, we will use data augmentation to generate more training samples:
In [ ]:
img_datagen = ImageDataGenerator(
rotation_range=90,
vertical_flip = True,
horizontal_flip=True,
width_shift_range = 0.2,
height_shift_range = 0.2,
zoom_range = 0.2)
mask_datagen = ImageDataGenerator(
rotation_range=90,
vertical_flip = True,
horizontal_flip=True,
width_shift_range = 0.2,
height_shift_range = 0.2,
zoom_range = 0.2)
In [ ]:
img_datagen.fit(x_train, augment=True,seed=1200)
mask_datagen.fit(y_train, augment=True,seed=1200)
In [ ]:
train_generator=img_datagen.flow(x_train,y_train,batch_size=8,seed=1200)
/usr/local/lib/python3.10/dist-packages/keras/src/legacy/preprocessing/image.py:619: UserWarning: NumpyArrayIterator is set to use the data format convention "channels_last" (channels on axis 3), i.e. expected either 1, 3, or 4 channels on axis 3. However, it was passed an array with shape (379, 512, 512, 2) (2 channels). warnings.warn(
In [ ]:
steps_per_epoch = len(x_train)//8
validation_steps = len(x_test)//8
So let's implement U-Net:
In [ ]:
upconv = True
droprate = 0.5
inputs = Input(shape=x_train.shape[1:])
conv1 = Conv2D(32, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(inputs)
conv1 = Conv2D(32, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
pool1 = BatchNormalization()(pool1)
conv2 = Conv2D(64, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(pool1)
conv2 = Conv2D(64, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
pool2 = Dropout(droprate)(pool2)
pool2 = BatchNormalization()(pool2)
conv3 = Conv2D(128, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(pool2)
conv3 = Conv2D(128, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
pool3 = Dropout(droprate)(pool3)
pool3 = BatchNormalization()(pool3)
conv4 = Conv2D(256, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(pool3)
conv4 = Conv2D(256, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2))(conv4)
pool4 = Dropout(droprate)(pool4)
pool4 = BatchNormalization()(pool4)
conv5 = Conv2D(512, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(pool4)
conv5 = Conv2D(512, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv5)
pool5 = MaxPooling2D(pool_size=(2, 2))(conv5)
pool5 = Dropout(droprate)(pool5)
pool5 = BatchNormalization()(pool5)
conv6 = Conv2D(1024, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(pool5)
conv6 = Conv2D(1024, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv6)
conv6 = Dropout(droprate)(conv6)
if upconv:
up6 = concatenate([Conv2DTranspose(512, (2, 2), strides=(2, 2), padding='same')(conv6), conv5])
else:
up6 = concatenate([UpSampling2D(size=(2, 2))(conv6), conv5])
up6 = BatchNormalization()(up6)
conv7 = Conv2D(512, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(up6)
conv7 = Conv2D(512, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv7)
conv7 = Dropout(droprate)(conv7)
if upconv:
up7 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(conv7), conv4])
else:
up7 = concatenate([UpSampling2D(size=(2, 2))(conv7), conv4])
up7 = BatchNormalization()(up7)
conv8 = Conv2D(256, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(up7)
conv8 = Conv2D(256, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv8)
conv8 = Dropout(droprate)(conv8)
if upconv:
up8 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(conv8), conv3])
else:
up8 = concatenate([UpSampling2D(size=(2, 2))(conv8), conv3])
up8 = BatchNormalization()(up8)
conv9 = Conv2D(128, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(up8)
conv9 = Conv2D(128, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv9)
conv9 = Dropout(droprate)(conv9)
if upconv:
up9 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(conv9), conv2])
else:
up9 = concatenate([UpSampling2D(size=(2, 2))(conv9), conv2])
up9 = BatchNormalization()(up9)
conv10 = Conv2D(64, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(up9)
conv10 = Conv2D(64, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv10)
conv10 = Dropout(droprate)(conv10)
if upconv:
up10 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv10), conv1])
else:
up10 = concatenate([UpSampling2D(size=(2, 2))(conv10), conv1])
up10 = BatchNormalization()(up10)
conv11 = Conv2D(32, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(up10)
conv11 = Conv2D(32, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same')(conv11)
conv12 = Conv2D(1, (1, 1), activation='sigmoid')(conv11)
model = Model(inputs=inputs, outputs=conv12)
model.compile(optimizer=Adam(learning_rate = 1e-5), loss = Dice, metrics=['accuracy'])
model.summary()
Model: "functional"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩ │ input_layer (InputLayer) │ (None, 512, 512, 2) │ 0 │ - │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d (Conv2D) │ (None, 512, 512, 32) │ 608 │ input_layer[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_1 (Conv2D) │ (None, 512, 512, 32) │ 9,248 │ conv2d[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ max_pooling2d │ (None, 256, 256, 32) │ 0 │ conv2d_1[0][0] │ │ (MaxPooling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization │ (None, 256, 256, 32) │ 128 │ max_pooling2d[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_2 (Conv2D) │ (None, 256, 256, 64) │ 18,496 │ batch_normalization[0… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_3 (Conv2D) │ (None, 256, 256, 64) │ 36,928 │ conv2d_2[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ max_pooling2d_1 │ (None, 128, 128, 64) │ 0 │ conv2d_3[0][0] │ │ (MaxPooling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout (Dropout) │ (None, 128, 128, 64) │ 0 │ max_pooling2d_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_1 │ (None, 128, 128, 64) │ 256 │ dropout[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_4 (Conv2D) │ (None, 128, 128, 128) │ 73,856 │ batch_normalization_1… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_5 (Conv2D) │ (None, 128, 128, 128) │ 147,584 │ conv2d_4[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ max_pooling2d_2 │ (None, 64, 64, 128) │ 0 │ conv2d_5[0][0] │ │ (MaxPooling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_1 (Dropout) │ (None, 64, 64, 128) │ 0 │ max_pooling2d_2[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_2 │ (None, 64, 64, 128) │ 512 │ dropout_1[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_6 (Conv2D) │ (None, 64, 64, 256) │ 295,168 │ batch_normalization_2… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_7 (Conv2D) │ (None, 64, 64, 256) │ 590,080 │ conv2d_6[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ max_pooling2d_3 │ (None, 32, 32, 256) │ 0 │ conv2d_7[0][0] │ │ (MaxPooling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_2 (Dropout) │ (None, 32, 32, 256) │ 0 │ max_pooling2d_3[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_3 │ (None, 32, 32, 256) │ 1,024 │ dropout_2[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_8 (Conv2D) │ (None, 32, 32, 512) │ 1,180,160 │ batch_normalization_3… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_9 (Conv2D) │ (None, 32, 32, 512) │ 2,359,808 │ conv2d_8[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ max_pooling2d_4 │ (None, 16, 16, 512) │ 0 │ conv2d_9[0][0] │ │ (MaxPooling2D) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_3 (Dropout) │ (None, 16, 16, 512) │ 0 │ max_pooling2d_4[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_4 │ (None, 16, 16, 512) │ 2,048 │ dropout_3[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_10 (Conv2D) │ (None, 16, 16, 1024) │ 4,719,616 │ batch_normalization_4… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_11 (Conv2D) │ (None, 16, 16, 1024) │ 9,438,208 │ conv2d_10[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_4 (Dropout) │ (None, 16, 16, 1024) │ 0 │ conv2d_11[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_transpose │ (None, 32, 32, 512) │ 2,097,664 │ dropout_4[0][0] │ │ (Conv2DTranspose) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate (Concatenate) │ (None, 32, 32, 1024) │ 0 │ conv2d_transpose[0][0… │ │ │ │ │ conv2d_9[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_5 │ (None, 32, 32, 1024) │ 4,096 │ concatenate[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_12 (Conv2D) │ (None, 32, 32, 512) │ 4,719,104 │ batch_normalization_5… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_13 (Conv2D) │ (None, 32, 32, 512) │ 2,359,808 │ conv2d_12[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_5 (Dropout) │ (None, 32, 32, 512) │ 0 │ conv2d_13[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_transpose_1 │ (None, 64, 64, 256) │ 524,544 │ dropout_5[0][0] │ │ (Conv2DTranspose) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate_1 │ (None, 64, 64, 512) │ 0 │ conv2d_transpose_1[0]… │ │ (Concatenate) │ │ │ conv2d_7[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_6 │ (None, 64, 64, 512) │ 2,048 │ concatenate_1[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_14 (Conv2D) │ (None, 64, 64, 256) │ 1,179,904 │ batch_normalization_6… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_15 (Conv2D) │ (None, 64, 64, 256) │ 590,080 │ conv2d_14[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_6 (Dropout) │ (None, 64, 64, 256) │ 0 │ conv2d_15[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_transpose_2 │ (None, 128, 128, 128) │ 131,200 │ dropout_6[0][0] │ │ (Conv2DTranspose) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate_2 │ (None, 128, 128, 256) │ 0 │ conv2d_transpose_2[0]… │ │ (Concatenate) │ │ │ conv2d_5[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_7 │ (None, 128, 128, 256) │ 1,024 │ concatenate_2[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_16 (Conv2D) │ (None, 128, 128, 128) │ 295,040 │ batch_normalization_7… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_17 (Conv2D) │ (None, 128, 128, 128) │ 147,584 │ conv2d_16[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_7 (Dropout) │ (None, 128, 128, 128) │ 0 │ conv2d_17[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_transpose_3 │ (None, 256, 256, 64) │ 32,832 │ dropout_7[0][0] │ │ (Conv2DTranspose) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate_3 │ (None, 256, 256, 128) │ 0 │ conv2d_transpose_3[0]… │ │ (Concatenate) │ │ │ conv2d_3[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_8 │ (None, 256, 256, 128) │ 512 │ concatenate_3[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_18 (Conv2D) │ (None, 256, 256, 64) │ 73,792 │ batch_normalization_8… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_19 (Conv2D) │ (None, 256, 256, 64) │ 36,928 │ conv2d_18[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ dropout_8 (Dropout) │ (None, 256, 256, 64) │ 0 │ conv2d_19[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_transpose_4 │ (None, 512, 512, 32) │ 8,224 │ dropout_8[0][0] │ │ (Conv2DTranspose) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ concatenate_4 │ (None, 512, 512, 64) │ 0 │ conv2d_transpose_4[0]… │ │ (Concatenate) │ │ │ conv2d_1[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ batch_normalization_9 │ (None, 512, 512, 64) │ 256 │ concatenate_4[0][0] │ │ (BatchNormalization) │ │ │ │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_20 (Conv2D) │ (None, 512, 512, 32) │ 18,464 │ batch_normalization_9… │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_21 (Conv2D) │ (None, 512, 512, 32) │ 9,248 │ conv2d_20[0][0] │ ├───────────────────────────┼────────────────────────┼────────────────┼────────────────────────┤ │ conv2d_22 (Conv2D) │ (None, 512, 512, 1) │ 33 │ conv2d_21[0][0] │ └───────────────────────────┴────────────────────────┴────────────────┴────────────────────────┘
Total params: 31,106,113 (118.66 MB)
Trainable params: 31,100,161 (118.64 MB)
Non-trainable params: 5,952 (23.25 KB)
In [ ]:
history = model.fit(train_generator,steps_per_epoch=steps_per_epoch, validation_steps=validation_steps,
epochs=100, validation_data=(x_test,y_test))
<ipython-input-34-f0bb3fd73806>:1: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators. history = model.fit_generator(train_generator,steps_per_epoch=steps_per_epoch, validation_steps=validation_steps,
Epoch 1/100 50/50 [==============================] - 75s 1s/step - loss: 0.8184 - jaccard_coef: 0.1816 - accuracy: 0.6871 - val_loss: 0.8425 - val_jaccard_coef: 0.1575 - val_accuracy: 0.8386 Epoch 2/100 50/50 [==============================] - 71s 1s/step - loss: 0.7010 - jaccard_coef: 0.2983 - accuracy: 0.7782 - val_loss: 0.8300 - val_jaccard_coef: 0.1700 - val_accuracy: 0.8394 Epoch 3/100 50/50 [==============================] - 55s 1s/step - loss: 0.5908 - jaccard_coef: 0.4106 - accuracy: 0.8358 - val_loss: 0.8244 - val_jaccard_coef: 0.1756 - val_accuracy: 0.8396 Epoch 4/100 50/50 [==============================] - 54s 1s/step - loss: 0.5426 - jaccard_coef: 0.4569 - accuracy: 0.8609 - val_loss: 0.8363 - val_jaccard_coef: 0.1637 - val_accuracy: 0.8387 Epoch 5/100 50/50 [==============================] - 55s 1s/step - loss: 0.5232 - jaccard_coef: 0.4787 - accuracy: 0.8670 - val_loss: 0.8286 - val_jaccard_coef: 0.1714 - val_accuracy: 0.8384 Epoch 6/100 50/50 [==============================] - 55s 1s/step - loss: 0.5010 - jaccard_coef: 0.4980 - accuracy: 0.8876 - val_loss: 0.8229 - val_jaccard_coef: 0.1771 - val_accuracy: 0.8386 Epoch 7/100 50/50 [==============================] - 55s 1s/step - loss: 0.4713 - jaccard_coef: 0.5281 - accuracy: 0.8953 - val_loss: 0.8026 - val_jaccard_coef: 0.1974 - val_accuracy: 0.8404 Epoch 8/100 50/50 [==============================] - 55s 1s/step - loss: 0.4782 - jaccard_coef: 0.5209 - accuracy: 0.8996 - val_loss: 0.8124 - val_jaccard_coef: 0.1876 - val_accuracy: 0.8399 Epoch 9/100 50/50 [==============================] - 55s 1s/step - loss: 0.3988 - jaccard_coef: 0.6027 - accuracy: 0.9195 - val_loss: 0.7610 - val_jaccard_coef: 0.2390 - val_accuracy: 0.8446 Epoch 10/100 50/50 [==============================] - 55s 1s/step - loss: 0.4307 - jaccard_coef: 0.5680 - accuracy: 0.9020 - val_loss: 0.7448 - val_jaccard_coef: 0.2552 - val_accuracy: 0.8476 Epoch 11/100 50/50 [==============================] - 55s 1s/step - loss: 0.4420 - jaccard_coef: 0.5590 - accuracy: 0.9064 - val_loss: 0.6679 - val_jaccard_coef: 0.3321 - val_accuracy: 0.8638 Epoch 12/100 50/50 [==============================] - 54s 1s/step - loss: 0.4107 - jaccard_coef: 0.5883 - accuracy: 0.9186 - val_loss: 0.4851 - val_jaccard_coef: 0.5149 - val_accuracy: 0.9159 Epoch 13/100 50/50 [==============================] - 55s 1s/step - loss: 0.3922 - jaccard_coef: 0.6096 - accuracy: 0.9164 - val_loss: 0.5718 - val_jaccard_coef: 0.4282 - val_accuracy: 0.8983 Epoch 14/100 50/50 [==============================] - 55s 1s/step - loss: 0.3854 - jaccard_coef: 0.6152 - accuracy: 0.9219 - val_loss: 0.4788 - val_jaccard_coef: 0.5212 - val_accuracy: 0.9172 Epoch 15/100 50/50 [==============================] - 55s 1s/step - loss: 0.4261 - jaccard_coef: 0.5728 - accuracy: 0.9100 - val_loss: 0.4114 - val_jaccard_coef: 0.5886 - val_accuracy: 0.9297 Epoch 16/100 50/50 [==============================] - 55s 1s/step - loss: 0.4010 - jaccard_coef: 0.6003 - accuracy: 0.9164 - val_loss: 0.4166 - val_jaccard_coef: 0.5834 - val_accuracy: 0.9276 Epoch 17/100 50/50 [==============================] - 54s 1s/step - loss: 0.3526 - jaccard_coef: 0.6478 - accuracy: 0.9324 - val_loss: 0.4388 - val_jaccard_coef: 0.5612 - val_accuracy: 0.9237 Epoch 18/100 50/50 [==============================] - 54s 1s/step - loss: 0.3828 - jaccard_coef: 0.6177 - accuracy: 0.9186 - val_loss: 0.4008 - val_jaccard_coef: 0.5992 - val_accuracy: 0.9301 Epoch 19/100 50/50 [==============================] - 54s 1s/step - loss: 0.3818 - jaccard_coef: 0.6197 - accuracy: 0.9161 - val_loss: 0.3965 - val_jaccard_coef: 0.6035 - val_accuracy: 0.9316 Epoch 20/100 50/50 [==============================] - 55s 1s/step - loss: 0.3935 - jaccard_coef: 0.6062 - accuracy: 0.9212 - val_loss: 0.3870 - val_jaccard_coef: 0.6130 - val_accuracy: 0.9316 Epoch 21/100 50/50 [==============================] - 54s 1s/step - loss: 0.3814 - jaccard_coef: 0.6180 - accuracy: 0.9296 - val_loss: 0.3578 - val_jaccard_coef: 0.6422 - val_accuracy: 0.9377 Epoch 22/100 50/50 [==============================] - 54s 1s/step - loss: 0.3692 - jaccard_coef: 0.6306 - accuracy: 0.9205 - val_loss: 0.3507 - val_jaccard_coef: 0.6493 - val_accuracy: 0.9364 Epoch 23/100 50/50 [==============================] - 54s 1s/step - loss: 0.3331 - jaccard_coef: 0.6664 - accuracy: 0.9320 - val_loss: 0.3846 - val_jaccard_coef: 0.6154 - val_accuracy: 0.9289 Epoch 24/100 50/50 [==============================] - 54s 1s/step - loss: 0.3853 - jaccard_coef: 0.6132 - accuracy: 0.9248 - val_loss: 0.3833 - val_jaccard_coef: 0.6167 - val_accuracy: 0.9325 Epoch 25/100 50/50 [==============================] - 54s 1s/step - loss: 0.3527 - jaccard_coef: 0.6467 - accuracy: 0.9297 - val_loss: 0.3748 - val_jaccard_coef: 0.6252 - val_accuracy: 0.9342 Epoch 26/100 50/50 [==============================] - 54s 1s/step - loss: 0.3476 - jaccard_coef: 0.6532 - accuracy: 0.9353 - val_loss: 0.4003 - val_jaccard_coef: 0.5997 - val_accuracy: 0.9283 Epoch 27/100 50/50 [==============================] - 55s 1s/step - loss: 0.3365 - jaccard_coef: 0.6643 - accuracy: 0.9262 - val_loss: 0.3916 - val_jaccard_coef: 0.6084 - val_accuracy: 0.9304 Epoch 28/100 50/50 [==============================] - 55s 1s/step - loss: 0.3561 - jaccard_coef: 0.6450 - accuracy: 0.9301 - val_loss: 0.3484 - val_jaccard_coef: 0.6516 - val_accuracy: 0.9382 Epoch 29/100 50/50 [==============================] - 54s 1s/step - loss: 0.3761 - jaccard_coef: 0.6240 - accuracy: 0.9282 - val_loss: 0.3462 - val_jaccard_coef: 0.6538 - val_accuracy: 0.9373 Epoch 30/100 50/50 [==============================] - 55s 1s/step - loss: 0.3446 - jaccard_coef: 0.6561 - accuracy: 0.9366 - val_loss: 0.3376 - val_jaccard_coef: 0.6624 - val_accuracy: 0.9388 Epoch 31/100 50/50 [==============================] - 54s 1s/step - loss: 0.3285 - jaccard_coef: 0.6721 - accuracy: 0.9339 - val_loss: 0.3480 - val_jaccard_coef: 0.6520 - val_accuracy: 0.9385 Epoch 32/100 50/50 [==============================] - 55s 1s/step - loss: 0.3242 - jaccard_coef: 0.6770 - accuracy: 0.9376 - val_loss: 0.3435 - val_jaccard_coef: 0.6565 - val_accuracy: 0.9348 Epoch 33/100 50/50 [==============================] - 54s 1s/step - loss: 0.3329 - jaccard_coef: 0.6658 - accuracy: 0.9299 - val_loss: 0.3322 - val_jaccard_coef: 0.6678 - val_accuracy: 0.9393 Epoch 34/100 50/50 [==============================] - 54s 1s/step - loss: 0.3444 - jaccard_coef: 0.6558 - accuracy: 0.9351 - val_loss: 0.3438 - val_jaccard_coef: 0.6562 - val_accuracy: 0.9300 Epoch 35/100 50/50 [==============================] - 54s 1s/step - loss: 0.3578 - jaccard_coef: 0.6431 - accuracy: 0.9285 - val_loss: 0.3465 - val_jaccard_coef: 0.6535 - val_accuracy: 0.9283 Epoch 36/100 50/50 [==============================] - 54s 1s/step - loss: 0.3458 - jaccard_coef: 0.6531 - accuracy: 0.9332 - val_loss: 0.3319 - val_jaccard_coef: 0.6681 - val_accuracy: 0.9395 Epoch 37/100 50/50 [==============================] - 54s 1s/step - loss: 0.3326 - jaccard_coef: 0.6669 - accuracy: 0.9407 - val_loss: 0.3303 - val_jaccard_coef: 0.6697 - val_accuracy: 0.9378 Epoch 38/100 50/50 [==============================] - 55s 1s/step - loss: 0.3189 - jaccard_coef: 0.6813 - accuracy: 0.9344 - val_loss: 0.3329 - val_jaccard_coef: 0.6671 - val_accuracy: 0.9387 Epoch 39/100 50/50 [==============================] - 54s 1s/step - loss: 0.3406 - jaccard_coef: 0.6605 - accuracy: 0.9377 - val_loss: 0.3413 - val_jaccard_coef: 0.6587 - val_accuracy: 0.9381 Epoch 40/100 50/50 [==============================] - 54s 1s/step - loss: 0.3301 - jaccard_coef: 0.6713 - accuracy: 0.9384 - val_loss: 0.4064 - val_jaccard_coef: 0.5936 - val_accuracy: 0.9029 Epoch 41/100 50/50 [==============================] - 54s 1s/step - loss: 0.3277 - jaccard_coef: 0.6732 - accuracy: 0.9376 - val_loss: 0.3580 - val_jaccard_coef: 0.6420 - val_accuracy: 0.9308 Epoch 42/100 50/50 [==============================] - 54s 1s/step - loss: 0.3654 - jaccard_coef: 0.6337 - accuracy: 0.9332 - val_loss: 0.3400 - val_jaccard_coef: 0.6600 - val_accuracy: 0.9311 Epoch 43/100 50/50 [==============================] - 54s 1s/step - loss: 0.3035 - jaccard_coef: 0.6965 - accuracy: 0.9402 - val_loss: 0.3413 - val_jaccard_coef: 0.6587 - val_accuracy: 0.9309 Epoch 44/100 50/50 [==============================] - 55s 1s/step - loss: 0.3428 - jaccard_coef: 0.6581 - accuracy: 0.9361 - val_loss: 0.3495 - val_jaccard_coef: 0.6505 - val_accuracy: 0.9267 Epoch 45/100 50/50 [==============================] - 54s 1s/step - loss: 0.3065 - jaccard_coef: 0.6936 - accuracy: 0.9401 - val_loss: 0.3342 - val_jaccard_coef: 0.6658 - val_accuracy: 0.9370 Epoch 46/100 50/50 [==============================] - 55s 1s/step - loss: 0.3316 - jaccard_coef: 0.6687 - accuracy: 0.9404 - val_loss: 0.3306 - val_jaccard_coef: 0.6694 - val_accuracy: 0.9350 Epoch 47/100 50/50 [==============================] - 54s 1s/step - loss: 0.3169 - jaccard_coef: 0.6822 - accuracy: 0.9399 - val_loss: 0.3329 - val_jaccard_coef: 0.6671 - val_accuracy: 0.9387 Epoch 48/100 50/50 [==============================] - 55s 1s/step - loss: 0.2998 - jaccard_coef: 0.7002 - accuracy: 0.9438 - val_loss: 0.3315 - val_jaccard_coef: 0.6685 - val_accuracy: 0.9383 Epoch 49/100 50/50 [==============================] - 54s 1s/step - loss: 0.3239 - jaccard_coef: 0.6760 - accuracy: 0.9389 - val_loss: 0.3256 - val_jaccard_coef: 0.6744 - val_accuracy: 0.9382 Epoch 50/100 50/50 [==============================] - 55s 1s/step - loss: 0.2870 - jaccard_coef: 0.7139 - accuracy: 0.9450 - val_loss: 0.3208 - val_jaccard_coef: 0.6792 - val_accuracy: 0.9405 Epoch 51/100 50/50 [==============================] - 55s 1s/step - loss: 0.3005 - jaccard_coef: 0.7002 - accuracy: 0.9415 - val_loss: 0.3228 - val_jaccard_coef: 0.6772 - val_accuracy: 0.9403 Epoch 52/100 50/50 [==============================] - 55s 1s/step - loss: 0.3046 - jaccard_coef: 0.6954 - accuracy: 0.9442 - val_loss: 0.3404 - val_jaccard_coef: 0.6596 - val_accuracy: 0.9273 Epoch 53/100 50/50 [==============================] - 54s 1s/step - loss: 0.3128 - jaccard_coef: 0.6868 - accuracy: 0.9406 - val_loss: 0.3446 - val_jaccard_coef: 0.6554 - val_accuracy: 0.9372 Epoch 54/100 50/50 [==============================] - 54s 1s/step - loss: 0.2839 - jaccard_coef: 0.7154 - accuracy: 0.9468 - val_loss: 0.3281 - val_jaccard_coef: 0.6719 - val_accuracy: 0.9345 Epoch 55/100 50/50 [==============================] - 54s 1s/step - loss: 0.3000 - jaccard_coef: 0.6992 - accuracy: 0.9413 - val_loss: 0.3155 - val_jaccard_coef: 0.6845 - val_accuracy: 0.9415 Epoch 56/100 50/50 [==============================] - 55s 1s/step - loss: 0.3190 - jaccard_coef: 0.6799 - accuracy: 0.9421 - val_loss: 0.3201 - val_jaccard_coef: 0.6799 - val_accuracy: 0.9376 Epoch 57/100 50/50 [==============================] - 54s 1s/step - loss: 0.3004 - jaccard_coef: 0.6991 - accuracy: 0.9410 - val_loss: 0.3200 - val_jaccard_coef: 0.6800 - val_accuracy: 0.9395 Epoch 58/100 50/50 [==============================] - 54s 1s/step - loss: 0.3290 - jaccard_coef: 0.6709 - accuracy: 0.9401 - val_loss: 0.3337 - val_jaccard_coef: 0.6663 - val_accuracy: 0.9352 Epoch 59/100 50/50 [==============================] - 54s 1s/step - loss: 0.3157 - jaccard_coef: 0.6837 - accuracy: 0.9420 - val_loss: 0.3679 - val_jaccard_coef: 0.6321 - val_accuracy: 0.9220 Epoch 60/100 50/50 [==============================] - 54s 1s/step - loss: 0.3010 - jaccard_coef: 0.6983 - accuracy: 0.9450 - val_loss: 0.3571 - val_jaccard_coef: 0.6429 - val_accuracy: 0.9270 Epoch 61/100 50/50 [==============================] - 54s 1s/step - loss: 0.3100 - jaccard_coef: 0.6906 - accuracy: 0.9417 - val_loss: 0.3202 - val_jaccard_coef: 0.6798 - val_accuracy: 0.9373 Epoch 62/100 50/50 [==============================] - 55s 1s/step - loss: 0.3070 - jaccard_coef: 0.6922 - accuracy: 0.9432 - val_loss: 0.3568 - val_jaccard_coef: 0.6432 - val_accuracy: 0.9216 Epoch 63/100 50/50 [==============================] - 54s 1s/step - loss: 0.2926 - jaccard_coef: 0.7071 - accuracy: 0.9455 - val_loss: 0.3210 - val_jaccard_coef: 0.6790 - val_accuracy: 0.9353 Epoch 64/100 50/50 [==============================] - 54s 1s/step - loss: 0.3109 - jaccard_coef: 0.6896 - accuracy: 0.9448 - val_loss: 0.3322 - val_jaccard_coef: 0.6678 - val_accuracy: 0.9387 Epoch 65/100 50/50 [==============================] - 54s 1s/step - loss: 0.3095 - jaccard_coef: 0.6901 - accuracy: 0.9409 - val_loss: 0.3179 - val_jaccard_coef: 0.6821 - val_accuracy: 0.9376 Epoch 66/100 50/50 [==============================] - 55s 1s/step - loss: 0.2853 - jaccard_coef: 0.7148 - accuracy: 0.9467 - val_loss: 0.3258 - val_jaccard_coef: 0.6742 - val_accuracy: 0.9394 Epoch 67/100 50/50 [==============================] - 54s 1s/step - loss: 0.2988 - jaccard_coef: 0.7017 - accuracy: 0.9428 - val_loss: 0.3378 - val_jaccard_coef: 0.6622 - val_accuracy: 0.9384 Epoch 68/100 50/50 [==============================] - 54s 1s/step - loss: 0.3003 - jaccard_coef: 0.7002 - accuracy: 0.9424 - val_loss: 0.3222 - val_jaccard_coef: 0.6778 - val_accuracy: 0.9363 Epoch 69/100 50/50 [==============================] - 54s 1s/step - loss: 0.2824 - jaccard_coef: 0.7182 - accuracy: 0.9477 - val_loss: 0.3185 - val_jaccard_coef: 0.6815 - val_accuracy: 0.9389 Epoch 70/100 50/50 [==============================] - 55s 1s/step - loss: 0.3280 - jaccard_coef: 0.6720 - accuracy: 0.9364 - val_loss: 0.4129 - val_jaccard_coef: 0.5871 - val_accuracy: 0.8958 Epoch 71/100 50/50 [==============================] - 55s 1s/step - loss: 0.2630 - jaccard_coef: 0.7362 - accuracy: 0.9517 - val_loss: 0.3219 - val_jaccard_coef: 0.6781 - val_accuracy: 0.9358 Epoch 72/100 50/50 [==============================] - 55s 1s/step - loss: 0.2751 - jaccard_coef: 0.7254 - accuracy: 0.9408 - val_loss: 0.3997 - val_jaccard_coef: 0.6003 - val_accuracy: 0.9088 Epoch 73/100 50/50 [==============================] - 55s 1s/step - loss: 0.2895 - jaccard_coef: 0.7114 - accuracy: 0.9482 - val_loss: 0.3170 - val_jaccard_coef: 0.6830 - val_accuracy: 0.9358 Epoch 74/100 50/50 [==============================] - 54s 1s/step - loss: 0.3112 - jaccard_coef: 0.6891 - accuracy: 0.9398 - val_loss: 0.3603 - val_jaccard_coef: 0.6397 - val_accuracy: 0.9356 Epoch 75/100 50/50 [==============================] - 55s 1s/step - loss: 0.2835 - jaccard_coef: 0.7165 - accuracy: 0.9478 - val_loss: 0.3277 - val_jaccard_coef: 0.6723 - val_accuracy: 0.9347 Epoch 76/100 50/50 [==============================] - 54s 1s/step - loss: 0.2830 - jaccard_coef: 0.7158 - accuracy: 0.9457 - val_loss: 0.3240 - val_jaccard_coef: 0.6760 - val_accuracy: 0.9379 Epoch 77/100 50/50 [==============================] - 54s 1s/step - loss: 0.3079 - jaccard_coef: 0.6924 - accuracy: 0.9411 - val_loss: 0.3227 - val_jaccard_coef: 0.6773 - val_accuracy: 0.9360 Epoch 78/100 50/50 [==============================] - 55s 1s/step - loss: 0.3151 - jaccard_coef: 0.6847 - accuracy: 0.9419 - val_loss: 0.3167 - val_jaccard_coef: 0.6833 - val_accuracy: 0.9389 Epoch 79/100 50/50 [==============================] - 54s 1s/step - loss: 0.2802 - jaccard_coef: 0.7186 - accuracy: 0.9463 - val_loss: 0.3255 - val_jaccard_coef: 0.6745 - val_accuracy: 0.9363 Epoch 80/100 50/50 [==============================] - 55s 1s/step - loss: 0.3101 - jaccard_coef: 0.6900 - accuracy: 0.9455 - val_loss: 0.3585 - val_jaccard_coef: 0.6415 - val_accuracy: 0.9191 Epoch 81/100 50/50 [==============================] - 54s 1s/step - loss: 0.2963 - jaccard_coef: 0.7031 - accuracy: 0.9396 - val_loss: 0.3115 - val_jaccard_coef: 0.6885 - val_accuracy: 0.9375 Epoch 82/100 50/50 [==============================] - 55s 1s/step - loss: 0.3126 - jaccard_coef: 0.6874 - accuracy: 0.9436 - val_loss: 0.3034 - val_jaccard_coef: 0.6966 - val_accuracy: 0.9408 Epoch 83/100 50/50 [==============================] - 54s 1s/step - loss: 0.3089 - jaccard_coef: 0.6898 - accuracy: 0.9469 - val_loss: 0.3048 - val_jaccard_coef: 0.6952 - val_accuracy: 0.9409 Epoch 84/100 50/50 [==============================] - 55s 1s/step - loss: 0.2812 - jaccard_coef: 0.7196 - accuracy: 0.9456 - val_loss: 0.3054 - val_jaccard_coef: 0.6946 - val_accuracy: 0.9396 Epoch 85/100 50/50 [==============================] - 55s 1s/step - loss: 0.2942 - jaccard_coef: 0.7057 - accuracy: 0.9451 - val_loss: 0.3164 - val_jaccard_coef: 0.6836 - val_accuracy: 0.9364 Epoch 86/100 50/50 [==============================] - 55s 1s/step - loss: 0.2949 - jaccard_coef: 0.7053 - accuracy: 0.9455 - val_loss: 0.3034 - val_jaccard_coef: 0.6966 - val_accuracy: 0.9402 Epoch 87/100 50/50 [==============================] - 54s 1s/step - loss: 0.2585 - jaccard_coef: 0.7409 - accuracy: 0.9516 - val_loss: 0.3074 - val_jaccard_coef: 0.6926 - val_accuracy: 0.9379 Epoch 88/100 50/50 [==============================] - 54s 1s/step - loss: 0.2712 - jaccard_coef: 0.7295 - accuracy: 0.9453 - val_loss: 0.3021 - val_jaccard_coef: 0.6979 - val_accuracy: 0.9413 Epoch 89/100 50/50 [==============================] - 54s 1s/step - loss: 0.3129 - jaccard_coef: 0.6875 - accuracy: 0.9445 - val_loss: 0.3190 - val_jaccard_coef: 0.6810 - val_accuracy: 0.9374 Epoch 90/100 50/50 [==============================] - 54s 1s/step - loss: 0.3162 - jaccard_coef: 0.6832 - accuracy: 0.9389 - val_loss: 0.3353 - val_jaccard_coef: 0.6647 - val_accuracy: 0.9265 Epoch 91/100 50/50 [==============================] - 55s 1s/step - loss: 0.2903 - jaccard_coef: 0.7093 - accuracy: 0.9470 - val_loss: 0.3621 - val_jaccard_coef: 0.6379 - val_accuracy: 0.9176 Epoch 92/100 50/50 [==============================] - 54s 1s/step - loss: 0.2554 - jaccard_coef: 0.7458 - accuracy: 0.9512 - val_loss: 0.3029 - val_jaccard_coef: 0.6971 - val_accuracy: 0.9401 Epoch 93/100 50/50 [==============================] - 54s 1s/step - loss: 0.2854 - jaccard_coef: 0.7139 - accuracy: 0.9449 - val_loss: 0.3491 - val_jaccard_coef: 0.6509 - val_accuracy: 0.9366 Epoch 94/100 50/50 [==============================] - 54s 1s/step - loss: 0.2974 - jaccard_coef: 0.7032 - accuracy: 0.9469 - val_loss: 0.3071 - val_jaccard_coef: 0.6929 - val_accuracy: 0.9426 Epoch 95/100 50/50 [==============================] - 54s 1s/step - loss: 0.2811 - jaccard_coef: 0.7189 - accuracy: 0.9463 - val_loss: 0.3056 - val_jaccard_coef: 0.6944 - val_accuracy: 0.9396 Epoch 96/100 50/50 [==============================] - 55s 1s/step - loss: 0.2939 - jaccard_coef: 0.7067 - accuracy: 0.9434 - val_loss: 0.3080 - val_jaccard_coef: 0.6920 - val_accuracy: 0.9421 Epoch 97/100 50/50 [==============================] - 55s 1s/step - loss: 0.2528 - jaccard_coef: 0.7466 - accuracy: 0.9482 - val_loss: 0.3127 - val_jaccard_coef: 0.6873 - val_accuracy: 0.9429 Epoch 98/100 50/50 [==============================] - 54s 1s/step - loss: 0.2955 - jaccard_coef: 0.7033 - accuracy: 0.9447 - val_loss: 0.3293 - val_jaccard_coef: 0.6707 - val_accuracy: 0.9370 Epoch 99/100 50/50 [==============================] - 55s 1s/step - loss: 0.2637 - jaccard_coef: 0.7373 - accuracy: 0.9509 - val_loss: 0.3196 - val_jaccard_coef: 0.6804 - val_accuracy: 0.9383 Epoch 100/100 50/50 [==============================] - 55s 1s/step - loss: 0.2861 - jaccard_coef: 0.7129 - accuracy: 0.9443 - val_loss: 0.3043 - val_jaccard_coef: 0.6957 - val_accuracy: 0.9386
In [ ]:
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='lower right')
plt.show()
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper right')
plt.show()
In [ ]:
predict = model.predict(x_test)
5/5 [==============================] - 30s 673ms/step
In [ ]:
pred = np.round(predict)
In [ ]:
accuracy = accuracy_score(y_test.flatten(),pred.flatten())
print(accuracy)
0.9385725189657772
Finally, let's plot an example of the predicted result compared to the original mask:
In [ ]:
i = 2
plt.figure(figsize=(20,8))
plt.subplot(1,3,1),
plt.imshow(x_test[i,:,:,0])
plt.subplot(1,3,2),
plt.imshow(y_test[i,:,:])
plt.subplot(1,3,3),
plt.imshow(pred[i,:,:,0])
Out[ ]:
<matplotlib.image.AxesImage at 0x7d1ad5d74610>
In [ ]:
save_path = '/content/Results_img'
if not os.path.isdir(save_path):
os.mkdir(save_path)
In [ ]:
for i in range(len(predict)):
im = np.round(predict[i])
full_im = im * 255
full_im = full_im.astype('uint8')
full_save_path = os.path.join(save_path,'Img_'+ str(i) + '.png')
print(full_save_path)
cv2.imwrite(full_save_path, full_im)
/content/Results_img/Img_0.png /content/Results_img/Img_1.png /content/Results_img/Img_2.png /content/Results_img/Img_3.png /content/Results_img/Img_4.png /content/Results_img/Img_5.png /content/Results_img/Img_6.png /content/Results_img/Img_7.png /content/Results_img/Img_8.png /content/Results_img/Img_9.png /content/Results_img/Img_10.png /content/Results_img/Img_11.png /content/Results_img/Img_12.png /content/Results_img/Img_13.png /content/Results_img/Img_14.png /content/Results_img/Img_15.png /content/Results_img/Img_16.png /content/Results_img/Img_17.png /content/Results_img/Img_18.png /content/Results_img/Img_19.png /content/Results_img/Img_20.png /content/Results_img/Img_21.png /content/Results_img/Img_22.png /content/Results_img/Img_23.png /content/Results_img/Img_24.png /content/Results_img/Img_25.png /content/Results_img/Img_26.png /content/Results_img/Img_27.png /content/Results_img/Img_28.png /content/Results_img/Img_29.png /content/Results_img/Img_30.png /content/Results_img/Img_31.png /content/Results_img/Img_32.png /content/Results_img/Img_33.png /content/Results_img/Img_34.png /content/Results_img/Img_35.png /content/Results_img/Img_36.png /content/Results_img/Img_37.png /content/Results_img/Img_38.png /content/Results_img/Img_39.png /content/Results_img/Img_40.png /content/Results_img/Img_41.png /content/Results_img/Img_42.png /content/Results_img/Img_43.png /content/Results_img/Img_44.png /content/Results_img/Img_45.png /content/Results_img/Img_46.png /content/Results_img/Img_47.png /content/Results_img/Img_48.png /content/Results_img/Img_49.png /content/Results_img/Img_50.png /content/Results_img/Img_51.png /content/Results_img/Img_52.png /content/Results_img/Img_53.png /content/Results_img/Img_54.png /content/Results_img/Img_55.png /content/Results_img/Img_56.png /content/Results_img/Img_57.png /content/Results_img/Img_58.png /content/Results_img/Img_59.png /content/Results_img/Img_60.png /content/Results_img/Img_61.png /content/Results_img/Img_62.png /content/Results_img/Img_63.png /content/Results_img/Img_64.png /content/Results_img/Img_65.png /content/Results_img/Img_66.png /content/Results_img/Img_67.png /content/Results_img/Img_68.png /content/Results_img/Img_69.png /content/Results_img/Img_70.png /content/Results_img/Img_71.png /content/Results_img/Img_72.png /content/Results_img/Img_73.png /content/Results_img/Img_74.png /content/Results_img/Img_75.png /content/Results_img/Img_76.png /content/Results_img/Img_77.png /content/Results_img/Img_78.png /content/Results_img/Img_79.png /content/Results_img/Img_80.png /content/Results_img/Img_81.png /content/Results_img/Img_82.png /content/Results_img/Img_83.png /content/Results_img/Img_84.png /content/Results_img/Img_85.png /content/Results_img/Img_86.png /content/Results_img/Img_87.png /content/Results_img/Img_88.png /content/Results_img/Img_89.png /content/Results_img/Img_90.png /content/Results_img/Img_91.png /content/Results_img/Img_92.png /content/Results_img/Img_93.png /content/Results_img/Img_94.png /content/Results_img/Img_95.png /content/Results_img/Img_96.png /content/Results_img/Img_97.png /content/Results_img/Img_98.png /content/Results_img/Img_99.png /content/Results_img/Img_100.png /content/Results_img/Img_101.png /content/Results_img/Img_102.png /content/Results_img/Img_103.png /content/Results_img/Img_104.png /content/Results_img/Img_105.png /content/Results_img/Img_106.png /content/Results_img/Img_107.png /content/Results_img/Img_108.png /content/Results_img/Img_109.png /content/Results_img/Img_110.png /content/Results_img/Img_111.png /content/Results_img/Img_112.png /content/Results_img/Img_113.png /content/Results_img/Img_114.png /content/Results_img/Img_115.png /content/Results_img/Img_116.png /content/Results_img/Img_117.png /content/Results_img/Img_118.png /content/Results_img/Img_119.png /content/Results_img/Img_120.png /content/Results_img/Img_121.png /content/Results_img/Img_122.png /content/Results_img/Img_123.png /content/Results_img/Img_124.png /content/Results_img/Img_125.png /content/Results_img/Img_126.png /content/Results_img/Img_127.png /content/Results_img/Img_128.png /content/Results_img/Img_129.png /content/Results_img/Img_130.png /content/Results_img/Img_131.png /content/Results_img/Img_132.png /content/Results_img/Img_133.png /content/Results_img/Img_134.png /content/Results_img/Img_135.png
PSPNet¶
a) Input Image
Input image of any shape usually with dimensions greater than (256, 256) is fed to the network.
b) Feature Map
It takes the input image and constructs feature maps for the image. Feature maps are extracted by feeding the image using transfer learning or zero-area network with dilated convolutions. Since large size kernels extract more useful information than small size kernel but the computation cost is higher, dilated convolutions gather large size area information with small size kernel for higher dilation rates to keep the dimensions same as the input image. Generally, residual blocks with dilations are used to construct the feature maps.No.of feature maps N is a hyperparameter and needs to be tuned for good result.
c) Pyramid Pooling Module
An image contains objects of sizes ranging from small area to large area in different regions. Fully Convolutional Network (FCN), U-Net and other networks construct feature maps by upscaling and segmenting at different levels for segmenting all objects in all regions. But in PSPNet to correctly segment all size objects, feature maps are averagely pooled into different pool sizes. Sub-region average pooling is done at different scales like Global Average Pooling,(2x2),(3x3),(4x4),(6x6),(8x8). After average pooling of N feature maps with n different sizes, the feature maps at each level were reduced to N/n feature maps by performing 1*1 convolutions. For example, if N=512 feature maps and n=4 sizes like Global Average Pooling,(2x2),(4x4),(8x8) then at each level 512 feature maps are reduced to 126 feature maps. N/n feature maps at each sampling level to have same dimensions as input image. For upsampling, bilinear_interpolation or convolution_transpose methods used instead of simple upsampling.
The output of Pyramid Pooling Module is a concatenation of base feature maps from b and upsampled average pooled feature maps from c.
d) Final Prediction
The 2*N feature maps are fed to convolution layer and final prediction of classes is generated as output layer is constructed, say different channels for different objects or single channel.
PSPNet can be implemented using the following codes:
In [ ]:
def conv_block(input_tensor, filters, strides, d_rates):
x = Conv2D(filters[0], kernel_size=1, kernel_initializer='he_uniform', dilation_rate=d_rates[0])(input_tensor)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2D(filters[1], kernel_size=3, strides=strides, kernel_initializer='he_uniform', padding='same', dilation_rate=d_rates[1])(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2D(filters[2], kernel_size=1, kernel_initializer='he_uniform', dilation_rate=d_rates[2])(x)
x = BatchNormalization()(x)
shortcut = Conv2D(filters[2], kernel_size=1, kernel_initializer='he_uniform', strides=strides)(input_tensor)
shortcut = BatchNormalization()(shortcut)
x = add([x, shortcut])
x = Activation('relu')(x)
return x
def identity_block(input_tensor, filters, d_rates):
x = Conv2D(filters[0], kernel_size=1, kernel_initializer='he_uniform', dilation_rate=d_rates[0])(input_tensor)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2D(filters[1], kernel_size=3, kernel_initializer='he_uniform', padding='same', dilation_rate=d_rates[1])(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2D(filters[2], kernel_size=1, kernel_initializer='he_uniform', dilation_rate=d_rates[2])(x)
x = BatchNormalization()(x)
x = add([x, input_tensor])
x = Activation('relu')(x)
return x
def pyramid_pooling_block(input_tensor, bin_sizes):
concat_list = [input_tensor]
h = input_tensor.shape[1].value
w = input_tensor.shape[2].value
for bin_size in bin_sizes:
x = AveragePooling2D(pool_size=(h//bin_size, w//bin_size), strides=(h//bin_size, w//bin_size))(input_tensor)
x = Conv2D(512, kernel_size=1, kernel_initializer='he_uniform')(x)
x = Lambda(lambda x: tf.image.resize_images(x, (h, w)))(x)
concat_list.append(x)
return concatenate(concat_list)
In [ ]:
droprate = 0.5
inputs = Input((640, 640, 14))
conv_1 = Conv2D(32, (3, 3), strides=(1, 1), kernel_initializer='he_uniform', padding='same')(inputs)
conv_1 = BatchNormalization()(conv_1)
conv_1 = Activation("relu")(conv_1)
conv_2 = Conv2D(64, (3, 3), strides=(1, 1), kernel_initializer='he_uniform', padding='same')(conv_1)
conv_2 = BatchNormalization()(conv_2)
conv_2 = Activation("relu")(conv_2)
conv_3 = Conv2D(128, (3, 3), strides=(1, 1), kernel_initializer='he_uniform', padding='same')(conv_2)
conv_3 = BatchNormalization()(conv_3)
conv_3 = Activation("relu")(conv_3)
pool_1 = MaxPooling2D((2, 2), strides=(2, 2),)(conv_3)
conv_block1 = conv_block(pool_1, filters=[64, 64, 128], strides=(1, 1), d_rates=[1, 1, 1])
identity_block1 = identity_block(conv_block1, filters=[64, 64, 128], d_rates=[1, 1, 1])
identity_block2 = identity_block(identity_block1, filters=[64, 64, 128], d_rates=[1, 1, 1])
conv_block2 = conv_block(identity_block2, filters=[128, 128, 256], strides=(2, 2), d_rates=[1, 1, 1])
identity_block3 = identity_block(conv_block2, filters=[128, 128, 256], d_rates=[1, 1, 1])
identity_block4 = identity_block(identity_block3, filters=[128, 128, 256], d_rates=[1, 1, 1])
identity_block5 = identity_block(identity_block4, filters=[128, 128, 256], d_rates=[1, 1, 1])
conv_block3 = conv_block(identity_block5, filters=[256, 256, 512], strides=(2, 2), d_rates=[1, 1, 1])
identity_block6 = identity_block(conv_block3, filters=[256, 256, 512], d_rates=[1, 2, 1])
identity_block7 = identity_block(identity_block6, filters=[256, 256, 512], d_rates=[1, 2, 1])
identity_block8 = identity_block(identity_block7, filters=[256, 256, 512], d_rates=[1, 2, 1])
identity_block9 = identity_block(identity_block8, filters=[256, 256, 512], d_rates=[1, 2, 1])
identity_block10 = identity_block(identity_block9, filters=[256, 256, 512], d_rates=[1, 2, 1])
conv_block4 = conv_block(identity_block10, filters=[512, 512, 1024], strides=(1, 1), d_rates=[1, 4, 1])
identity_block11 = identity_block(conv_block4, filters=[512, 512, 1024], d_rates=[1, 4, 1])
identity_block12 = identity_block(identity_block11, filters=[512, 512, 1024], d_rates=[1, 4, 1])
pyramid_pooling_block1 = pyramid_pooling_block(identity_block12, [1, 2, 4, 8])
conv_4 = Conv2D(512, (3, 3), kernel_initializer='he_uniform', padding='same')(pyramid_pooling_block1)
conv_4 = BatchNormalization()(conv_4)
conv_4 = Activation('relu')(conv_4)
conv_5 = Conv2D(1, (1, 1), kernel_initializer='he_uniform', padding='same')(conv_4)
conv_5 = Conv2DTranspose(1, (16, 16), strides=(8, 8), kernel_initializer='he_uniform', padding='same')(conv_5)
outputs = Activation('softmax')(conv_5)
model = Model(inputs=inputs, outputs=outputs)
model.compile(optimizer=Adam(learning_rate = 1e-5), loss = Dice, metrics=['accuracy'])
model.summary()
Model: "model_7"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_7 (InputLayer) (None, 640, 640, 14) 0
__________________________________________________________________________________________________
conv2d_367 (Conv2D) (None, 640, 640, 32) 4064 input_7[0][0]
__________________________________________________________________________________________________
batch_normalization_337 (BatchN (None, 640, 640, 32) 128 conv2d_367[0][0]
__________________________________________________________________________________________________
activation_319 (Activation) (None, 640, 640, 32) 0 batch_normalization_337[0][0]
__________________________________________________________________________________________________
conv2d_368 (Conv2D) (None, 640, 640, 64) 18496 activation_319[0][0]
__________________________________________________________________________________________________
batch_normalization_338 (BatchN (None, 640, 640, 64) 256 conv2d_368[0][0]
__________________________________________________________________________________________________
activation_320 (Activation) (None, 640, 640, 64) 0 batch_normalization_338[0][0]
__________________________________________________________________________________________________
conv2d_369 (Conv2D) (None, 640, 640, 128 73856 activation_320[0][0]
__________________________________________________________________________________________________
batch_normalization_339 (BatchN (None, 640, 640, 128 512 conv2d_369[0][0]
__________________________________________________________________________________________________
activation_321 (Activation) (None, 640, 640, 128 0 batch_normalization_339[0][0]
__________________________________________________________________________________________________
max_pooling2d_7 (MaxPooling2D) (None, 320, 320, 128 0 activation_321[0][0]
__________________________________________________________________________________________________
conv2d_370 (Conv2D) (None, 320, 320, 64) 8256 max_pooling2d_7[0][0]
__________________________________________________________________________________________________
batch_normalization_340 (BatchN (None, 320, 320, 64) 256 conv2d_370[0][0]
__________________________________________________________________________________________________
activation_322 (Activation) (None, 320, 320, 64) 0 batch_normalization_340[0][0]
__________________________________________________________________________________________________
conv2d_371 (Conv2D) (None, 320, 320, 64) 36928 activation_322[0][0]
__________________________________________________________________________________________________
batch_normalization_341 (BatchN (None, 320, 320, 64) 256 conv2d_371[0][0]
__________________________________________________________________________________________________
activation_323 (Activation) (None, 320, 320, 64) 0 batch_normalization_341[0][0]
__________________________________________________________________________________________________
conv2d_372 (Conv2D) (None, 320, 320, 128 8320 activation_323[0][0]
__________________________________________________________________________________________________
conv2d_373 (Conv2D) (None, 320, 320, 128 16512 max_pooling2d_7[0][0]
__________________________________________________________________________________________________
batch_normalization_342 (BatchN (None, 320, 320, 128 512 conv2d_372[0][0]
__________________________________________________________________________________________________
batch_normalization_343 (BatchN (None, 320, 320, 128 512 conv2d_373[0][0]
__________________________________________________________________________________________________
add_97 (Add) (None, 320, 320, 128 0 batch_normalization_342[0][0]
batch_normalization_343[0][0]
__________________________________________________________________________________________________
activation_324 (Activation) (None, 320, 320, 128 0 add_97[0][0]
__________________________________________________________________________________________________
conv2d_374 (Conv2D) (None, 320, 320, 64) 8256 activation_324[0][0]
__________________________________________________________________________________________________
batch_normalization_344 (BatchN (None, 320, 320, 64) 256 conv2d_374[0][0]
__________________________________________________________________________________________________
activation_325 (Activation) (None, 320, 320, 64) 0 batch_normalization_344[0][0]
__________________________________________________________________________________________________
conv2d_375 (Conv2D) (None, 320, 320, 64) 36928 activation_325[0][0]
__________________________________________________________________________________________________
batch_normalization_345 (BatchN (None, 320, 320, 64) 256 conv2d_375[0][0]
__________________________________________________________________________________________________
activation_326 (Activation) (None, 320, 320, 64) 0 batch_normalization_345[0][0]
__________________________________________________________________________________________________
conv2d_376 (Conv2D) (None, 320, 320, 128 8320 activation_326[0][0]
__________________________________________________________________________________________________
batch_normalization_346 (BatchN (None, 320, 320, 128 512 conv2d_376[0][0]
__________________________________________________________________________________________________
add_98 (Add) (None, 320, 320, 128 0 batch_normalization_346[0][0]
activation_324[0][0]
__________________________________________________________________________________________________
activation_327 (Activation) (None, 320, 320, 128 0 add_98[0][0]
__________________________________________________________________________________________________
conv2d_377 (Conv2D) (None, 320, 320, 64) 8256 activation_327[0][0]
__________________________________________________________________________________________________
batch_normalization_347 (BatchN (None, 320, 320, 64) 256 conv2d_377[0][0]
__________________________________________________________________________________________________
activation_328 (Activation) (None, 320, 320, 64) 0 batch_normalization_347[0][0]
__________________________________________________________________________________________________
conv2d_378 (Conv2D) (None, 320, 320, 64) 36928 activation_328[0][0]
__________________________________________________________________________________________________
batch_normalization_348 (BatchN (None, 320, 320, 64) 256 conv2d_378[0][0]
__________________________________________________________________________________________________
activation_329 (Activation) (None, 320, 320, 64) 0 batch_normalization_348[0][0]
__________________________________________________________________________________________________
conv2d_379 (Conv2D) (None, 320, 320, 128 8320 activation_329[0][0]
__________________________________________________________________________________________________
batch_normalization_349 (BatchN (None, 320, 320, 128 512 conv2d_379[0][0]
__________________________________________________________________________________________________
add_99 (Add) (None, 320, 320, 128 0 batch_normalization_349[0][0]
activation_327[0][0]
__________________________________________________________________________________________________
activation_330 (Activation) (None, 320, 320, 128 0 add_99[0][0]
__________________________________________________________________________________________________
conv2d_380 (Conv2D) (None, 320, 320, 128 16512 activation_330[0][0]
__________________________________________________________________________________________________
batch_normalization_350 (BatchN (None, 320, 320, 128 512 conv2d_380[0][0]
__________________________________________________________________________________________________
activation_331 (Activation) (None, 320, 320, 128 0 batch_normalization_350[0][0]
__________________________________________________________________________________________________
conv2d_381 (Conv2D) (None, 160, 160, 128 147584 activation_331[0][0]
__________________________________________________________________________________________________
batch_normalization_351 (BatchN (None, 160, 160, 128 512 conv2d_381[0][0]
__________________________________________________________________________________________________
activation_332 (Activation) (None, 160, 160, 128 0 batch_normalization_351[0][0]
__________________________________________________________________________________________________
conv2d_382 (Conv2D) (None, 160, 160, 256 33024 activation_332[0][0]
__________________________________________________________________________________________________
conv2d_383 (Conv2D) (None, 160, 160, 256 33024 activation_330[0][0]
__________________________________________________________________________________________________
batch_normalization_352 (BatchN (None, 160, 160, 256 1024 conv2d_382[0][0]
__________________________________________________________________________________________________
batch_normalization_353 (BatchN (None, 160, 160, 256 1024 conv2d_383[0][0]
__________________________________________________________________________________________________
add_100 (Add) (None, 160, 160, 256 0 batch_normalization_352[0][0]
batch_normalization_353[0][0]
__________________________________________________________________________________________________
activation_333 (Activation) (None, 160, 160, 256 0 add_100[0][0]
__________________________________________________________________________________________________
conv2d_384 (Conv2D) (None, 160, 160, 128 32896 activation_333[0][0]
__________________________________________________________________________________________________
batch_normalization_354 (BatchN (None, 160, 160, 128 512 conv2d_384[0][0]
__________________________________________________________________________________________________
activation_334 (Activation) (None, 160, 160, 128 0 batch_normalization_354[0][0]
__________________________________________________________________________________________________
conv2d_385 (Conv2D) (None, 160, 160, 128 147584 activation_334[0][0]
__________________________________________________________________________________________________
batch_normalization_355 (BatchN (None, 160, 160, 128 512 conv2d_385[0][0]
__________________________________________________________________________________________________
activation_335 (Activation) (None, 160, 160, 128 0 batch_normalization_355[0][0]
__________________________________________________________________________________________________
conv2d_386 (Conv2D) (None, 160, 160, 256 33024 activation_335[0][0]
__________________________________________________________________________________________________
batch_normalization_356 (BatchN (None, 160, 160, 256 1024 conv2d_386[0][0]
__________________________________________________________________________________________________
add_101 (Add) (None, 160, 160, 256 0 batch_normalization_356[0][0]
activation_333[0][0]
__________________________________________________________________________________________________
activation_336 (Activation) (None, 160, 160, 256 0 add_101[0][0]
__________________________________________________________________________________________________
conv2d_387 (Conv2D) (None, 160, 160, 128 32896 activation_336[0][0]
__________________________________________________________________________________________________
batch_normalization_357 (BatchN (None, 160, 160, 128 512 conv2d_387[0][0]
__________________________________________________________________________________________________
activation_337 (Activation) (None, 160, 160, 128 0 batch_normalization_357[0][0]
__________________________________________________________________________________________________
conv2d_388 (Conv2D) (None, 160, 160, 128 147584 activation_337[0][0]
__________________________________________________________________________________________________
batch_normalization_358 (BatchN (None, 160, 160, 128 512 conv2d_388[0][0]
__________________________________________________________________________________________________
activation_338 (Activation) (None, 160, 160, 128 0 batch_normalization_358[0][0]
__________________________________________________________________________________________________
conv2d_389 (Conv2D) (None, 160, 160, 256 33024 activation_338[0][0]
__________________________________________________________________________________________________
batch_normalization_359 (BatchN (None, 160, 160, 256 1024 conv2d_389[0][0]
__________________________________________________________________________________________________
add_102 (Add) (None, 160, 160, 256 0 batch_normalization_359[0][0]
activation_336[0][0]
__________________________________________________________________________________________________
activation_339 (Activation) (None, 160, 160, 256 0 add_102[0][0]
__________________________________________________________________________________________________
conv2d_390 (Conv2D) (None, 160, 160, 128 32896 activation_339[0][0]
__________________________________________________________________________________________________
batch_normalization_360 (BatchN (None, 160, 160, 128 512 conv2d_390[0][0]
__________________________________________________________________________________________________
activation_340 (Activation) (None, 160, 160, 128 0 batch_normalization_360[0][0]
__________________________________________________________________________________________________
conv2d_391 (Conv2D) (None, 160, 160, 128 147584 activation_340[0][0]
__________________________________________________________________________________________________
batch_normalization_361 (BatchN (None, 160, 160, 128 512 conv2d_391[0][0]
__________________________________________________________________________________________________
activation_341 (Activation) (None, 160, 160, 128 0 batch_normalization_361[0][0]
__________________________________________________________________________________________________
conv2d_392 (Conv2D) (None, 160, 160, 256 33024 activation_341[0][0]
__________________________________________________________________________________________________
batch_normalization_362 (BatchN (None, 160, 160, 256 1024 conv2d_392[0][0]
__________________________________________________________________________________________________
add_103 (Add) (None, 160, 160, 256 0 batch_normalization_362[0][0]
activation_339[0][0]
__________________________________________________________________________________________________
activation_342 (Activation) (None, 160, 160, 256 0 add_103[0][0]
__________________________________________________________________________________________________
conv2d_393 (Conv2D) (None, 160, 160, 256 65792 activation_342[0][0]
__________________________________________________________________________________________________
batch_normalization_363 (BatchN (None, 160, 160, 256 1024 conv2d_393[0][0]
__________________________________________________________________________________________________
activation_343 (Activation) (None, 160, 160, 256 0 batch_normalization_363[0][0]
__________________________________________________________________________________________________
conv2d_394 (Conv2D) (None, 80, 80, 256) 590080 activation_343[0][0]
__________________________________________________________________________________________________
batch_normalization_364 (BatchN (None, 80, 80, 256) 1024 conv2d_394[0][0]
__________________________________________________________________________________________________
activation_344 (Activation) (None, 80, 80, 256) 0 batch_normalization_364[0][0]
__________________________________________________________________________________________________
conv2d_395 (Conv2D) (None, 80, 80, 512) 131584 activation_344[0][0]
__________________________________________________________________________________________________
conv2d_396 (Conv2D) (None, 80, 80, 512) 131584 activation_342[0][0]
__________________________________________________________________________________________________
batch_normalization_365 (BatchN (None, 80, 80, 512) 2048 conv2d_395[0][0]
__________________________________________________________________________________________________
batch_normalization_366 (BatchN (None, 80, 80, 512) 2048 conv2d_396[0][0]
__________________________________________________________________________________________________
add_104 (Add) (None, 80, 80, 512) 0 batch_normalization_365[0][0]
batch_normalization_366[0][0]
__________________________________________________________________________________________________
activation_345 (Activation) (None, 80, 80, 512) 0 add_104[0][0]
__________________________________________________________________________________________________
conv2d_397 (Conv2D) (None, 80, 80, 256) 131328 activation_345[0][0]
__________________________________________________________________________________________________
batch_normalization_367 (BatchN (None, 80, 80, 256) 1024 conv2d_397[0][0]
__________________________________________________________________________________________________
activation_346 (Activation) (None, 80, 80, 256) 0 batch_normalization_367[0][0]
__________________________________________________________________________________________________
conv2d_398 (Conv2D) (None, 80, 80, 256) 590080 activation_346[0][0]
__________________________________________________________________________________________________
batch_normalization_368 (BatchN (None, 80, 80, 256) 1024 conv2d_398[0][0]
__________________________________________________________________________________________________
activation_347 (Activation) (None, 80, 80, 256) 0 batch_normalization_368[0][0]
__________________________________________________________________________________________________
conv2d_399 (Conv2D) (None, 80, 80, 512) 131584 activation_347[0][0]
__________________________________________________________________________________________________
batch_normalization_369 (BatchN (None, 80, 80, 512) 2048 conv2d_399[0][0]
__________________________________________________________________________________________________
add_105 (Add) (None, 80, 80, 512) 0 batch_normalization_369[0][0]
activation_345[0][0]
__________________________________________________________________________________________________
activation_348 (Activation) (None, 80, 80, 512) 0 add_105[0][0]
__________________________________________________________________________________________________
conv2d_400 (Conv2D) (None, 80, 80, 256) 131328 activation_348[0][0]
__________________________________________________________________________________________________
batch_normalization_370 (BatchN (None, 80, 80, 256) 1024 conv2d_400[0][0]
__________________________________________________________________________________________________
activation_349 (Activation) (None, 80, 80, 256) 0 batch_normalization_370[0][0]
__________________________________________________________________________________________________
conv2d_401 (Conv2D) (None, 80, 80, 256) 590080 activation_349[0][0]
__________________________________________________________________________________________________
batch_normalization_371 (BatchN (None, 80, 80, 256) 1024 conv2d_401[0][0]
__________________________________________________________________________________________________
activation_350 (Activation) (None, 80, 80, 256) 0 batch_normalization_371[0][0]
__________________________________________________________________________________________________
conv2d_402 (Conv2D) (None, 80, 80, 512) 131584 activation_350[0][0]
__________________________________________________________________________________________________
batch_normalization_372 (BatchN (None, 80, 80, 512) 2048 conv2d_402[0][0]
__________________________________________________________________________________________________
add_106 (Add) (None, 80, 80, 512) 0 batch_normalization_372[0][0]
activation_348[0][0]
__________________________________________________________________________________________________
activation_351 (Activation) (None, 80, 80, 512) 0 add_106[0][0]
__________________________________________________________________________________________________
conv2d_403 (Conv2D) (None, 80, 80, 256) 131328 activation_351[0][0]
__________________________________________________________________________________________________
batch_normalization_373 (BatchN (None, 80, 80, 256) 1024 conv2d_403[0][0]
__________________________________________________________________________________________________
activation_352 (Activation) (None, 80, 80, 256) 0 batch_normalization_373[0][0]
__________________________________________________________________________________________________
conv2d_404 (Conv2D) (None, 80, 80, 256) 590080 activation_352[0][0]
__________________________________________________________________________________________________
batch_normalization_374 (BatchN (None, 80, 80, 256) 1024 conv2d_404[0][0]
__________________________________________________________________________________________________
activation_353 (Activation) (None, 80, 80, 256) 0 batch_normalization_374[0][0]
__________________________________________________________________________________________________
conv2d_405 (Conv2D) (None, 80, 80, 512) 131584 activation_353[0][0]
__________________________________________________________________________________________________
batch_normalization_375 (BatchN (None, 80, 80, 512) 2048 conv2d_405[0][0]
__________________________________________________________________________________________________
add_107 (Add) (None, 80, 80, 512) 0 batch_normalization_375[0][0]
activation_351[0][0]
__________________________________________________________________________________________________
activation_354 (Activation) (None, 80, 80, 512) 0 add_107[0][0]
__________________________________________________________________________________________________
conv2d_406 (Conv2D) (None, 80, 80, 256) 131328 activation_354[0][0]
__________________________________________________________________________________________________
batch_normalization_376 (BatchN (None, 80, 80, 256) 1024 conv2d_406[0][0]
__________________________________________________________________________________________________
activation_355 (Activation) (None, 80, 80, 256) 0 batch_normalization_376[0][0]
__________________________________________________________________________________________________
conv2d_407 (Conv2D) (None, 80, 80, 256) 590080 activation_355[0][0]
__________________________________________________________________________________________________
batch_normalization_377 (BatchN (None, 80, 80, 256) 1024 conv2d_407[0][0]
__________________________________________________________________________________________________
activation_356 (Activation) (None, 80, 80, 256) 0 batch_normalization_377[0][0]
__________________________________________________________________________________________________
conv2d_408 (Conv2D) (None, 80, 80, 512) 131584 activation_356[0][0]
__________________________________________________________________________________________________
batch_normalization_378 (BatchN (None, 80, 80, 512) 2048 conv2d_408[0][0]
__________________________________________________________________________________________________
add_108 (Add) (None, 80, 80, 512) 0 batch_normalization_378[0][0]
activation_354[0][0]
__________________________________________________________________________________________________
activation_357 (Activation) (None, 80, 80, 512) 0 add_108[0][0]
__________________________________________________________________________________________________
conv2d_409 (Conv2D) (None, 80, 80, 256) 131328 activation_357[0][0]
__________________________________________________________________________________________________
batch_normalization_379 (BatchN (None, 80, 80, 256) 1024 conv2d_409[0][0]
__________________________________________________________________________________________________
activation_358 (Activation) (None, 80, 80, 256) 0 batch_normalization_379[0][0]
__________________________________________________________________________________________________
conv2d_410 (Conv2D) (None, 80, 80, 256) 590080 activation_358[0][0]
__________________________________________________________________________________________________
batch_normalization_380 (BatchN (None, 80, 80, 256) 1024 conv2d_410[0][0]
__________________________________________________________________________________________________
activation_359 (Activation) (None, 80, 80, 256) 0 batch_normalization_380[0][0]
__________________________________________________________________________________________________
conv2d_411 (Conv2D) (None, 80, 80, 512) 131584 activation_359[0][0]
__________________________________________________________________________________________________
batch_normalization_381 (BatchN (None, 80, 80, 512) 2048 conv2d_411[0][0]
__________________________________________________________________________________________________
add_109 (Add) (None, 80, 80, 512) 0 batch_normalization_381[0][0]
activation_357[0][0]
__________________________________________________________________________________________________
activation_360 (Activation) (None, 80, 80, 512) 0 add_109[0][0]
__________________________________________________________________________________________________
conv2d_412 (Conv2D) (None, 80, 80, 512) 262656 activation_360[0][0]
__________________________________________________________________________________________________
batch_normalization_382 (BatchN (None, 80, 80, 512) 2048 conv2d_412[0][0]
__________________________________________________________________________________________________
activation_361 (Activation) (None, 80, 80, 512) 0 batch_normalization_382[0][0]
__________________________________________________________________________________________________
conv2d_413 (Conv2D) (None, 80, 80, 512) 2359808 activation_361[0][0]
__________________________________________________________________________________________________
batch_normalization_383 (BatchN (None, 80, 80, 512) 2048 conv2d_413[0][0]
__________________________________________________________________________________________________
activation_362 (Activation) (None, 80, 80, 512) 0 batch_normalization_383[0][0]
__________________________________________________________________________________________________
conv2d_414 (Conv2D) (None, 80, 80, 1024) 525312 activation_362[0][0]
__________________________________________________________________________________________________
conv2d_415 (Conv2D) (None, 80, 80, 1024) 525312 activation_360[0][0]
__________________________________________________________________________________________________
batch_normalization_384 (BatchN (None, 80, 80, 1024) 4096 conv2d_414[0][0]
__________________________________________________________________________________________________
batch_normalization_385 (BatchN (None, 80, 80, 1024) 4096 conv2d_415[0][0]
__________________________________________________________________________________________________
add_110 (Add) (None, 80, 80, 1024) 0 batch_normalization_384[0][0]
batch_normalization_385[0][0]
__________________________________________________________________________________________________
activation_363 (Activation) (None, 80, 80, 1024) 0 add_110[0][0]
__________________________________________________________________________________________________
conv2d_416 (Conv2D) (None, 80, 80, 512) 524800 activation_363[0][0]
__________________________________________________________________________________________________
batch_normalization_386 (BatchN (None, 80, 80, 512) 2048 conv2d_416[0][0]
__________________________________________________________________________________________________
activation_364 (Activation) (None, 80, 80, 512) 0 batch_normalization_386[0][0]
__________________________________________________________________________________________________
conv2d_417 (Conv2D) (None, 80, 80, 512) 2359808 activation_364[0][0]
__________________________________________________________________________________________________
batch_normalization_387 (BatchN (None, 80, 80, 512) 2048 conv2d_417[0][0]
__________________________________________________________________________________________________
activation_365 (Activation) (None, 80, 80, 512) 0 batch_normalization_387[0][0]
__________________________________________________________________________________________________
conv2d_418 (Conv2D) (None, 80, 80, 1024) 525312 activation_365[0][0]
__________________________________________________________________________________________________
batch_normalization_388 (BatchN (None, 80, 80, 1024) 4096 conv2d_418[0][0]
__________________________________________________________________________________________________
add_111 (Add) (None, 80, 80, 1024) 0 batch_normalization_388[0][0]
activation_363[0][0]
__________________________________________________________________________________________________
activation_366 (Activation) (None, 80, 80, 1024) 0 add_111[0][0]
__________________________________________________________________________________________________
conv2d_419 (Conv2D) (None, 80, 80, 512) 524800 activation_366[0][0]
__________________________________________________________________________________________________
batch_normalization_389 (BatchN (None, 80, 80, 512) 2048 conv2d_419[0][0]
__________________________________________________________________________________________________
activation_367 (Activation) (None, 80, 80, 512) 0 batch_normalization_389[0][0]
__________________________________________________________________________________________________
conv2d_420 (Conv2D) (None, 80, 80, 512) 2359808 activation_367[0][0]
__________________________________________________________________________________________________
batch_normalization_390 (BatchN (None, 80, 80, 512) 2048 conv2d_420[0][0]
__________________________________________________________________________________________________
activation_368 (Activation) (None, 80, 80, 512) 0 batch_normalization_390[0][0]
__________________________________________________________________________________________________
conv2d_421 (Conv2D) (None, 80, 80, 1024) 525312 activation_368[0][0]
__________________________________________________________________________________________________
batch_normalization_391 (BatchN (None, 80, 80, 1024) 4096 conv2d_421[0][0]
__________________________________________________________________________________________________
add_112 (Add) (None, 80, 80, 1024) 0 batch_normalization_391[0][0]
activation_366[0][0]
__________________________________________________________________________________________________
activation_369 (Activation) (None, 80, 80, 1024) 0 add_112[0][0]
__________________________________________________________________________________________________
average_pooling2d_25 (AveragePo (None, 1, 1, 1024) 0 activation_369[0][0]
__________________________________________________________________________________________________
average_pooling2d_26 (AveragePo (None, 2, 2, 1024) 0 activation_369[0][0]
__________________________________________________________________________________________________
average_pooling2d_27 (AveragePo (None, 4, 4, 1024) 0 activation_369[0][0]
__________________________________________________________________________________________________
average_pooling2d_28 (AveragePo (None, 8, 8, 1024) 0 activation_369[0][0]
__________________________________________________________________________________________________
conv2d_422 (Conv2D) (None, 1, 1, 512) 524800 average_pooling2d_25[0][0]
__________________________________________________________________________________________________
conv2d_423 (Conv2D) (None, 2, 2, 512) 524800 average_pooling2d_26[0][0]
__________________________________________________________________________________________________
conv2d_424 (Conv2D) (None, 4, 4, 512) 524800 average_pooling2d_27[0][0]
__________________________________________________________________________________________________
conv2d_425 (Conv2D) (None, 8, 8, 512) 524800 average_pooling2d_28[0][0]
__________________________________________________________________________________________________
lambda_25 (Lambda) (None, 80, 80, 512) 0 conv2d_422[0][0]
__________________________________________________________________________________________________
lambda_26 (Lambda) (None, 80, 80, 512) 0 conv2d_423[0][0]
__________________________________________________________________________________________________
lambda_27 (Lambda) (None, 80, 80, 512) 0 conv2d_424[0][0]
__________________________________________________________________________________________________
lambda_28 (Lambda) (None, 80, 80, 512) 0 conv2d_425[0][0]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 80, 80, 3072) 0 activation_369[0][0]
lambda_25[0][0]
lambda_26[0][0]
lambda_27[0][0]
lambda_28[0][0]
__________________________________________________________________________________________________
conv2d_426 (Conv2D) (None, 80, 80, 512) 14156288 concatenate_7[0][0]
__________________________________________________________________________________________________
batch_normalization_392 (BatchN (None, 80, 80, 512) 2048 conv2d_426[0][0]
__________________________________________________________________________________________________
activation_370 (Activation) (None, 80, 80, 512) 0 batch_normalization_392[0][0]
__________________________________________________________________________________________________
conv2d_427 (Conv2D) (None, 80, 80, 1) 513 activation_370[0][0]
__________________________________________________________________________________________________
conv2d_transpose_7 (Conv2DTrans (None, 640, 640, 1) 257 conv2d_427[0][0]
__________________________________________________________________________________________________
activation_371 (Activation) (None, 640, 640, 1) 0 conv2d_transpose_7[0][0]
==================================================================================================
Total params: 33,148,322
Trainable params: 33,112,802
Non-trainable params: 35,520
__________________________________________________________________________________________________