DL-FWI内训Day3

InversionNet-网络搭建

InversionNet是在2019年提出的一种完全端到端网络算法，在众多的端到端网络中InversionNet网络较为简单，原因是该算法采用了最基础的CNN网络。

InversionNet构建了一个编码器-解码器的卷积神经网络，来模拟地震数据和地下速度结构的对应关系。

结构分析

卷积层

网络中，每一个卷积层包含了3个部分：卷积计算、批归一化和激活函数。

• 卷积承担输入信号的责任，同时担任滤波器的作用以提取有意义的特征。
• 批量归一化（Batch Normalization）的理论依据是：如果网络的输入具有零均值、单位方差和去相关，那么深层网络的收敛速度会加快。因此，BN层的作用是对馈送到网络中中间层的数据子集进行归一化操作后输出。
• LeakyReLU激活函数用于解决ReLU激活函数中神经元死亡的问题，在该网络中有较好的效果。

class ConvBlock(nn.Module):
    def __init__(
        self,
        in_fea,
        out_fea,
        kernel_size=3,
        stride=1,
        padding=1,
        norm=nn.BatchNorm2d,
        relu_slop=0.2,
        dropout=None,
    ):
        """
        Standard convolution operation
        [Affiliated with InversionNet]

        :param in_fea:          Number of channels of input
        :param out_fea:         Number of channels of output
        :param kernel_size:     Size of the convolution kernel
        :param stride:          Step size of the convolution
        :param padding:         Zero-fill width
        :param norm:            The means of normalization
        :param relu_slop:       Parameters of relu
        :param dropout:         Whether to apply dropout
        """
        super(ConvBlock, self).__init__()
        layers = [
            nn.Conv2d(
                in_channels=in_fea,
                out_channels=out_fea,
                kernel_size=kernel_size,
                stride=stride,
                padding=padding,
            )
        ]
        layers.append(norm(out_fea))
        layers.append(nn.LeakyReLU(relu_slop, inplace=True))
        if dropout:
            layers.append(nn.Dropout2d(0.8))
        self.layers = nn.Sequential(*layers)

    def forward(self, X):
        return self.layers(X)

编码器

编码器的部分主要由卷积层构建，编码器从地震数据中提取特征，将它们压缩成单一的高维向量。由于地震数据时间维度的大小$T$远大于空间维度$R$，网络中先使用了非方形的卷积核（$7\times 1$、$3\times 1$等尺寸）以及步长（$2\times 1$）对地震波形图进行卷积计算。这样，经过卷积计算后的地震数据矩阵尺寸更接近于传统神经网络中使用的方形矩阵，便于后续的特征提取。

地震数据尺寸经过上述计算转化为方形后，再使用尺寸为$3\times 3$和$8\times 8$的卷积核进行卷积计算，最终将尺寸为None, 5, 32, 1000的地震数据编码为尺寸为None, 512, 1, 1的高维向量。这样的操作被认为是合理的压缩方式，因为我们通常认为没有必要保留时间和空间的相关关系。

解码器

高维向量一步步升维为最终速度模型图的过程由若干个反卷积（转置卷积）计算完成。类似可以实现升维效果的操作还有反池化，但在这里的地震反演任务中，反卷积较反池化有更好的效果。

反卷积：一种特殊的卷积，先通过padding来扩大图像尺寸，紧接着跟正向卷积一样，旋转卷积核180度，再进行卷积计算。

反池化：池化的逆操作，无法通过反池化的结果还原出全部的原始数据。因为池化的过程就是只保留主要信息，舍去部分信息。如想从池化后的这些主要信息恢复出全部信息，则存在信息缺失，这时只能通过补位来实现最大程度的信息完整。包含最大反池化和平均反池化。

class InversionNet(nn.Module):
    def __init__(self, dim1=32, dim2=64, dim3=128, dim4=256, dim5=512, **kwargs):
        """
        Network architecture of InversionNet

        :param dim1:            Number of channels in the 1st layer
        :param dim2:            Number of channels in the 2nd layer
        :param dim3:            Number of channels in the 3rd layer
        :param dim4:            Number of channels in the 4th layer
        :param dim5:            Number of channels in the 5th layer
        :param sample_spatial:  Scale parameters for sampling in space
        """
        super(InversionNet, self).__init__()
        self.convblock1 = ConvBlock(5, dim1, kernel_size=(7, 1), stride=(2, 1), padding=(3, 0))
        self.convblock2_1 = ConvBlock(dim1, dim2, kernel_size=(3, 1), stride=(2, 1), padding=(1, 0))
        self.convblock2_2 = ConvBlock(dim2, dim2, kernel_size=(3, 1), stride=1, padding=(1, 0))
        self.convblock3_1 = ConvBlock(dim2, dim2, kernel_size=(3, 1), stride=(2, 1), padding=(1, 0))
        self.convblock3_2 = ConvBlock(dim2, dim2, kernel_size=(3, 1), stride=1, padding=(1, 0))
        self.convblock4_1 = ConvBlock(dim2, dim3, kernel_size=(3, 1), stride=(2, 1), padding=(1, 0))
        self.convblock4_2 = ConvBlock(dim3, dim3, kernel_size=(3, 1), stride=1, padding=(1, 0))
        self.convblock5_1 = ConvBlock(dim3, dim3, kernel_size=(3, 1), stride=(2, 1), padding=(1, 0))
        self.convblock5_2 = ConvBlock(dim3, dim3, kernel_size=(3, 1), stride=1, padding=(1, 0))
        self.convblock6_1 = ConvBlock(dim3, dim4, kernel_size=(3, 3), stride=2, padding=1)
        self.convblock6_2 = ConvBlock(dim4, dim4, kernel_size=(3, 3), stride=1, padding=1)
        self.convblock7_1 = ConvBlock(dim4, dim4, kernel_size=(3, 3), stride=2, padding=1)
        self.convblock7_2 = ConvBlock(dim4, dim4, kernel_size=(3, 3), stride=1, padding=1)
        self.convblock8 = ConvBlock(dim4, dim5, kernel_size=8, padding=0)

        self.deconv1_1 = DeconvBlock(dim5, dim5, kernel_size=5, stride=1, padding=0)
        self.deconv1_2 = ConvBlock(dim5, dim5, kernel_size=3, stride=1)
        self.deconv2_1 = DeconvBlock(dim5, dim4, kernel_size=4, stride=2, padding=1)
        self.deconv2_2 = ConvBlock(dim4, dim4, kernel_size=3, stride=1)
        self.deconv3_1 = DeconvBlock(dim4, dim3, kernel_size=4, stride=2, padding=1)
        self.deconv3_2 = ConvBlock(dim3, dim3, kernel_size=3, stride=1)
        self.deconv4_1 = DeconvBlock(dim3, dim2, kernel_size=4, stride=2, padding=1)
        self.deconv4_2 = ConvBlock(dim2, dim2, kernel_size=3, stride=1)
        self.deconv5_1 = DeconvBlock(dim2, dim1, kernel_size=4, stride=2, padding=1)
        self.deconv5_2 = ConvBlock(dim1, dim1, kernel_size=3, stride=1)
        self.deconv6 = ConvBlock_Tanh(dim1, 1)

    def forward(self, x):
        # Encoder Part
        x = self.convblock1(x)  # (None, 32, 500, 32)
        x = self.convblock2_1(x)  # (None, 64, 250, 32)
        x = self.convblock2_2(x)  # (None, 64, 250, 32)
        x = self.convblock3_1(x)  # (None, 64, 125, 32)
        x = self.convblock3_2(x)  # (None, 64, 125, 32)
        x = self.convblock4_1(x)  # (None, 128, 63, 32)
        x = self.convblock4_2(x)  # (None, 128, 63, 32)
        x = self.convblock5_1(x)  # (None, 128, 32, 32)
        x = self.convblock5_2(x)  # (None, 128, 32, 32)
        x = self.convblock6_1(x)  # (None, 256, 16, 16)
        x = self.convblock6_2(x)  # (None, 256, 16, 16)
        x = self.convblock7_1(x)  # (None, 256, 8, 8)
        x = self.convblock7_2(x)  # (None, 256, 8, 8)
        x = self.convblock8(x)  # (None, 512, 1, 1)

        # Decoder Part
        x = self.deconv1_1(x)  # (None, 512, 5, 5)
        x = self.deconv1_2(x)  # (None, 512, 5, 5)
        x = self.deconv2_1(x)  # (None, 256, 10, 10)
        x = self.deconv2_2(x)  # (None, 256, 10, 10)
        x = self.deconv3_1(x)  # (None, 128, 20, 20)
        x = self.deconv3_2(x)  # (None, 128, 20, 20)
        x = self.deconv4_1(x)  # (None, 64, 40, 40)
        x = self.deconv4_2(x)  # (None, 64, 40, 40)
        x = self.deconv5_1(x)  # (None, 32, 80, 80)
        x = self.deconv5_2(x)  # (None, 32, 80, 80)
        x = F.pad(x, [-2, -3, -2, -3], mode="constant", value=0)
        # (None, 32, 75, 75) 125, 100
        x = self.deconv6(x)  # (None, 1, 75, 75)
        return x