【AI】[BERT]mat1 and mat2 shapes cannot be ・・・

■事象

BERTのbert-base-uncasedをbert-large-uncasedに変更したら下記エラー

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x1024 and 768x1)

■原因

bert-base-uncasedの隠れ層次元が768次元ですが、

bert-large-uncasedの隠れ層次元は1024次元のため

■解決方法

【当エラーが発生するコード】

class BERTModel(nn.Module):
    def __init__(self, dropout_rate):
        super(BERTModel, self).__init__()
        self.bert = BertModel.from_pretrained('bert-large-uncased')
        self.dropout_rate = dropout_rate
        self.dropout = nn.Dropout(self.dropout_rate)
        self.fc = nn.Linear(768, 1)
        self.sig = nn.Sigmoid()

        後略

【正解コード】

＜bert-base-uncaseの場合＞

class BERTModel(nn.Module):
    def __init__(self, dropout_rate):
        super(BERTModel, self).__init__()
        self.bert = BertModel.from_pretrained('bert-base-uncased')
        self.dropout_rate = dropout_rate
        self.dropout = nn.Dropout(self.dropout_rate)
        self.fc = nn.Linear(768, 1)
        self.sig = nn.Sigmoid()

        後略

＜bert-large-uncaseの場合＞

class BERTModel(nn.Module):
    def __init__(self, dropout_rate):
        super(BERTModel, self).__init__()
        self.bert = BertModel.from_pretrained('bert-large-uncased')
        self.dropout_rate = dropout_rate
        self.dropout = nn.Dropout(self.dropout_rate)
        self.fc = nn.Linear(1024, 1)
        self.sig = nn.Sigmoid()

        後略