■事象
BERTのbert-base-uncasedをbert-large-uncasedに変更したら下記エラー
RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x1024 and 768x1)
■原因
bert-base-uncasedの隠れ層次元が768次元ですが、
bert-large-uncasedの隠れ層次元は1024次元のため
■解決方法
【当エラーが発生するコード】
class BERTModel(nn.Module):
def __init__(self, dropout_rate):
super(BERTModel, self).__init__()
self.bert = BertModel.from_pretrained('bert-large-uncased')
self.dropout_rate = dropout_rate
self.dropout = nn.Dropout(self.dropout_rate)
self.fc = nn.Linear(768, 1)
self.sig = nn.Sigmoid()
後略
【正解コード】
<bert-base-uncaseの場合>
class BERTModel(nn.Module):
def __init__(self, dropout_rate):
super(BERTModel, self).__init__()
self.bert = BertModel.from_pretrained('bert-base-uncased')
self.dropout_rate = dropout_rate
self.dropout = nn.Dropout(self.dropout_rate)
self.fc = nn.Linear(768, 1)
self.sig = nn.Sigmoid()
後略
<bert-large-uncaseの場合>
class BERTModel(nn.Module):
def __init__(self, dropout_rate):
super(BERTModel, self).__init__()
self.bert = BertModel.from_pretrained('bert-large-uncased')
self.dropout_rate = dropout_rate
self.dropout = nn.Dropout(self.dropout_rate)
self.fc = nn.Linear(1024, 1)
self.sig = nn.Sigmoid()
後略