免费A级毛片无码专区网站-成人国产精品视频一区二区-啊 日出水了 用力乖乖在线-国产黑色丝袜在线观看下-天天操美女夜夜操美女-日韩网站在线观看中文字幕-AV高清hd片XXX国产-亚洲av中文字字幕乱码综合-搬开女人下面使劲插视频

  1. 首頁 > 生活 > >

BERT模型源碼解析

生活百科

BERT模型源碼解析
modeling.py
目錄
屬性
類
class BertConfig(object)   BERT模型配置參數(shù)類
class BertModel(object)   BERT模型類
函數(shù)
def gelu(x)  格魯激活函數(shù)
def get_activation(activation_string) 通過名稱獲取激活函數(shù)
def get_assignment_map_from_checkpoint 讀取檢查點函數(shù)
def dropout(input_tensor, dropout_prob) 丟棄函數(shù)，按一定比例丟棄權(quán)重數(shù)據(jù)
def layer_norm(input_tensor, name=None) 數(shù)據(jù)標(biāo)準(zhǔn)化
def layer_norm_and_dropout 先標(biāo)準(zhǔn)化，再丟棄
def create_initializer(initializer_range=0.02) 數(shù)據(jù)初始化
def embedding_lookup 嵌入查找函數(shù)
def embedding_postprocessor 嵌入處理函數(shù)
def create_attention_mask_from_input_mask 創(chuàng)建注意力掩碼
def attention_layer 注意力層 處理函數(shù)
def transformer_model    transformer模型
def get_shape_list 獲取張量的形狀參數(shù)列表
def reshape_to_matrix(input_tensor) 將張量轉(zhuǎn)換為二維矩陣
def reshape_from_matrix(output_tensor, orig_shape_list) 將二維張量轉(zhuǎn)換為指定維數(shù)
def assert_rank(tensor, expected_rank, name=None) 斷言 張量的維數(shù)
源碼
許可信息
# coding=utf-8 編碼使用utf-8
# Copyright 2018 The Google AI Language Team Authors.版權(quán)術(shù)語谷歌語言團隊的作者
#
# Licensed under the Apache License, Version 2.0 (the "License");根據(jù)Apache許可證進行許可
# you may not use this file except in compliance with the License.
如不符合許可證的規(guī)定，則不可使用本文件
# You may obtain a copy of the License at 可以通過下面的網(wǎng)址獲取許可證副本
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The main BERT model and related functions."""
導(dǎo)入依賴
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import collections
import copy
import json
import math
import re
import numpy as np
import six
import tensorflow as tf
模型配置
構(gòu)造函數(shù)
參數(shù)說明
class BertConfig(object):
"""Configuration for `BertModel`."""對BERT模型進行參數(shù)配置
def __init__(self,
vocab_size,
hidden_size=768,
num_hidden_layers=12,
num_attention_heads=12,
intermediate_size=3072,
hidden_act="gelu",
hidden_dropout_prob=0.1,
attention_probs_dropout_prob=0.1,
max_position_embeddings=512,
type_vocab_size=16,
initializer_range=0.02):
"""Constructs BertConfig.構(gòu)造函數(shù)
Args:參數(shù)說明
vocab_size: Vocabulary size of `inputs_ids` in `BertModel`.
inputs_ids集合的大小
hidden_size: Size of the encoder layers and the pooler layer.
編碼層和池化層的大小
num_hidden_layers: Number of hidden layers in the Transformer encoder.
Transformer 編碼器中隱藏層個數(shù)
num_attention_heads: Number of attention heads for each attention layer in
the Transformer encoder.
Transformer 編碼器中每個注意層的頭數(shù)
intermediate_size: The size of the "intermediate" (i.e., feed-forward)
layer in the Transformer encoder.
Transformer 編碼器中中間層個數(shù)
hidden_act: The non-linear activation function (function or string) in the
encoder and pooler.
編碼器和池化器的激活函數(shù)
hidden_dropout_prob: The dropout probability for all fully connected

經(jīng)驗總結(jié)擴展閱讀