Deep & Cross算法&实现

背景

DCN由2017年斯坦福大学和谷歌的研究人员提出的，针对Wide&Deep中Wide部分进行改进得到；

原理

Cross Network

与Wide&Deep差异之处在于Cross层，其中特征交叉的数学表达式为：

图形表达式：

优缺点

优点：

引入 cross layer 显示的构造有限阶特征组合，无需特征工程，可端到端训练；

cross layer 具有线性复杂度，可累加多层构造高阶特征交互，并且因为其类似残差连接的计算方式，使其累加多层也不会产生梯度消失问题；

跟 deepfm 相同，两个分支共享输入，可更精确的训练学习。

缺点：

cross layer 是以 bit-wise 方式构造特征组合的，最小粒度是特征向量中的每个元素，这样导致 DCN 不会考虑域的概念，属于同一特征的各个元素应同等对待；

代码实现

recommender_system/deepCross.py at main · Hcyand/recommender_system

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters You can't perform that action at this time. You signed in with another tab or window.

https://github.com/Hcyand/recommender_system/blob/main/algorithm/deep_learning/model/deepCross.py


class CrossLayer(Layer):
    def __init__(self, layer_num, reg_w=1e-4, reg_b=1e-4):
        super(CrossLayer, self).__init__()
        self.layer_num = layer_num
        self.reg_w = reg_w
        self.reg_b = reg_b

    def build(self, input_shape):
        self.cross_weight = [
            self.add_weight(name='w' + str(i),
                            shape=(input_shape[1], 1),
                            initializer=tf.random_normal_initializer(),
                            regularizer=l2(self.reg_w),
                            trainable=True)
            for i in range(self.layer_num)
        ]

        self.cross_bias = [
            self.add_weight(name='b' + str(i),
                            shape=(input_shape[1], 1),
                            initializer=tf.random_normal_initializer(),
                            regularizer=l2(self.reg_b),
                            trainable=True)
            for i in range(self.layer_num)
        ]

    def call(self, inputs, *args, **kwargs):
        x0 = tf.expand_dims(inputs, axis=2)
        x1 = x0
        for i in range(self.layer_num):
            x1_w = tf.matmul(tf.transpose(x1, [0, 2, 1]), self.cross_weight[i])
            x1 = tf.matmul(x0, x1_w) + self.cross_bias[i] + x1

        output = tf.squeeze(x1, axis=2)
        return output

参考

推荐算法(五)--谷歌经典 Deep&Cross Network原理及代码实践

https://zhuanlan.zhihu.com/p/368381633