Deep Crossing算法&实现

背景

微软与2016年提出的Deep Crossing模型是一次深度学习框架在推荐系统中的完整应用。Deep Crossing模型完整地解决了从特征工程、稀疏向量稠密化、多层神经网络进行优化目标拟合等一系列深度学习在推荐系统中地应用问题。

论文

www.kdd.org

https://www.kdd.org/kdd2016/papers/files/adf0975-shanA.pdf

模型结构

如上述图中，其网络结构分为4层——Embedding层、Stacking层、Multiple Residual Units层和Scoring层；

Embedding层：将稀疏地类别特征转换为稠密地Embedding向量；

Stacking层：连接层，将不同地Embedding特征和数值型特征拼接在一起；

Multiple Residual Units层：主要结构为多层感知机，采用多层残差网络作为MLP地具体实现；通过多层残差网络对特征向量各个维度进行充分的交叉组合，使模型能够抓取到更多的非线性特征和组合特征的信息，进而使深度学习模型在表达能力上较传统机器学习模型大为增强。

Scoring层：作为输出层，就是为了拟合优化目标而存在的。对于CTR预估这类二分类问题，Scoring层往往使用的是逻辑回归模型；

代码实现

recommender_system/deepCrossing.py at main · Hcyand/recommender_system

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters You can't perform that action at this time. You signed in with another tab or window.

https://github.com/Hcyand/recommender_system/blob/main/algorithm/deep_learning/model/deepCrossing.py


class ResLayer(Layer):
    """hidden layer"""

    def __init__(self, hidden_units):
        super(ResLayer, self).__init__()
        self.dense_layer = [Dense(i, activation='relu') for i in hidden_units]

    def build(self, input_shape):
        self.output_layer = Dense(input_shape[-1], activation=None)

    def call(self, inputs, *args, **kwargs):
        x = inputs
        for layer in self.dense_layer:
            x = layer(x)
        x = self.output_layer(x)

        output = inputs + x
        return tf.nn.relu(output)

class DeepCrossing(Model):
    def __init__(self, feature_columns, k, hidden_units, res_layer_num):
        super(DeepCrossing, self).__init__()
        self.dense_feature_columns, self.sparse_feature_columns = feature_columns
        self.embed_layer = EmbedLayer(self.sparse_feature_columns)
        self.res_layer = [ResLayer(hidden_units) for _ in range(res_layer_num)]
        self.output_layer = Dense(1, activation='sigmoid')

    def call(self, inputs, training=None, mask=None):
        dense_inputs, sparse_inputs = inputs[:, :13], inputs[:, 13:]
        emb = self.embed_layer(sparse_inputs)
        x = tf.concat([dense_inputs, emb], axis=-1)
        for layer in self.res_layer:
            x = layer(x)
        output = self.output_layer(x)
        return output

参考

[论文笔记]Deep Crossing模型原理

论文不是我写的......书也不是我写的......我只是论文的搬运工......专栏简介都写的水硕的学习笔记，有问题你问我，我大概率也不清楚，我太菜了...... 本文参考了王喆大佬的《深度学习推荐系统》一书3.3节[1] 如果说AutoRec模型是将深度学习的思想应用于推荐系统的初步尝试，那么微软于2016年提出的Depp ...

https://zhuanlan.zhihu.com/p/130455031