O'Reilly、Cloudera 主办
Make Data Work
2017年7月12-13日:培训
2017年7月13-15日:会议
北京,中国

基于深度学习的网络表示 (Network representations based on deep learning)

此演讲使用中文 (This will be presented in Chinese)

张铭 (北京大学)
14:00–14:40 Saturday, 2017-07-15
AI应用 (AI applications)
地点: 报告厅(Auditorium) 观众水平 (Level): Intermediate
平均得分:: **...
(2.50, 2 次得分)

必要预备知识 (Prerequisite Knowledge)

了解人工智能、深度学习的基本原理

您将学到什么 (What you'll learn)

大规模的网络结构数据和丰富的网络节点信息对相关的研究方法提出了新的挑战,受到了学术界和工业界的广泛关注。听众可以了解学习网络的低维网络表示,了解Line和LargeVis这两个开源工具的基本理论和应用。

描述 (Description)

网络结构在现实世界中无处不在(如航线网络、通信网络、论文引用网络、世界万维网和社交网络等),大规模的网络结构数据和丰富的网络节点信息对相关的研究方法提出了新的挑战,受到了学术界和工业界的广泛关注。本报告重点介绍北大博士毕业生唐建和导师张铭团队合作的系列工作。

学习网络的低维网络表示,在不同应用领域中体现出很好的效率和效果,近年来受到了学术界和工业界的密切关注。本报告将对基于神经网络的网络表示方法进行了介绍,
,相比传统的独热表示(one-hot representation),表示学习能够抓住数据之间的相似性同时缓解数据稀疏性问题(data sparsity)。这些方法可以处理现实世界中拥有百万级节点和十亿级边的网络结构,主要考虑了网络结构信息和网络节点自身信息(如文本信息和属性信息等)。

LINE模型提出了一种适用于不同类别网络图结构(有向图、无向图和加权图)的网络学习模型LINE。具体上,LINE模型从一阶相似性(first-order proximity)和二阶相似性(second-order proximity)两方面设计目标函数。基于一阶或者二阶相似性,LINE可以分别学习到一种网络表示。为了同时使用这两种相似性,LINE模型将一阶节点向量和二阶节点向量拼接起来作为最终的节点表示。LINE模型很好地抓住了词之间的全局共现信息,学习词的向量表示,相比现在流行的Skip-gram词向量模型效率更高而且效果更好。

LargeVis研究如何将庞大的信息网络植入到低维空间并进行可视化分析。首先根据数据构造一个准确的K近邻图,然后再在低维空间对图进行布局。LargeVis显著降低了计算成本,有效地优化通过异步的随机梯度下降法达到了线性时间复杂度,整个过程因此很容易扩展到数百万高维数据点,使得在二维或者三维空间上直观地观察和理解高维数据成为可能。

LINE和LargeVis的研究论文先后发表在WWW 2015和 WWW 2016上,获得WWW 2016最佳论文奖提名(最终排名第二),累计他引已经超过200篇次,在深度学习相关领域得到了广泛的应用。

参考文献:
1. 张铭,尹伊淳,唐建,基于深度学习的网络表示研究进展,人工智能通讯,2016.03.31,6(3):1~6.
2. Jian Tang,Jingzhou Liu, Ming Zhang, Qiaozhu Mei,Visualizing Large-scale and High-dimensional (#) Ming Zhang(*) Data,Proceedings of the 25th International Conference on World Wide Web,Montreal,2016.04.11-2016.04.15
3. Jian Tang,Jingzhou Liu,Ming Zhang,Qiaozhu Mei,Visualizing Large-scale and High-dimensional Data,25th International Conference on World Wide Web,Montreal, Canada,2016.04.11-2016.04.15.


Information networks are ubiquitous in the real world, in the airline, communication, and publishing industries, on the World Wide Web, and within social networks. The methodology for studying network-related topics has been challenged by large-scale network structure data and information-rich vertices in a network and has been attracting increasing attention in both academia and industry. Ming Zhang explores the LINE model and LargeVis, projects that came out of work done by Ming and her advisee Jian Tan.

Learning networks’ low-dimensional representation has achieved high efficiency and effectiveness in a variety of domains. Ming introduces network representation methods based on neural networks. Compared with traditional one-hot representation, representation learning can preserve the similarity among data while alleviating the problem of data sparsity. These methods can tackle real-world networks with millions of vertices and billions of edges, and they take both the network structure information and vertices’ own information, such as text and attributes, into account.

The LINE model proposes a network representation learning model that suits arbitrary types of information networks (e.g., undirected, directed, and/or weighted). Specifically, the LINE model has a carefully designed objective function that preserves both the first- and second-order proximities. Based on the two proximities, LINE learns two network representations separately. To utilize the first-order and second-order proximities, the LINE model concatenates the vector representation learned into a longer vector. The LINE model better captures the global structure of word co-occurrence, learning the word vector representation. Therefore, it outperforms the popular skip-gram word embedding model in terms of both effectiveness and efficiency.

LargeVis studies the problem of visualizing large-scale and high-dimensional data in a low-dimensional space (typically 2D or 3D). It first constructs an accurately approximated k-nearest neighbor graph from the data and then lays out the graph in the low-dimensional space. LargeVis significantly reduces the computational cost of the graph construction step and can be effectively optimized through asynchronous stochastic gradient descent with a linear time complexity. The whole procedure thus easily scales to millions of high-dimensional data points, making it possible to intuitively observe and understand high-dimensional data in a 2D or 3D space.

Photo of 张铭

张铭

北京大学

张铭,北京大学信息科学技术学院教授,博士生导师,ACM Education Council惟一的中国委员兼任中国ACM教育专委会主 席,是ACM/IEEE IT2017学科规范起草小组成员。自1984年考入北京大学,分别获得学士、硕士和博士学位。研究方向为文本挖掘、社会网络分析、教育大数据等,目前主持国家自然科学基金和教育部博士点基金在研项目,合作发表科研学术论文100多篇(ICML, KDD, AAAI, IJCAI, ACL, WWW, TKDE等A类会议和期刊),获得ICML 2014最佳论文奖。发表了SIGCSE、L@S等教学研究论文,出版学术专著1部,获软件著作权6项,获发明专利3项。主编多部教材,其中2部教材为国家“十一五”规划教材,《数据结构与算法》获北京市精品教材奖并得到国家“十二五”规划教材支持。主持的“数据结构与算法”被评选为国家级和北京市级精品课程,也是教育部精品资源共享课程。

联系OReillyData

关注OReillyData微信号获取最新会议信息并浏览前沿数据文章。

WeChat QRcode

 

Stay Connected Image 1
Stay Connected Image 3
Stay Connected Image 2

阅读关于大数据的最新理念。

ORB Data Site