面向深度学习的多模态融合技术研究综述_何俊.pdf

智慧社区解决方案

HOPC的matlab代码、实验数据和相关论文，该方法可实现可见光、红外、SAR和LiDAR和栅格地图等多种多模态遥感影像的精确匹配和配准

自从2010年以来，深度学习技术对语音，语言，视觉等子领域的推动，在语言和视觉跨模态交叉学科领域我们也取得了很多激动人心的进展，包括跨语言与图像的理解、推理和生成。多模态智能旨在融合多种模态的信息进行处理实现智能应用，在5G时代将会是重要的热点技术之一。最近IEEE Fellow何晓东和邓力等作者撰写关于多模态智能的综述论文《Multimodal Intelligence: Representation Learning, Information Fusion, and Applications》，详述了多模态智能研究进展，涵盖259篇参考文献，本文从学习多模态表示、多模态信号在不同层次上的融合以及多模态应用三个新角度对多模态深度学习的最新研究成果进行了综合分析。

基于深度学习的机器人抓取检测 采用康奈尔大学抓取数据集 We consider the problem of detecting robotic grasps in an RGB-D view of a scene containing objects. In this work, we apply a deep learning approach to solve this problem, which avoids time-consuming hand-design of features. This presents two main challenges. First, we need to evaluate a huge number of candidate grasps. In order to make detection fast and robust, we present a two-step cascaded system with two deep networks, where the top detections from the first are re-evaluated by the second. The first network has fewer features, is faster to run, and can effectively prune out unlikely candidate grasps. The second, with more features, is slower but has to run only on the top few detections. Second, we need to handle multimodal inputs effectively, for which we present a method that applies structured regularization on the weights based on multimodal group regularization. We show that our method improves performance on an RGBD robotic grasping dataset, and can be used to successfully execute grasps on two different robotic platforms.

多模态配准方法和应用综述 武汉大学电子信息学院

本文针对多模态情绪识别这一新兴领域进行综述。首先从情绪描述模型及情绪诱发方式两个方面对情绪识别的研究基础进行了综述。接着针对多模态情绪识别中的信息融合这一重难点问题，从数据级融合、特征级融合、决策级融合、模型级融合4种融合层次下的主流高效信息融合策略进行了介绍。

基于深度模态融合网络的多模态情感识别，陈坤，夏海轮，情感识别在情感计算领域是一个具有挑战性的课题，因为从音视频多模态数据中提取出具有判别性的特征来识别人类情感中的细微差距仍

关于环形拓扑多模态多目标粒子群算法的代码，直接在MATLAB上运行即可

原文pdf & Code