CVSE:用于图像-文本匹配的共识感知视觉语义嵌入论文的官方源代码(ECCV 2020)-源码

上传者: 42174176 | 上传时间: 2021-07-14 15:46:05 | 文件大小: 214.1MB | 文件类型: ZIP
介绍 这是共识感知视觉语义嵌入(CVSE) ,这是论文《的官方源代码。 它基于PyTorch中的构建。 抽象的: 图像文本匹配在桥接视觉和语言方面起着核心作用。 大多数现有方法仅依靠图像-文本实例对来学习它们的表示,从而利用它们的匹配关系并进行相应的对齐。 这样的方法只是利用实例成对数据中包含的表面关联,而没有考虑任何外部常识知识,这可能会阻碍它们推理图像和文本之间更高层次关系的能力。 在本文中,我们提出了一种共识感知视觉语义嵌入(CVSE)模型,以将共识信息(即两种模式之间共享的常识知识)整合到图像文本匹配中。 具体而言,通过计算来自图像字幕语料库的语义概念之间的统计共现相关性,并部署构造的概念相关图以产生共识感知概念(CAC)表示,来利用共识信息。 之后,CVSE基于所利用的共识以及两种模式的实例级表示形式,学习图像与文本之间的关联和对齐方式。 在两个公共数据集上进行的大量实验证明

文件下载

资源详情

[{"title":"( 64 个子文件 214.1MB ) CVSE:用于图像-文本匹配的共识感知视觉语义嵌入论文的官方源代码(ECCV 2020)-源码","children":[{"title":"CVSE-master","children":[{"title":"runs","children":[{"title":"f30k","children":[{"title":"CVSE_f30k","children":[{"title":"log","children":[{"title":"events.out.tfevents.1608974921.DESKTOP-VNJSTH5 <span style='color:#111;'> 40B </span>","children":null,"spread":false}],"spread":true},{"title":"model_best.pth.tar <span style='color:#111;'> 96.57MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"coco","children":[{"title":"CVSE_COCO","children":[{"title":"model_best.pth.tar <span style='color:#111;'> 98.32MB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true},{"title":"figures","children":[{"title":"framework_CVSE.jpg <span style='color:#111;'> 471.94KB </span>","children":null,"spread":false}],"spread":true},{"title":"train_coco.py <span style='color:#111;'> 14.15KB </span>","children":null,"spread":false},{"title":"data","children":[{"title":"f30k_annotations","children":[{"title":"Concept_annotations","children":[{"title":"f30k_concepts_glove_word2vec.pkl <span style='color:#111;'> 703.28KB </span>","children":null,"spread":false},{"title":"category_concepts.json <span style='color:#111;'> 4.11KB </span>","children":null,"spread":false},{"title":"all_f30k_concept_label.json <span style='color:#111;'> 2.95MB </span>","children":null,"spread":false},{"title":"f30k_adj_concepts.pkl <span style='color:#111;'> 1.03MB </span>","children":null,"spread":false},{"title":"vocab_f30k_concepts.pkl <span style='color:#111;'> 6.21KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"f30k","children":[{"title":"dataset_flickr30k.json <span style='color:#111;'> 36.54MB </span>","children":null,"spread":false}],"spread":true},{"title":"coco_annotations","children":[{"title":"Concept_annotations","children":[{"title":"category_concepts.json <span style='color:#111;'> 4.11KB </span>","children":null,"spread":false},{"title":"coco_adj_concepts.pkl <span style='color:#111;'> 1.03MB </span>","children":null,"spread":false},{"title":"trainval_concept_label.json <span style='color:#111;'> 16.01MB </span>","children":null,"spread":false},{"title":"vocab_trainval_concepts.pkl <span style='color:#111;'> 6.21KB </span>","children":null,"spread":false},{"title":"coco_concepts_glove_word2vec.pkl <span style='color:#111;'> 703.28KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"coco_to_f30k_annotations","children":[{"title":"Concept_annotations","children":[{"title":"category_concepts.json <span style='color:#111;'> 4.11KB </span>","children":null,"spread":false},{"title":"Flickr30k_test_concept_label.json <span style='color:#111;'> 88.77KB </span>","children":null,"spread":false},{"title":"coco_adj_concepts.pkl <span style='color:#111;'> 1.03MB </span>","children":null,"spread":false},{"title":"vocab_trainval_concepts.pkl <span style='color:#111;'> 6.21KB </span>","children":null,"spread":false},{"title":"coco_concepts_glove_word2vec.pkl <span style='color:#111;'> 703.28KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"coco","children":[{"title":"annotations","children":[{"title":"coco_test_ids.npy <span style='color:#111;'> 195.39KB </span>","children":null,"spread":false},{"title":"coco_restval_ids.npy <span style='color:#111;'> 1.16MB </span>","children":null,"spread":false},{"title":"coco_dev_ids.npy <span style='color:#111;'> 195.39KB </span>","children":null,"spread":false},{"title":"coco_train_ids.npy <span style='color:#111;'> 3.16MB </span>","children":null,"spread":false},{"title":"captions_train2014.json <span style='color:#111;'> 63.69MB </span>","children":null,"spread":false},{"title":"image_info_test2014.json <span style='color:#111;'> 7.50MB </span>","children":null,"spread":false},{"title":"captions_val2014.json <span style='color:#111;'> 30.92MB </span>","children":null,"spread":false}],"spread":true},{"title":"download.sh <span style='color:#111;'> 402B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"model_CVSE.py <span style='color:#111;'> 36.06KB </span>","children":null,"spread":false},{"title":"__pycache__","children":[{"title":"model_CVSE.cpython-36.pyc <span style='color:#111;'> 21.03KB </span>","children":null,"spread":false},{"title":"evaluation.cpython-36.pyc <span style='color:#111;'> 12.68KB </span>","children":null,"spread":false},{"title":"data.cpython-36.pyc <span style='color:#111;'> 11.03KB </span>","children":null,"spread":false},{"title":"vocab.cpython-36.pyc <span style='color:#111;'> 3.25KB </span>","children":null,"spread":false}],"spread":true},{"title":"evaluation.py <span style='color:#111;'> 19.42KB </span>","children":null,"spread":false},{"title":"evaluate.py <span style='color:#111;'> 4.21KB </span>","children":null,"spread":false},{"title":".idea","children":[{"title":"$PRODUCT_WORKSPACE_FILE$ <span style='color:#111;'> 461B </span>","children":null,"spread":false},{"title":"$CACHE_FILE$ <span style='color:#111;'> 467B </span>","children":null,"spread":false},{"title":"misc.xml <span style='color:#111;'> 197B </span>","children":null,"spread":false},{"title":"vcs.xml <span style='color:#111;'> 180B </span>","children":null,"spread":false},{"title":"modules.xml <span style='color:#111;'> 308B </span>","children":null,"spread":false},{"title":"dictionaries <span style='color:#111;'> 163B </span>","children":null,"spread":false},{"title":"workspace.xml <span style='color:#111;'> 6.51KB </span>","children":null,"spread":false},{"title":"Code_github_ECCV2020_revised.iml <span style='color:#111;'> 453B </span>","children":null,"spread":false},{"title":"inspectionProfiles","children":[{"title":"profiles_settings.xml <span style='color:#111;'> 128B </span>","children":null,"spread":false}],"spread":false}],"spread":true},{"title":"vocab","children":[{"title":"coco_vocab.pkl <span style='color:#111;'> 315.87KB </span>","children":null,"spread":false},{"title":"f30k_vocab.json <span style='color:#111;'> 292.58KB </span>","children":null,"spread":false},{"title":"f8k_precomp_vocab.pkl <span style='color:#111;'> 82.65KB </span>","children":null,"spread":false},{"title":"f8k_vocab.pkl <span style='color:#111;'> 88.31KB </span>","children":null,"spread":false},{"title":"coco_precomp_vocab.pkl <span style='color:#111;'> 267.90KB </span>","children":null,"spread":false},{"title":"coco_precomp_vocab.json <span style='color:#111;'> 392.13KB </span>","children":null,"spread":false},{"title":"f30k_vocab.pkl <span style='color:#111;'> 230.42KB </span>","children":null,"spread":false},{"title":"f30k_precomp_vocab.json <span style='color:#111;'> 14B </span>","children":null,"spread":false},{"title":"f30k_precomp_vocab.pkl <span style='color:#111;'> 227.45KB </span>","children":null,"spread":false}],"spread":true},{"title":"train_f30k.py <span style='color:#111;'> 14.36KB </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 5.05KB </span>","children":null,"spread":false},{"title":"util","children":[{"title":"utils.py <span style='color:#111;'> 4.01KB </span>","children":null,"spread":false},{"title":"__pycache__","children":[{"title":"C_GCN.cpython-36.pyc <span style='color:#111;'> 4.03KB </span>","children":null,"spread":false},{"title":"util_C_GCN.cpython-36.pyc <span style='color:#111;'> 3.35KB </span>","children":null,"spread":false},{"title":"utils.cpython-36.pyc <span style='color:#111;'> 3.67KB </span>","children":null,"spread":false}],"spread":false},{"title":"util_C_GCN.py <span style='color:#111;'> 6.04KB </span>","children":null,"spread":false},{"title":"C_GCN.py <span style='color:#111;'> 4.15KB </span>","children":null,"spread":false}],"spread":true},{"title":"data.py <span style='color:#111;'> 17.72KB </span>","children":null,"spread":false},{"title":"vocab.py <span style='color:#111;'> 2.90KB </span>","children":null,"spread":false}],"spread":false}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明