NLP项目，主要包括atten-seq2seq和transformer模型，实现机器翻译以及对话系统
使用说明
分对话系统和机器翻译两部分
data为数据集
model为训练的模型
translation文件夹下又分了Seq2Seq和transformer两个模型，大家按需查看使用
以transformer文件夹为例，attention.py主要实现了注意力机制，transformer.py实现了transformer的主体架构，data.py为数据的预处理以及生成了词典、dataset、dataloader，readdata.py运行可以查看数据形状，train.py为训练模型，predict.py为预测，config.py为一些参数的定义。
transformer机器翻译的模型是用cuda:1训练的，如果要使用可能需要修改代码
如：gpu->cpu，即在CPU上使用
torch.load('trans_encoder.mdl', map_location= lambda storage, loc: storage)
torch.load('trans_decoder.mdl', map_location= lambda storage, loc: storage)
文件下载
资源详情
[{"title":"（ 92 个子文件 89.09MB ） NLP项目，主要包括atten-seq2seq和transformer模型，实现机器翻译以及对话系统","children":[{"title":"nlp-master","children":[{"title":"dialogue","children":[{"title":"transformer","children":[{"title":"train (1).ipynb <span style='color:#111;'> 129.06KB </span>","children":null,"spread":false},{"title":"readdata.py <span style='color:#111;'> 498B </span>","children":null,"spread":false},{"title":"BPE.py <span style='color:#111;'> 1.65KB </span>","children":null,"spread":false},{"title":"predict.py <span style='color:#111;'> 2.14KB </span>","children":null,"spread":false},{"title":"train1.ipynb <span style='color:#111;'> 34.35KB </span>","children":null,"spread":false},{"title":"train.ipynb <span style='color:#111;'> 78.74KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 4.18KB </span>","children":null,"spread":false},{"title":"train-sampling.py <span style='color:#111;'> 6.79KB </span>","children":null,"spread":false},{"title":"transformer.py <span style='color:#111;'> 8.63KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 4.29KB </span>","children":null,"spread":false},{"title":"test.py <span style='color:#111;'> 4.84KB </span>","children":null,"spread":false},{"title":"attention.py <span style='color:#111;'> 5.91KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 1.01KB </span>","children":null,"spread":false}],"spread":false},{"title":"Seq2Seq","children":[{"title":"predict.py <span style='color:#111;'> 1.78KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 2.94KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 5.27KB </span>","children":null,"spread":false},{"title":"Seq2SeqModel.py <span style='color:#111;'> 3.07KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 685B </span>","children":null,"spread":false}],"spread":true},{"title":"bert","children":[{"title":"bert.py <span style='color:#111;'> 4.81KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 8.53KB </span>","children":null,"spread":false},{"title":"use_bert.py <span style='color:#111;'> 953B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"data","children":[{"title":"ijcnlp_dailydialog.zip <span style='color:#111;'> 4.27MB </span>","children":null,"spread":false},{"title":"eng-cmn","children":[{"title":"eng-chi.txt <span style='color:#111;'> 1.25MB </span>","children":null,"spread":false},{"title":"eng-cmn.txt <span style='color:#111;'> 1.25MB </span>","children":null,"spread":false},{"title":"_about.txt <span style='color:#111;'> 1.41KB </span>","children":null,"spread":false},{"title":"dataset <span style='color:#111;'> 2.85MB </span>","children":null,"spread":false},{"title":"zh_vocab <span style='color:#111;'> 125.73KB </span>","children":null,"spread":false},{"title":"eng_vocab <span style='color:#111;'> 86.73KB </span>","children":null,"spread":false},{"title":"pairs <span style='color:#111;'> 1.79MB </span>","children":null,"spread":false},{"title":"eng.lang <span style='color:#111;'> 218.11KB </span>","children":null,"spread":false},{"title":"cmn.lang <span style='color:#111;'> 357.98KB </span>","children":null,"spread":false}],"spread":true},{"title":"PersonaChat","children":[{"title":"4_turns","children":[{"title":"valid.txt <span style='color:#111;'> 2.61MB </span>","children":null,"spread":false},{"title":"test.txt <span style='color:#111;'> 2.48MB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 21.31MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"ijcnlp_dailydialog","children":[{"title":".DS_Store <span style='color:#111;'> 6.00KB </span>","children":null,"spread":false},{"title":"validation.zip <span style='color:#111;'> 186.79KB </span>","children":null,"spread":false},{"title":"train","children":[{"title":"dialogues_emotion_train.txt <span style='color:#111;'> 181.11KB </span>","children":null,"spread":false},{"title":"dialogues_act_train.txt <span style='color:#111;'> 181.11KB </span>","children":null,"spread":false},{"title":"dialogues_train.txt <span style='color:#111;'> 5.76MB </span>","children":null,"spread":false}],"spread":true},{"title":"dialogues_topic.txt <span style='color:#111;'> 26.09KB </span>","children":null,"spread":false},{"title":"dialogues_act.txt <span style='color:#111;'> 213.94KB </span>","children":null,"spread":false},{"title":"dialogues_emotion.txt <span style='color:#111;'> 213.94KB </span>","children":null,"spread":false},{"title":"pairs <span style='color:#111;'> 3.76MB </span>","children":null,"spread":false},{"title":"test.zip <span style='color:#111;'> 184.44KB </span>","children":null,"spread":false},{"title":"dialogues_text.txt <span style='color:#111;'> 6.81MB </span>","children":null,"spread":false},{"title":"train.zip <span style='color:#111;'> 1.85MB </span>","children":null,"spread":false},{"title":"test","children":[{"title":"dialogues_act_test.txt <span style='color:#111;'> 16.09KB </span>","children":null,"spread":false},{"title":"dialogues_test.txt <span style='color:#111;'> 531.45KB </span>","children":null,"spread":false},{"title":"dialogues_emotion_test.txt <span style='color:#111;'> 16.09KB </span>","children":null,"spread":false}],"spread":false},{"title":"dialog.lang <span style='color:#111;'> 500.61KB </span>","children":null,"spread":false},{"title":"readme.txt <span style='color:#111;'> 1.05KB </span>","children":null,"spread":false},{"title":"validation","children":[{"title":"dialogues_validation.txt <span style='color:#111;'> 544.95KB </span>","children":null,"spread":false},{"title":"dialogues_emotion_validation.txt <span style='color:#111;'> 16.74KB </span>","children":null,"spread":false},{"title":"dialogues_act_validation.txt <span style='color:#111;'> 16.74KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"DailyDialog","children":[{"title":"valid.txt <span style='color:#111;'> 1.12MB </span>","children":null,"spread":false},{"title":"test.txt <span style='color:#111;'> 1.13MB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 8.98MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"LICENSE <span style='color:#111;'> 1.50KB </span>","children":null,"spread":false},{"title":"zh_core_web_sm-3.2.0-py3-none-any.whl <span style='color:#111;'> 47.17MB </span>","children":null,"spread":false},{"title":"model","children":[{"title":"translation","children":[{"title":"trans_decoder.mdl <span style='color:#111;'> 4.79MB </span>","children":null,"spread":false},{"title":"trans_encoder.mdl <span style='color:#111;'> 1.98MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"requirements.txt <span style='color:#111;'> 154B </span>","children":null,"spread":false},{"title":"en_core_web_sm-3.2.0-py3-none-any.whl <span style='color:#111;'> 13.26MB </span>","children":null,"spread":false},{"title":"translation","children":[{"title":"transformer","children":[{"title":"train0.py <span style='color:#111;'> 5.27KB </span>","children":null,"spread":false},{"title":"readdata.py <span style='color:#111;'> 528B </span>","children":null,"spread":false},{"title":"predict.py <span style='color:#111;'> 2.42KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 3.89KB </span>","children":null,"spread":false},{"title":"transformer.py <span style='color:#111;'> 8.65KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 3.86KB </span>","children":null,"spread":false},{"title":"test.py <span style='color:#111;'> 1.88KB </span>","children":null,"spread":false},{"title":"attention.py <span style='color:#111;'> 5.91KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 710B </span>","children":null,"spread":false}],"spread":true},{"title":"Seq2Seq","children":[{"title":"predict.py <span style='color:#111;'> 1.64KB </span>","children":null,"spread":false},{"title":"predict1.py <span style='color:#111;'> 1.81KB </span>","children":null,"spread":false},{"title":"ACmodel.py <span style='color:#111;'> 3.22KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 3.44KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 5.39KB </span>","children":null,"spread":false},{"title":"train1.py <span style='color:#111;'> 5.44KB </span>","children":null,"spread":false},{"title":"Seq2SeqModel.py <span style='color:#111;'> 3.07KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 784B </span>","children":null,"spread":false}],"spread":true},{"title":"transformer1","children":[{"title":"train0.py <span style='color:#111;'> 5.22KB </span>","children":null,"spread":false},{"title":"predict.py <span style='color:#111;'> 1.64KB </span>","children":null,"spread":false},{"title":"predata.py <span style='color:#111;'> 255B </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 2.16KB </span>","children":null,"spread":false},{"title":"transformer.py <span style='color:#111;'> 3.53KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 6.61KB </span>","children":null,"spread":false},{"title":"Seq2SeqModel.py <span style='color:#111;'> 3.07KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 592B </span>","children":null,"spread":false}],"spread":true},{"title":"models","children":[{"title":"LSTM.py <span style='color:#111;'> 2.16KB </span>","children":null,"spread":false},{"title":"RNN.py <span style='color:#111;'> 3.11KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"img.png <span style='color:#111;'> 13.28KB </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 2.92KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}]
评论信息
其他资源
- 思迅美世家餐饮管理系统V4
- 微机原理课程设计及代码
- Multisim10实验电路图（26个）（原创）
- 故障分类svm代码
- 股票查询系统
- 2019全国最新景点POI数据.csv
- 基于Android的移动医疗终端系统
- 同时发起多次批量POST请求
- Polygon Mesh Processing
- VideoScribe2.3.4中文汉化可输入汉字
- c# 验证码识别 OCR图片识别 准确率非常高
- 2019年全国职业院校技能大赛“分布式光伏系统的装调与运维”任务书.pdf
- 微球测井仪屡烧检波板厚膜电路的问题解决
- 室内空气智能调控系统
- 基于字典学习与稀疏模型的SAR图像压缩技术研究_占新
- 《细说PHP(第2版)》
- 用VB写的语音识别的代码
- Flash+ as3.0一个拖动的拼图游戏
- 天猫评论爬虫
- arinc429 label270解析
- ABFramework文檔與基礎包源碼.rar
- 微型伺服马达原理与控制.doc
免责申明
【只为小站】的资源来自网友分享，仅供学习研究，请务必在下载后24小时内给予删除，不得用于其他任何用途，否则后果自负。基于互联网的特殊性，【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查；无论 【只为小站】 经营者是否已进行审查，用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场，基于网友分享，根据中国法律《信息网络传播权保护条例》第二十二条之规定，若资源存在侵权或相关问题请联系本站客服人员，zhiweidada#qq.com，请把#换成@，本站将给予最大的支持与配合，做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明
本站所有资源不代表本站的观点或立场，基于网友分享，根据中国法律《信息网络传播权保护条例》第二十二条之规定，若资源存在侵权或相关问题请联系本站客服人员，zhiweidada#qq.com，请把#换成@，本站将给予最大的支持与配合，做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明