NLP项目,主要包括atten-seq2seq和transformer模型,实现机器翻译以及对话系统

上传者: 63553972 | 上传时间: 2024-05-12 21:03:10 | 文件大小: 89.09MB | 文件类型: ZIP
使用说明 分对话系统和机器翻译两部分 data为数据集 model为训练的模型 translation文件夹下又分了Seq2Seq和transformer两个模型,大家按需查看使用 以transformer文件夹为例,attention.py主要实现了注意力机制,transformer.py实现了transformer的主体架构,data.py为数据的预处理以及生成了词典、dataset、dataloader,readdata.py运行可以查看数据形状,train.py为训练模型,predict.py为预测,config.py为一些参数的定义。 transformer机器翻译的模型是用cuda:1训练的,如果要使用可能需要修改代码 如:gpu->cpu,即在CPU上使用 torch.load('trans_encoder.mdl', map_location= lambda storage, loc: storage) torch.load('trans_decoder.mdl', map_location= lambda storage, loc: storage)

文件下载

资源详情

[{"title":"( 92 个子文件 89.09MB ) NLP项目,主要包括atten-seq2seq和transformer模型,实现机器翻译以及对话系统","children":[{"title":"nlp-master","children":[{"title":"dialogue","children":[{"title":"transformer","children":[{"title":"train (1).ipynb <span style='color:#111;'> 129.06KB </span>","children":null,"spread":false},{"title":"readdata.py <span style='color:#111;'> 498B </span>","children":null,"spread":false},{"title":"BPE.py <span style='color:#111;'> 1.65KB </span>","children":null,"spread":false},{"title":"predict.py <span style='color:#111;'> 2.14KB </span>","children":null,"spread":false},{"title":"train1.ipynb <span style='color:#111;'> 34.35KB </span>","children":null,"spread":false},{"title":"train.ipynb <span style='color:#111;'> 78.74KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 4.18KB </span>","children":null,"spread":false},{"title":"train-sampling.py <span style='color:#111;'> 6.79KB </span>","children":null,"spread":false},{"title":"transformer.py <span style='color:#111;'> 8.63KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 4.29KB </span>","children":null,"spread":false},{"title":"test.py <span style='color:#111;'> 4.84KB </span>","children":null,"spread":false},{"title":"attention.py <span style='color:#111;'> 5.91KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 1.01KB </span>","children":null,"spread":false}],"spread":false},{"title":"Seq2Seq","children":[{"title":"predict.py <span style='color:#111;'> 1.78KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 2.94KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 5.27KB </span>","children":null,"spread":false},{"title":"Seq2SeqModel.py <span style='color:#111;'> 3.07KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 685B </span>","children":null,"spread":false}],"spread":true},{"title":"bert","children":[{"title":"bert.py <span style='color:#111;'> 4.81KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 8.53KB </span>","children":null,"spread":false},{"title":"use_bert.py <span style='color:#111;'> 953B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"data","children":[{"title":"ijcnlp_dailydialog.zip <span style='color:#111;'> 4.27MB </span>","children":null,"spread":false},{"title":"eng-cmn","children":[{"title":"eng-chi.txt <span style='color:#111;'> 1.25MB </span>","children":null,"spread":false},{"title":"eng-cmn.txt <span style='color:#111;'> 1.25MB </span>","children":null,"spread":false},{"title":"_about.txt <span style='color:#111;'> 1.41KB </span>","children":null,"spread":false},{"title":"dataset <span style='color:#111;'> 2.85MB </span>","children":null,"spread":false},{"title":"zh_vocab <span style='color:#111;'> 125.73KB </span>","children":null,"spread":false},{"title":"eng_vocab <span style='color:#111;'> 86.73KB </span>","children":null,"spread":false},{"title":"pairs <span style='color:#111;'> 1.79MB </span>","children":null,"spread":false},{"title":"eng.lang <span style='color:#111;'> 218.11KB </span>","children":null,"spread":false},{"title":"cmn.lang <span style='color:#111;'> 357.98KB </span>","children":null,"spread":false}],"spread":true},{"title":"PersonaChat","children":[{"title":"4_turns","children":[{"title":"valid.txt <span style='color:#111;'> 2.61MB </span>","children":null,"spread":false},{"title":"test.txt <span style='color:#111;'> 2.48MB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 21.31MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"ijcnlp_dailydialog","children":[{"title":".DS_Store <span style='color:#111;'> 6.00KB </span>","children":null,"spread":false},{"title":"validation.zip <span style='color:#111;'> 186.79KB </span>","children":null,"spread":false},{"title":"train","children":[{"title":"dialogues_emotion_train.txt <span style='color:#111;'> 181.11KB </span>","children":null,"spread":false},{"title":"dialogues_act_train.txt <span style='color:#111;'> 181.11KB </span>","children":null,"spread":false},{"title":"dialogues_train.txt <span style='color:#111;'> 5.76MB </span>","children":null,"spread":false}],"spread":true},{"title":"dialogues_topic.txt <span style='color:#111;'> 26.09KB </span>","children":null,"spread":false},{"title":"dialogues_act.txt <span style='color:#111;'> 213.94KB </span>","children":null,"spread":false},{"title":"dialogues_emotion.txt <span style='color:#111;'> 213.94KB </span>","children":null,"spread":false},{"title":"pairs <span style='color:#111;'> 3.76MB </span>","children":null,"spread":false},{"title":"test.zip <span style='color:#111;'> 184.44KB </span>","children":null,"spread":false},{"title":"dialogues_text.txt <span style='color:#111;'> 6.81MB </span>","children":null,"spread":false},{"title":"train.zip <span style='color:#111;'> 1.85MB </span>","children":null,"spread":false},{"title":"test","children":[{"title":"dialogues_act_test.txt <span style='color:#111;'> 16.09KB </span>","children":null,"spread":false},{"title":"dialogues_test.txt <span style='color:#111;'> 531.45KB </span>","children":null,"spread":false},{"title":"dialogues_emotion_test.txt <span style='color:#111;'> 16.09KB </span>","children":null,"spread":false}],"spread":false},{"title":"dialog.lang <span style='color:#111;'> 500.61KB </span>","children":null,"spread":false},{"title":"readme.txt <span style='color:#111;'> 1.05KB </span>","children":null,"spread":false},{"title":"validation","children":[{"title":"dialogues_validation.txt <span style='color:#111;'> 544.95KB </span>","children":null,"spread":false},{"title":"dialogues_emotion_validation.txt <span style='color:#111;'> 16.74KB </span>","children":null,"spread":false},{"title":"dialogues_act_validation.txt <span style='color:#111;'> 16.74KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"DailyDialog","children":[{"title":"valid.txt <span style='color:#111;'> 1.12MB </span>","children":null,"spread":false},{"title":"test.txt <span style='color:#111;'> 1.13MB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 8.98MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"LICENSE <span style='color:#111;'> 1.50KB </span>","children":null,"spread":false},{"title":"zh_core_web_sm-3.2.0-py3-none-any.whl <span style='color:#111;'> 47.17MB </span>","children":null,"spread":false},{"title":"model","children":[{"title":"translation","children":[{"title":"trans_decoder.mdl <span style='color:#111;'> 4.79MB </span>","children":null,"spread":false},{"title":"trans_encoder.mdl <span style='color:#111;'> 1.98MB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"requirements.txt <span style='color:#111;'> 154B </span>","children":null,"spread":false},{"title":"en_core_web_sm-3.2.0-py3-none-any.whl <span style='color:#111;'> 13.26MB </span>","children":null,"spread":false},{"title":"translation","children":[{"title":"transformer","children":[{"title":"train0.py <span style='color:#111;'> 5.27KB </span>","children":null,"spread":false},{"title":"readdata.py <span style='color:#111;'> 528B </span>","children":null,"spread":false},{"title":"predict.py <span style='color:#111;'> 2.42KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 3.89KB </span>","children":null,"spread":false},{"title":"transformer.py <span style='color:#111;'> 8.65KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 3.86KB </span>","children":null,"spread":false},{"title":"test.py <span style='color:#111;'> 1.88KB </span>","children":null,"spread":false},{"title":"attention.py <span style='color:#111;'> 5.91KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 710B </span>","children":null,"spread":false}],"spread":true},{"title":"Seq2Seq","children":[{"title":"predict.py <span style='color:#111;'> 1.64KB </span>","children":null,"spread":false},{"title":"predict1.py <span style='color:#111;'> 1.81KB </span>","children":null,"spread":false},{"title":"ACmodel.py <span style='color:#111;'> 3.22KB </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 3.44KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 5.39KB </span>","children":null,"spread":false},{"title":"train1.py <span style='color:#111;'> 5.44KB </span>","children":null,"spread":false},{"title":"Seq2SeqModel.py <span style='color:#111;'> 3.07KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 784B </span>","children":null,"spread":false}],"spread":true},{"title":"transformer1","children":[{"title":"train0.py <span style='color:#111;'> 5.22KB </span>","children":null,"spread":false},{"title":"predict.py <span style='color:#111;'> 1.64KB </span>","children":null,"spread":false},{"title":"predata.py <span style='color:#111;'> 255B </span>","children":null,"spread":false},{"title":"data.py <span style='color:#111;'> 2.16KB </span>","children":null,"spread":false},{"title":"transformer.py <span style='color:#111;'> 3.53KB </span>","children":null,"spread":false},{"title":"train.py <span style='color:#111;'> 6.61KB </span>","children":null,"spread":false},{"title":"Seq2SeqModel.py <span style='color:#111;'> 3.07KB </span>","children":null,"spread":false},{"title":"config.py <span style='color:#111;'> 592B </span>","children":null,"spread":false}],"spread":true},{"title":"models","children":[{"title":"LSTM.py <span style='color:#111;'> 2.16KB </span>","children":null,"spread":false},{"title":"RNN.py <span style='color:#111;'> 3.11KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"img.png <span style='color:#111;'> 13.28KB </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 2.92KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明