bert_distill:BERT蒸馏(基于BERT的蒸馏实验)-源码

上传者: 42146086 | 上传时间: 2021-06-10 13:46:28 | 文件大小: 30.23MB | 文件类型: ZIP
基于BERT的蒸馏实验 参考论文《从BERT提取任务特定的知识到简单神经网络》 分别采用keras和pytorch基于textcnn和bilstm(gru)进行了实验 实验数据分割成1(有标签训练):8(无标签训练):1(测试) 在情感2分类服装的数据集上初步结果如下: 小模型(textcnn&bilstm)准确率在0.80〜0.81 BERT模型准确率在0.90〜0.91 蒸馏模型准确率在0.87〜0.88 实验结果与论文某些基本一致,与预期相符 后续将尝试其他更有效的蒸馏方案 使用方法 首先finetune BERT python ptbert.py 然后把BERT的知识蒸馏到小模型里

文件下载

资源详情

[{"title":"( 31 个子文件 30.23MB ) bert_distill:BERT蒸馏(基于BERT的蒸馏实验)-源码","children":[{"title":"bert_distill-master","children":[{"title":"utils.py <span style='color:#111;'> 5.43KB </span>","children":null,"spread":false},{"title":"readme.md <span style='color:#111;'> 926B </span>","children":null,"spread":false},{"title":"data","children":[{"title":"pda","children":[{"title":"test.txt <span style='color:#111;'> 121.74KB </span>","children":null,"spread":false},{"title":"pda.txt <span style='color:#111;'> 1.23MB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 126.22KB </span>","children":null,"spread":false},{"title":"dev.txt <span style='color:#111;'> 1008.36KB </span>","children":null,"spread":false}],"spread":true},{"title":"hotel","children":[{"title":"hotel.txt <span style='color:#111;'> 3.03MB </span>","children":null,"spread":false},{"title":"test.txt <span style='color:#111;'> 304.68KB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 298.92KB </span>","children":null,"spread":false},{"title":"dev.txt <span style='color:#111;'> 2.44MB </span>","children":null,"spread":false}],"spread":true},{"title":"cache","children":[{"title":"word2vec.gz <span style='color:#111;'> 24.28MB </span>","children":null,"spread":false},{"title":"t_de <span style='color:#111;'> 62.66KB </span>","children":null,"spread":false},{"title":"t_tr <span style='color:#111;'> 7.96KB </span>","children":null,"spread":false}],"spread":true},{"title":"shampoo","children":[{"title":"test.txt <span style='color:#111;'> 112.10KB </span>","children":null,"spread":false},{"title":"shampoo.txt <span style='color:#111;'> 1.10MB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 110.65KB </span>","children":null,"spread":false},{"title":"dev.txt <span style='color:#111;'> 898.70KB </span>","children":null,"spread":false}],"spread":true},{"title":"fruit","children":[{"title":"fruit.txt <span style='color:#111;'> 1.08MB </span>","children":null,"spread":false},{"title":"test.txt <span style='color:#111;'> 112.08KB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 113.82KB </span>","children":null,"spread":false},{"title":"dev.txt <span style='color:#111;'> 877.01KB </span>","children":null,"spread":false}],"spread":true},{"title":"clothing","children":[{"title":"test.txt <span style='color:#111;'> 88.67KB </span>","children":null,"spread":false},{"title":"train.txt <span style='color:#111;'> 86.32KB </span>","children":null,"spread":false},{"title":"dev.txt <span style='color:#111;'> 729.53KB </span>","children":null,"spread":false},{"title":"clothing.txt <span style='color:#111;'> 904.52KB </span>","children":null,"spread":false}],"spread":true},{"title":"split.py <span style='color:#111;'> 528B </span>","children":null,"spread":false}],"spread":true},{"title":"test.py <span style='color:#111;'> 3.52KB </span>","children":null,"spread":false},{"title":"small.py <span style='color:#111;'> 4.27KB </span>","children":null,"spread":false},{"title":"distill.py <span style='color:#111;'> 4.42KB </span>","children":null,"spread":false},{"title":"__init__.py <span style='color:#111;'> 0B </span>","children":null,"spread":false},{"title":"ptbert.py <span style='color:#111;'> 7.99KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明