longformer:加长型

上传者: 42156940 | 上传时间: 2022-09-20 14:06:52 | 文件大小: 591KB | 文件类型: ZIP
Longformer Longformer和LongformerEncoderDecoder LongformerEncoderDecoder (LED)是用于长文档的预训练变压器模型。 ***** 2020年12月1日新版:LongformerEncoderDecoder ***** LongformerEncoderDecoder (LED)模型现在可用。它支持长输入的seq2seq任务。使用渐变检查点,fp16和48GB gpu,输入长度最多可达到16K令牌。检查更新的纸张以获取模型的详细信息和评估。 训练有素的模型:1) 16384,2) 要求:确保使用的huggingface /变压器在叉指定requirements.txt 。它增加了对梯度检查点的支持,并允许输入和输出具有不同的最大序列长度。您还可以运行pip install git+https://github.c

文件下载

资源详情

[{"title":"( 53 个子文件 591KB ) longformer:加长型","children":[{"title":"longformer-master","children":[{"title":"longformer","children":[{"title":"diagonaled_mm_tvm.py <span style='color:#111;'> 17.03KB </span>","children":null,"spread":false},{"title":"longformer_encoder_decoder.py <span style='color:#111;'> 3.39KB </span>","children":null,"spread":false},{"title":"longformer.py <span style='color:#111;'> 16.08KB </span>","children":null,"spread":false},{"title":"sliding_chunks.py <span style='color:#111;'> 8.04KB </span>","children":null,"spread":false},{"title":"__init__.py <span style='color:#111;'> 265B </span>","children":null,"spread":false},{"title":"lib","children":[{"title":"lib_diagonaled_mm_float32_cuda.so <span style='color:#111;'> 40.79KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"tvm","children":[{"title":"ndarray.py <span style='color:#111;'> 4.80KB </span>","children":null,"spread":false},{"title":"_ffi","children":[{"title":"ndarray.py <span style='color:#111;'> 11.42KB </span>","children":null,"spread":false},{"title":"runtime_ctypes.py <span style='color:#111;'> 8.46KB </span>","children":null,"spread":false},{"title":"libinfo.py <span style='color:#111;'> 7.38KB </span>","children":null,"spread":false},{"title":"_ctypes","children":[{"title":"ndarray.py <span style='color:#111;'> 4.66KB </span>","children":null,"spread":false},{"title":"types.py <span style='color:#111;'> 3.33KB </span>","children":null,"spread":false},{"title":"__init__.py <span style='color:#111;'> 829B </span>","children":null,"spread":false},{"title":"node.py <span style='color:#111;'> 3.48KB </span>","children":null,"spread":false},{"title":"vmobj.py <span style='color:#111;'> 1.60KB </span>","children":null,"spread":false},{"title":"function.py <span style='color:#111;'> 9.97KB </span>","children":null,"spread":false}],"spread":true},{"title":"__init__.py <span style='color:#111;'> 1.11KB </span>","children":null,"spread":false},{"title":"node.py <span style='color:#111;'> 3.68KB </span>","children":null,"spread":false},{"title":"function.py <span style='color:#111;'> 9.23KB </span>","children":null,"spread":false},{"title":"node_generic.py <span style='color:#111;'> 3.59KB </span>","children":null,"spread":false},{"title":"base.py <span style='color:#111;'> 7.97KB </span>","children":null,"spread":false}],"spread":true},{"title":"module.py <span style='color:#111;'> 9.46KB </span>","children":null,"spread":false},{"title":"__init__.py <span style='color:#111;'> 895B </span>","children":null,"spread":false},{"title":"libtvm_runtime.so <span style='color:#111;'> 1.31MB </span>","children":null,"spread":false},{"title":"contrib","children":[{"title":"dlpack.py <span style='color:#111;'> 1.99KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"tests","children":[{"title":"test_readme.py <span style='color:#111;'> 2.51KB </span>","children":null,"spread":false},{"title":"test_integration.py <span style='color:#111;'> 2.14KB </span>","children":null,"spread":false},{"title":"test_sliding_chunks.py <span style='color:#111;'> 3.35KB </span>","children":null,"spread":false},{"title":"test_var_global_attn.py <span style='color:#111;'> 2.76KB </span>","children":null,"spread":false}],"spread":true},{"title":"LICENSE <span style='color:#111;'> 11.09KB </span>","children":null,"spread":false},{"title":"requirements.txt <span style='color:#111;'> 274B </span>","children":null,"spread":false},{"title":"setup.py <span style='color:#111;'> 394B </span>","children":null,"spread":false},{"title":".gitignore <span style='color:#111;'> 1.73KB </span>","children":null,"spread":false},{"title":"tvm_docker <span style='color:#111;'> 1.34KB </span>","children":null,"spread":false},{"title":"longformer_on_beaker.sh <span style='color:#111;'> 361B </span>","children":null,"spread":false},{"title":"experiment.yml <span style='color:#111;'> 573B </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 8.05KB </span>","children":null,"spread":false},{"title":"scripts","children":[{"title":"hp-splits.json <span style='color:#111;'> 3.08KB </span>","children":null,"spread":false},{"title":"convert_bart_to_longformerencoderdecoder.py <span style='color:#111;'> 6.26KB </span>","children":null,"spread":false},{"title":"summarization.py <span style='color:#111;'> 17.26KB </span>","children":null,"spread":false},{"title":"convert_model_to_long.ipynb <span style='color:#111;'> 29.81KB </span>","children":null,"spread":false},{"title":"mem_profiler.py <span style='color:#111;'> 2.45KB </span>","children":null,"spread":false},{"title":"triviaqa.py <span style='color:#111;'> 42.92KB </span>","children":null,"spread":false},{"title":"test_tpu.py <span style='color:#111;'> 1.25KB </span>","children":null,"spread":false},{"title":"__init__.py <span style='color:#111;'> 2B </span>","children":null,"spread":false},{"title":"hp_preprocess.py <span style='color:#111;'> 3.14KB </span>","children":null,"spread":false},{"title":"cheatsheet.txt <span style='color:#111;'> 4.99KB </span>","children":null,"spread":false},{"title":"triviaqa_utils","children":[{"title":"dataset_utils.py <span style='color:#111;'> 2.02KB </span>","children":null,"spread":false},{"title":"convert_to_squad_format.py <span style='color:#111;'> 4.43KB </span>","children":null,"spread":false},{"title":"evaluation_utils.py <span style='color:#111;'> 5.58KB </span>","children":null,"spread":false},{"title":"__init__.py <span style='color:#111;'> 0B </span>","children":null,"spread":false},{"title":"file_utils.py <span style='color:#111;'> 799B </span>","children":null,"spread":false}],"spread":false},{"title":"pretrain.py <span style='color:#111;'> 21.16KB </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明