大模型微调，插件包：deepspeed-0.11.2+cuda121-cp310-cp310-win_amd64.whl
2024-05-30 12:09:43
25.71MB
1
通过wheel安装deepspeed,其中cp39为python 3.9 cp310为python 3.10
2024-04-09 16:17:41
32.84MB
1
1.克隆源码 https://github.com/microsoft/DeepSpeed.git
2.在win10系统22H2版本上安装Visual Studio 2019，在支持GTX1060显卡的NVIDIA Studio Driver462.31驱动上安装cuda11.6+cudnn8.7，进入在python3.7的虚拟环境中，安装依赖pytorch1.12.0
3.更改源码
3.1.setyup.py增加环境变量
os.environ['DS_BUILD_AIO']='0'
os.environ['DS_BUILD_SPARSE_ATTN']='0'
3.2.csrc/includes/memory_access_utils.h增加头文件
#include
4.执行编译命令 python setup.py bdist_wheel ，大约20分钟后在dist目录生成deepspeed-0.8.2+81b4d5db-cp37-cp37m-win_amd64.whl文件
2023-04-10 22:31:28
15.97MB
1
DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters
2021-10-25 18:13:17
774KB
1