文章目录前言Qwen官网第一步新建虚拟环境第二步设置加速源第三步安装基础环境创建Qwen文件夹并新建requirements.txt把下面内容复制到里面。安装需要的包第四步离线下载flash-attention第五步下载模型权重近期用huggface镜像下载会出现不稳定的情况请改用modelscope下载权重第六步运行代码修改对应模型权重路径flash-attn下载存在问题的读者请直接跳转并阅读第四步离线下载flash-attention。前言由于LLM参数量巨大、且国内网络不佳在服务器部署Qwen系列模型会存在缓存模型权重难等问题。依照Qwen官方的README部署模型会存在问题比如模型下载过程中进度卡死、flash-attn相关库难以安装等问题。因为为了解决上述问题将整理一个教程快速部署Qwen系列模型。Qwen官网https://github.com/QwenLM/Qwen2.5-VL第一步新建虚拟环境conda create-nQwenpython3.10.16 conda activate Qwen第二步设置加速源pip configsetglobal.index-url https://mirrors.aliyun.com/pypi/simple/第三步安装基础环境创建Qwen文件夹并新建requirements.txt把下面内容复制到里面。requirements.txt中的内容如下复制下面内容到txt文件中torch2.6.0torchvision0.21.0transformers4.51.3 accelerate qwen-vl-utils[decord]安装需要的包pipinstall-rrequirements.txt第四步离线下载flash-attention下载地址https://github.com/Dao-AILab/flash-attention/releases/找到你对应的版本flash_attn-2.74.postcuda我是安装的12.0其它的步骤一样就可以选择下面这个包记得选择False这个版本。选择自己对应的离线版本即可。下载版本flash_attn-2.7.4.post1cu12torch2.6cxx11abiFALSE-cp310-cp310-linux_x86_64.whlCSDN下载https://download.csdn.net/download/weixin_43312117/91177518?spm1001.2014.3001.5501把下载的离线文件上传到服务器当前用户的Download文件夹中然后安装该文件pipinstallflash_attn-2.7.4.post1cu12torch2.6cxx11abiFALSE-cp310-cp310-linux_x86_64.whl第五步下载模型权重近期用huggface镜像下载会出现不稳定的情况请改用modelscope下载权重# 用魔塔社区下载pipinstallmodelscope# 下载时指定路径加载模型的时候要指向到该路径modelscope download--modelQwen/Qwen2.5-VL-3B-Instruct--local_dir./Qwen2.5-VL-3B-Instruct下载完成第六步运行代码修改对应模型权重路径from transformersimportQwen2_5_VLForConditionalGeneration, AutoProcessor from qwen_vl_utilsimportprocess_vision_info# default: Load the model on the available device(s)modelQwen2_5_VLForConditionalGeneration.from_pretrained(./Qwen2.5-VL-3B-Instruct,torch_dtypeauto,device_mapauto)# We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.# model Qwen2_5_VLForConditionalGeneration.from_pretrained(# Qwen/Qwen2.5-VL-7B-Instruct,# torch_dtypetorch.bfloat16,# attn_implementationflash_attention_2,# device_mapauto,# )# default processorprocessorAutoProcessor.from_pretrained(./Qwen2.5-VL-3B-Instruct)# The default range for the number of visual tokens per image in the model is 4-16384.# You can set min_pixels and max_pixels according to your needs, such as a token range of 256-1280, to balance performance and cost.# min_pixels 256*28*28# max_pixels 1280*28*28# processor AutoProcessor.from_pretrained(Qwen/Qwen2.5-VL-7B-Instruct, min_pixelsmin_pixels, max_pixelsmax_pixels)messages[{role:user,content:[{type:image,image:https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg,},{type:text,text:Describe this image.},],}]# Preparation for inferencetextprocessor.apply_chat_template(messages,tokenizeFalse,add_generation_promptTrue)image_inputs, video_inputsprocess_vision_info(messages)inputsprocessor(text[text],imagesimage_inputs,videosvideo_inputs,paddingTrue,return_tensorspt,)inputsinputs.to(model.device)# Inference: Generation of the outputgenerated_idsmodel.generate(**inputs,max_new_tokens128)generated_ids_trimmed[out_ids[len(in_ids):]forin_ids, out_idsinzip(inputs.input_ids, generated_ids)]output_textprocessor.batch_decode(generated_ids_trimmed,skip_special_tokensTrue,clean_up_tokenization_spacesFalse)print(output_text)轻松运行得到结果如何和我设置一样就直接运行不一致就把加载模型路径切换到你下载权重的地方。