Qwen-7B部署教程Python调用+web端部署

见贤思齐

前提操作系统为Ubuntu22.04.4LTS安装Anaconda（本人安装教程如下）Ubuntu22.04.4LTS系统/安装Anaconda【GPU版】-CSDN博客安装python3.9/pytorch/torchvision（本人安装教程如下）Ubuntu22.04.4系统/安装python3.9/pytorch/torchvision【GPU版】-CSDN博客1、conda虚拟环境condaactivateQwenChat2、下载Qwen代码gitclonehttps://github.com/QwenLM/Qwen.git3、安装依赖环境cdQwenpipinstall-rrequirements.txt-ihttps://pypi.tuna.tsinghua.edu.cn/simplepipinstall-rrequirements_web_demo.txt-ihttps://pypi.tuna.tsinghua.edu.cn/simple4、安装魔搭pipinstallmodelscopetransformers-ihttps://pypi.tuna.tsinghua.edu.cn/simple5、测试模型5.1Python调用5.1.1创建chat-7b.pytouchchat-7b.py5.1.2编辑chat-7b.pyvichat-7b.pychat-7b.py文件内容如下frommodelscopeimportAutoModelForCausalLM,AutoTokenizerfrommodelscopeimportGenerationConfig#可选的模型包括:"qwen/Qwen-7B-Chat","qwen/Qwen-14B-Chat"tokenizer=AutoTokenizer.from_pretrained("qwen/Qwen-7B-Chat",trust_remote_code=True)model=AutoModelForCausalLM.from_pretrained("qwen/Qwen-7B-Chat",device_map="auto",trust_remote_code=True,fp16=True).eval()model.generation_config=GenerationConfig.from_pretrained("Qwen/Qwen-7B-Chat",trust_remote_code=True)#可指定不同的生成长度、top_p等相关超参response,history=model.chat(tokenizer,"你好",history=None)print(response)response,history=model.chat(tokenizer,"上海好玩吗？",history=history)print(response)response,history=model.chat(tokenizer,"七月份去上海，有推荐的旅游攻略吗？",history=history)print(response)5.1.3运行chat-7b.pypythonchat-7b.py5.1.4 测试成功5.2web端部署5.2.1 编辑web_demo.pyviweb_demo.py修改内容如下，将transformers修改为modelscope...importgradioasgrimportmdtex2htmlimporttorch#fromtransformersimportAutoModelForCausalLM,AutoTokenizer#fromtransformers.generationimportGenerationConfigfrommodelscopeimportAutoModelForCausalLM,AutoTokenizer,GenerationConfigDEFAULT_CKPT_PATH='qwen/Qwen-7B-Chat-Int4'...5.2.2其他安装需求pipinstallauto-gptqpipinstalloptimumpipinstall--upgradegradio5.2.3运行web_demo.pypythonweb_demo.py5.2.4 测试成功本人安装浏览器的教程，自取链接如下Ubuntu下载安装chrome浏览器-CSDN博客参考链接【Ubuntu20.04部署通义千问Qwen-7B，实测成功】_ubuntu上部署qwen-CSDN博客^v100^pc_search_result_base2&spm=1018.2226.3001.4187