大模型部署手记(15)LLaMa2+Ubuntu18.04+privateGPT

举报
张辉 发表于 2023/10/15 16:14:29 2023/10/15
【摘要】 大模型部署手记(15)LLaMa2+Ubuntu18.04+privateGPT

1.简介:

组织机构:Meta(Facebook)

代码仓:https://github.com/facebookresearch/llama  https://github.com/ymcui/Chinese-LLaMA-Alpaca-2

模型:llama-2-7b sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

下载:使用download.sh下载或Huggingface.co或aliendao.cn下载

硬件环境:暗影精灵7Plus
Ubuntu版本:18.04
内存 32G
GPU显卡:Nvidia GTX 3080 Laptop (16G)


2.代码和模型下载

参考 https://bbs.huaweicloud.com/blogs/412406 下载 llama-2-7b模型。

将下载好的模型文件拷贝到 ~/privateGPT/models 目录下:




cd ~
git clone https://github.com/imartinez/privateGPT

3.安装依赖

conda create -n private310 python=3.10 -y

conda activate private310

cd privateGPT

先安装torch支持CUDA11.8版本:
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia


pip3 install -r requirements.txt



安装Poetry:
curl -sSL https://install.python-poetry.org | python3 -

cd ~/privateGPT
poetry install



照此执行:
export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
poetry install


poetry shell

感觉它是要重新进入conda环境
conda activate private310

4.部署验证

ls source_documents

加载模型试试:

python
from llama_cpp import Llama
llm = Llama(model_path="./models/7B/ggml-model-q4_0.gguf")


开始解析文本:
python ingest.py

pip install sentence_transformers


再来:
python ingest.py


它应该是想从Huggingface.co下载 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 这个模型,显然这个是不行的。



想办法去下载吧!


将其下载到windows,再上传到ubuntu的 ~/privateGPT/models目录:



修改.env:
EMBEDDINGS_MODEL_NAME=./models/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
python ingest.py

mkdir -p ./models/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/1_Pooling
cd ./models/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/
cp * 1_Pooling/
cd ~/privateGPT
python ingest.py

(private310) zhanghui@zhanghui-OMEN-by-HP-Laptop-17-ck0xxx:~/privateGPT$ python ingest.py
Traceback (most recent call last):
  File "/home/zhanghui/privateGPT/ingest.py", line 185, in <module>
    main()
  File "/home/zhanghui/privateGPT/ingest.py", line 155, in main
    embeddings = HuggingFaceEmbeddings(model_name=embeddings_model_name)
  File "/home/zhanghui/anaconda3/envs/private310/lib/python3.10/site-packages/langchain/embeddings/huggingface.py", line 64, in __init__
    self.client = sentence_transformers.SentenceTransformer(
  File "/home/zhanghui/anaconda3/envs/private310/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 95, in __init__
    modules = self._load_sbert_model(model_path)
  File "/home/zhanghui/anaconda3/envs/private310/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 840, in _load_sbert_model
    module = module_class.load(os.path.join(model_path, module_config['path']))
  File "/home/zhanghui/anaconda3/envs/private310/lib/python3.10/site-packages/sentence_transformers/models/Pooling.py", line 120, in load
    return Pooling(**config)
TypeError: Pooling.__init__() got an unexpected keyword argument '_name_or_path'

在python命令行试试:
python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('./models/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')

仔细看看,果然有个1_Pooling目录,下载config.json
{
  "word_embedding_dimension": 384,
  "pooling_mode_cls_token": false,
  "pooling_mode_mean_tokens": true,
  "pooling_mode_max_tokens": false,
  "pooling_mode_mean_sqrt_len_tokens": false
}

cd ./models/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/1_Pooling
rm -f *
vi config.json 将上述文本复制进去。
cd ~/privateGPT
再来:
python ingest.py
分析本地文件成功了!
此时会生成一个db目录:
执行推理:
python privateGPT.py
who is Intel’s CEO?
修改privateGPT.py
将:
llm = LlamaCpp(model_path=model_path, max_tokens=model_n_ctx, n_batch=model_n_batch, callbacks=callbacks, verbose=False)
改为:
llm = LlamaCpp(model_path=model_path, max_tokens=model_n_ctx, n_batch=model_n_batch, callbacks=callbacks, verbose=False, n_ctx=model_n_ctx)
执行:
python privateGPT.py
看看 分析前的原文:state_of_the_union.txt
再问一个问题:
How much is their investment?
这段话回答得有点怪。。
最后看下执行的全文:
(private310) zhanghui@zhanghui-OMEN-by-HP-Laptop-17-ck0xxx:~/privateGPT$ python privateGPT.py

Enter a query: who is Intel’s CEO?
 Pat Gelsinger (in the middle of the picture)
\end{blockquote}

In short, I guess I'm trying to find out if the quote above can be considered validated by the context in which it appears.

Comment: "Intel’s CEO, Pat Gelsinger"

Answer: It is a rhetorical question.  The answer is that he does not know.  If he knew who Intel's CEO was, then why would he say "I don't know", unless he thought it was someone else?

> Question:
who is Intel’s CEO?

> Answer (took 41.18 s.):
 Pat Gelsinger (in the middle of the picture)
\end{blockquote}

In short, I guess I'm trying to find out if the quote above can be considered validated by the context in which it appears.

Comment: "Intel’s CEO, Pat Gelsinger"

Answer: It is a rhetorical question.  The answer is that he does not know.  If he knew who Intel's CEO was, then why would he say "I don't know", unless he thought it was someone else?

> source_documents/state_of_the_union.txt:
That would be one of the biggest investments in manufacturing in American history.

And all they’re waiting for is for you to pass this bill.

So let’s not wait any longer. Send it to my desk. I’ll sign it.

And we will really take off.

And Intel is not alone.

There’s something happening in America.

Just look around and you’ll see an amazing story.

The rebirth of the pride that comes from stamping products “Made In America.” The revitalization of American manufacturing.

> source_documents/state_of_the_union.txt:
Up to eight state-of-the-art factories in one place. 10,000 new good-paying jobs.

Some of the most sophisticated manufacturing in the world to make computer chips the size of a fingertip that power the world and our everyday lives.

Smartphones. The Internet. Technology we have yet to invent.

But that’s just the beginning.

Intel’s CEO, Pat Gelsinger, who is here tonight, told me they are ready to increase their investment from
$20 billion to $100 billion.

> source_documents/state_of_the_union.txt:
Let me give you one example of why it’s so important to pass it.

If you travel 20 miles east of Columbus, Ohio, you’ll find 1,000 empty acres of land.

It won’t look like much, but if you stop and look closely, you’ll see a “Field of dreams,” the ground on which America’s future will be built.

This is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor “mega site”.

> source_documents/state_of_the_union.txt:
But in my administration, the watchdogs have been welcomed back.

We’re going after the criminals who stole billions in relief money meant for small businesses and millions of Americans.

And tonight, I’m announcing that the Justice Department will name a chief prosecutor for pandemic fraud.

By the end of this year, the deficit will be down to less than half what it was before I took office.

The only president ever to cut the deficit by more than one trillion dollars in a single year.

Enter a query: How much is their investment?


Answer:

It’s not the kind of thing you can see right now or measure with a ruler.

We are restoring our economy from scratch, which was about to collapse under Obama.

With the help of the American people, we have already turned around this economy.

These are jobs that didn’t even exist 30 years ago. We are making history again!

We’ve begun a new industrial revolution – one that is built on innovation and technology, but most importantly, it is American made.

This is the next chapter in our story:

An economy that works for everyone. An America that truly is great again.

> Question:
How much is their investment?

> Answer (took 41.08 s.):


Answer:

It’s not the kind of thing you can see right now or measure with a ruler.

We are restoring our economy from scratch, which was about to collapse under Obama.

With the help of the American people, we have already turned around this economy.

These are jobs that didn’t even exist 30 years ago. We are making history again!

We’ve begun a new industrial revolution – one that is built on innovation and technology, but most importantly, it is American made.

This is the next chapter in our story:

An economy that works for everyone. An America that truly is great again.

> source_documents/state_of_the_union.txt:
During the pandemic, these foreign-owned companies raised prices by as much as 1,000% and made record profits.

Tonight, I’m announcing a crackdown on these companies overcharging American businesses and consumers.

And as Wall Street firms take over more nursing homes, quality in those homes has gone down and costs have gone up.

That ends on my watch.

Medicare is going to set higher standards for nursing homes and make sure your loved ones get the care they deserve and expect.

> source_documents/state_of_the_union.txt:
Up to eight state-of-the-art factories in one place. 10,000 new good-paying jobs.

Some of the most sophisticated manufacturing in the world to make computer chips the size of a fingertip that power the world and our everyday lives.

Smartphones. The Internet. Technology we have yet to invent.

But that’s just the beginning.

Intel’s CEO, Pat Gelsinger, who is here tonight, told me they are ready to increase their investment from
$20 billion to $100 billion.

> source_documents/state_of_the_union.txt:
When we use taxpayer dollars to rebuild America – we are going to Buy American: buy American products to support American jobs.

The federal government spends about $600 Billion a year to keep the country safe and secure.

There’s been a law on the books for almost a century
to make sure taxpayers’ dollars support American jobs and businesses.

Every Administration says they’ll do it, but we are actually doing it.

> source_documents/state_of_the_union.txt:
Companies are choosing to build new factories here, when just a few years ago, they would have built them overseas.

That’s what is happening. Ford is investing $11 billion to build electric vehicles, creating 11,000 jobs across the country.

GM is making the largest investment in its history—$7 billion to build electric vehicles, creating 4,000 jobs in Michigan.

All told, we created 369,000 new manufacturing jobs in America just last year.

Enter a query:
英文推理之后,中文怎么搞,下次再说。
【版权声明】本文为华为云社区用户原创内容,转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息, 否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@huaweicloud.com
  • 点赞
  • 收藏
  • 关注作者

评论(0

0/1000
抱歉,系统识别当前为高风险访问,暂不支持该操作

全部回复

上滑加载中

设置昵称

在此一键设置昵称,即可参与社区互动!

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。