spaCy 是由 Explosion AI 开发的工业级自然语言处理库，设计理念是"开箱即用、极快速度"。它提供多语言预训练模型，一条 Pipeline 完成分词、词性标注、命名实体识别、依存句法分析等全套 NLP 任务。
核心价值：
nlp = spacy.load("zh_core_web_sm") 即可处理中文spacy-transformers 接入 BERT 提升精度适用场景： 文本预处理、信息抽取、知识图谱构建、合规审查、简历解析。
pip install spacy
# 英文（小/中/大/Transformer）
python -m spacy download en_core_web_sm # 12 MB，适合入门
python -m spacy download en_core_web_md # 40 MB，含词向量
python -m spacy download en_core_web_trf # 460 MB，BERT 精度
# 中文
python -m spacy download zh_core_web_sm # 中文小模型
python -m spacy download zh_core_web_trf # 中文 Transformer
# 日文/德文/法文等
python -m spacy download ja_core_news_sm
python -m spacy download de_core_news_sm
python -m spacy download fr_core_news_sm
pip install spacy-transformers
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("Apple is looking at buying U.K. startup for $1 billion.")
for ent in doc.ents:
print(ent.text, ent.label_)
Can't find model 'en_core_web_sm'未下载模型。运行
python -m spacy download en_core_web_sm。如网络问题，从 https://github.com/explosion/spacy-models/releases 下载.whl手动安装。
zh_core_web_sm 分词不准小模型基于规则分词，精度有限。换用
zh_core_web_trf（BERT 级分词）或 jieba 预处理。
OSError: [E050] Can't find model虚拟环境切换后模型不可见。重新在目标环境
python -m spacy download ...
用 spaCy 一条 Pipeline 完成：分词、词性标注、命名实体识别 (NER)、依存句法分析，并用 displaCy 可视化。
import spacy
from spacy import displacy
# ─── 1. 加载模型 ───
nlp = spacy.load("en_core_web_sm")
# ─── 2. 处理文本 ───
text = "Elon Musk announced that Tesla will build a new factory in Shanghai next year, investing $2 billion."
doc = nlp(text)
# ─── 3. 分词 + 词性 + 依存分析 ───
print("=" * 70)
print(f"{'Token':<12} {'POS':<10} {'依存关系':<16} {'Head':<12}")
print("=" * 70)
for token in doc:
print(f"{token.text:<12} {token.pos_:<10} {token.dep_:<16} {token.head.text:<12}")
# ─── 4. 命名实体识别 ───
print("\n" + "=" * 40)
print("命名实体 (NER):")
print("=" * 40)
for ent in doc.ents:
print(f" {ent.text:<25} | {ent.label_:<10} | {spacy.explain(ent.label_)}")
# ─── 5. 名词短语 ───
print("\n名词短语:")
for chunk in doc.noun_chunks:
print(f" → {chunk.text}")
# ─── 6. 依存句法可视化 ───
displacy.render(doc, style="dep", jupyter=False, options={"compact": True})
# 或保存到文件:
# displacy.serve(doc, style="dep") # 启动 Web 服务器
# ─── 7. NER 可视化 ───
displacy.render(doc, style="ent", jupyter=False)
# displacy.serve(doc, style="ent")
pip install spacy
python -m spacy download en_core_web_sm
python spacy_demo.py
======================================================================
Token POS 依存关系 Head
======================================================================
Elon PROPN compound Musk
Musk PROPN nsubj announced
announced VERB ROOT announced
that SCONJ mark build
...
Shanghai PROPN pobj in
...
$ SYM quantmod billion
2 NUM compound billion
billion NUM pobj of
. PUNCT punct announced
============================================
命名实体 (NER):
============================================
Elon Musk | PERSON | People, including fictional
Tesla | ORG | Companies, agencies
Shanghai | GPE | Countries, cities, states
next year | DATE | Absolute or relative dates
$2 billion | MONEY | Monetary values
spaCy 采用模块化 Pipeline 设计，每个组件有确定的输入和输出：
Text → tokenizer → tagger → parser → ner → ... → Doc
查看当前 Pipeline：
import spacy
nlp = spacy.load("en_core_web_sm")
print(nlp.pipe_names)
# ['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'lemmatizer', 'ner']
| 组件 | 功能 | 输出 |
|---|---|---|
| tokenizer | 分词 | Token 序列 |
| tok2vec | 词→向量 | Token 向量 |
| tagger | 词性标注 | token.pos_, token.tag_ |
| parser | 依存句法 | token.dep_, token.head |
| ner | 命名实体 | doc.ents |
| lemmatizer | 词形还原 | token.lemma_ |
from spacy.matcher import Matcher
nlp = spacy.load("en_core_web_sm")
matcher = Matcher(nlp.vocab)
# 模式：形容词 + 名词
pattern = [{"POS": "ADJ"}, {"POS": "NOUN"}]
matcher.add("AdjNoun", [pattern])
doc = nlp("The quick brown fox jumps over the lazy dog")
matches = matcher(doc)
for match_id, start, end in matches:
print(doc[start:end]) # quick fox, brown fox, lazy dog
from spacy.pipeline import EntityRuler
nlp = spacy.load("en_core_web_sm")
ruler = nlp.add_pipe("entity_ruler", before="ner")
patterns = [
{"label": "PRODUCT", "pattern": "iPhone 15"},
{"label": "PRODUCT", "pattern": [{"LOWER": "macbook"}, {"LOWER": "pro"}]},
{"label": "SKILL", "pattern": [{"LOWER": {"IN": ["python", "java", "rust"]}}]},
]
ruler.add_patterns(patterns)
doc = nlp("I use Python and Rust on my MacBook Pro.")
print([(ent.text, ent.label_) for ent in doc.ents])
# [('Python', 'SKILL'), ('Rust', 'SKILL'), ('MacBook Pro', 'PRODUCT')]
import spacy
from spacy.training import Example
import random
# 从空白模型开始
nlp = spacy.blank("en")
ner = nlp.add_pipe("ner")
ner.add_label("TECH")
# 训练数据
TRAIN_DATA = [
("I love PyTorch", {"entities": [(7, 14, "TECH")]}),
("TensorFlow is from Google", {"entities": [(0, 10, "TECH")]}),
("FastAPI makes APIs easy", {"entities": [(0, 7, "TECH")]}),
]
# 开始训练
optimizer = nlp.begin_training()
for epoch in range(30):
random.shuffle(TRAIN_DATA)
losses = {}
for text, annots in TRAIN_DATA:
doc = nlp.make_doc(text)
example = Example.from_dict(doc, annots)
nlp.update([example], drop=0.5, losses=losses)
if epoch % 10 == 0:
print(f"Epoch {epoch}: Loss {losses['ner']:.4f}")
# 测试
doc = nlp("I started learning PyTorch yesterday")
print([(ent.text, ent.label_) for ent in doc.ents])
| 维度 | spaCy | NLTK | Stanza |
|---|---|---|---|
| 速度 | ⭐⭐⭐ 极快 | ⭐ 慢 | ⭐⭐ 中等 |
| 易用性 | ⭐⭐⭐ | ⭐⭐ | ⭐⭐ |
| 学术支持 | 较少 | 丰富（教材） | 丰富（Stanford） |
| 生产就绪 | ⭐⭐⭐ | ⭐ | ⭐⭐ |
| 多语言 | 75+ | 有限 | 70+ |
doc.ents 和 Matcher 都是提取实体，什么时候用哪个？ner 放在 parser 前面会怎样？暂无参考文献