text2vec

text2vec is an open-source Python toolkit that turns text into vectors and computes text similarity out of the box. Maintained by shibing624 and holding just under 5,000 stars on GitHub, it wraps several proven representation methods, from Word2Vec and RankBM25 to Sentence-BERT and CoSENT, behind one consistent API, with strong support for Chinese as well as English text.

Overview

Embeddings are the foundation of semantic search, recommendation, clustering and retrieval-augmented generation, but each model family has its own loading and inference quirks. text2vec removes that friction: load a model, encode sentences into dense vectors, and compare them with cosine similarity in a few lines. It ships pretrained Chinese and English models, training scripts for fine-tuning on your own pairs, and utilities for building semantic search over document collections. The project is part of a broader family of open NLP tools by the same author, and its models are also published on Hugging Face and ModelScope, which makes it a default choice in many Chinese enterprise search and customer-service matching systems.

Key Features

• Unified API for Word2Vec, RankBM25, Sentence-BERT, CoSENT and pooled BERT embeddings
• Pretrained Chinese and English models ready to download and run
• Sentence similarity and semantic search utilities built in
• Fine-tuning scripts for training on custom sentence-pair datasets
• Batch encoding with GPU support for large corpora
• Word vector training and averaged word-vector sentence representations for lightweight deployments
• Actively maintained, with releases continuing into 2026

💳 Pricing

Free and open source under the Apache 2.0 license, distributed via PyPI. There is no paid tier; models run entirely on your own infrastructure.

Who Is It For

NLP developers and data scientists, especially those working with Chinese text, who need dependable sentence embeddings and similarity scoring for search, matching, deduplication or RAG pipelines.