Voyage AI provides cutting-edge embedding/vectorizations models.Let’s load the Voyage AI Embedding class. (Install the LangChain partner package with
pip install langchain-voyageai
)
voyage-3-large
voyage-3
voyage-3-lite
voyage-large-2
voyage-code-2
voyage-2
voyage-law-2
voyage-large-2-instruct
voyage-finance-2
voyage-multilingual-2
embed_documents
to get their embeddings.
embed_query
to embed the query.
A minimalist retrieval system
The main feature of the embeddings is that the cosine similarity between two embeddings captures the semantic relatedness of the corresponding original passages. This allows us to use the embeddings to do semantic retrieval / search. We can find a few closest embeddings in the documents embeddings based on the cosine similarity, and retrieve the corresponding document using theKNNRetriever
class from LangChain.