transformers
library. We then split this data into smaller chunks, which will form our searchable knowledge base.
docs_processed
is a structured collection of document chunks, ready to be used by a retriever tool.
Note: You can replace BM25 with a vector-based retriever for better semantic matching.
retriever_tool
to gather relevant context from the knowledge base.