If you’ve found your way here, you’re probably already excited about the potential of the RAGFlow project and eager to see it in action. I was in the same position, so I delved into the codebase to see how it could work with custom large language models (LLMs). This post will walk you through my findings and show you how to get RAGFlow running with your own LLM.As of now (November 8, 2024), RAGFlow offers limited support for local LLMs, and vLLM isn’t yet compatible. In our case, we needed to test it on a company project with strict dat...