Large Language Models with our own documents - RAG for LLM

RAG = Retrieval Augmented Generation - can be done locally as well as on cloud - explanatory video - 

Feed Your OWN Documents to a Local Large Language Model!

In the video, openwebui is used to run the local LLM - 

https://docs.openwebui.com/features/

If using docker (for ease of setup), and we want to copy our documents to the LLM's docker container, Docker copy command - 

Copying Files to and from Docker Containers - GeeksforGeeks

docker container cp source_local container:/path

sudo docker cp ~/Desktop/to-be-copied.txt 135950565ad8:/to-be-copied.txt

(the container id like 135950565ad8 above, is usually seen as a root@containerid prompt in the running container)

And apparently in the latest versions of openwebui, there is integration inside the UI itself for uploading documents - https://docs.openwebui.com/features/rag#local-and-remote-rag-integration

"Local documents must first be uploaded via the Documents section of the Workspace area to access them using the # symbol before a query. Click on the formatted URL in the that appears above the chat box. Once selected, a document icon appears above Send a message, indicating successful retrieval."

See this post for some initial setup, experiments and benchmarks - https://hnsws.blogspot.com/2025/04/experiments-with-llms-and-docker.html