Vector embeddings with ChromaDB. Basically you pre compute the word embeddings of every row / table / whatever granularity you want and then stick that into a vector DB. Then you do an embedding computation of your query and compare similarity. You can either return the table / row / whatever you want that’s most similar (“semantic search”) or you use that as context for an LLM (“RAG”)
Honestly this is pretty much it. Sometimes you have to be pretty aggressive to get companies to do the thing you need; they will take advantage of the social friction required to keep you in predatory arrangements. They literally design it to be frustrating so you’ll give up. Like you, I try to make it clear to the person I’m speaking with I have no problem with them just the business. But if the corporations require me to get mad to do the right thing I will get mad.