RAG Interview Questions

What should I focus on for a RAG interview?

Can I explain how a query becomes an answer?

The end-to-end pipeline question was the most asked question, appearing in 31 of 147 interviews. Check whether you can explain the workflow, not just name its tools.

Reviews of real answers found a specific gap: candidates missed how a query retrieves relevant passages and how those passages become context for the LLM to generate an answer.

• Can you trace a query through retrieval, context assembly, and answer generation?
• Can you explain when you would choose RAG rather than fine-tuning or prompt engineering?
• Can you show where tools and structured data fit into your design?

Can I justify my chunk size and overlap?

Reviews of chunking answers found that candidates missed a concrete method for choosing and justifying chunk sizes. A useful check is whether you can describe how document boundaries guide your choices, then explain how you would test candidate sizes on queries that reflect actual use.

• Can you explain why you chose a chunk size and overlap?
• Can you test whether retrieval finds relevant passages and whether those passages contain enough information to answer?
• Can you describe how you would extract PDF content, keep metadata, and handle new or changed documents?

Can I compare retrieval choices?

Retrieval and reranking appeared in 34 of 147 interviews. The questions covered keyword and hybrid search, query rewriting, filtering, and reranking. Check whether you can compare approaches rather than only define semantic search.

• Can you explain what embeddings represent and how you would choose them?
• Can you compare a dedicated vector database with alternatives?
• Can you explain when you would use semantic search rather than keyword search?
• Can you name the metrics you would use to compare retrieval and reranking approaches?

Can I test quality and handle missing evidence?

Check whether you can describe an evaluation dataset and explain how you would measure retrieval quality separately from answer quality. Include how you would test for regressions and investigate incorrect results.

• Can you explain what the system should do when retrieved evidence does not support an answer?
• Can you describe how citations connect an answer to its evidence?
• Can you explain how you would keep data current and control which data each user can access?

Can I write the code behind semantic search?

Coding tasks appeared in 2 of 147 interviews. Check whether you can create a vector and use it for semantic search. Also check whether you can implement and explain a cosine similarity function using NumPy. These were the coding tasks recorded in these interviews.

Who gets asked RAG

The 20 most asked RAG questions

Counted by interview, not by mention: a question asked twice in one conversation counts once.

1. How would you design and implement an end-to-end RAG pipeline for an AI assistant?

   Asked in 31 of 147 interviews · last asked Oct 21 · Design · Mid-level

   Show answer

   RAG retrieves documents and uses them to ground an LLM’s answer. I would clean, chunk, embed, and index documents with source metadata. For each query, I would retrieve relevant chunks, rerank them, and include them in the LLM’s prompt. I would request citations, handle missing evidence, and evaluate retrieval and answer quality.

   • Use RAG for current or private knowledge; document updates do not require model retraining.
   • Combine keyword and vector search when useful; tune chunk size, retrieval count, and reranking.
   • Enforce document access permissions during retrieval and refresh the index when sources change.
   • Test retrieval relevance, answer correctness, citation support, latency, and cost using representative questions.

   What the interviewer checks: The interviewer checks whether you can connect document ingestion, retrieval, generation, and evaluation into a working system.

   Common mistake (in 12 of 15 reviewed answers): Many answers forget that retrieved passages must become context in the LLM’s prompt, not just search results.

   Other ways it was asked

   • “What is retrieval-augmented generation (RAG), and when would you use it?”
   • “What is retrieval-augmented generation (RAG)?”
   • “How can you optimize a retrieval-augmented generation (RAG) system, and why might you choose RAG over another approach?”

2. What is RAG, how does it work, and why is it used?

   Asked in 20 of 147 interviews · last asked Oct 23 · Concept · Junior

   Show answer

   RAG means retrieval-augmented generation. It finds relevant information in an external source, adds it to the model’s prompt, and asks the model to answer using that information. It is used when answers need current, private, or specialized knowledge without retraining the model. It can reduce made-up answers, but it does not prevent them completely.

   • Retrieval finds relevant content, often using keyword search, vector search, or both.
   • Retrieved content gives the model evidence to use when writing its answer.
   • Source content can be updated without changing the model’s weights.
   • Answer quality depends on finding good sources and using them correctly.

   What the interviewer checks: The interviewer checks whether you understand the retrieval and generation steps, their purpose, and their limits.

   Other ways it was asked

   • “What is RAG?”
   • “How does RAG work?”
   • “What is RAG, and what does it do?”

3. How do you choose a chunking strategy, chunk size, and overlap for a RAG system?

   Asked in 13 of 147 interviews · last asked Oct 22 · Scenario · Mid-level

   Show answer

   I start with document boundaries, such as headings and paragraphs, and keep related information together. I test several chunk sizes in tokens on representative queries, checking retrieval quality, context completeness, and answer accuracy. I keep chunks within embedding limits and retrieved context within the generation budget. I add overlap only when boundary splits lose useful context.

   • Match chunks to document structure and expected questions; preserve tables and code blocks where practical.
   • Small chunks improve focus; large chunks preserve context but may include unrelated text.
   • Compare candidate sizes and overlaps using the same queries and retrieval settings.
   • Overlap can protect boundary context, but increases storage and duplicate results.

   What the interviewer checks: The interviewer wants to see a practical, tested choice rather than one fixed chunk size for every system.

   Common mistake (in 3 of 5 reviewed answers): Many answers forget that chunk sizes need justification through document structure and tests of retrieval quality and context completeness.

   Other ways it was asked

   • “How do you choose a document-chunking strategy for a RAG system?”
   • “What factors should you consider when designing chunking strategies for a RAG system?”
   • “How do you choose a chunking strategy and chunk size for a RAG system?”

4. How would you evaluate retrieval and answer quality in a RAG system before production?

   Asked in 11 of 147 interviews · last asked Oct 19 · Concept · Mid-level

   Show answer

   I would test retrieval and answers separately on realistic questions with labeled relevant sources and expected answers. For retrieval, I would measure recall@k and ranking quality. For answers, I would check correctness, support from sources, and citation accuracy, using human review to validate automated scores. I would also test missing evidence, access rules, latency, and cost before release.

   • Use held-out questions from each customer, including hard cases and questions the documents cannot answer.
   • Recall@k measures relevant sources found; ranking metrics check whether useful sources appear early.
   • Check answer correctness and source support separately; a supported answer can still be incomplete.
   • Set release thresholds and compare against a baseline after each pipeline change.

   What the interviewer checks: The interviewer wants to see whether you can separate retrieval failures from answer failures and define reliable release checks.

   Other ways it was asked

   • “How would you implement a RAG system step by step?”
   • “How would you evaluate a RAG pipeline for each customer?”
   • “How would you implement a RAG system end to end?”

5. What are embeddings, how do they work, and why are they used?

   Asked in 9 of 147 interviews · last asked Oct 23 · Concept · Junior

   Show answer

   Embeddings are lists of numbers that represent text, images, or other data. A trained model creates them so that similar items tend to have similar vectors. They help systems find related items even when the words differ. In RAG, embeddings help retrieve relevant documents for the model to use.

   • Training teaches the model which patterns and relationships to represent.
   • Vector search compares embeddings using measures such as cosine similarity.
   • Queries and documents need embeddings from the same model or compatible models.
   • Embeddings support semantic search, recommendations, and clustering, but similarity does not guarantee relevance.

   What the interviewer checks: The interviewer checks whether you understand how learned vectors capture similarity and support retrieval in RAG.

   Other ways it was asked

   • “What is your understanding of embeddings?”
   • “What are embeddings, and when are they used?”
   • “What are embeddings, and where are they used?”

6. What is a vector database, and why would you use one in a RAG system?

   Asked in 9 of 147 interviews · last asked Oct 13 · Concept · Junior

   Show answer

   A vector database stores embeddings, which represent data as lists of numbers, and searches for similar ones. In RAG, it helps find document chunks related to a question, even when the wording differs. The system gives those chunks to the language model as context for its answer.

   • The question and document chunks need embeddings from compatible models.
   • Vector indexes make similarity search fast, often trading some accuracy for speed.
   • Metadata filters can limit results by source, date, or access rights.
   • A vector store is a broad term; it may be a database or a simpler storage library.

   What the interviewer checks: The interviewer checks whether you understand embeddings, similarity search, and how retrieved content supports a RAG answer.

   Other ways it was asked

   • “What is a vector database?”
   • “What is a vector database, and what is it used for?”
   • “What vector databases are currently available?”

7. How does semantic search differ from keyword search, and when would you use it?

   Asked in 7 of 147 interviews · last asked Oct 13 · Concept · Junior

   Show answer

   Semantic search finds results by meaning, often using embeddings, while keyword search mainly matches words. Use semantic search when users express the same idea differently from the documents. Use keyword search for exact names, codes, or phrases, and combine both when you need both kinds of matching.

   • Semantic search helps with natural-language questions, synonyms, and paraphrases.
   • Keyword search is useful when the exact wording matters.
   • Similarity search finds nearby items under a chosen metric; semantic search aims to match meaning.
   • Hybrid search combines keyword and semantic results.

   What the interviewer checks: The interviewer checks whether you can choose a search method based on user questions and document content.

   Other ways it was asked

   • “What do you mean by semantic search?”
   • “What do you mean by semantic search, and how is it different from normal search?”
   • “What is semantic search, and when would you use it?”

8. When would you choose RAG, fine-tuning, or prompt engineering?

   Asked in 4 of 147 interviews · last asked Oct 21 · Concept · Mid-level

   Show answer

   Use prompt engineering when clear instructions and examples are enough. Choose RAG when answers need external facts, changing information, or sources you can cite. Choose fine-tuning when you need consistent task behavior, style, or format and have good training examples. You can combine these methods.

   • Prompt engineering is usually the simplest starting point and does not change model weights.
   • RAG retrieves information at runtime; its quality depends on finding useful sources.
   • Fine-tuning changes model weights but is not a reliable way to keep facts current.
   • Fine-tuning needs training data and evaluation; RAG adds retrieval cost and latency.

   What the interviewer checks: The interviewer is checking whether you can separate knowledge needs from behavior needs and choose a practical approach.

   Other ways it was asked

   • “When would you choose RAG over fine-tuning?”
   • “What is the basic difference between RAG and fine-tuning, and when would you use fine-tuning or RAG?”
   • “Can you briefly explain the difference between RAG and fine-tuning?”

9. What is cosine similarity, and how does it work on two vectors?

   Asked in 4 of 147 interviews · last asked Oct 20 · Concept · Junior

   Show answer

   Cosine similarity measures how closely two nonzero vectors point in the same direction. It is the cosine of the angle between them: cos_sim(a, b) = (a · b) / (||a|| × ||b||). Compute their dot product, then divide by the product of their lengths.

   • Scores range from -1 to 1: 1 means same direction, 0 means perpendicular, and -1 means opposite directions.
   • It compares direction, not magnitude; multiplying a vector by a positive number does not change the score.
   • It is undefined for zero vectors because the denominator is zero.

   What the interviewer checks: The interviewer checks whether you understand the formula, its geometric meaning, and the zero-vector edge case.

   Other ways it was asked

   • “What is cosine similarity?”
   • “What do you mean by cosine similarity?”
   • “How does cosine similarity work on two numeric vectors?”

10. How would you evaluate and compare retrieval and reranking approaches, and which metrics would you use?

    Asked in 4 of 147 interviews · last asked Oct 22 · Concept · Mid-level

    Show answer

    I would compare approaches on the same held-out set of real questions with labeled relevant passages. For retrieval, I would measure Recall@k; for reranking, nDCG@k and MRR. I would also compare latency, cost, and final answer quality. I would test rerankers on identical candidate sets, then compare complete pipelines.

    • Recall@k measures the fraction of known relevant passages that appear in the top k results.
    • nDCG@k rewards placing highly relevant passages near the top and supports graded relevance labels.
    • MRR measures how early the first relevant result appears; it does not measure coverage.
    • Check results across question types; reranking cannot recover relevant passages missing from its candidate set.

    What the interviewer checks: The interviewer is checking whether you can compare approaches fairly and separate retrieval coverage from ranking quality.

    Other ways it was asked

    • “How would you evaluate a retrieval pipeline after embedding and reranking, and which metrics would you use?”
    • “What's your evaluation strategy for retrieval?”
    • “How would you evaluate reranking and retrieval approaches as potential solutions for an enterprise AI platform?”

11. Why does RAG use document chunks instead of whole documents, and what should each chunk contain?

    Asked in 3 of 147 interviews · last asked Sep 21 · Concept · Junior

    Show answer

    RAG usually splits documents into smaller pieces called chunks to retrieve relevant text without sending whole documents. This reduces irrelevant text and helps fit the model’s context limit. Each chunk should contain a meaningful passage with enough context to understand it, plus metadata such as its source and section.

    • Keep related ideas together; avoid splitting in the middle of an explanation.
    • Small chunks give precise matches; larger chunks preserve more context.
    • Some overlap can preserve context across chunk boundaries, but adds duplicate text.
    • Short documents may work well as whole documents; chunking is not always needed.

    What the interviewer checks: The interviewer checks whether you can balance retrieval accuracy, context limits, and the meaning preserved in each chunk.

    Other ways it was asked

    • “What is chunking, and why is it required?”
    • “Can you explain chunking?”
    • “Why is data chunked in a RAG pipeline, and what content should each chunk contain?”

12. How do you choose an embedding model for RAG and decide which documents to embed?

    Asked in 3 of 147 interviews · last asked May 12 · Concept · Senior

    Show answer

    I choose an embedding model by testing retrieval on real questions with known relevant documents. I compare language and domain fit, cost, speed, and hosting needs. I embed clean, current, trusted content that users need, usually split into chunks, and remove duplicates and irrelevant material. I keep access permissions with each chunk and enforce them during retrieval.

    • Compare models such as E5, BGE, and OpenAI embeddings on your own retrieval tests.
    • Use compatible query and document encoders, following the model’s required input format.
    • Choose chunk sizes that preserve meaning and fit the model’s input limit.
    • Changing embedding models usually requires embedding the documents again.

    What the interviewer checks: The interviewer checks whether you can test model choices and select useful, safe content for retrieval.

    Other ways it was asked

    • “Which embedding models do you know for RAG?”
    • “How do you choose an embedding model for a RAG system, what documents should be embedded, and why?”
    • “How would you choose an embedding model for a document retrieval system?”

13. How would you compare and choose vector databases for a production application?

    Asked in 3 of 147 interviews · last asked Oct 14 · Concept · Mid-level

    Show answer

    I would choose the database that meets our search quality, speed, reliability, and cost needs on real workload tests. I would compare candidates using our embeddings, filters, data size, and expected peak traffic. User count alone is not enough; I would measure recall, tail latency, throughput, and total operating cost.

    • Compare recall and p95/p99 latency at the same load and search quality target.
    • Check embedding dimensions, distance metrics, metadata filters, hybrid search, and update freshness.
    • Test scaling, backups, recovery, access controls, and tenant isolation.
    • Rank options by hard requirements first, then cost and operational effort, including managed versus self-hosted deployment.

    What the interviewer checks: The interviewer is checking whether you can turn production needs into fair tests and explain the trade-offs.

    Other ways it was asked

    • “How would you choose a vector database for storing and querying document embeddings?”
    • “Which database would you choose for vector embeddings to handle production scale of 10,000 users?”
    • “How would you compare vector databases for production use, and which parameters would you use to rank them?”

14. What are large language models (LLMs) and retrieval-augmented generation (RAG), and how do they differ?

    Asked in 2 of 147 interviews · last asked Sep 22 · Concept · Mid-level

    Show answer

    An LLM is a model trained on large amounts of text to understand and generate language. RAG is a method that retrieves relevant information from external sources and gives it to an LLM to help answer a question. The difference is that an LLM is the model, while RAG adds a retrieval step to support its answers.

    • Without retrieval, an LLM relies on its training and the context supplied to it.
    • RAG can use current or private information without retraining the LLM.
    • RAG depends on retrieval quality and does not guarantee correct answers.

    What the interviewer checks: The interviewer checks whether you can distinguish a language model from a system that combines retrieval and generation.

    Other ways it was asked

    • “What are large language models (LLMs) and retrieval-augmented generation (RAG)?”

15. Can you explain what LLM and RAG are, and what two types of tokens are available in agentic AI?

    Asked in 2 of 147 interviews · last asked Jul 17 · Concept · Junior

    Show answer

    An LLM, or large language model, learns patterns in text to understand and generate language. RAG, or retrieval-augmented generation, retrieves external information and gives it to the model as context. The two main token categories are input tokens, which the model reads, and output tokens, which it generates. Tokens can be words or parts of words.

    • RAG supplies relevant information without retraining the model.
    • Input tokens include instructions, conversation history, retrieved text, and tool results sent to the model.
    • Output tokens include generated answers and tool calls.
    • Input and output tokens apply to LLMs generally, not only agentic AI.

    What the interviewer checks: The interviewer checks whether you can distinguish the model, retrieval, and token usage in an agent workflow.

    Other ways it was asked

    • “Can you explain LLMs, RAG, and tokens?”

16. How would you create a vector and use it for semantic search?

    Asked in 2 of 147 interviews · last asked Aug 19 · Write code · Mid-level

    Show answer

    I would use an embedding model to turn each document into a vector that represents its meaning. I would embed the search query with the same model, compare it with the document vectors, and return the closest matches. For large collections, I would use a vector index to make search faster.

    • Use the same embedding model for documents and queries.
    • Cosine similarity compares vector direction; dot product gives the same score when vectors have unit length.
    • Vector indexes can speed up search, sometimes at the cost of missing relevant matches.

    PythonCopy

    # Requires: pip install sentence-transformers
    from sentence_transformers import SentenceTransformer
    model = SentenceTransformer("all-MiniLM-L6-v2")
    documents = ["Dogs make good pets.", "Baking bread takes time.", "Cats are household companions."]
    vectors = model.encode(documents, normalize_embeddings=True)
    query_vector = model.encode("Animals to keep at home", normalize_embeddings=True)
    scores = vectors @ query_vector
    for i in scores.argsort()[::-1][:2]:
        print(documents[i], float(scores[i]))

    What the interviewer checks: The interviewer is checking whether you can connect embedding creation, similarity scoring, and retrieval.

    Other ways it was asked

    • “How would you write code to create a vector and perform semantic search on it?”
    • “How can vector embeddings be used for semantic search?”

17. How does vector search find and score embeddings similar to a query?

    Asked in 2 of 147 interviews · last asked Sep 9 · Concept · Mid-level

    Show answer

    Vector search embeds the query using the same embedding model as the stored vectors. It finds nearby vectors using an exact scan or an approximate nearest neighbor index. It ranks results using a measure such as cosine similarity, dot product, or Euclidean distance.

    • Cosine similarity compares vector direction; dot product also depends on vector length.
    • Higher similarity means closer matches; lower distance means closer matches.
    • Approximate indexes speed up search but may miss some of the closest vectors.
    • Returned scores may transform the raw similarity or distance; check the database’s definition.

    What the interviewer checks: The interviewer checks whether you understand query embedding, nearest neighbor search, scoring, and the speed-versus-accuracy trade-off.

    Other ways it was asked

    • “How does a vector search system calculate similarity scores between a query and stored embeddings?”
    • “How does vector search work?”

18. How can you implement similarity search without a vector database, and why might you use one?

    Asked in 2 of 147 interviews · last asked May 22 · Concept · Mid-level

    Show answer

    You can store chunk embeddings in memory or files, then compare them with a query embedding using cosine similarity. Use NumPy to scan all embeddings, or a library like FAISS for indexed search. A vector database helps manage storage, metadata filters, updates, and search at scale.

    • Embed queries and chunks with the same compatible embedding model.
    • Keep a mapping from each embedding to its source chunk.
    • A full scan is simple and exact, but gets slower as the collection grows.
    • Approximate indexes trade some retrieval accuracy for speed; a vector database is optional.

    What the interviewer checks: The interviewer checks whether you understand similarity search separately from the storage and scaling features of vector databases.

    Other ways it was asked

    • “Can you perform semantic search without a vector store, and why are vector stores used to retrieve chunks or embeddings in RAG systems?”
    • “Can a RAG system perform similarity search without a vector database?”
    • “How can you implement similarity search without a vector database?”

19. How would you implement hybrid search combining vector similarity and keyword search?

    Asked in 2 of 147 interviews · last asked Jul 16 · Design · Senior

    Show answer

    I would run vector search and keyword search, such as BM25, on the same query, usually in parallel. Then I would merge results by document or chunk ID and combine their ranks using reciprocal rank fusion. Finally, I would optionally rerank the merged candidates and return the top results.

    • Vector search finds related meaning; keyword search catches exact terms, names, and codes.
    • Rank fusion avoids directly adding scores from different scales.
    • Apply the same access filters to both searches and remove duplicate results.
    • Tune candidate counts and fusion weights using representative queries and relevance labels.

    What the interviewer checks: The interviewer checks whether you can combine complementary retrieval methods while handling incompatible scores, duplicates, and access filters.

    Other ways it was asked

    • “How would you implement hybrid search for a user query, combining semantic similarity search with keyword search?”
    • “How would you build a hybrid search system that combines vector retrieval with traditional keyword search?”

20. What is query rewriting in retrieval, how does it work, and when would you use it?

    Asked in 2 of 147 interviews · last asked Sep 9 · Concept · Mid-level

    Show answer

    Query rewriting turns a user's question into a clearer search query before retrieval. Rules or an LLM can add context, expand terms, or split a complex question into smaller queries. In vector search, the rewritten text is embedded and used to find documents. Use it for vague questions, follow-up questions, or wording that differs from your documents.

    • Keep the user's meaning; do not add unsupported assumptions.
    • Rewriting changes the search input, not the stored documents.
    • Multiple queries can improve coverage, but add cost and may retrieve irrelevant results.
    • Check whether rewriting improves retrieval quality; clear queries may not need it.

    CodeCopy

    Previous question: What is our refund policy?
    Follow-up: Does it cover sale items?
    Rewritten query: What is our refund policy for sale items?

    What the interviewer checks: The interviewer checks whether you understand how rewriting improves retrieval and when it may hurt.

    Other ways it was asked

    • “What is query rewriting in retrieval with a vector database, and when would you use it?”
    • “How does query rewriting work?”

Related interview questions

• AI Engineer interview questions
• Software Engineer interview questions
• Data Scientist interview questions
• Data Engineer interview questions
• Machine Learning Engineer interview questions
• Product Manager interview questions
• Cybersecurity interview questions
• Machine Learning interview questions
• Deep Learning interview questions
• Computer Vision interview questions
• Generative AI and LLM interview questions
• Agentic AI interview questions
• MLOps interview questions

How this list is made

Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.

Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.

The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every RAG interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.

147 interviews, 98 candidates, Nov 2025 to Oct 2026. Updated every month with new interviews.