What should I focus on for a Generative AI and LLM interview?
Which topics should I review first?
Evaluation was the most common topic in these interviews, appearing in 92 of 291 interviews. Safety and security followed, appearing in 80 of 291 interviews. LLM basics came next. Give testing and safety attention alongside model definitions.
Can you explain how you would judge an answer, check the evidence behind it, and protect private data? Use those checks to decide which questions to review first.
Can I explain how an LLM works?
“What is a large language model?” was the most repeated question, asked in 15 of 291 interviews. Transformer architecture and what makes a good prompt were also among the most asked questions.
Check whether your answer covers the process, not just the terms:
- Can you explain tokens, embeddings, and attention in plain words?
- Can you walk through what happens when a model receives a prompt and generates a response?
- Can you write a prompt with clear context, constraints, and a required output format?
Can I explain how I would test an application?
The questions cover evaluating LLM applications, agent behavior, and RAG systems using customer-specific test cases. Check whether you can describe what you would measure and how you would judge the results.
- What ground truth would you use for comparison?
- What would your scoring rules check, and where would you use human or model judges?
- How would you repeat tests and check whether a change made results worse?
- How would you test for hallucinations, conflicting evidence, prompt injection, and access-control problems?
Can I explain my RAG, training, and agent choices?
RAG questions cover retrieval, chunking, knowledge sources, and production failures. Fine-tuning questions include choosing between training, prompting, and RAG. Agent questions cover tools, workflows, and memory.
Can you explain how you would evaluate retrieval separately from the final answer? Can you say when you would choose fine-tuning rather than prompting or RAG? For an agent, describe how you would manage tool use, conversation summaries, and context limits.
Can I take a design into production?
System design questions connect product requirements with model choice, API integration, deployment, and monitoring. Can you explain what you would monitor and how you would investigate a production problem? For inference, can you discuss how caching, token budgets, and model routing affect cost, response time, and quality?
Coding tasks appeared in 2 of 291 interviews. As a concrete check, can you write a Python function that takes a PDF filename, extracts its contents, and summarizes them using an OpenAI model?
Who gets asked Generative AI and LLM
The 20 most asked Generative AI and LLM questions
Counted by interview, not by mention: a question asked twice in one conversation counts once.
What is a large language model?
Asked in 15 of 291 interviews · last asked Oct 22 · Concept · Junior
Show answer
A large language model (LLM) is a neural network trained on large amounts of text to learn patterns in language. Most modern LLMs use transformers and generate text by predicting the next token, a word or part of a word. They can answer questions, summarize text, and write code, but can also produce convincing wrong answers.
- Tokens are pieces of text, such as words, word parts, or punctuation.
- During training, the model adjusts its parameters to improve predictions.
- When generating text, it uses the available context to predict one token at a time.
- Fluent output does not guarantee correct facts.
What the interviewer checks: The interviewer is checking whether you can explain what an LLM is, how it generates text, and its limits.
Other ways it was asked
- “What is an LLM?”
- “What are LLMs?”
- “Can you explain how an LLM works?”
What is the Transformer architecture?
Asked in 8 of 291 interviews · last asked Oct 14 · Concept · Junior
Show answer
A Transformer is a neural network architecture that uses attention to learn relationships between tokens. Each layer combines attention with a feed-forward network to build context-aware representations. Position information tells it where tokens occur in the sequence. Transformers can use encoders, decoders, or both, depending on the task.
- Self-attention lets each token use information from other allowed tokens; multiple heads learn different relationships.
- Layers also use residual connections and normalization to support training.
- GPT-style models use decoder blocks with masking to prevent attention to future tokens.
- Training can process token positions in parallel, but text generation usually produces one token at a time.
What the interviewer checks: The interviewer checks whether you understand attention, token order, and the main Transformer variants.
Other ways it was asked
- “What is transformer architecture, and how does it work?”
- “What are transformer architectures, and how do they work?”
- “What is transformer architecture?”
What makes a good prompt, and how do you write one?
Asked in 7 of 291 interviews · last asked Oct 21 · Concept · Mid-level
Show answer
A good prompt clearly states the task, gives relevant context, and defines what a useful answer should look like. I include constraints, the output format, and examples when needed. I test it on realistic inputs, check the results, and revise unclear instructions.
- Give enough context to avoid guessing, but leave out unrelated details.
- State requirements such as language, length, format, and software versions.
- Use examples when the desired behavior is hard to describe.
- Test and refine prompts; clear instructions do not guarantee correct output.
CodeWrite a Python function that removes duplicate strings from a list. Keep the first occurrence of each string, in its original order. Use only the standard library. Do not change the input list. Return the function and tests for empty input, duplicates, and unique values.What the interviewer checks: The interviewer is checking whether you can turn a vague goal into clear instructions and evaluate the results.
Other ways it was asked
- “How do you write prompts for AI tools to support software development?”
- “How do you write effective prompts for AI systems?”
- “How do you write effective prompts for generative AI tools, and how did you develop that skill?”
How would you evaluate an AI agent's behavior and correctness, and which metrics would you use?
Asked in 7 of 291 interviews · last asked Oct 16 · Concept · Senior
Show answer
I would test the agent on realistic tasks with clear success rules, including difficult and unsafe requests. I would check both the final result and its actions, such as tool calls, using automated checks and human review. I would track task success, answer correctness, tool errors, safety violations, response time, and cost, and repeat tests to check reliability.
- Check whether the task was completed correctly, not just whether the answer sounds right.
- Review tool choices, arguments, and actions for correctness and permission limits.
- Repeat tests to measure consistency; include tool failures and unexpected inputs.
- Measure speed and cost alongside quality and safety.
What the interviewer checks: The interviewer wants to see whether you can evaluate both outcomes and actions with clear, repeatable tests.
Other ways it was asked
- “How do you evaluate an AI agent?”
- “Which metrics would you use to measure an AI agent's effectiveness?”
- “Which metrics would you use to evaluate an AI agent?”
How would you evaluate an LLM application?
Asked in 7 of 291 interviews · last asked Oct 6 · Scenario · Mid-level
Show answer
I would evaluate the application against clear task goals using a held-out test set of realistic inputs. I would combine automated checks with human review to measure correctness, relevance, safety, latency, and cost. Before release, I would compare results with a baseline, then monitor failures and user outcomes in production.
- Test the full application, including retrieval and tools, not just the model.
- Include common requests, edge cases, and attempts to bypass safety rules.
- Check factual claims against trusted sources; validate LLM-based judges against human ratings.
- Run regression tests after changes to prompts, models, tools, or data.
What the interviewer checks: The interviewer is checking whether you can turn product goals into reliable tests and track production quality.
Other ways it was asked
- “How would you evaluate an LLM system?”
- “What parameters or techniques would you use to evaluate an LLM application?”
- “How would you design and implement an LLM application?”
How would you evaluate a RAG system using customer-specific test cases and metrics?
Asked in 5 of 291 interviews · last asked Oct 7 · Concept · Mid-level
Show answer
I would build a test set for each customer using real questions, trusted answers, and expected source documents. I would test retrieval and answer quality separately, including questions the documents cannot answer. I would also measure latency and cost under expected traffic, set customer-specific targets, and rerun tests after changes.
- Measure retrieval recall@k: the fraction of relevant documents found in the top k results.
- Score answer correctness, support from sources, citation accuracy, and correct refusal when evidence is missing.
- Include customer terms, access rules, and hard cases; have domain experts review answers.
- Track tail latency, throughput, error rates, and cost per query under load.
What the interviewer checks: The interviewer checks whether you can turn customer needs into repeatable tests for quality and production performance.
Other ways it was asked
- “How would you evaluate a RAG pipeline for each customer?”
- “How would you deploy and evaluate a RAG system, including its accuracy, latency, and performance?”
- “What are some methods for evaluating a RAG system?”
What happens step by step when an LLM receives a prompt and generates a response?
Asked in 4 of 291 interviews · last asked Oct 7 · Concept · Mid-level
Show answer
The system formats the prompt and chat history, then splits them into tokens. The model converts tokens into vectors and processes them through transformer layers to predict next-token probabilities. A decoding rule selects a token, and generation repeats using each new token until a stop condition. Output tokens are decoded into text, which may be streamed.
- Tokens can represent words, word parts, or punctuation.
- Position information tracks token order; attention uses the available context.
- Decoding may choose the highest-probability token or sample; temperature affects sampling.
- Generation stops at an end token or another limit; normal inference does not update model weights.
What the interviewer checks: The interviewer checks whether you understand tokenization, transformer processing, next-token selection, and repeated generation.
Other ways it was asked
- “When you submit a prompt to a chat-based language model, what happens behind the scenes before it returns a response?”
- “Could you describe what happens under the hood when a model like GPT-4 generates a response to a prompt?”
- “Can you briefly explain the LLM inference process step by step?”
What is the difference between LLM and AI?
Asked in 4 of 291 interviews · last asked Oct 22 · Concept · Junior
Show answer
AI is the broad field of building systems that perform tasks such as learning, reasoning, and understanding language. An LLM is a type of AI trained on large amounts of text to understand and generate language. An AI agent is a system that takes actions toward a goal, often using an LLM with tools.
- Every LLM is AI, but not all AI uses LLMs.
- AI also includes systems for image recognition, prediction, and robotics.
- An LLM generates responses; an agent adds a process for choosing and taking actions.
- Agents can use LLMs, but do not require them.
What the interviewer checks: The interviewer checks whether you can distinguish the broad field, a language model, and an action-taking system.
Other ways it was asked
- “What is the difference between agentic AI and an LLM?”
- “What is the difference between an AI agent and an LLM?”
- “Is an LLM a form of AI?”
What is prompt injection, and how would you protect an AI application against it?
Asked in 4 of 291 interviews · last asked Oct 13 · Scenario · Senior
Show answer
Prompt injection is an attempt to make an AI follow malicious instructions in user input or external content instead of its intended rules. I treat that content as untrusted data, limit tool permissions, and check actions in application code before execution. Prompt wording alone cannot reliably stop these attacks.
- Attacks can come directly from users or indirectly through documents, websites, and tool results.
- Separate trusted instructions from untrusted content, but do not rely on separation alone.
- Enforce access controls outside the model and require approval for sensitive actions.
- Test with attack examples and monitor for suspicious tool use or data leaks.
What the interviewer checks: The interviewer checks whether you understand the attack and use layered controls rather than trusting prompts alone.
Other ways it was asked
- “What's your approach to mitigating risks like prompt injection in AI applications?”
- “What is prompt injection, and how can you protect a system against it?”
- “What is prompt injection?”
What is KV caching, and how does it work during LLM inference?
Asked in 4 of 291 interviews · last asked Oct 6 · Concept · Mid-level
Show answer
KV caching stores the attention key and value vectors for earlier tokens at each transformer layer. The prompt first fills the cache. During generation, each new token adds its keys and values, while its query attends to the cached and current keys and values. This avoids recomputing earlier tokens, making generation faster but using more memory.
- The cache stores key and value vectors, not token IDs or attention scores.
- Past queries are not needed for future tokens, so they are not cached.
- Cache memory grows with sequence length and batch size.
- Caching avoids repeated computation, but each new query still attends over the available context.
What the interviewer checks: The interviewer checks whether you understand what attention reuses during generation and the speed versus memory trade-off.
Other ways it was asked
- “What is a KV cache, and what does it store during transformer inference?”
- “What is KV cache in LLMs, and what exactly does it store?”
- “What is KV caching, and how does it work in LLM inference?”
What are tokens in a large language model?
Asked in 3 of 291 interviews · last asked Oct 1 · Concept · Junior
Show answer
Tokens are the units of text that a large language model processes. A token can be a word, part of a word, punctuation, or whitespace, depending on the tokenizer. The tokenizer converts text into token IDs, which the model uses to read input and generate output.
- A token is not always a whole word.
- Different tokenizers can split the same text differently.
- Context limits and usage costs are often measured in tokens.
What the interviewer checks: The interviewer is checking whether you understand how text becomes model input and why token counts matter.
Other ways it was asked
- “What are tokens?”
- “In the context of large language models, what is a token?”
What is RAG, how does it work, and how does it differ from using an LLM alone?
Asked in 3 of 291 interviews · last asked Sep 22 · Concept · Junior
Show answer
RAG means retrieval-augmented generation: it retrieves relevant information and gives it to an LLM to help answer a question. Documents are usually split into chunks and indexed; matching chunks are added to the prompt. An LLM alone relies on its learned knowledge and given prompt, without retrieving sources. RAG can use current or private information without retraining the model.
- An LLM generates text; RAG is a system that adds retrieval around an LLM.
- Retrieval can use keyword search, vector search, or both.
- RAG can support source citations, but poor retrieval can still produce wrong answers.
- Retrieval adds latency, cost, and system complexity.
What the interviewer checks: The interviewer checks whether you understand the retrieval-to-generation flow, its benefits, and its limits.
Other ways it was asked
- “What is RAG, how is it architected, and how does it differ from using an LLM without retrieval?”
- “What are large language models (LLMs) and retrieval-augmented generation (RAG)?”
- “What are large language models (LLMs) and retrieval-augmented generation (RAG), and how do they differ?”
What is the basic difference between RAG and fine-tuning, and when would you use fine-tuning or RAG?
Asked in 3 of 291 interviews · last asked Oct 21 · Concept · Mid-level
Show answer
RAG retrieves relevant information and gives it to the model as context, without changing its weights. Fine-tuning trains the model on examples and changes its weights. Use RAG for current or private knowledge that needs source references. Use fine-tuning for consistent style, output format, or task behavior.
- RAG lets you update source documents without retraining the model.
- Fine-tuning needs good training examples; it is not a reliable way to store changing facts.
- RAG depends on retrieval quality and can still produce unsupported answers.
- You can combine RAG for knowledge with fine-tuning for behavior.
What the interviewer checks: The interviewer is checking whether you can separate knowledge access from learned behavior and choose the right approach.
Other ways it was asked
- “When would you choose RAG over fine-tuning?”
- “Can you briefly explain the difference between RAG and fine-tuning?”
How would you measure whether fine-tuning improved the model?
Asked in 3 of 291 interviews · last asked Sep 27 · Scenario · Mid-level
Show answer
I would compare the original and fine-tuned models on the same held-out test set that reflects real use. I would use task-specific metrics and blind human review, with the same prompts and generation settings. I would check whether gains are reliable, test for regressions, and compare latency and cost.
- Keep test data separate from training and model selection to avoid leakage.
- Choose metrics that match the task, such as accuracy, factual correctness, or instruction following.
- Use enough test cases and confidence intervals to separate real gains from noise.
- Check important user groups, safety, and general skills, not just the average score.
What the interviewer checks: The interviewer is checking whether you can design a fair comparison and show useful gains without hiding regressions.
Other ways it was asked
- “How would you determine whether fine-tuning actually improves a model?”
- “How would you systematically compare a model's performance before and after fine-tuning?”
How would you manage growing conversation history and decide what context to keep, summarize, or drop?
Asked in 3 of 291 interviews · last asked Oct 6 · Design · Senior
Show answer
I would set a token budget, leaving room for the model’s reply and tool results. I would keep system instructions, recent messages, and facts needed for the current task. Before nearing that budget, I would summarize older decisions, constraints, and unfinished work, and drop repeated or irrelevant content. I would store older details separately and retrieve them when needed.
- Choose context by relevance, not just age; keep earlier facts the task still needs.
- Preserve user corrections and check that summaries do not change important facts.
- Shorten large tool outputs, keeping relevant results and references to the originals.
- Trigger summaries based on token usage, not only message count.
What the interviewer checks: The interviewer is checking whether you can control context size without losing information needed to finish the task.
Other ways it was asked
- “How would you manage context in a long-running chat with a user?”
- “How would you manage context in a long-running chat, including recent messages, summaries, and when to generate a summary?”
- “In a multi-turn agent, how would you keep the conversation useful as history grows, and how would you decide what to drop?”
Why do language models hallucinate?
Asked in 3 of 291 interviews · last asked Aug 19 · Concept · Mid-level
Show answer
Language models hallucinate because they generate likely text, not verified facts. Gaps or errors in training data, unclear prompts, and missing context can lead to plausible but false answers. They may also give confident answers when they lack enough information.
- Predicting the next token does not guarantee factual accuracy.
- Training data can be incomplete, outdated, conflicting, or wrong.
- Missing context and false assumptions in prompts can trigger errors.
- Retrieval and fact-checking can reduce hallucinations, but cannot fully prevent them.
What the interviewer checks: The interviewer is checking whether you understand why fluent, confident output can still be factually wrong.
Other ways it was asked
- “Why do LLMs hallucinate, and what causes these hallucinations?”
- “Why do LLMs hallucinate sometimes?”
- “What do you believe triggers hallucinations?”
How would you implement PII protection and related security controls in an LLM application?
Asked in 3 of 291 interviews · last asked Sep 30 · Scenario · Mid-level
Show answer
I would send only the data the model needs and remove or mask PII before sending it. I would protect any mapping back to real values with encryption and strict access controls. I would also check outputs and logs for sensitive data, verify provider retention and training settings, and limit what tools the model can use.
- Combine pattern checks and PII detection; neither catches every sensitive value.
- Check access rights before retrieving private data, not just before showing the answer.
- Treat prompts and retrieved content as untrusted; enforce tool permissions outside the model.
- Protect data in transit and storage, limit retention, and test for leaks and prompt injection.
What the interviewer checks: The interviewer is checking whether you protect sensitive data across the whole system, rather than relying only on masking.
Other ways it was asked
- “How would you protect users' personally identifiable information when sending data to an LLM?”
- “How have you implemented PII protection and related security controls in an AI system?”
- “Besides PII masking, what other guardrails have you implemented in an AI system?”
How would you reduce hallucinations in an AI system and evaluate whether your approach works?
Asked in 3 of 291 interviews · last asked Sep 25 · Scenario · Senior
Show answer
I would ground answers in trusted sources, check factual claims against evidence, and make the system abstain when evidence is missing. For an agent, I would also validate tool results before using them. I would compare the original and updated systems on the same test set, measuring unsupported claims, answer accuracy, and unnecessary refusals.
- Use retrieval or tools for facts, but check source quality and whether evidence supports each claim.
- Ask for citations and verify them; citations alone do not prove an answer is correct.
- Test missing evidence, conflicting sources, and misleading inputs, not just easy questions.
- Use human review to check automated scores, and monitor failures after release.
What the interviewer checks: The interviewer checks whether you can combine evidence-based safeguards with evaluation that measures both factual reliability and usefulness.
Other ways it was asked
- “How do you mitigate hallucinations in AI models?”
- “How would you prevent hallucinations in an AI agent and keep its responses grounded in evidence?”
How would you reduce token usage in an LLM application?
Asked in 3 of 291 interviews · last asked Sep 8 · Scenario · Mid-level
Show answer
I would measure input and output tokens, then remove repeated instructions and irrelevant context. I would retrieve only useful passages, trim or summarize chat history, and request concise answers. I would test these changes on real tasks to check quality, and cache repeated results to avoid extra model calls.
- Keep instructions and context needed for a correct answer.
- Summaries save tokens but may lose important details.
- Set output limits carefully; low limits can cut answers short.
- Prompt caching may lower cost without reducing token counts; support and pricing vary by provider.
What the interviewer checks: The interviewer checks whether you can reduce token waste while measuring the effect on answer quality.
Other ways it was asked
- “What strategies do you use to reduce token usage in an LLM application?”
- “How can you reduce LLM token costs without sacrificing quality?”
What are embeddings in an LLM?
Asked in 2 of 291 interviews · last asked Aug 27 · Concept · Junior
Show answer
Embeddings are vectors, or lists of numbers, that represent tokens or text. An LLM gets its starting token embeddings from a lookup table learned during training. For search, a model trained to create text embeddings maps whole texts to vectors, placing similar meanings close together. This may be separate from the LLM generating answers.
- Token embeddings are learned representations, not token IDs.
- An LLM’s layers use context to build richer representations from token embeddings.
- Text embeddings support similarity search, retrieval, and clustering.
- A text embedding model can be based on a language model.
What the interviewer checks: The interviewer is checking whether you understand embeddings and distinguish internal token embeddings from text embeddings used for search.
Other ways it was asked
- “What are embeddings, and how are they created? Does the LLM create them, or does a different model?”
Generative AI and LLM basics interview questions
Start with questions about generative AI, tokens, embeddings, transformers, attention, response generation, model families, and differences between AI platforms. 74 questions.
What are the main challenges facing language models, and how would you address them?
Asked in 2 of 291 interviews · last asked Oct 21 · Scenario · Mid-level
Other ways it was asked
- “What challenges do language models present, and how would you address one of them?”
- “What do you see as the key challenges currently impacting the effectiveness of AI systems, particularly within language models?”
What do you know about mixture of experts?
Asked in 2 of 291 interviews · last asked Jul 16 · Concept · Senior
Other ways it was asked
- “What is mixture of experts?”
How would you explain a generative AI system end to end?
Asked in 2 of 291 interviews · last asked Aug 22 · Design · Mid-level
Other ways it was asked
- “How would you explain the flow of an end-to-end generative AI project?”
Can you explain what LLM and RAG are, and what two types of tokens are available in agentic AI?
Asked in 2 of 291 interviews · last asked Jul 17 · Concept · Junior
Other ways it was asked
- “Can you explain LLMs, RAG, and tokens?”
What common mistakes do LLMs make when solving mathematical problems?
Asked once · last asked May 19 · Concept · Mid-level
Other ways it was asked
- “What kinds of errors do LLMs make in the steps of a mathematical solution?”
How do transformer-based next-word predictors use context windows to generate successive predictions?
Asked once · last asked Apr 11 · Concept · Mid-level
Why can one word correspond to multiple tokens, and how would you calculate the number of tokens in a sentence?
Asked once · last asked Jun 12 · Concept · Junior
Is tokenization deterministic, or can it change over time for the same input?
Asked once · last asked Jun 12 · Concept · Mid-level
What is the difference between a chatbot, a RAG application, and an AI agent?
Asked once · last asked Jul 16 · Concept · Mid-level
At a high level, how does a transformer work?
Asked once · last asked Jul 16 · Concept · Junior
What is the difference between decoder-only, encoder-only, and encoder-decoder architectures, and which type is GPT?
Asked once · last asked Jul 16 · Concept · Mid-level
How does reasoning in LLMs work, and how is it implemented?
Asked once · last asked Jul 16 · Concept · Senior
Prompt engineering interview questions
Explore prompting techniques, context, constraints, output formats, and prompt versioning for writing, coding, voice, and multimodal tasks. 76 questions.
What is the difference between one-shot, few-shot, and zero-shot prompting?
Asked in 2 of 291 interviews · last asked Aug 19 · Concept · Mid-level
Other ways it was asked
- “What are zero-shot, one-shot, and few-shot prompting, and when would you use each?”
Tell me about a work task where an AI tool produced useful output end to end. What input did you provide, and how did you validate and use the output?
Asked in 2 of 291 interviews · last asked Oct 18 · Behavioral · Mid-level
Other ways it was asked
- “Tell me about a task where an AI tool produced a particularly useful result. What prompt or workflow did you use, and how did you use the output?”
How would you design and manage the system prompt for an LLM agent?
Asked in 2 of 291 interviews · last asked Jul 16 · Concept · Mid-level
Other ways it was asked
- “How would you prompt an LLM using a system prompt?”
Write a prompt for a multilingual voice AI agent with agentic tool orchestration.
Asked once · last asked Jul 27 · Mid-level
How would you structure a four-turn prompting process to improve an executive KPI narrative without adding facts or changing KPI definitions?
Asked once · last asked Sep 16 · Scenario · Mid-level
Can you walk me through the process or approach you generally follow to craft a prompt from scratch to achieve a high-quality, specific output?
Asked once · last asked Mar 5 · Tooling · Mid-level
How would you design a complex, multistep prompt or chain-of-thought workflow?
Asked once · last asked Mar 5 · Behavioral · Senior
Other ways it was asked
- “Can you describe a complex, multistep prompt or chain-of-thought workflow you have designed?”
Can you think of a complex, real-world insurance scenario and draft a prompt with multiple requirements for a model acting as an insurance analyst?
Asked once · last asked Mar 5 · Case · Mid-level
If you had to write a 500-word, SEO-friendly blog, what prompt would you give ChatGPT or another LLM tool?
Asked once · last asked Apr 8 · Tooling · Mid-level
Other ways it was asked
- “How would you prompt an LLM to write a 500-word, SEO-friendly blog on a topic provided to you?”
How do you use prompt files to provide context or instructions to an AI coding assistant?
Asked once · last asked Apr 14 · Tooling · Mid-level
What context or style guidance do you give an AI coding assistant, such as repository code samples, and do you use ad hoc prompts?
Asked once · last asked Apr 14 · Tooling · Mid-level
If a model keeps repeating the same answer instead of generating improved or varied responses, how would you address that?
Asked once · last asked Apr 20 · Scenario · Mid-level
RAG interview questions
RAG questions cover retrieval, chunking, knowledge sources, system architecture, evaluation, and finding the cause of production failures. 41 questions.
How do you monitor a RAG system in production over time, and what do you do if you find issues?
Asked once · last asked Oct 14 · Scenario · Senior
Other ways it was asked
- “How do you monitor a RAG system in production, identify issues, and address them?”
What is chunking, and how is it related to the context window?
Asked once · last asked Feb 5 · Concept · Junior
How would you evaluate a retrieval pipeline after embedding and reranking, and which metrics would you use?
Asked once · last asked May 12 · Scenario · Senior
How does RAG help reduce hallucinations?
Asked once · last asked Jun 10 · Concept · Mid-level
Which metrics would you use to measure reductions in hallucinations and improvements in retrieval accuracy?
Asked once · last asked Jun 29 · Concept · Mid-level
How does the Ragas framework work, particularly for measuring faithfulness?
Asked once · last asked Jun 29 · Concept · Mid-level
How would you test a chatbot's data-chunking strategy to ensure it does not degrade answer quality?
Asked once · last asked Jul 16 · Scenario · Mid-level
Can you explain RAG and LLMs, including LLM hallucinations?
Asked once · last asked Jul 17 · Concept · Junior
What's your evaluation strategy for retrieval?
Asked once · last asked Jul 20 · Scenario · Mid-level
How would you build an evaluation dataset for a RAG system that retrieves drug aliases?
Asked once · last asked Jul 24 · Scenario · Senior
Why are citations important in a RAG workflow, and what happens without them?
Asked once · last asked Jul 27 · Concept · Mid-level
How would you provide an LLM with information about relationships within a large dataset without overwhelming its context window?
Asked once · last asked Aug 7 · Design · Senior
Other ways it was asked
- “How would you give an LLM the interconnectedness of a large dataset without overwhelming it?”
LLM fine-tuning and training interview questions
Compare fine-tuning with prompting and RAG, then review LoRA, small language models, reinforcement learning, preference training, and measuring improvements. 28 questions.
What is model fine-tuning, and when would you use it?
Asked in 2 of 291 interviews · last asked Jul 29 · Concept · Mid-level
How do you fine-tune small language models?
Asked once · last asked Sep 25 · Concept · Mid-level
Other ways it was asked
- “What are the different ways to fine-tune small language models?”
How would you handle an agent that uses both static company-policy information and dynamic patient or insurance data, and where would fine-tuning fit?
Asked once · last asked Feb 5 · Design · Senior
What is LoRA (low-rank adaptation), and how does it work?
Asked once · last asked Jun 12 · Concept · Mid-level
How do you format prompts and training examples for fine-tuning a model to produce structured outputs?
Asked once · last asked Jul 16 · Scenario · Mid-level
What is the difference between full fine-tuning and LoRA?
Asked once · last asked Jul 16 · Concept · Mid-level
What is the difference between training a model and changing its system prompt?
Asked once · last asked Jul 16 · Concept · Mid-level
What are the main considerations when training or retraining an LLM?
Asked once · last asked Jul 16 · Concept · Senior
When do you choose fine-tuning versus RAG versus prompt engineering?
Asked once · last asked Jul 22 · Concept · Senior
How would you approach a model customization process from a high level, and what information would you gather before starting?
Asked once · last asked Jul 27 · Scenario · Senior
Why would you fine-tune an LLM, and what process would you follow?
Asked once · last asked Aug 13 · Concept · Mid-level
Other ways it was asked
- “Why did you fine-tune the LLM, and what was the process?”
How would you fine-tune an LLM for code generation, and when would you use a fine-tuned model instead of a base LLM?
Asked once · last asked Aug 13 · Concept · Mid-level
Show 14 more
An engineering team is fine-tuning a model using quantized LoRA (QLoRA) on limited GPU infrastructure and repeatedly encounters out-of-memory errors. How would you mitigate these errors?
Asked once · last asked Aug 20 · Scenario · Senior
Other ways it was asked
- “The engineering team is doing a quantized LoRA for a model on limited GPU infrastructure and keeps getting out-of-memory errors. How could we help them mitigate these errors?”
What is the difference between fine-tuning and in-context learning for document processing?
Asked once · last asked Aug 25 · Concept · Mid-level
How would you fine-tune a model when no training dataset is available?
Asked once · last asked Sep 5 · Scenario · Senior
In which scenarios should we use supervised fine-tuning, reinforcement learning, GRPO, or PPO for small language models?
Asked once · last asked Sep 25 · Concept · Senior
For an LLM answering questions about frequently changing internal policies, would you fine-tune it or use another approach?
Asked once · last asked Sep 27 · Scenario · Senior
When building an enterprise AI feature, what key factors help you decide between using advanced RAG and fine-tuning?
Asked once · last asked Oct 20 · Design · Senior
How would you adapt a six-item pass/fail rubric to an RLHF preference task with two candidate responses so annotators consistently choose A or B?
Asked once · last asked Oct 21 · Scenario · Senior
Other ways it was asked
- “How would you adapt those same six pass or fail items to an RLHF preference task where the model gives two candidate responses? What would annotators look for to choose A or B consistently?”
How would you design a preference tool and workflow for RLHF to keep rater consistency high over time, including rubric, calibration, goldens, adjudication, and measuring drift in rater behavior?
Asked once · last asked Oct 22 · Design
When training a reward model from human preference labels, how would you detect and mitigate rater bias or position bias so the reward does not overfit annotator quirks?
Asked once · last asked Oct 22 · Scenario · Senior
Other ways it was asked
- “When training a reward model from those preferences, how would you detect and mitigate rater bias or position bias so the reward does not overfit annotator quirks?”
In RLHF, how would you build an evaluation suite to detect reward hacking and regression in safety beyond just tracking average reward score?
Asked once · last asked Oct 22 · Scenario · Senior
What does fine-tuning mean in the context of an LLM?
Asked once · last asked Feb 5 · Concept · Junior
What is LoRA?
Asked once · last asked Jul 21 · Concept · Junior
What is agentic reinforcement learning?
Asked once · last asked Sep 25 · Concept · Senior
Why would you choose RAG rather than fine-tuning for frequently changing internal policies?
Asked once · last asked Sep 27 · Concept · Senior
Other ways it was asked
- “What are additional reasons to use RAG rather than fine-tuning for frequently changing internal policies?”
AI agents and context management interview questions
Agent questions explore workflows, tools, memory, conversation summaries, context limits, feedback loops, and repeated task steps. 56 questions.
How would you provide an AI agent with relevant data sources and context for a task?
Asked in 2 of 291 interviews · last asked Oct 21 · Scenario · Mid-level
Other ways it was asked
- “How would you provide an AI agent with the context it needs?”
How would you measure whether a summarization or pruning strategy is harming performance?
Asked once · last asked Oct 5 · Scenario · Senior
Other ways it was asked
- “How would you measure whether a summarization or pruning strategy is harming performance, and what offline or online metrics and tests would you use?”
How do you use prompt engineering with LLMs, and what are the key steps in building an AI agent?
Asked once · last asked Feb 17 · Concept
In LLM applications, how do you manage context and memory over a long interaction when the full context cannot fit in the model's context window?
Asked once · last asked Apr 28 · Design · Senior
How would you use feedback to help an AI agent avoid repeating incorrect results?
Asked once · last asked Jun 25 · Scenario · Senior
How would you store short-term, medium-term, and long-term memory in a multi-turn chatbot?
Asked once · last asked Jul 16 · Design · Senior
Other ways it was asked
- “How would a multi-turn chatbot decide which medium-term memory to retrieve?”
What memory types does LangChain provide, and when would you use each?
Asked once · last asked Jul 17 · Concept · Mid-level
How would you use multistep prompt chains with Zapier or Make, and how would you assess the outcomes?
Asked once · last asked Jul 20 · Tooling · Senior
Other ways it was asked
- “How have you used multistep prompt chains with Zapier or Make, and what outcomes did they produce?”
What separates an agent from a regular LLM chatbot?
Asked once · last asked Jul 22 · Concept · Mid-level
Do multiple AI agents run into context-window limitations, and how would you address them?
Asked once · last asked Jul 22 · Scenario · Senior
How would you design chat summarization, including prompts, key information, summary structure, and token limits?
Asked once · last asked Jul 28 · Design · Senior
Other ways it was asked
- “How would you summarize a long chat, and how would you prompt an LLM to extract the relevant information?”
- “How would you design a chat summarization process, including the prompt, key-information selection, summary structure and size, token limits, and use of the user's input?”
How would you build or optimize an LLM-powered or agentic application, including prompt design, tool use, and orchestration?
Asked once · last asked Aug 2 · Design · Senior
Other ways it was asked
- “How have you approached building or optimizing LLM-powered or agentic applications? Walk me through one example, including how you handled prompt design, tool use, or orchestration.”
LLM evaluation and benchmarking interview questions
Check how you would evaluate LLM outputs using metrics, ground truth, benchmarks, scoring rules, human or model judges, and regression tests. 197 questions.
When evaluating AI-generated legal responses, how do you assess their accuracy and quality?
Asked in 2 of 291 interviews · last asked Oct 14 · Scenario · Senior
Other ways it was asked
- “What are some key factors you consider when assessing the quality of AI-generated responses?”
- “When you are evaluating those AI-generated legal responses, how do you assess their accuracy and quality?”
When designing a benchmark for an AI agent, what are the essential components of a good rubric?
Asked in 2 of 291 interviews · last asked Aug 29 · Scenario · Senior
Other ways it was asked
- “When you design an evaluation rubric for an AI agent, what are the minimum required rubric components to make scoring consistent and auditable?”
When you're offering a deliberately hard Q&A, how do you choose the one ambiguity to preserve to make it challenging while eliminating all other ambiguity that could make grading subjective?
Asked in 2 of 291 interviews · last asked Oct 23 · Scenario · Senior
Other ways it was asked
- “When you design a deliberately challenging question-and-answer pair, how do you preserve one intentional ambiguity while removing ambiguity that could make grading subjective?”
How do LLM evaluation frameworks calculate scores, and who or what assigns them?
Asked once · last asked Jul 30 · Concept · Mid-level
Other ways it was asked
- “What results does an LLM evaluation produce, and how are its scores calculated?”
- “What scores or score mappings can LLM evaluation frameworks produce?”
How would you test an LLM output when the acceptance criteria and expected behavior are known?
Asked once · last asked Jul 30 · Scenario · Mid-level
How would you compare two videos against a prompt about a golden retriever catching a red Frisbee on the beach at sunset and justify which video follows it more closely?
Asked once · last asked Oct 17 · Scenario · Junior
Other ways it was asked
- “How would you compare two videos against a prompt and justify which one follows it more closely?”
How would you assess whether an LLM is performing well, and what metrics or parameters would you use to validate it?
Asked once · last asked Feb 5 · Scenario · Mid-level
How would you evaluate an AI model’s response to a prompt?
Asked once · last asked Feb 13 · Scenario · Mid-level
How do you design evaluation rubrics or scoring mechanisms for model outputs, and which dimensions do you prioritize?
Asked once · last asked Mar 5 · Concept · Mid-level
Can you describe a challenging case you encountered while evaluating an AI model’s outputs, particularly when spotting hallucinations or checking logical consistency? What issue did you identify, and what steps did you take to address it?
Asked once · last asked Apr 22 · Behavioral · Mid-level
Can you share a specific example from your AI evaluation work where you found a significant issue or inconsistency? How did you pinpoint it, and what impact did it have on the broader project?
Asked once · last asked Apr 22 · Behavioral · Mid-level
When evaluating the logical consistency of an LLM’s output, how do you determine whether its reasoning holds up?
Asked once · last asked Apr 27 · Scenario · Senior
LLM hallucinations, safety and security interview questions
Review questions on hallucinations, conflicting evidence, bias, prompt injection, privacy, access controls, adversarial testing, human approvals, and governance. 96 questions.
Describe a time an AI assistant gave an incorrect answer on a task other than document review, including its output, how you detected the error, and what might have happened if you had trusted it.
Asked in 2 of 291 interviews · last asked Oct 18 · Behavioral · Mid-level
Other ways it was asked
- “Tell me about a recent time an AI assistant gave you an incorrect answer. What did it output, how did you catch the error, and what might have happened if you had missed it?”
- “Tell me about another time an AI assistant gave an incorrect answer on a different task. What did it output, how did you detect the error, and what might have happened if you had trusted it?”
- “Tell me about a time an AI assistant made a mistake or needed your intervention. What did it do, and how did you respond?”
What AI threats and risks should adversarial testing assess?
Asked once · last asked Sep 10 · Concept · Senior
Other ways it was asked
- “Which AI cybersecurity risks should an adversarial testing harness detect, and how would you test for them?”
How would you decide which source to trust when tools return conflicting information?
Asked once · last asked Sep 22 · Scenario · Senior
How would you design a prompt or input data to test whether an AI produces a plausible guess instead of flagging a real discrepancy?
Asked once · last asked Sep 25 · Scenario · Senior
What prompts would you try to make a museum assistant contradict these facts: it opens Tuesday through Sunday, 10 a.m. to 5 p.m., closes Mondays, and charges 200 rupees per ticket?
Asked once · last asked Oct 12 · Scenario · Mid-level
Other ways it was asked
- “What other prompts would you try to make the museum assistant contradict its stated hours or ticket price?”
- “What are some examples of prompts that could make the museum assistant contradict its stated hours or ticket price?”
- “What prompts would you use to test whether a museum assistant can be made to contradict its stated hours of Tuesday through Sunday, 10 a.m. to 5 p.m., Monday closure, or 200-rupee ticket price?”
What is the Chain-of-Verification method for reducing hallucinations in large language models?
Asked once · last asked Feb 5 · Concept · Mid-level
Other ways it was asked
- “What is the Chain-of-Verification method, discussed in a recent paper from Meta, for reducing hallucinations in large language models?”
What settings or deployment choices can prevent data processed by an LLM from being sent externally or made public?
Asked once · last asked Feb 5 · Scenario · Mid-level
Other ways it was asked
- “What settings or deployment choices can prevent an LLM's data from being sent externally or made public?”
What techniques or parameters would you use to reduce hallucinations?
Asked once · last asked Feb 5 · Concept · Mid-level
Can the temperature parameter help reduce hallucinations?
Asked once · last asked Feb 5 · Concept · Mid-level
What benefits might generative AI create, and how should policymakers account for them when designing regulation?
Asked once · last asked Mar 10 · Case · Mid-level
How can AI hallucinations be reduced, and why might you use RAG to address them?
Asked once · last asked Apr 17 · Concept · Senior
How would you address the common problem of hallucinations in models?
Asked once · last asked Apr 20 · Scenario · Mid-level
Generative AI system design interview questions
Work through use-case selection, product requirements, model choices, application architecture, API integration, deployment, monitoring, and production troubleshooting. 65 questions.
How would you design a maintainable, evaluable LLM system?
Asked in 2 of 291 interviews · last asked Sep 9 · Design · Mid-level
Other ways it was asked
- “How would you build an evaluation framework for a production LLM system?”
How would you design a chatbot's architecture, including backend entities, chat flow, streaming, and context management?
Asked in 2 of 291 interviews · last asked Jul 28 · Design · Senior
Other ways it was asked
- “Can you describe the technical architecture of an AI chatbot and explain how you would design and implement it?”
- “How would you design the backend for a chatbot, including its core entities, chat flow, streaming, and context management?”
How would you debug a chatbot that is slow and sometimes gives incorrect answers about product specifications?
Asked once · last asked Jul 16 · Scenario · Mid-level
How would you design and implement an internal low-code/no-code generative AI application to accelerate functional scoping for projects?
Asked once · last asked Sep 20 · Design · Senior
If the application has direct database access, why use an LLM, and how would you design the solution?
Asked once · last asked Sep 25 · Design · Senior
Other ways it was asked
- “If we have direct database access, why use an LLM to query the database, and why make the design complicated?”
How should a company respond when its customer support AI gives a customer an incorrect answer?
Asked once · last asked Apr 20 · Scenario · Mid-level
Other ways it was asked
- “How should a company respond when Zendesk's AI gives a customer an incorrect answer?”
When you integrate an LLM into a CMS to generate on-brand SEO copy, what exact guardrails do you implement to prevent keyword stuffing and voice drift before anything is published?
Asked once · last asked May 28 · Scenario · Senior
Write a Python function that takes a PDF filename, extracts its contents, and summarizes them using an OpenAI model.
Asked once · last asked Jun 12 · Write code · Mid-level
What challenges arise when integrating AI with Playwright to create tests, how would you address them, and what safeguards would you use to limit errors and hallucinations?
Asked once · last asked Jun 18 · Behavioral · Senior
Other ways it was asked
- “What challenges have you encountered when integrating AI with Playwright to create tests, how did you address them, and what safeguards do you use to limit errors and hallucinations?”
How do you debug AI-generated results in API tests and applications?
Asked once · last asked Jun 18 · Scenario · Senior
If you were implementing a generative AI solution for personalized marketing content at scale, what key architectural components would you design, and how would you address data privacy, model bias, and content diversity?
Asked once · last asked Jul 16 · Design · Senior
How do you work with FastAPI and LangChain, and how are they interconnected?
Asked once · last asked Jul 16 · Concept · Mid-level
LLM inference optimization interview questions
Optimization questions cover caching, token budgets, model routing, reasoning effort, serving infrastructure, concurrency, and trade-offs between cost, response time, and quality. 33 questions.
What techniques can reduce LLM API costs, and how would you measure the savings?
Asked in 2 of 291 interviews · last asked Oct 7 · Scenario · Senior
Other ways it was asked
- “What techniques can reduce latency and API costs in an LLM application?”
How would you compare the token efficiency of a transformer-based next-word predictor with modern chatbot models?
Asked once · last asked Apr 11 · Concept · Senior
If a customer needs a RAG system for about 1,000–2,000 users, with 100 concurrent users, how would you determine the LLM's tokens-per-minute configuration?
Asked once · last asked Apr 24 · Scenario · Senior
How would you manage token limits when an AI application runs out of available tokens?
Asked once · last asked Jun 25 · Scenario · Mid-level
What techniques can you use to reduce latency and memory usage in models and AI agents?
Asked once · last asked Jul 16 · Concept · Senior
How do you decide when a task requires a more expensive AI model and when a cheaper model is sufficient?
Asked once · last asked Aug 14 · Scenario · Mid-level
The team is self-hosting an open-weight model and experiencing unacceptable time to first token and throughput bottlenecks during traffic spikes. What inference optimization strategies would you suggest?
Asked once · last asked Aug 20 · Scenario · Senior
What context caching techniques are available, and when would you use them?
Asked once · last asked Aug 20 · Concept · Mid-level
Can you tell me more about how model routing works within the system? Does it involve using several models for different tasks?
Asked once · last asked Aug 22 · Design · Senior
What pricing model does OpenAI use, and what cautions should you consider when using its services?
Asked once · last asked Aug 25 · Concept · Mid-level
When comparing OpenAI models for an AI chatbot, how do you account for concurrency, and what factors guide your choice?
Asked once · last asked Aug 25 · Scenario · Senior
When two users use an AI chatbot at the same time, how are token limits and usage allocated?
Asked once · last asked Aug 25 · Concept · Mid-level
Show 19 more
How would you reduce token waste in an agent system when taking screenshots or observations, especially when delays are involved?
Asked once · last asked Aug 25 · Scenario · Senior
How do you avoid wasting tokens and manage costs when building AI agents, especially when users retry actions repeatedly?
Asked once · last asked Aug 25 · Scenario · Senior
Would you use the same LLM for every task in a pipeline or select different models for different tasks?
Asked once · last asked Sep 2 · Design · Senior
How would you compare LLM latency for an information extraction task and choose a model?
Asked once · last asked Sep 4 · Scenario · Mid-level
How would you change the architecture to reduce LLM costs for a chatbot handling 14,000 incident resolution requests per day?
Asked once · last asked Sep 8 · Design · Senior
How can resolved LLM query data be used to reduce future LLM costs and handle recurring questions internally?
Asked once · last asked Sep 8 · Design · Senior
LLM token costs have doubled. How would you diagnose the increase and optimize the system?
Asked once · last asked Sep 8 · Scenario · Mid-level
What types of AI workloads are vLLM and Ray Serve used for?
Asked once · last asked Sep 24 · Concept · Mid-level
How will you optimize LLM inference?
Asked once · last asked Sep 25 · Scenario · Senior
How do GGUF and quantization support running language models locally?
Asked once · last asked Sep 29 · Concept · Mid-level
How can changing the model and reasoning effort affect LLM output quality, cost, and latency?
Asked once · last asked Oct 6 · Scenario · Mid-level
How would you evaluate trade-offs among cost, latency, and answer quality in an LLM routing system, and what metrics would you track?
Asked once · last asked Oct 6 · Scenario · Senior
Why might an application use both Qwen and Gemini Flash instead of a single model?
Asked once · last asked Oct 11 · Concept · Mid-level
Why is it called a key-value (KV) cache?
Asked once · last asked Jul 16 · Concept · Junior
Does an LLM token have a fixed price, such as two cents?
Asked once · last asked Jul 16 · Concept · Junior
How would you complete a coding task while using a limited number of tokens with an AI coding assistant?
Asked once · last asked Sep 21 · Scenario · Senior
How do you reduce token usage when working with an AI coding assistant such as Amazon Kiro?
Asked once · last asked Sep 21 · Tooling · Senior
What does “juice” refer to in LLM inference?
Asked once · last asked Oct 6 · Concept · Mid-level
What does cache mean in the LLM context?
Asked once · last asked Oct 6 · Concept · Junior
Related interview questions
- AI Engineer interview questions
- AI Trainer interview questions
- Software Engineer interview questions
- Machine Learning Engineer interview questions
- Data Scientist interview questions
- Data Engineer interview questions
- QA Engineer interview questions
- Machine Learning interview questions
- Deep Learning interview questions
- Computer Vision interview questions
- RAG interview questions
- Agentic AI interview questions
- MLOps interview questions
How this list is made
Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.
Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.
The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every Generative AI and LLM interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.
291 interviews, 178 candidates, Feb 2026 to Oct 2026. Updated every month with new interviews.
Or add the Chrome extension