All 74 questions
What is a large language model?
Asked in 15 of 291 interviews · last asked Oct 22 · Concept · Junior
Show answer
A large language model (LLM) is a neural network trained on large amounts of text to learn patterns in language. Most modern LLMs use transformers and generate text by predicting the next token, a word or part of a word. They can answer questions, summarize text, and write code, but can also produce convincing wrong answers.
- Tokens are pieces of text, such as words, word parts, or punctuation.
- During training, the model adjusts its parameters to improve predictions.
- When generating text, it uses the available context to predict one token at a time.
- Fluent output does not guarantee correct facts.
What the interviewer checks: The interviewer is checking whether you can explain what an LLM is, how it generates text, and its limits.
Other ways it was asked
- “What is an LLM?”
- “What are LLMs?”
- “Can you explain how an LLM works?”
What is the Transformer architecture?
Asked in 8 of 291 interviews · last asked Oct 14 · Concept · Junior
Show answer
A Transformer is a neural network architecture that uses attention to learn relationships between tokens. Each layer combines attention with a feed-forward network to build context-aware representations. Position information tells it where tokens occur in the sequence. Transformers can use encoders, decoders, or both, depending on the task.
- Self-attention lets each token use information from other allowed tokens; multiple heads learn different relationships.
- Layers also use residual connections and normalization to support training.
- GPT-style models use decoder blocks with masking to prevent attention to future tokens.
- Training can process token positions in parallel, but text generation usually produces one token at a time.
What the interviewer checks: The interviewer checks whether you understand attention, token order, and the main Transformer variants.
Other ways it was asked
- “What is transformer architecture, and how does it work?”
- “What are transformer architectures, and how do they work?”
- “What is transformer architecture?”
What happens step by step when an LLM receives a prompt and generates a response?
Asked in 4 of 291 interviews · last asked Oct 7 · Concept · Mid-level
Show answer
The system formats the prompt and chat history, then splits them into tokens. The model converts tokens into vectors and processes them through transformer layers to predict next-token probabilities. A decoding rule selects a token, and generation repeats using each new token until a stop condition. Output tokens are decoded into text, which may be streamed.
- Tokens can represent words, word parts, or punctuation.
- Position information tracks token order; attention uses the available context.
- Decoding may choose the highest-probability token or sample; temperature affects sampling.
- Generation stops at an end token or another limit; normal inference does not update model weights.
What the interviewer checks: The interviewer checks whether you understand tokenization, transformer processing, next-token selection, and repeated generation.
Other ways it was asked
- “When you submit a prompt to a chat-based language model, what happens behind the scenes before it returns a response?”
- “Could you describe what happens under the hood when a model like GPT-4 generates a response to a prompt?”
- “Can you briefly explain the LLM inference process step by step?”
What is the difference between LLM and AI?
Asked in 4 of 291 interviews · last asked Oct 22 · Concept · Junior
Show answer
AI is the broad field of building systems that perform tasks such as learning, reasoning, and understanding language. An LLM is a type of AI trained on large amounts of text to understand and generate language. An AI agent is a system that takes actions toward a goal, often using an LLM with tools.
- Every LLM is AI, but not all AI uses LLMs.
- AI also includes systems for image recognition, prediction, and robotics.
- An LLM generates responses; an agent adds a process for choosing and taking actions.
- Agents can use LLMs, but do not require them.
What the interviewer checks: The interviewer checks whether you can distinguish the broad field, a language model, and an action-taking system.
Other ways it was asked
- “What is the difference between agentic AI and an LLM?”
- “What is the difference between an AI agent and an LLM?”
- “Is an LLM a form of AI?”
What are tokens in a large language model?
Asked in 3 of 291 interviews · last asked Oct 1 · Concept · Junior
Show answer
Tokens are the units of text that a large language model processes. A token can be a word, part of a word, punctuation, or whitespace, depending on the tokenizer. The tokenizer converts text into token IDs, which the model uses to read input and generate output.
- A token is not always a whole word.
- Different tokenizers can split the same text differently.
- Context limits and usage costs are often measured in tokens.
What the interviewer checks: The interviewer is checking whether you understand how text becomes model input and why token counts matter.
Other ways it was asked
- “What are tokens?”
- “In the context of large language models, what is a token?”
What are embeddings in an LLM?
Asked in 2 of 291 interviews · last asked Aug 27 · Concept · Junior
Show answer
Embeddings are vectors, or lists of numbers, that represent tokens or text. An LLM gets its starting token embeddings from a lookup table learned during training. For search, a model trained to create text embeddings maps whole texts to vectors, placing similar meanings close together. This may be separate from the LLM generating answers.
- Token embeddings are learned representations, not token IDs.
- An LLM’s layers use context to build richer representations from token embeddings.
- Text embeddings support similarity search, retrieval, and clustering.
- A text embedding model can be based on a language model.
What the interviewer checks: The interviewer is checking whether you understand embeddings and distinguish internal token embeddings from text embeddings used for search.
Other ways it was asked
- “What are embeddings, and how are they created? Does the LLM create them, or does a different model?”
What are the main challenges facing language models, and how would you address them?
Asked in 2 of 291 interviews · last asked Oct 21 · Scenario · Mid-level
Other ways it was asked
- “What challenges do language models present, and how would you address one of them?”
- “What do you see as the key challenges currently impacting the effectiveness of AI systems, particularly within language models?”
What do you know about mixture of experts?
Asked in 2 of 291 interviews · last asked Jul 16 · Concept · Senior
Other ways it was asked
- “What is mixture of experts?”
How would you explain a generative AI system end to end?
Asked in 2 of 291 interviews · last asked Aug 22 · Design · Mid-level
Other ways it was asked
- “How would you explain the flow of an end-to-end generative AI project?”
Can you explain what LLM and RAG are, and what two types of tokens are available in agentic AI?
Asked in 2 of 291 interviews · last asked Jul 17 · Concept · Junior
Other ways it was asked
- “Can you explain LLMs, RAG, and tokens?”
What common mistakes do LLMs make when solving mathematical problems?
Asked once · last asked May 19 · Concept · Mid-level
Other ways it was asked
- “What kinds of errors do LLMs make in the steps of a mathematical solution?”
How do transformer-based next-word predictors use context windows to generate successive predictions?
Asked once · last asked Apr 11 · Concept · Mid-level
Why can one word correspond to multiple tokens, and how would you calculate the number of tokens in a sentence?
Asked once · last asked Jun 12 · Concept · Junior
Is tokenization deterministic, or can it change over time for the same input?
Asked once · last asked Jun 12 · Concept · Mid-level
What is the difference between a chatbot, a RAG application, and an AI agent?
Asked once · last asked Jul 16 · Concept · Mid-level
At a high level, how does a transformer work?
Asked once · last asked Jul 16 · Concept · Junior
What is the difference between decoder-only, encoder-only, and encoder-decoder architectures, and which type is GPT?
Asked once · last asked Jul 16 · Concept · Mid-level
How does reasoning in LLMs work, and how is it implemented?
Asked once · last asked Jul 16 · Concept · Senior
What does the temperature parameter control in an LLM, and how does it affect the output?
Asked once · last asked Jul 16 · Concept · Junior
If a chatbot uses a knowledge base and you change temperature from 0.2 to 1.0, how will that affect its responses?
Asked once · last asked Jul 16 · Scenario · Mid-level
Is there any difference between the normal OpenAI API and Azure OpenAI?
Asked once · last asked Jul 16 · Concept · Mid-level
How do the OpenAI API and Azure OpenAI compare in latency and response quality?
Asked once · last asked Jul 16 · Concept · Mid-level
How does a model have 100 billion-plus parameters but only activate a small number at a time?
Asked once · last asked Jul 16 · Concept · Senior
Can you explain the difference between RAG and tokens, and also explain the two types of tokens?
Asked once · last asked Jul 17 · Concept · Junior
Can you explain the two types of tokens in an LLM and the overall concept of tokens?
Asked once · last asked Jul 17 · Concept · Junior
How does a model accessed through the Gemini API process a PDF document?
Asked once · last asked Jul 17 · Concept · Mid-level
What is a transformer, and what are its building blocks?
Asked once · last asked Jul 20 · Concept · Mid-level
What are Claude's main limitations, and how do its strengths and weaknesses for AI generation compare with Gemini?
Asked once · last asked Jul 24 · Concept · Mid-level
How does ChatGPT work internally when a user asks a question?
Asked once · last asked Jul 25 · Concept · Mid-level
What is the difference between GPT-3.5, GPT-4, and open-source LLMs?
Asked once · last asked Jul 25 · Concept · Mid-level
How is a text prompt converted into numerical representations that a language model can process?
Asked once · last asked Jul 31 · Concept · Junior
Can you tell me the architecture of LLMs?
Asked once · last asked Aug 7 · Concept · Junior
How do you link tokens, embeddings, and transformers together?
Asked once · last asked Aug 7 · Concept · Junior
How would you compare human attention with the attention mechanism in AI?
Asked once · last asked Aug 12 · Concept · Mid-level
Can you explain how forward propagation and backward propagation work in a transformer model?
Asked once · last asked Aug 13 · Concept · Mid-level
Why might the same prompt yield different responses if asked twice?
Asked once · last asked Aug 19 · Concept · Mid-level
What size of LLM do you prefer, and how do Claude and Gemini differ?
Asked once · last asked Sep 4 · Concept · Mid-level
Can you explain the concept of transformer architecture and why it's become such a significant advancement in the field of deep learning?
Asked once · last asked Sep 4 · Concept · Junior
Explain multi-head attention.
Asked once · last asked Sep 4 · Concept · Junior
What is an LLM, and how can AI support agent-based applications?
Asked once · last asked Sep 5 · Concept · Mid-level
How would you explain the differences between generative AI, AI agents, and agentic AI to a layperson?
Asked once · last asked Sep 16 · Concept · Mid-level
What kinds of tasks reveal limitations in ChatGPT, such as errors or missing context?
Asked once · last asked Sep 21 · Concept · Mid-level
What is the difference between Snowflake Cortex, ChatGPT, and an LLM?
Asked once · last asked Sep 21 · Concept · Mid-level
Other ways it was asked
- “What is the difference between Cortex, ChatGPT, and an LLM?”
What is the difference between Snowflake Cortex and OpenAI's offerings?
Asked once · last asked Sep 21 · Concept · Mid-level
What advantages do embeddings offer over one-hot encoding or a basic lookup table for representing words in language models?
Asked once · last asked Oct 1 · Concept · Junior
How do tensor shapes, dtypes, and device placements change from tokenizer output through a model's forward pass to logits, including batch and sequence dimensions and padding or attention masks?
Asked once · last asked Oct 5 · Concept · Senior
What is attention in a transformer exactly?
Asked once · last asked Oct 7 · Concept · Mid-level
What are the query, key, and value vectors in transformer attention, and how are they used?
Asked once · last asked Oct 7 · Concept · Mid-level
How would you compare the strengths and weaknesses of at least two locally run AI models, and when would you choose one over the other?
Asked once · last asked Oct 15 · Concept · Senior
Other ways it was asked
- “How do you compare the strengths and weaknesses of at least two local AI models you have used, and when do you choose one over the other?”
What is self-attention?
Asked once · last asked Aug 2 · Concept · Junior
What are tokenization and embeddings?
Asked once · last asked Apr 28 · Concept · Mid-level
Why is context needed for an LLM?
Asked once · last asked Jun 12 · Concept · Junior
How would you convert a sentence into tokens?
Asked once · last asked Jun 12 · Tooling · Junior
What is Claude?
Asked once · last asked Jun 15 · Concept · Junior
Can you explain LLMs in basic wording, as simple as you can?
Asked once · last asked Jul 16 · Concept · Junior
What is deep learning, and how is an LLM related to it?
Asked once · last asked Jul 16 · Concept · Junior
A thousand words is how many tokens?
Asked once · last asked Jul 16 · Concept · Junior
What are tokens in an LLM, and what are the two types of prompts?
Asked once · last asked Jul 17 · Concept · Junior
Why must text input be converted into numerical representations before a language model processes it?
Asked once · last asked Jul 31 · Concept · Junior
What are LLMs, generative AI, and agentic AI, and what are some practical applications?
Asked once · last asked Aug 7 · Concept · Junior
What do tokens, embeddings, and transformers do?
Asked once · last asked Aug 7 · Concept · Junior
What is the maximum context window size supported by a Claude model?
Asked once · last asked Aug 13 · Concept · Mid-level
What is the difference between ChatGPT and cloud computing?
Asked once · last asked Sep 14 · Concept · Junior
What are transformers in machine learning?
Asked once · last asked Sep 15 · Concept · Junior
What are Transformers?
Asked once · last asked Sep 16 · Concept · Junior
In an LLM application, what do tokens refer to when dividing content into chunks?
Asked once · last asked Sep 22 · Concept · Mid-level
What is context in a large language model?
Asked once · last asked Oct 1 · Concept · Junior
What is the difference between input and output tokens?
Asked once · last asked Oct 1 · Concept · Junior
What does temperature control in a language model?
Asked once · last asked Oct 6 · Concept · Junior
What types of content can ChatGPT generate?
Asked once · last asked Oct 9 · Concept · Junior
What closed-source AI models are available on the market?
Asked once · last asked Oct 13 · Concept · Junior
What is a VLM?
Asked once · last asked Oct 14 · Concept · Junior
What feature does self-attention add to Transformer architecture?
Asked once · last asked Oct 14 · Concept · Junior
Who are the globally recognized competitors in the LLM market?
Asked once · last asked Oct 22 · Concept · Junior
More Generative AI and LLM interview questions
- All Generative AI and LLM interview questions
- Prompt Engineering Interview Questions
- RAG Interview Questions
- AI Agents and Context Management Interview Questions
- LLM Evaluation and Benchmarking Interview Questions
- LLM Hallucinations, Safety and Security Interview Questions
- Generative AI System Design Interview Questions
How this list is made
Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.
Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.
The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every Generative AI and LLM interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.
291 interviews, 178 candidates, Feb 2026 to Oct 2026. Updated every month with new interviews.
Or add the Chrome extension