All 54 questions

  1. How do you use AI in testing, and where is human judgment still essential?

    Asked in 5 of 345 interviews · last asked Nov 20 · Behavioral · Mid-level

    Show answer

    I use AI to suggest test cases, draft test data, and summarize logs or bug reports. I check its output against requirements and actual system behavior before using it. Human judgment is still essential for setting priorities, exploring unexpected behavior, judging usability, and deciding whether a risk is acceptable.

    • Treat AI output as a draft, not proof that a feature works.
    • Review suggested tests for missing cases, wrong assumptions, and relevance.
    • Use approved tools and avoid sharing sensitive data without permission.
    • People remain responsible for test decisions and risk acceptance.

    What the interviewer checks: The interviewer wants to see practical uses of AI, clear limits, and human responsibility for testing decisions.

    Other ways it was asked
    • “How are you currently using AI tools in your testing work, or how might you use them in the future?”
    • “How does AI fit into your QA workflow?”
    • “How have you used AI to improve your testing workflow, and where do you think human judgment is still essential in QA?”
  2. How would you test an LLM output when the acceptance criteria and expected behavior are known?

    Asked once · last asked Jul 30 · Scenario · Mid-level

  3. How would you use an LLM to turn Jira requirements into test cases, and how would you choose between a local model and an online model?

    Asked once · last asked Apr 23 · Tooling · Mid-level

  4. What would you include in a data science QA tracker to monitor the quality of outputs?

    Asked once · last asked Jun 15 · Scenario · Mid-level

  5. How would you build a test suite for a chatbot that answers questions about a company's product?

    Asked once · last asked Jul 16 · Scenario · Mid-level

  6. How will you test that a multi-tenant AI CRM never leaks contact data between agency clients?

    Asked once · last asked Jul 16 · Scenario · Senior

  7. How would you build a QA dashboard for tracking AI product quality over time?

    Asked once · last asked Jul 16 · Design · Senior

  8. What is a feedback loop in an AI product, and how can production feedback be used to improve test cases?

    Asked once · last asked Jul 16 · Concept · Mid-level

  9. How would you reduce production defects as a sole tester, and where could AI help?

    Asked once · last asked Jul 16 · Scenario · Senior

  10. What constraints should you account for when testing an AI agent?

    Asked once · last asked Jul 30 · Scenario · Senior

  11. How would you test an AI chatbot using TestNG and TestRail?

    Asked once · last asked Jul 30 · Tooling · Mid-level

  12. How would you use AI coding tools and manual QA during a software migration?

    Asked once · last asked Jul 31 · Scenario · Senior

  13. What should the software testing life cycle for agentic AI systems look like, especially in regulated insurance environments?

    Asked once · last asked Aug 19 · Scenario · Senior

  14. What do you see as the key differences between testing an AI-enabled application versus a traditional application that isn't leveraging AI?

    Asked once · last asked Sep 10 · Concept · Senior

  15. How can AI be integrated into software testing or validation?

    Asked once · last asked Sep 10 · Scenario · Mid-level

  16. How would you validate a real estate chatbot before and after launch?

    Asked once · last asked Sep 21 · Behavioral · Mid-level

    Other ways it was asked
    • “How did you validate a real estate chatbot before and after launch?”
  17. What is your process for using AI to create test reports or summaries?

    Asked once · last asked Sep 23 · Tooling · Mid-level

  18. How do you measure the time saved by using AI for test reporting, and how do you verify its results?

    Asked once · last asked Sep 23 · Scenario · Mid-level

  19. How would you structure the test matrix across agents, versions, settings, and seeds to make comparisons fair and reproducible?

    Asked once · last asked Oct 5 · Scenario · Senior

  20. If the same prompt produces different failures across five runs, how would you document those failures and assess the agent's reliability?

    Asked once · last asked Oct 5 · Scenario · Senior

  21. How would you use Zephyr to estimate the effort required to automate tests with AI?

    Asked once · last asked Oct 8 · Tooling · Senior

  22. What concrete failures would you look for when using LLMs to generate test ideas?

    Asked once · last asked Oct 9 · Behavioral · Senior

    Other ways it was asked
    • “When you tried using LLMs to generate test ideas, what concrete failures did you see most?”
  23. How would you design the CI gate to distinguish an invalid LLM-generated test sequence from a genuine game bug, and how would you classify and triage failures?

    Asked once · last asked Oct 9 · Design · Senior

  24. Would you use a fixed test suite or create new tests each time to check whether an AI agent's guardrails trigger correctly?

    Asked once · last asked Oct 13 · Scenario · Senior

  25. How would you identify gaps in an AI-generated quality control plan for a financial trading platform and account for real-time market volatility's impact on trade settlement accuracy?

    Asked once · last asked Oct 14 · Scenario · Senior

  26. How would you test an AI chatbot and build an evaluation dataset for it?

    Asked once · last asked Oct 22 · Scenario · Senior

  27. What is your practical approach to quality assurance for AI systems?

    Asked once · last asked Oct 26 · Scenario · Mid-level

  28. How would you validate test cases generated by an LLM?

    Asked once · last asked Oct 27 · Scenario · Mid-level

  29. How would you use simulation, synthetic environments, or world models to test an agent before it touches real data?

    Asked once · last asked Nov 2 · Scenario · Senior

  30. How would you define pass/fail criteria for an AI agent using specific, measurable metrics?

    Asked once · last asked Nov 2 · Scenario · Senior

  31. What minimal evidence would you require to trust a failure flagged by an AI agent, such as a reproducible scene, frame index, bone or value differences, screenshots or video, or Unity logs?

    Asked once · last asked Nov 2 · Scenario · Senior

  32. How do you handle a privacy-sensitive image in bug reports or model performance tickets so engineers can reproduce issues without exposing the original, for example redaction, minimization, and access controls?

    Asked once · last asked Nov 5 · Scenario · Senior

  33. When you build a regression stress suite, how do you ensure broad coverage of adaptive adversaries' paraphrases, multi-turn setup, encoding, and tool-chain pivots without overfitting to your known attack templates?

    Asked once · last asked Nov 5 · Scenario · Senior

  34. How would you implement a maintainable versioning strategy for a jailbreak regression suite—including IDs, taxonomy, expected and allowed outcomes, and change labels—so results stay comparable across model and policy updates?

    Asked once · last asked Nov 5 · Design · Senior

    Other ways it was asked
    • “Implement a maintainable versioning strategy for this jailbreak regression suite—IDs, taxonomy, expected and allowed outcomes, and change labels—so results stay comparable across model and policy updates.”
  35. When a policy update changes the expected outcome for some tests, how do you prevent it from looking like a security regression in your dashboards while keeping an auditable trail?

    Asked once · last asked Nov 5 · Scenario · Senior

  36. What quality KPIs would you define for a team where agents write most of the tests?

    Asked once · last asked Nov 6 · Scenario

  37. How would you document a hallucinated statistic for the model or prompt owner so the issue is actionable, and what fields would your defect report include?

    Asked once · last asked Nov 7 · Scenario · Mid-level

  38. What process would you follow when supporting the testing of a chatbot?

    Asked once · last asked Nov 11 · Behavioral · Mid-level

    Other ways it was asked
    • “When you supported testing of your chatbot, what process did you follow?”
  39. How would you ensure comprehensive testing of a chatbot?

    Asked once · last asked Nov 11 · Behavioral · Mid-level

    Other ways it was asked
    • “How did you ensure that testing of your chatbot was done completely?”
  40. How would you generate a comprehensive document of an application's touchpoints, test cases, and domain knowledge using an exploratory agent?

    Asked once · last asked Nov 12 · Design · Senior

  41. How would you test an AI application before releasing it?

    Asked once · last asked Nov 13 · Scenario · Mid-level

  42. If a product changes weekly, how do you keep a human-verified golden set up to date, and who should own it?

    Asked once · last asked Nov 17 · Scenario · Senior

  43. How should QA and development teams divide responsibility for building, integrating, and approving AI agents?

    Asked once · last asked Nov 20 · Scenario · Senior

  44. What should you test when deploying an AI agent to production, and how would you prevent it from taking unintended actions?

    Asked once · last asked Nov 20 · Scenario · Senior

  45. What does a metric such as “40 defects found using AI” indicate about testing effectiveness?

    Asked once · last asked Nov 20 · Senior

  46. How would you determine whether defects found by AI testing had been missed by manual testers?

    Asked once · last asked Nov 20 · Scenario · Senior

  47. What are the three most common failure points in a chatbot that answers product questions?

    Asked once · last asked Jul 16 · Concept · Mid-level

  48. How would you test an AI chatbot, and what tools could you use?

    Asked once · last asked Jul 30 · Tooling · Mid-level

  49. Should test-case creation be included in the technical specification for an AI agent?

    Asked once · last asked Aug 7 · Concept · Mid-level

  50. What does exploratory testing of AI inference workloads on Kubernetes involve?

    Asked once · last asked Sep 24 · Tooling · Mid-level

  51. What reasons might lead a client to test AI use cases themselves?

    Asked once · last asked Oct 22 · Scenario · Mid-level

  52. What kinds of errors can occur in an AI-powered checkout flow?

    Asked once · last asked Nov 5 · Scenario · Mid-level

  53. How long should an exploratory agent be allowed to explore an application?

    Asked once · last asked Nov 12 · Scenario · Mid-level

  54. What should an exploratory agent include in its final report?

    Asked once · last asked Nov 12 · Scenario · Mid-level

More Manual Testing interview questions

How this list is made

Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.

Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.

The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every Manual Testing interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.

345 interviews, 203 candidates, Jan 2026 to Nov 2026. Updated every month with new interviews.