Anonymous session readout

What the room
understood.

18questions revealed

22anonymous votes

55%overall accuracy

By domain

Confidence map

CONTEXT MANAGEMENT & RELIABILITY3 questions · 4 votes

50%

TOOL DESIGN & MCP INTEGRATION3 questions · 4 votes

75%

CLAUDE CODE CONFIGURATION & WORKFLOWS4 questions · 5 votes

60%

AGENTIC ARCHITECTURE & ORCHESTRATION4 questions · 5 votes

20%

PROMPT ENGINEERING & STRUCTURED OUTPUT4 questions · 4 votes

75%

CONTEXT MANAGEMENT & RELIABILITY

A customer support agent searches for a customer named 'Sarah Johnson' and the lookup tool returns three matching accounts with the same name but different addresses and account ages. The agent selects the most recently active account and proceeds with the refund. Later, the customer calls back because the refund was applied to the wrong account. What should the agent have done differently?

2 votes · correct answer D

TOOL DESIGN & MCP INTEGRATION

A customer support team configures an MCP server for their internal CRM system. The agent frequently ignores the CRM MCP tool and instead uses the built-in Grep tool to search local log files for customer information, producing incomplete results. The MCP tool's description reads: 'CRM tool'. What should the team do first?

2 votes · correct answer D

50%

CLAUDE CODE CONFIGURATION & WORKFLOWS

A fintech company requires that all API response payloads are normalised to snake_case before being logged, and that any file write to the src/api/ directory is automatically linted. They want these enforcements to be deterministic and not rely on the model remembering instructions. Which hook configuration achieves both requirements?

2 votes · correct answer C

50%

AGENTIC ARCHITECTURE & ORCHESTRATION

A developer-tool agent has two guardrails: a PreToolUse hook blocking file writes outside the project directory (enforced 100% of the time), and a system prompt instruction 'Always create a backup before overwriting existing files' (followed 88% of the time). A senior engineer wants all guardrails converted to hooks for consistency. What is the correct assessment?

2 votes · correct answer D

PROMPT ENGINEERING & STRUCTURED OUTPUT

Your CI/CD pipeline uses a system prompt with a tool named 'check_security' and also includes instructions that say 'check the security of each function'. Developers report that the model sometimes produces a text-based security analysis instead of calling the check_security tool. What is the most likely cause?

1 votes · correct answer C

100%

CONTEXT MANAGEMENT & RELIABILITY

A synthesis agent receives findings from three subagents: solar investment grew 15% (Bloomberg, January 2024), solar investment grew 22% (IEA, March 2024), and cost-per-watt data. The synthesis agent currently picks the IEA figure because it is more recent. What is the correct approach?

1 votes · correct answer B

100%

CLAUDE CODE CONFIGURATION & WORKFLOWS

Project-level `.claude/CLAUDE.md` says 'use 4-space indentation matching the existing codebase.' A senior architect has 'use 2-space indentation' in their user-level `~/.claude/CLAUDE.md`. In recent sessions the architect's code has come back in 2 spaces and broken the build. The team needs a guarantee that 4-space indentation is applied on every save. What should they do?

1 votes · correct answer A

100%

PROMPT ENGINEERING & STRUCTURED OUTPUT

Your Claude Code agent generates API endpoint implementations. You provide detailed instructions specifying error handling conventions, but the generated code inconsistently handles async errors: sometimes using try/catch, sometimes using .catch(), and sometimes omitting error handling entirely. Adding more detailed instructions did not resolve the inconsistency. What is the most effective next step?

1 votes · correct answer D

100%

TOOL DESIGN & MCP INTEGRATION

A customer support agent calls a payment processing tool to check a refund status. The tool returns: {"status": "success", "data": [], "message": "No refunds found for this transaction"}. The agent's recovery logic treats this as an error and retries the request. What is wrong with the agent's behaviour?

1 votes · correct answer B

100%

AGENTIC ARCHITECTURE & ORCHESTRATION

During the monolith-to-microservices refactoring, each extracted service must follow a consistent Java package naming convention (com.company.service.<service-name>). The team notices that subagents sometimes use inconsistent package names like com.company.app.<service-name> or com.company.<service-name>. A team member proposes adding stronger instructions to the system prompt. What is the correct approach?

1 votes · correct answer C

CLAUDE CODE CONFIGURATION & WORKFLOWS

A team is debugging a complex distributed system issue. The lead developer wants to use Claude Code to explore logs, trace request flows, and form hypotheses without making any changes to the codebase. Partway through the investigation, they identify a one-line fix in a configuration file and want to apply it immediately. What is the optimal workflow?

1 votes · correct answer A

TOOL DESIGN & MCP INTEGRATION

A platform has `search_knowledge_base` ('Searches help articles') and `process_action` ('Handles customer actions like refunds and plan changes'). The agent picks `search_knowledge_base` for 'cancel my subscription' requests. After adding 'subscription cancellations' to `process_action`'s description, it now picks `process_action` for knowledge queries like 'how does cancellation work?'. What is the most effective solution?

1 votes · correct answer A

100%

PROMPT ENGINEERING & STRUCTURED OUTPUT

Your Claude Code agent generates unit tests in the CI pipeline. When given detailed instructions alone, it produces tests with inconsistent assertion styles: sometimes using expect().toBe(), sometimes assert.equal(), and occasionally mixing both in the same file. Adding more detailed instructions about assertion style did not fix the problem. What should you do next?

1 votes · correct answer C

100%

AGENTIC ARCHITECTURE & ORCHESTRATION

A synthesis agent produces a report with several claims that have no source attribution. The web search and document analysis subagents are working correctly and returning well-sourced results. What is the most likely root cause?

1 votes · correct answer A

CONTEXT MANAGEMENT & RELIABILITY

A contract extraction system reports 96% overall accuracy. The team plans to auto-approve all extractions where model confidence exceeds 90%. A pilot reveals party name extraction achieves 99% accuracy but indemnification clause extraction only 71%, even though the model reports high confidence on both. What should they implement before automating?

1 votes · correct answer C

100%

AGENTIC ARCHITECTURE & ORCHESTRATION

A customer support agent must redact credit card numbers from its responses before they reach the customer. The current system prompt instructs the agent to replace card numbers with asterisks, but QA testing reveals that 6% of responses still contain unredacted card numbers. What guardrail approach should the architect use?

1 votes · correct answer C

100%

PROMPT ENGINEERING & STRUCTURED OUTPUT

A CI/CD system prompt defines two review categories with the instructions 'Check for security vulnerabilities in each function' and 'Check for performance issues in each loop'. The model frequently calls `performance_check` for security issues found inside loops, and `security_check` for performance issues in security-sensitive functions. What is the root cause and best fix?

1 votes · correct answer D

CLAUDE CODE CONFIGURATION & WORKFLOWS

The team wants to ensure that every Java file written by Claude Code during refactoring is automatically formatted with the project's Checkstyle rules before being saved. Developers occasionally forget to run the formatter manually. What is the correct hook configuration?

1 votes · correct answer C

100%