Prompt Engineering Assessments: Testing Applied AI Skills

Executive Summary

Prompt engineering is the core AI fluency skill, yet most organizations don't test it systematically. This guide provides performance-based assessment frameworks for evaluating prompt quality, iteration strategy, and output evaluation across common workplace tasks. Learn to design realistic challenges with clear scoring criteria that predict real-world AI effectiveness.

What you'll learn:

• The 5 dimensions of prompt engineering competency to assess
• Task design for email drafting, data analysis, content creation, and problem-solving
• Scoring rubrics for prompt clarity, context provision, and iteration quality
• Time constraints that simulate real work pressure
• Common failure patterns and diagnostic insights

Expected outcome: Validated prompt engineering assessments that identify who can actually use AI effectively in their daily work.

Why Prompt Engineering Can't Be Tested with Multiple Choice

Knowing prompt engineering concepts ≠ Being able to write effective prompts

Knowledge test example:

Q: Which technique improves prompt clarity? A) Chain-of-thought reasoning ✓

What this measures: Recall

What it doesn't measure: Can they apply chain-of-thought in real tasks?

Performance assessment example:

Task: You have 10 minutes. Use AI to analyze this sales data (30 rows) and identify the top 3 concerns for the executive team. Deliverables: (1) Your prompts, (2) AI output, (3) Final 3-bullet summary.

What this measures: Applied skill under realistic constraints

The gap: Someone can pass the knowledge test but fail the performance task.

The fix: Test what people DO, not what they know.

The 5 Dimensions of Prompt Engineering Competency

Effective assessments measure all five:

1. Prompt Clarity

Definition: Instructions are unambiguous and complete

Good example:

"Summarize this 20-page report in 200 words. Focus on: (1) key findings, (2) recommendations, (3) implementation timeline. Use bullet points. Avoid technical jargon—audience is non-technical executives."

Bad example:

"Summarize this."

Assessment criteria:

• Task clearly stated
• Output format specified
• Audience/purpose provided
• Constraints defined (length, tone, structure)

2. Context Provision

Definition: Relevant background information included in prompt

Good example:

"You are an HR advisor for a 500-person tech company. Draft a policy on remote work. Company culture values flexibility but has concerns about collaboration. Include: eligibility, days/week allowed, technology requirements, performance expectations."

Bad example:

"Write a remote work policy."

Assessment criteria:

• Role/perspective provided ("You are...")
• Situational context given
• Constraints/considerations mentioned
• Key details from source material incorporated

3. Iteration Strategy

Definition: Systematic refinement of prompts based on output quality

Effective iteration:

1. Initial prompt: Generic output
2. Add specificity: Better but still generic examples
3. Provide actual data/context: Now output is useful

Ineffective iteration:

1. Initial prompt: Poor output
2. Rephrase same vague request: Still poor
3. Give up or accept unusable output

Assessment criteria:

• Each iteration adds new information (not just rephrasing)
• Clear improvement trajectory across attempts
• Stops iterating when output meets requirements (doesn't over-engineer)

4. Output Evaluation

Definition: Critically assesses AI output for accuracy, relevance, and appropriateness

Strong evaluation:

• Catches factual errors
• Identifies missing information
• Recognizes tone mismatches
• Makes appropriate edits before using

Weak evaluation:

• Accepts AI output at face value
• Doesn't verify facts
• Misses obvious errors
• Uses AI output verbatim without review

Assessment criteria:

• Identified all major errors in AI output
• Made necessary edits
• Final submission is accurate and appropriate
• Didn't introduce new errors during editing

5. Efficiency

Definition: Achieves quality output in reasonable time with minimal wasted effort

Efficient workflow:

• Clear initial prompt (reduces iterations needed)
• Targeted refinements
• Completes task within time limit
• 1-3 iterations typical

Inefficient workflow:

• Vague initial prompt (forces many iterations)
• Random trial-and-error
• Runs out of time or barely finishes

• 5 iterations

Assessment criteria:

• Time to completion
• Number of prompt iterations
• Ratio of iteration effort to output quality improvement

Sample Assessment: Email Response Task

Scenario

You received this customer complaint:

"I've been waiting 3 weeks for a refund after returning a defective product. Your customer service keeps saying 'it's being processed' but I see no progress. This is completely unacceptable. I'm posting about this experience on social media if I don't get my money back by Friday."

Task

Use AI to draft a response email that:

1. Acknowledges the customer's frustration appropriately
2. Explains the situation (you have context: refund was submitted 2 weeks ago, processing takes 10-14 business days, should arrive within 3 days)
3. Offers appropriate compensation (you can offer: expedited processing, 15% credit on next purchase, or priority customer service line)
4. Maintains professional, empathetic tone
5. Prevents social media escalation without sounding defensive

Time Limit

10 minutes

Deliverables

1. All prompts you used (copy-paste each one)
2. AI outputs (for each prompt)
3. Final edited email (ready to send to customer)

Scoring Rubric

Dimension 1: Prompt Quality (0-5 points)

Dimension 2: Output Evaluation (0-5 points)

Dimension 3: Efficiency (0-5 points)

Total: 15 points possible
Pass threshold: ≥11 points (73%)

Sample Assessment: Data Analysis Task

Scenario

You're preparing for Monday's executive team meeting. The CEO wants insights on Q4 sales performance.

Task

Use AI to analyze this sales data (CSV with 40 rows: Product, Region, Q4 Revenue, Q3 Revenue, Target) and create a 3-bullet executive summary highlighting:

1. Overall performance vs. target
2. Top performers (products or regions)
3. Areas of concern

Data Provided

[CSV file with columns: Product, Region, Q4_Revenue, Q3_Revenue, Q4_Target]

Time Limit

12 minutes

Deliverables

1. Prompts used
2. AI analysis output
3. Final 3-bullet executive summary (50-75 words total)

Scoring Rubric

Dimension 1: Prompt Quality (0-5)

Dimension 2: Accuracy (0-5)

Dimension 3: Relevance (0-5)

Total: 15 points
Pass threshold: ≥11 points

Common Failure Patterns

Use assessment results to diagnose skill gaps:

Pattern 1: "The Lazy Prompter"

Symptoms:

• Minimal initial prompts ("Summarize this")
• Many iterations trying to fix poor initial prompt
• Low scores on Prompt Quality dimension
• Often runs out of time

Diagnosis: Hasn't learned to invest effort upfront in detailed prompts

Intervention:

• Teach "frontload specificity" principle
• Show examples of good vs. bad initial prompts
• Practice exercises: "Add 5 details to this prompt"

Pattern 2: "The Uncritical Acceptor"

Symptoms:

• Accepts AI output with minimal review
• Doesn't catch factual errors
• High scores on Efficiency, low on Output Evaluation
• Final submissions contain obvious mistakes

Diagnosis: Over-trusts AI; lacks critical evaluation skills

Intervention:

• "Spot the error" exercises with flawed AI outputs
• Emphasize "AI as draft, human as editor" workflow
• Checklist for output review

Pattern 3: "The Perfectionist"

Symptoms:

• Excessive iterations (>5)
• Over-edits AI output
• High quality but low Efficiency scores
• Often doesn't finish in time

Diagnosis: Doesn't recognize "good enough"; wastes time on marginal improvements

Intervention:

• Teach "80/20 rule" for AI outputs
• Time management practice ("Stop at 70% complete")
• Examples of diminishing returns in iteration

Pattern 4: "The Context Skimper"

Symptoms:

• Prompts lack necessary background
• Generic outputs that don't fit scenario
• Moderate scores across all dimensions
• Never achieves excellence

Diagnosis: Doesn't understand importance of context

Intervention:

• Side-by-side comparison: output from minimal vs. context-rich prompts
• Template: "You are [role]. Your goal is [goal]. Context: [details]."
• Practice adding 3 layers of context to generic prompts

Key Takeaways

1. Performance-based assessment is essential—prompt engineering cannot be measured with knowledge tests alone.
2. Test all 5 dimensions: Clarity, Context, Iteration, Evaluation, Efficiency—weakness in any dimension undermines overall effectiveness.
3. Realistic constraints matter: Time limits and authentic tasks reveal how people actually work under pressure.
4. Failure patterns are diagnostic: Use scores to identify specific skill gaps and target interventions.
5. Rubrics must be behaviorally anchored: Descriptive criteria reduce scorer subjectivity and improve consistency.

Next Steps

This week:

1. Choose 3 common workplace tasks for your organization (email, analysis, content, etc.)
2. Draft scenarios and prompts for each task
3. Create scoring rubrics with behavioral anchors

This month:

1. Pilot assessments with 20 employees
2. Validate inter-rater reliability (2 scorers, same submissions)
3. Analyze failure patterns to refine rubrics

This quarter:

1. Deploy prompt engineering assessment as part of AI Fluency certification
2. Track scores over time to measure skill development
3. Use diagnostic insights to improve training

Partner with Pertama Partners to design prompt engineering assessments that accurately measure real-world AI capability and diagnose specific skill development needs.