The Debugging Test
A broken Python script with three real bugs. Two models asked to find them all, explain what's wrong, and ship a fixed version. Time to working code: very different.
1
The exact prompt we used
prompt.txt — fed identically to both models
The following Python script is supposed to read a list of users, keep only the adults (18 and older), and print each adult's name and age. It has bugs. Find every bug, explain what each one does wrong, and return a corrected version.
```python
from datetime import date
users = [
{"name": "Ada", "birth_year": 1995},
{"name": "Bram", "birth_year": 2010},
{"name": "Cleo", "birth_year": 2008},
{"name": "Dana", "birth_year": 1988},
]
CURRENT_YEAR = date.today().year
def get_adults(people):
adults = []
for person in people:
age = CURRENT_YEAR - person["birth_year"]
if age > 18:
adults.append(person)
return adults
adults = get_adults(users)
print("Adults found:", len(users))
for person in adults:
print(person["name"], "is", person["age"], "years old")
```Don’t take our word for it. Paste this into any AI model and compare your result against the outputs below.
2
What each model wrote — the exact answers, side by side
Claude Opus 4.8Model answer
There are three bugs. The script runs but returns the wrong adults, prints the wrong count, and then crashes.
- Off-by-one in the adult test.
if age > 18drops anyone who is exactly 18, but the goal is “18 and older.” Cleo (born 2008) is 18 and gets wrongly excluded. Useage >= 18. - Wrong count printed.
len(users)is the size of the whole list, not the adults — it should belen(adults). - Missing
agekey →KeyError.ageis computed insideget_adultsbut never saved onto the person, soperson["age"]in the print loop raisesKeyError: 'age'. Store it on the record before appending.
Corrected version:
from datetime import date
users = [
{"name": "Ada", "birth_year": 1995},
{"name": "Bram", "birth_year": 2010},
{"name": "Cleo", "birth_year": 2008},
{"name": "Dana", "birth_year": 1988},
]
CURRENT_YEAR = date.today().year
def get_adults(people):
adults = []
for person in people:
age = CURRENT_YEAR - person["birth_year"]
if age >= 18: # Fix 1: 18 counts as an adult
person["age"] = age # Fix 3: store age so it can be printed
adults.append(person)
return adults
adults = get_adults(users)
print("Adults found:", len(adults)) # Fix 2: count adults, not all users
for person in adults:
print(person["name"], "is", person["age"], "years old")
Model answer — exact text returned
GPT-4oModel answer
The given script has several bugs and mistakes. I'll outline the issues and provide a corrected version of the script:
Bugs and Issues:
- Off-by-One Error in Age Calculation:
- The conditional check
if age > 18:incorrectly filters out users who are exactly 18 years old. It should beif age >= 18:to include users who are 18.
- The conditional check
- Accessing Non-existent Key:
- In the
printstatement,person["age"]is being accessed, but the dictionary does not have anagekey. We need to compute the age again or store it in the dictionary.
- In the
- Misleading Length Display:
- The output
print("Adults found:", len(users))incorrectly prints the total number of users, rather than the number of adults. It should printlen(adults).
- The output
Corrected Version:
from datetime import date
users = [
{"name": "Ada", "birth_year": 1995},
{"name": "Bram", "birth_year": 2010},
{"name": "Cleo", "birth_year": 2008},
{"name": "Dana", "birth_year": 1988},
]
CURRENT_YEAR = date.today().year
def get_adults(people):
adults = []
for person in people:
age = CURRENT_YEAR - person["birth_year"]
if age >= 18: # Corrected to include 18-year-olds
person["age"] = age # Store age in the dictionary
adults.append(person)
return adults
adults = get_adults(users)
print("Adults found:", len(adults)) # Corrected to show the number of adults found
for person in adults:
print(person["name"], "is", person["age"], "years old") # Corrected to use person["age"]
Changes Made:
- Updated the condition to
age >= 18to correctly include those who are exactly 18. - Modified the function to store the calculated age in the dictionary with the key
"age". - Adjusted the print statement to correctly display the number of adults found by using
len(adults). - Ensured the print statement in the loop accesses the correct
"age"key that we added to the dictionary entry.
Model answer — exact text returned
3
What to look for
Try this
Did both models find all three bugs? Which one explained WHY each bug exists — not just what to change?
A new comparison lands every week.
47,000+ business pros get it in their inbox first.