AI
pro AI in coding
- quickly solves a problem
- works in unfamiliar coding areas, concepts, languages
- suggests unknown functions and approaches
contra AI in coding
- doesn’t always follow best practices
- is hard to maintain and has security issues (see below)
- needs experienced senior developers to catch issues
- lowers debugging skills
- reduces learning to think in a structured manner (see below)
solution
- learn basic programming yourself
- develop good coding habits from the beginning
- make mistakes, debug, struggle, research, think, build problem solving skills
- implement AI-assisted code critically
AI in learning
don’t bring a forklift to a fitness studio to lift weights
- Outsourcing work to AI hurts long-term growth of learning skills
- AI stunts intellectual development as engineer + reasoning about complex concepts
- Chatbots threaten critical thinking and cognitive development skills
- Students using AI lose critical skills and become dependent on unreliable tools
- LLM-assisted writing lowers cognitive activity, engagement, memory recall, ownership
detailed problems
(sorted by publication time, hoping that some of the older problems get solved over time)
- 2026-07 when OpenAI guardrails are circumvented, agents use zero-day-exploits
- 2026-07: developers have decreasing trust in AI coding tools output
- 2026-07 jailbreaking still is a problem and may remain one forever
- 2026-06: AI has faulty medical reasoning and hallucinations, uses inappropriate shortcuts
- 2026-06: AI support system used to hijack over 20,000 accounts
- 2026-06: OpenAI and Anthropic do not trust their own AI and hire humans instead
- 2026-05: real business impact of AI is low but it de-skills engineers and erodes career growth
- 2026-05: autonomous agents are vulnerable to tool-chaining attacks
- 2026-05: AI code looks correct but isn’t, engineers don’t trust it but don’t check it either
- 2026-04: AI-generated code gets rewritten/reverted at higher rates than human code
- 2026-04: AI agents still destroy production databases and cannot be trusted
- 2026-04: incidents per pull request and bugs per developer are rising at increasing pace
- 2026-04: sycophantic AI reduces healthy skepticism, AI agents need clear specs
- 2026-04: as a society, we may forget how to code and have a hard time re-learning
- 2026-04: widespread AI usage creates additional maintenance load on senior developers
- 2026-04: Claude code itself has “corner-cutting and wilful blindness at scale”
- 2026-04: ChatGPT cannot reliably keep track of time
- 2026-03: AI code caused Amazon outage
- 2026-03: AI adoption decreases delivery speed and causes a drop in system stability
- 2026-03: Medical X-ray benchmark scores are worthless (top rank without using an image)
- 2026-03: measured productivity gains are lower than perceived
- 2026-03: Gen-AI-assisted changes cause outages, decrease maintainability
- 2026-02: AI adoption stalls, performance gains plateau, AI fails to deliver durable impact
- 2026-01: engineers who lean on AI to write code understand it less well
- 2025-12: AI-generated code contains more bugs and errors than human output
- 2025-11: With increasing complexity, it gets harder to check if AI output is correct
- 2025-11: AI output lowers skepticism during reviews, stunts junior devs knowledge gain
- 2025-11: LLMs are bad at conceptual programming
- 2025-10: AI-written code causes security breaches, hard to track down vulnerabilites
- 2025-09: 95% of companies see no return on AI investment (reasons why)
- 2025-09: AI assistants actually slow down software development (reproducible)
- 2025-08: prompt injection attacks remain a major concern
- 2025-08: Vibe coding without engineering discipline creates unmaintainable code
- 2025-08: Coding agents create cybersecurity vulnerabilities in numerous ways
- 2025-07: AI usage degrades our ability to catch bugs, creates a reliability problem
- 2025-07: AI coding tools chase phantom bugs and destroy real production databases
- 2024-12: AI coding collapses under real-world pressure, requires expertise for final 30%
- 2024-05: Coders with AI assistance write more security leaks (with less awareness of them)