The tech world is obsessed with labeling AI agents as "good" or "bad," but this binary framing is actively hindering effective tool selection. A recent essay on Hacker News, "What does good AI mean?", argues that the word "good" is not a universal adjective but a context-dependent judgment that changes based on the specific role being evaluated. For developers, this means that a model praised for generating boilerplate code in a weekend hackathon is not automatically "good" for maintaining a production-grade microservice architecture.

The Philosophical Flaw in AI Hype

The article draws on philosopher Bernard Williams’ concept of "good" to illustrate why cross-context comparisons fail. Williams notes that while "red" is a consistent property (a red car is a red vehicle), "good" is not (a good soccer player is not necessarily a good human being). Applied to AI, this means an agent that is "good" at writing essays is not inherently "good" at debugging race conditions. The properties that make a tool effective in one domain—like speed or verbosity—do not translate to domains requiring precision, security, or long-term maintainability.

Coding Is Not Just Generating Text

Developers often conflate code generation with programming. The source material emphasizes that true programming involves clarifying requirements, understanding legacy systems, coordinating with stakeholders, and reasoning about trade-offs. An AI agent might generate a patch in seconds, satisfying the metric of speed, but if that patch introduces subtle bugs that a human reviewer misses, the agent failed the actual task of maintaining system integrity. The article warns that measuring success solely by output volume or latency ignores the critical human elements of software engineering: persuasion, disagreement, and knowledge transfer.

Context Is the Only Metric That Matters

The piece argues that "good" only has meaning when shared context exists, much like friends recommending a restaurant. The danger arises when vendors claim their tool is "good" at a specific task, and teams assume this quality transfers to their completely different environment. A prototype script that works with zero verification is acceptable for a throwaway demo but catastrophic for a system serving millions. Without defining the environment, the user, and the acceptable failure conditions, any claim of AI "goodness" is marketing noise rather than engineering data.

Key Takeaways

  • "Good" is not a transferable adjective; an AI good at drafting is not automatically good at debugging.
  • Programming involves human-centric tasks like requirement clarification and stakeholder coordination, which are often ignored in AI benchmarks.
  • Vendor claims of AI performance often lack the context of the specific environment and success criteria of the buyer.
  • Effective evaluation requires defining specific baselines, acceptable failure modes, and the exact nature of the task before judging the tool.

The Bottom Line

Stop asking if an AI is good. Ask if it is good at the specific, narrow task you are trying to automate, and be ruthless about defining what failure looks like.