LLMs Fail Simple Task: Matching HTML5 Elements and TLDs

The author tested three commercially available LLMs on a seemingly simple task: identifying which top-level domains (TLDs) share names with valid HTML5 elements. The results were disappointing, with all three models producing inaccurate or incomplete results, highlighting the limitations of current LLMs even on tasks requiring basic comparison skills. The accuracy, it seems, is heavily dependent on the user's familiarity with the subject matter.
Read more