An AI engineering journal published by Eazler Inc.
We check what AI models and tools claim
against primary sources and run logs.
We read papers and official documents ourselves and keep what we verified apart from what we could not.
Reports with our own measurements state the conditions, and every report shows the date its sources were checked.
Featured report
Latest reports
An OpenAI agent breached a statistics portal. 6 of 50 data portals we checked publish a valid security.txt
IPv6-only servers: github.com, the Stripe API and Slack webhooks still have no IPv6 address — 114 endpoints checked
Claude Code silently skipped AGENTS.md when telemetry was disabled — what 2.1.281 fixed and what remains
Next.js OG image RCE (CVE-2026-94545) — why the same vulnerability scores 9.5 and 5.3
Browse by topic
All reports
38 reports · Updated 2026-10-06
Laya scored 47–48% in Korean, Qwen-based 0.8B decision models 71–81% — six Ollaya models tested on matched Korean and English sentences
Six Ollaya decision models on matched Korean and English requests: Laya fell from 56.9% to 47.0% in Korean, while Qwen-based 0.8B models scored 71–81%.
An OpenAI agent breached a statistics portal. 6 of 50 data portals we checked publish a valid security.txt
After Australia said OpenAI took 84 days to report an agent breach, we checked 50 public statistics portals: only 6 publish a valid security.txt.
IPv6-only servers: github.com, the Stripe API and Slack webhooks still have no IPv6 address — 114 endpoints checked
We checked 114 developer endpoints for IPv6 support: most package registries publish AAAA records; github.com, the Stripe API and Slack webhooks do not.
Claude Code silently skipped AGENTS.md when telemetry was disabled — what 2.1.281 fixed and what remains
Up to 2.1.280, Claude Code did not read AGENTS.md when telemetry was off. What 2.1.281 fixed and what still remains.
Next.js OG image RCE (CVE-2026-94545) — why the same vulnerability scores 9.5 and 5.3
CVE-2026-94545 in Next.js next/og scores 9.5 in one advisory and 5.3 in another. What decides it is what processes the generated SVG.
Claude's new tokenizer produced no extra tokens for pure Korean text — the cost after the switch, measured on 10 test texts
Four pure-Korean test texts gained zero tokens under Claude's new tokenizer, while English prose, code and JSON logs grew 1.227–1.420×.
Agent memory is moving to Markdown files — from Karpathy's LLM wiki to Google's OKF 0.2: what the spec covers and what it doesn't
Keeping an agent's long-term memory in a folder of Markdown files: from Karpathy's LLM wiki gist to Google's Open Knowledge Format (OKF) v0.2.
AI agents fabricate values when a failed tool call returns "ok"
When the tool returned status:error, 0% of answers were dishonest; status:ok with a redacted value, 45.3%. Measured on 1,024 items.
Can you trust an AI coding agent that says it reviewed every file?
AI coding agents skipped files they were asked to review in 67.9% of runs, and 80.4% of those runs did not say so.
How a solo developer runs 18 services — mrlatte.net's setup, as its service pages describe it
How one developer runs 18 services at mrlatte.net, as its own pages describe it: Firebase, Astro static builds and a 03:00 crawl scheduler.
ZCode uploaded the entire Git history — what AI coding tools send to the cloud
ZCode packed a 345MB workspace into a 313MB encrypted snapshot for upload, and 86.6% of it was .git. What five other AI coding tools document.
What is Jev? TypeSafe AI's System One model that does not generate text — the claims and their conditions
Jev, a model that does not generate text: the conditions behind TypeSafe's claims of 0.4 seconds per case and 67.8% agreement in its own evals.
AI coding agent usage statistics 2026 — why 90% and 59% are both right
JetBrains says 90%, Stack Overflow says 59%: why AI coding agent usage rates differ from one survey to the next.
GitHub Agentic Workflows guide — how much it restricts coding agents in GitHub Actions
Read-only agent jobs, declared safe outputs and a 1,000-credit ($10) cap per run: what GitHub Agentic Workflows' defaults stop, and what they don't.
Bash alone vs. typed tools for AI agents — adding typed tools to the shell did not raise the score
In a Microsoft study, bash alone beat typed tools and used 19–72% fewer tokens. Half of randomly drawn MCP servers failed to initialize.
Reward hacking in AI agents — how to keep success checks and execution logs out of the agent's control
Reward hacking in AI agents: separate write access to grading criteria, execution logs and shared resources. Checked against METR's incident report.
OpenRouter provider routing guide — provider selection, fallbacks and response validation
In OpenRouter, provider.order is a priority list, not an allowlist. How order, only and allow_fallbacks differ, and what restricting providers costs.
DeepSeek V4.1 Flash migration guide — API model names, pricing and Claude Code setup
DeepSeek V4.1 Flash and the September 14 switch: where the old API model names route, peak and off-peak prices, and Claude Code settings.
Subagents vs agent skills — subagents helped only when skills had input-output contracts
Running agent skills as subagents helped only when the skills had input-output contracts, and total tokens rose 1.3–1.7× (7 models, 64 tasks).
Supply chain security for AI coding tools — why provenance attestation cannot replace code review
Provenance attestation does not replace code review. Google GTIG's DUSTMAKER findings, checked against the SLSA, npm and GitHub documentation.
Agent retries and idempotency — a timeout is not proof of failure
A timeout is not proof of failure. What idempotency keys and checkpoints do and do not guarantee when an agent retries a call that changes external state.
The three layers of a coding agent sandbox — isolation strength was not the problem
Per file system operation, 69.0–98.6% of real command denylists missed a bypass. In coding agent sandboxes, isolation strength was not the problem.
Nothing in OpenTelemetry's GenAI semantic conventions is Stable yet — should you instrument agents now?
Every definition in OpenTelemetry's GenAI semantic conventions is still at Development stability. Should you instrument your agents now?
South Korea's AI Basic Act: no direct administrative fine for failing to label generative AI output — checked against the articles
Labeling generative AI output is a legal duty under Korea's AI Basic Act, but Article 43, which sets administrative fines, does not list it.
Where multi-agent LLM systems fail — 14 failure modes in 1,642 traces
1,642 multi-agent traces, 14 failure modes: the most common was repeating the same step (15.7%); inter-agent misalignment was under a third.
Prompt caching: a narrower scope paid off more — 12 conditions measured
Caching only the system prompt was cheapest for 3 of 4 models and fastest to first token for 3 of 4, across 12 measured agent conditions.
AI-generated code quality — the numbers disagree
Duplication up 81%, yet no general increase in code churn. Why the numbers on AI-generated code quality disagree, and what to measure in your own repo.
Benchmarks and LLM judges are both shaky — how to evaluate AI agents
OpenAI retired SWE-bench Verified: 59.4% of the 138 audited tasks had flawed tests or problem statements. LLM judges show high reliability, low validity.
More context is not the answer — measured context rot and what to do about it
All 18 models Chroma tested got worse as input grew longer, starting well before the context window was full. What to do about context rot.
Coding agent security in 2026 — why prompts cannot stop prompt injection
OWASP's 2026 agentic security report: 28 of the 53 projects it tracks are coding agents. What actually broke, and which defenses fail on their own.
AI agents: capability vs. enterprise scaling
AI agents hit 66.3% on the OSWorld computer-use benchmark (humans: 72.35%), but only about 2 in 10 McKinsey respondents report organization-wide scaling.
AI Max and Advantage+ defaults
Google AI Max defaults, Meta Advantage+ audience limits, the controls advertisers keep, and labeling rules for AI-generated ads, from official docs.
Ollama, Qdrant and Redis in Docker: no auth by default
Ollama, Qdrant and Redis in Docker accept requests without auth, and Compose publishes ports on 0.0.0.0. Bind to 127.0.0.1 and check agent defaults.
Five coding agents: config vs approvals
Claude Code, Codex, GitHub Copilot, Cursor and Gemini CLI compared from official docs: instruction files, skill folders, approval defaults, billing units.
Loop engineering has no standard definition
Loop engineering has no standard definition yet. What two papers and the Claude Code and Codex docs say about stop conditions, verification and budgets.
Agent Skills: six spec fields, tool-specific settings
SKILL.md structure, how Claude Code, Codex and VS Code differ on paths, auto-invocation and tool access, and adoption rules from gain and security studies.
GBrain’s graph-recall figure is an upper bound
GBrain's README checked against its code and the developer's eval reports: the 0.9791 graph recall is an upper bound, and Markdown alone is not a backup.
Running GEO/SEO with an AI agent
93% of app referrals came from ChatGPT while robots.txt blocked that crawler. Measured baselines, the assumptions that broke, and a checklist.
How the English edition works
Most Eazler AI Lab reports are written in Korean first, then translated into English and checked against the Korean original before they appear here: the numbers, dates and sources are the same in both, and each translated report links to its original. A few reports are written in English first; they say so at the top and have no Korean original. Editorial standards and method →