ENGINE NOTES / OCTOBER 19, 2026

Claude Code built-in

WebSearch returns links and a short summary; WebFetch hands the agent a note from a second model.

48 questions, one search each

One search per question
Engine	Versions / 30	This week / 10	Traps / 8	Total / 48	Median seconds	Median tokens	US$ / 1,000
Claude Code built-in	0	4	1	5	Not measured	565	$10.00

Three coding jobs

Three coding jobs, three runs each
Engine	Tests passed / 66	Median seconds per job	Median page tokens in context	US$ web calls per job
Claude Code built-in	66	35.9	1,983	$0.01

Every question

Open a cell to read its ranked results. Right and wrong refer to the agent’s answer.
Question	Claude Code built-in
latest version of Astroversions · Right answer: 7.4.1	Wrong Top five results # Software Updatesdocs.freeflysystems.com · Date not supplied Upgrade Astrodocs.astro.build · Date not supplied Upgrade Astrodocs.astro.build · Date not supplied Upgrade Astrodocs.astro.build · Date not supplied Upgrade Astrodocs.astro.build · Date not supplied
latest version of Cloudflare Wranglerversions · Right answer: 4.151.2	Wrong Top five results github cloudflare/wrangler-legacy v1.6.0-rc.0newreleases.io · Date not supplied github cloudflare/wrangler-legacy v1.5.0-rc.0newreleases.io · Date not supplied developers.cloudflare.comdevelopers.cloudflare.com · Date not supplied 2025 03 13 wrangler v4developers.cloudflare.com · Date not supplied 2025 03 13 wrangler v4developers.cloudflare.com · Date not supplied
latest version of Expo SDKversions · Right answer: 58.0.16	Wrong Top five results > ## Documentation Indexmoengage.com · Date not supplied docs.expo.devdocs.expo.dev · Date not supplied docs.expo.devdocs.expo.dev · Date not supplied Jan 18, 2024 byexpo.dev · Date not supplied CONTACT USpagepro.co · Date not supplied
latest version of openai npm packageversions · Right answer: 7.39.0	Wrong Top five results summary.ecosyste.mssummary.ecosyste.ms · Date not supplied npm openai 4.92.0newreleases.io · Date not supplied snyk.iosnyk.io · Date not supplied ecosystem.code.gouv.frecosystem.code.gouv.fr · Date not supplied depscope.devdepscope.dev · Date not supplied
latest version of Claude Codeversions · Right answer: 2.1.304	Wrong Top five results # How to Check and Update Your Claude Code Versionfast.io · Date not supplied What's new in Claude Codehavoptic.com · Date not supplied Claude needs to update and doesn't work when I doforum.cursor.com · Date not supplied AI & Agentsfast.io · Date not supplied version checktessl.io · Date not supplied
latest version of Vercel AI SDKversions · Right answer: 7.0.145	Wrong Top five results ai@3.1.1 vulnerabilitiessecurity.snyk.io · Date not supplied AI SDK by Vercel - The AI Toolkit for TypeScript and JavaScriptnpmjs.com · Date not supplied ai-sdk.devai-sdk.dev · Date not supplied vercel ai sdkinfoq.com · Date not supplied Writing/The Vercel AI SDK: A worthwhile investment in bleeding edge GenAIzackproser.com · Date not supplied
latest version of TanStack Queryversions · Right answer: 5.104.5	Wrong Top five results TanStack Query v5: A Leap Forward in Simplicity and Functionalitythisdot.co · Date not supplied introducing tanstack query v5 a leap forward in simplicity and functionalitylabs.thisdot.co · Date not supplied github TanStack/query v5.8.8newreleases.io · Date not supplied github TanStack/query v4.40.0newreleases.io · Date not supplied github TanStack/query v5.8.4newreleases.io · Date not supplied
latest version of FastAPIversions · Right answer: 0.150.0	Wrong Top five results data.safetycli.comdata.safetycli.com · Date not supplied fastapi restfulpyup.io · Date not supplied www.pepy.techpepy.tech · Date not supplied pepy.techpepy.tech · Date not supplied FastAPI 0.78.0: A Deep Dive Into This Python Web Frameworkgrandavehousing.calpoly.edu · Date not supplied
latest version of uvversions · Right answer: 0.13.2	Wrong Top five results safetycli.comdata.safetycli.com · Date not supplied pyup.iopyup.io · Date not supplied pepy.techpepy.tech · Date not supplied advanced version configurationgitea.psi.ch · Date not supplied data.safetycli.comdata.safetycli.com · Date not supplied
latest version of Hugging Face Transformersversions · Right answer: 5.20.2	Wrong Top five results KI für Ihr Unternehmen – Jetzt Demo buchenmind-verse.de · Date not supplied releasealert.devreleasealert.dev · Date not supplied Product Launchtransformers librarymultimodalhuggingfacemlopsletsdatascience.com · Date not supplied huggingface.curated.cohuggingface.curated.co · Date not supplied releasealert.devreleasealert.dev · Date not supplied
latest version of LiteLLMversions · Right answer: 1.106.0	Wrong Top five results Release Notesdocs.litellm.ai · Date not supplied github BerriAI/litellm v1.48.5-stablenewreleases.io · Date not supplied github BerriAI/litellm v1.39.5-stablenewreleases.io · Date not supplied github BerriAI/litellm 1.34.35-stablenewreleases.io · Date not supplied github BerriAI/litellm v1.48.7-stablenewreleases.io · Date not supplied
latest version of Electronversions · Right answer: 44.9.0	Wrong Top five results project news electron releases a new versionopenjsf.org · Date not supplied Zum Hauptteil springenelectronjs.org · Date not supplied endoflife.dateendoflife.date · Date not supplied openjs foundation incubating project electron releases a new versionopenjsf.org · Date not supplied project news electron ships v12openjsf.org · Date not supplied
latest version of ESLintversions · Right answer: 10.13.0	Wrong Top five results DEV Communitydev.to · Date not supplied ESLint v8.x reached end-of-life on 2024-10-05 and is no longer maintained. Upgrade or consider long-term support optionseslint.org · Date not supplied News and Updateseslint.org · Date not supplied ESLint v8.x reached end-of-life on 2024-10-05 and is no longer maintained. Upgrade or consider long-term support optionseslint.org · Date not supplied ESLint v9.x reached end-of-life on 2026-08-06 and is no longer maintained. Upgrade or consider long-term support optionseslint.org · Date not supplied
latest version of Turborepoversions · Right answer: 2.11.9	Wrong Top five results turborepo.orgTurborepo.org · Date not supplied turbo.buildturbo.build · Date not supplied yarn turbo 1.10.3newreleases.io · Date not supplied github vercel/turborepo v2.10.5-canary.1newreleases.io · Date not supplied turborepo releasesreleases.sh · Date not supplied
latest version of Nuxtversions · Right answer: 4.6.2	Wrong Top five results nuxt.comnuxt.com · Date not supplied Visit Nuxt 2 NES Home Pagedocs.herodevs.com · Date not supplied nuxt@3.4.1 vulnerabilitiessecurity.snyk.io · Date not supplied nuxt vulnerabilitiessnyk.io · Date not supplied title \| description \| navigation \|github.com · Date not supplied
latest version of Next.jsversions · Right answer: 16.4.2	Wrong Top five results Next.js 16.2 Released: What's New for Developers in April 2026hashnode.com · Date not supplied www.wildvisualizer.comwildvisualizer.com · Date not supplied Next.js: Releases, patches & end-of-lifeversio.io · Date not supplied Next.js 16.2 Released: What's New for Developers in April 2026hashnode.com · Date not supplied nextjs.orgnextjs.org · Date not supplied
latest version of Vitestversions · Right answer: 5.0.5	Wrong Top five results Vitest v0.23.0wallabyjs.com · Date not supplied Vitest v0.30.0wallabyjs.com · Date not supplied Support for Vitest v3.2.x+wallabyjs.com · Date not supplied Support for Vitest v1.4.x+wallabyjs.com · Date not supplied Support for Vitest v3.1.x+wallabyjs.com · Date not supplied
latest version of React Nativeversions · Right answer: 0.88.0	Wrong Top five results Back to all blogpoststsh.io · Date not supplied Why Is It Important to Always Run The Latest Version of React Native?callstack.com · Date not supplied www.mintlify.commintlify.com · Date not supplied microsoft.github.iomicrosoft.github.io · Date not supplied React Native versionsreactnative.dev · Date not supplied
latest version of Anthropic TypeScript SDKversions · Right answer: 0.135.0	Wrong Top five results anthropic sdk typescriptreleasealert.dev · Date not supplied cdn.jsdelivr.netcdn.jsdelivr.net · Date not supplied tessl.iotessl.io · Date not supplied Toggle navigationjsdelivr.com · Date not supplied anthropic ai sdkskills.cat · Date not supplied
latest version of OpenAI Codex CLIversions · Right answer: 0.163.0	Wrong Top five results github.comgithub.com · Date not supplied mapping openais full ai suite after codex cli debutvisualstudiomagazine.com · Date not supplied codex clidocs.digitalocean.com · Date not supplied Primary navigationdevelopers.openai.com · Date not supplied OpenAI Codex CLI Changelog & Release Noteshavoptic.com · Date not supplied
latest version of pnpmversions · Right answer: 12.14.0	Wrong Top five results data.safetycli.comdata.safetycli.com · Date not supplied How To Upgrade Pnpmsquash.io · Date not supplied How To Check Pnpm Versionsquash.io · Date not supplied npm pnpm 11.16.0newreleases.io · Date not supplied We recently updated the major version of pnpmdeveloper.woocommerce.com · Date not supplied
latest version of Ruffversions · Right answer: 0.17.1	Wrong Top five results data.safetycli.comdata.safetycli.com · Date not supplied docs.rsdocs.rs · Date not supplied Bug 2488001 - ruff-0.15.18 is availablebugzilla.redhat.com · Date not supplied RubyGems Navigation menurubygems.org · Date not supplied Bug 2447206 - ruff-0.15.7 is availablebugzilla.redhat.com · Date not supplied
latest version of anthropic Python SDKversions · Right answer: 1.15.0	Wrong Top five results summary.ecosyste.mssummary.ecosyste.ms · Date not supplied data.safetycli.comdata.safetycli.com · Date not supplied Last released Jun 9, 2025pypi.org · Date not supplied github.comgithub.com · Date not supplied cdn.jsdelivr.netcdn.jsdelivr.net · Date not supplied
latest version of Reactversions · Right answer: 19.3.1	Wrong Top five results Latest React Versiongurubase.io · Date not supplied What is the latest version of React, and what new features does it include?c-sharpcorner.com · Date not supplied React Versionszh-hant.react.dev · Date not supplied DEV Communitydev.to · Date not supplied startup house warsaw logostartup-house.com · Date not supplied
latest version of SvelteKitversions · Right answer: 3.1.0	Wrong Top five results SvelteKit 1.0 releasedsdtimes.com · Date not supplied SvelteKit 1.0 releasedsdtimes.com · Date not supplied Announcing SvelteKit 2svelte.technology · Date not supplied lireamp sveltekit 10 apporte le full stack a svelte 89082lemondeinformatique.fr · Date not supplied releases.shreleases.sh · Date not supplied
latest version of Playwrightversions · Right answer: 1.64.1	Wrong Top five results Latest version of com.microsoft.playwright:playwrightjavadoc.io · Date not supplied Latest version of com.microsoft.playwright:playwrightjavadoc.io · Date not supplied Latest version of com.microsoft.playwright:driverjavadoc.io · Date not supplied Latest version of com.microsoft.playwright:playwrightjavadoc.io · Date not supplied Playwright: Releases, patches & end-of-lifeversio.io · Date not supplied
latest version of Vueversions · Right answer: 3.5.44	Wrong Top five results gitlink.org.cngitlink.org.cn · Date not supplied This is the repo for Vue 2. For Vue 3, go to https://github.com/vuejs/corecode.morphllm.com · Date not supplied Embed presentationslideshare.net · Date not supplied Vue 2 has reached End of Lifelibraries.io · Date not supplied portal.gitnation.orgportal.gitnation.org · Date not supplied
latest version of Zodversions · Right answer: 4.6.7	Wrong Top five results github colinhacks/zod v4.0.1newreleases.io · Date not supplied zod@1.10.4 vulnerabilitiessecurity.snyk.io · Date not supplied data.safetycli.comdata.safetycli.com · Date not supplied @conform-to/zod vulnerabilitiessnyk.io · Date not supplied npm zod 3.14.5newreleases.io · Date not supplied
latest version of Pydanticversions · Right answer: 2.14.1	Wrong Top five results pypi.orgpypi.org · Date not supplied pypi pydantic 2.13.0newreleases.io · Date not supplied Path to this page:pkgsrc.se · Date not supplied www.eessi.ioeessi.io · Date not supplied pypi.orgpypi.org · Date not supplied
latest version of Bunversions · Right answer: 1.4.3	Wrong Top five results > ## Documentation Indexbun.sh · Date not supplied Upgrade Bun to the latest versionbun.com · Date not supplied # Upgrade Bun to the latest versionbun.com · Date not supplied Setting your Bun Versiondocs.render.com · Date not supplied > ## Documentation Indexbun.com · Date not supplied
Exa annualized revenue run rate announced October 2026week · Right answer: $100M ARR (Will Bryk, X, Oct 13)	Wrong Top five results Exail Technologies (EXA) H1 2026 earnings summaryquartr.com · Date not supplied Exa at $10M growing 11x YoYsacra.com · Date not supplied getlatka.comgetlatka.com · Date not supplied Exail Technologies (EXA) Q3 2026 TU earnings summaryquartr.com · Date not supplied Exail Technologies (EXA) Q2 2026 TU earnings summaryquartr.com · Date not supplied
Which company became the default search provider for Browserbase in October 2026week · Right answer: Parallel (Parag Agrawal, X, Oct 14)	Right Top five results browserbase search powered by parallelparallel.ai · Date not supplied digg.comdigg.com · Date not supplied support.mozilla.orgsupport.mozilla.org · Date not supplied support.mozilla.orgsupport.mozilla.org · Date not supplied press ukblog.mozilla.org · Date not supplied
Name of Firecrawl's company funding intelligence tool for ChatGPT and Claude launched October 2026week · Right answer: Fireround (Firecrawl, X, Oct 15)	Wrong Top five results
Which DeepSeek Harness version is recommended for DeepSeek-V4.1-Flash-1017week · Right answer: v0.2.1-rc.1 or later (DeepSeek, X, Oct 17)	Wrong Top five results deepseek v4.1 flashopenrouter.ai · Date not supplied deepseek v4.1 flash 20260910openrouter.ai · Date not supplied deepseek releases harness 0 1 5 with v4 1 flash support file uploads and sidebar previewstechnode.com · Date not supplied GPT-6.1 Sol are now live on CometAPI →cometapi.com · Date not supplied eu.36kr.comeu.36kr.com · Date not supplied
TinyFish free search API rate limit requests per minuteweek · Right answer: 30 requests per minute (Oct 18)	Right Top five results TinyFish: Free Web Search and Fetch API for Your AI Coding Agentsbitdoze.com · Date not supplied Free Web Search for AI Coding Agents: TinyFish Search + Fetch Setup Guidebitdoze.com · Date not supplied # TinyFish vs Parallel: operating websites or retrieving the webparallel.ai · Date not supplied AI EngineerGuideaiengineerguide.com · Date not supplied > ## Content Indextestingcatalog.com · Date not supplied
According to Parallel, which open model was the first to reach the top 10 of its Search Capability Leaderboardweek · Right answer: DeepSeek V4.1 Flash (1017) (Parallel, X, Oct 18)	Right Top five results search leaderboardparallel.ai · Date not supplied # Introducing the Search Capability Leaderboardparallel.ai · Date not supplied parallel search capabilitybenchlm.ai · Date not supplied parallel.aiparallel.ai · Date not supplied # Pareto 26.9 · Search Capability Leaderboardparallel.ai · Date not supplied
Which Firecrawl product gives Gemini agents live web data in the Firecrawl Google Cloud partnership announced October 19 2026week · Right answer: Alexandria (Firecrawl, X, Oct 19)	Wrong Top five results
Which Claude Code environment variable chooses the model used by the auto mode classifierweek · Right answer: CLAUDE_CODE_AUTO_MODE_MODEL (Claude Code v2.1.302, Oct 16)	Right Top five results Preflight Checklistgithub.com · Date not supplied Vibe Coding for Productionai-workshop.dometrain.com · Date not supplied [FEATURE] Allow configuring a separate model for auto mode classifier via environment variableclaudeissues.com · Date not supplied claude code auto mode desktop control and channels what you need to know right now c4f751ea0f33medium.com · Date not supplied what is claude code auto mode permission classifiermindstudio.ai · Date not supplied
Which new crypto function did Node.js 24.22.0 addweek · Right answer: crypto.parsePKCS12() (Node v24.22.0, Oct 14)	Wrong Top five results cdn.jsdelivr.netcdn.jsdelivr.net · Date not supplied Node.js v22.0.0-v8-canary20240201f4b4aa7f2e documentationnodejs.org · Date not supplied Node.js v24.7.0 (Current)nodejs.org · Date not supplied Node.js v22.0.0-nightly20240304b34512e38e documentationnodejs.org · Date not supplied Node.js v22.0.0-nightly20240127af3e2b2493 documentationnodejs.org · Date not supplied
Which new value for the Link prefetch prop did Next.js add in the 16.5 canary releasesweek · Right answer: "navigation" (<Link prefetch="navigation">, v16.5.0-canary.18, Oct 19)	Wrong Top five results
DeepSeek Search API rate limit QPS per accounttraps · Right answer: 100 QPS (docs; launch coverage said 5)	Wrong Top five results byteplus.combyteplus.com · Date not supplied # DeepSeek Message Limit: Web Chat Quotas, Rate Limits, and Workaroundsfast.io · Date not supplied # DeepSeek Rate Limits: API Quotas, Concurrency Caps, and MCP Workaroundsfast.io · Date not supplied rate limitapi-docs.deepseek.com · Date not supplied AI & Agentsfast.io · Date not supplied
Exact UTC time DeepSeek starts billing web search calls made through Claude Code and Codextraps · Right answer: Oct 31 2026 16:00 UTC (DeepSeek docs)	Wrong Top five results All AI updatessessionwatcher.com · Date not supplied docs.litellm.aidocs.litellm.ai · Date not supplied www.byteplus.combyteplus.com · Date not supplied www.byteplus.combyteplus.com · Date not supplied deepseek api price increaseaireiter.com · Date not supplied
Which DeepSeek model is required for Codex native web search through DeepSeek's APItraps · Right answer: deepseek-v4-pro (DeepSeek Codex guide; posts said search is free for any model)	Wrong Top five results deepseek june10 claude code web search docsdeepseekv4pro.com · Date not supplied Updated 2026-06-10deepseekv4pro.com · Date not supplied x codex ds - No need for a ChatGPT account! Zero threshold for connecting Codex to the DeepSeek AI modelx-cmd.com · Date not supplied deepseek june23 claude code official integrationdeepseekv4pro.com · Date not supplied Comment utiliser DeepSeek 3.1 dans Codex : de l'installation au premier lancementblogs.novita.ai · Date not supplied
Exa API header value needed to use dynamic highlightstraps · Right answer: Exa-Beta: dynamic-highlights-2026-08-28 (API docs; launch post calls it generally available)	Wrong Top five results # Breaking change: ExaTools no longer accepts “highlights”agno.com · Date not supplied # Align ExaTools with Exa API v2 by removing deprecated “highlights”agno.com · Date not supplied align exatools with exa api v2 by removing deprecated highlightsagno.com · Date not supplied align exatools with exa api v2 by removing deprecated highlightsagno.com · Date not supplied highlights restored js sdkexa.ai · Date not supplied
Exa 3.0 release datetraps · Right answer: Oct 19 2026 (Exa blog; some coverage said Oct 18)	Wrong Top five results
Which search provider is tied with Exa 3.0 for first place on the Artificial Analysis Search Indextraps · Right answer: Perplexity, both 80 (Artificial Analysis, Oct 19; Exa says #1 on every benchmark it tested)	Wrong Top five results All articlesartificialanalysis.ai · Date not supplied All articlesartificialanalysis.ai · Date not supplied Artificial Analysissimplify.jobs · Date not supplied octen search artificial analysis search indexorcarouter.ai · Date not supplied octen search artificial analysis search indexorcarouter.ai · Date not supplied
Who owns Tavilytraps · Right answer: Nebius (acquired Feb 2026)	Right Top five results NEBIUS WENT SHOPPINGrs.linkedin.com · Date not supplied oz jacobiil.linkedin.com · Date not supplied startupim.comstartupim.com · Date not supplied tavily and insight partnersgoldfarb.com · Date not supplied The Collectivethecollectivecommunity.substack.com · Date not supplied
Perplexity Decision API price per million input tokenstraps · Right answer: 2 cents (halved Oct 6, Aravind Srinivas)	Wrong Top five results pplx decider v1 27bgradually.ai · Date not supplied Perplexity Open-Sources 27B Decider, Edges Jev on 11-Test Panelaiweekly.co · Date not supplied perplexity decider v1 27baiagentstore.ai · Date not supplied pplx decider v1 27brequesty.ai · Date not supplied Table Of Contentscloudzero.com · Date not supplied

Back to the full results