{"id":287232,"date":"2026-07-08T12:39:00","date_gmt":"2026-07-08T16:39:00","guid":{"rendered":"https:\/\/news-you-need.com\/index.php\/2026\/07\/08\/google-updates-android-bench-with-new-llms-but-gemini-still-lags-behind\/"},"modified":"2026-07-08T12:55:08","modified_gmt":"2026-07-08T16:55:08","slug":"google-updates-android-bench-with-new-llms-but-gemini-still-lags-behind","status":"publish","type":"post","link":"https:\/\/news-you-need.com\/index.php\/2026\/07\/08\/google-updates-android-bench-with-new-llms-but-gemini-still-lags-behind\/","title":{"rendered":"Google updates Android Bench with new LLMs, but Gemini still lags behind"},"content":{"rendered":"<p><a href=\"https:\/\/arstechnica.com\/google\/2026\/07\/google-revamps-android-ai-dev-benchmark-adds-fable-5-and-other-agents\/\">Google updates Android Bench with new LLMs, but Gemini still lags behind<\/a><\/p>\n<p><a href=\"https:\/\/arstechnica.com\/google\/2026\/07\/google-revamps-android-ai-dev-benchmark-adds-fable-5-and-other-agents\/\">https:\/\/arstechnica.com\/google\/2026\/07\/google-revamps-android-ai-dev-benchmark-adds-fable-5-and-other-agents\/<\/a><\/p>\n<p>Publish Date: <a href=\"publish_date]\">2026-07-08 12:39:00<\/a><\/p>\n<p>Source Domain: <a href=\"arstechnica.com\">arstechnica.com<\/a><\/p>\n<p>              Google\u2019s updated leaderboard shows Gemini slipping to fifth place.<\/p>\n<p>\n                  Credit:<br \/>\n                                      Google\n                                  <\/p>\n<p>\n      Google\u2019s updated leaderboard shows Gemini slipping to fifth place.<\/p>\n<p>              <span class=\"caption-credit mt-2 text-xs\"><br \/>\n          Credit:<\/p>\n<p>          Google<\/p>\n<p>                  <\/span>\n          <\/p>\n<p>However, Fable 5 and GPT 5.5 also have extremely high operating costs, chewing through more than $130 in tokens for the 100-problem, 10-run benchmark. Gemini 3.1 Pro didn\u2019t score as high, but it only costs $87 to run the test. Gemini 3.5 Flash, which is supposed to be cheaper to run than other models, has the highest cost on the leaderboard because it took so much longer to complete the benchmark: $165 per run and a 28-hour runtime.<\/p>\n<p>The Android coding performance gap for Google\u2019s models is a problem as the company shifts many of its projects toward agentic development. Obviously, Google would prefer that Android developers use Google\u2019s tools in their workflows, which may be why Google has reportedly been offering to buy application source code from developers for AI training.<\/p>\n<h2>Community collaboration<\/h2>\n<p>Android Bench is supposed to evolve over time, adopting new workflows to test models. Google hopes that developers will want to contribute to Android Bench by sharing benchmarks and development tasks. To make that more feasible, Google is switching to the Harbor framework. According to the company, this testing sandbox makes it easy for developers to run, evaluate, and share results for Android Bench.<\/p>\n<p>Google re-ran all its previous tests with Harbor to get a new baseline for LLM performance. So there\u2019s been some shift in the previously reported scores even though the&#8230;<\/p>\n<p><a href=\"https:\/\/arstechnica.com\/google\/2026\/07\/google-revamps-android-ai-dev-benchmark-adds-fable-5-and-other-agents\/\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Google updates Android Bench with new LLMs, but Gemini still lags behind https:\/\/arstechnica.com\/google\/2026\/07\/google-revamps-android-ai-dev-benchmark-adds-fable-5-and-other-agents\/ Publish Date:&#8230;<\/p>\n","protected":false},"author":1,"featured_media":287233,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/cdn.arstechnica.net\/wp-content\/uploads\/2025\/07\/Android-statue-1152x648.jpg","fifu_image_alt":"","footnotes":""},"categories":[46],"tags":[70],"class_list":["post-287232","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-android","tag-google"],"_links":{"self":[{"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/posts\/287232"}],"collection":[{"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/comments?post=287232"}],"version-history":[{"count":1,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/posts\/287232\/revisions"}],"predecessor-version":[{"id":287234,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/posts\/287232\/revisions\/287234"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/media\/287233"}],"wp:attachment":[{"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/media?parent=287232"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/categories?post=287232"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news-you-need.com\/index.php\/wp-json\/wp\/v2\/tags?post=287232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}