TL;DR

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

An XDA Developers writer tested local language models on an 8GB Galaxy S23 FE and found that a model that crawled in backgrounded Termux ran at about 11 tokens per second when Termux was visible. The test suggests Android’s background CPU limits were a major factor, though the supplied account ends before confirming the full cause or explaining how the phone handles longer everyday use.

An XDA Developers writer found that a local language model that generated text at roughly 0.1 token per second on a Galaxy S23 FE reached 11.39 tokens per second in a later test when Termux remained in the foreground. The result suggests that app visibility and Android’s allocation of CPU cores can make a substantial difference to running an LLM on this phone, but it does not establish performance across other devices or usage conditions.

The writer tested an Exynos 2200 Galaxy S23 FE with 8 GB of RAM, of which Android showed about 7.3 GB as available. The phone already had 4.6 GB of memory in use before testing. PocketPal, a local-model chat app, loaded Gemma 3 1B, but repeatedly crashed when the writer tried to load Gemma 4 E2B, which PocketPal warned could exceed available memory.

The writer then ran the model through llama.cpp in Termux, using an Unsloth Q4_K_M build listed at 2.8 GB. With Termux in the background, a Gemma prompt generated at about 0.1 token per second. A separate short Qwen 3.5 2B test in the same backgrounded condition measured 0.17 tokens per second for generation. The writer checked the phone’s power settings and reported that battery use was unrestricted, power saving was off, and unused-app sleep was disabled.

In further tests, the writer kept Termux in front and ran llama-bench, including tests initiated over SSH from a PC. Gemma 4 E2B then generated at 11.39 tokens per second; Qwen 3.5 2B reached 13.33 tokens per second on a short test and 9.98 ± 0.00 tokens per second on a longer one. Separately, PocketPal ran Qwen 3.5 2B at about 9 tokens per second in the writer’s real-use tests. These are the writer’s results on one phone, not independent benchmark findings.

Foreground Use Changed Model Speed

The comparison shows why a phone’s stated memory capacity or a model file’s size alone may not predict how usable local AI will be. On this Galaxy S23 FE, the writer saw a sharp difference between foreground and background execution. For someone considering on-device AI to reduce reliance on cloud services, keeping the terminal app visible may be a practical requirement rather than a minor setup detail.

That finding comes with limits. Token-generation rates measure speed, not whether a model gives reliable or useful answers. The writer said PocketPal’s Qwen tests produced many hallucinations and invented facts, despite usable speed. The experience therefore points to two separate questions for local AI on a phone: whether it can respond quickly enough and whether its output is trustworthy enough for the task.

Two Ways to Run Models Locally

The writer’s aim was to move some everyday tasks, such as summarizing or replying to emails, away from cloud services including ChatGPT and Claude. They already used Gemma 4 E4B on a PC, but wanted a way to run a model when that computer was not available. The Galaxy test was an attempt to see whether local inference could cover some of that use.

The experiment compared a purpose-built chat interface, PocketPal, with a command-line setup using Termux and llama.cpp. PocketPal was simpler to use and offered a built-in chat screen, but it crashed on the model the writer initially wanted to try. Termux loaded that model, yet its performance varied depending on whether the app was visible. The article reports that all eight CPU cores were available while Termux was active, compared with cores 0–2 when it was in the background. The writer linked that observation to the speed difference.

“The 103-token prefill ran at 1.27 tok/s, and the actual generation was 0.11 tok/s.”

— Shekhar Vaidya, XDA Developers writer

The Cause Is Not Fully Established

The account reports that more CPU cores were available while Termux was visible and proposes that Android’s background-app restrictions contributed to the slowdown. That is the writer’s explanation based on the observed tests; the supplied material does not establish the full mechanism or rule out every other difference between runs. It also does not provide a controlled comparison across repeated sessions, other Galaxy models, or different Android versions.

Practical questions remain about heat, battery drain, and sustained performance, as well as how the phone performs during longer conversations or with other model sizes. The article also does not show whether the crashes in PocketPal can be avoided through different settings, or whether the reported hallucinations occurred consistently across prompts.

More Testing Needed for Daily Use

The immediate takeaway from the reported tests is that users trying local inference through Termux may need to check whether the app remains in the foreground before judging speed. The writer’s next step, as described in the account, was to investigate why backgrounding Termux reduced its available CPU cores. The supplied material ends before giving the rest of that explanation or a final recommendation.

Further testing would be needed to establish whether the foreground performance holds over longer sessions and whether it is practical for routine tasks. Until then, the reported token rates should be treated as one-device observations, and the writer’s concerns about answer accuracy remain relevant alongside speed.

Key Questions

What phone was used for the local LLM tests?

The writer tested an Exynos 2200 Samsung Galaxy S23 FE with 8 GB of RAM, about 7.3 GB of which Android showed as visible.

Why did Gemma 4 E2B initially run so slowly?

The writer recorded about 0.1 token per second with Termux in the background and later found fewer CPU cores available in that state. The article presents background CPU restrictions as a likely factor, not a fully confirmed explanation.

How fast was Gemma 4 E2B with Termux in the foreground?

In the writer’s short llama-bench test, Gemma 4 E2B generated at 11.39 tokens per second with Termux visible. That result applies to the tested phone and setup.

Did the faster model provide reliable answers?

Not necessarily. The writer said PocketPal’s Qwen tests produced many hallucinations and invented facts, showing that speed and answer quality are separate considerations.

Source: XDA Developers