KN001: Foundation Models in the Background

How Apple’s on-device Foundation Models limit apps running in the background, and how Karui copes.

September 28, 2026 · iOS 27.0 · Measured on iPhone 17 Pro

Overview

Karui is an on-device iPhone notification filter built on iOS 27’s new notification trigger.

iOS 27’s “When I receive a notification” Shortcuts trigger runs an automation for each notification from the apps you choose. Karui’s Shortcut automation calls an App Intent that runs in the background. The intent builds a prompt and makes one guided-generation request with LanguageModelSession:

let decision = try await session.respond(
    to: prompt,
    generating: TriageDecision.self
)

With Karui on screen this takes about 4 seconds, but in the background the request is often refused.

Apple documents that the limit applies only in the background, but not how large it is or how it recovers. The measurements below come from one iPhone’s logs over a few days.

Apple’s TN3193 covers the model’s context window. This note covers how often an app can use the model in the background in practice.

What the limit looks like

Refusals arrive as the rateLimited generation error in under 35 milliseconds, too fast for the model to have run. Apple’s documentation for the error says:

“This error will only happen if your app is running in the background and exceeds the system defined rate limit.”

In Karui’s logs the pattern was consistent: a burst of accepted calls, then hours of refusals.

All calls were in the background, with the iPhone unplugged. Three slow calls of 8 to 15 seconds got through after midnight on the first night.

In one case, refusals continued across a 74-minute gap, so I assume this isn’t a simple hourly quota.

Factors that did and didn’t affect refusals

Karui logs the device’s state with every model call. Refusals happened with the iPhone locked and unlocked, at nominal thermal state, with Low Power Mode off, and at 60 to 100 percent battery.

As Apple’s documentation says, calls with Karui on screen weren’t affected.

One night on the charger, all 10 background calls between 1:30 and 5:51 AM succeeded. They were also spread out, about two an hour. The first refusal came 15 minutes after the iPhone was unplugged.

Misleading symptoms

The error message

The rate-limit error message suggests using non-streaming requests in background activities. Karui never streams though.

Timeouts that are really suspensions

One call “timed out” after 70 seconds against a 20-second limit. ContinuousClock keeps counting while the process is suspended, and iOS had suspended Karui mid-call.

Other failures

Rarely, a call fails with an error from the safety layer’s sensitive content analysis instead.

Coping

Karui still delivers every notification when the model refuses:

1. It falls back to hard-coded rules. A deterministic classifier decides right away. Important contacts, mentions, one-time codes, and urgent terms were already decided by rules before the model ran.
2. It rechecks once it comes to the foreground. Notifications decided by rules because of a rate limit are marked and reclassified the next time Karui is opened.
3. It tries BGProcessingTask. Karui schedules one after a rate limit, and iOS ran them 15 minutes to 2 hours later, even unplugged. Whether model calls from a background task get a larger budget is still unknown.
4. It asks Private Cloud Compute instead, through the user’s shortcut. The triage intent returns a flag and a ready-made prompt, the shortcut’s Use Model action sends the prompt to Private Cloud Compute, and a second intent applies the answer. In Karui’s logs, all 8 rate-limited notifications one morning got an answer 2 to 3 seconds after the on-device refusal. For how Karui builds on this, see KN002.

Revision History

• 2026-09-28 First published.

← All Technical Notes