Core AI is Apple’s on-device inference runtime for models you bring to the app. It loads a .aimodel asset, specializes it for the current device, and runs one or more InferenceFunction graphs through a memory-safe Swift API. Use it when you need your own model on Apple silicon, not when the system language model or a higher-level ML API already covers the feature.
The production problem is not calling AIModel. It is keeping conversion, specialization, tensor shapes, and UI state from leaking into each other. Once those boundaries are explicit, Core AI is a runtime. Before they are, it becomes a pile of Python, Xcode, and SwiftUI timing bugs.
Choose Core AI against the rest of the stack
Reach for Core AI when you own the model and need explicit control over specialization, caching, and inference. Stay elsewhere when a higher layer already owns those decisions.
| Runtime | Use it when | Do not use it when |
|---|---|---|
| Core AI | You ship a .aimodel, care about specialization, states, or custom kernels |
The feature only needs Apple’s system model |
| Foundation Models | You want LanguageModelSession on the on-device system LLM |
You must run a specific converted architecture |
| Core ML | An existing Core ML pipeline already meets the latency and shape budget | You are starting a new model that Core AI already exports cleanly |
| MLX | You are iterating locally on a Mac endpoint or research loop | The feature has to ship inside an iOS app binary as a specialized asset |
A useful exception: Core AI’s language-model helpers can sit under a LanguageModelSession so the session API stays familiar while the weights are yours. That is still a Core AI deployment. It is not a reason to skip a Swift boundary around loading and tensor preparation.
Convert a PyTorch model before you specialize
The .aimodel is an interface. Export the dynamic dimensions the app will actually send, give tensors stable names, and compare converted outputs against the Python model before any SwiftUI work. A successful Xcode build can still ship the wrong numeric behavior.
Convert PyTorch models to Core AI covers the export contract, naming, and numerical checks. Treat those tests as release checks, not as a one-time notebook cell.
Load AIModel off the interaction path
AIModel inspects and loads functions from a URL. InferenceFunction is the runnable graph. NDArray is the input and output container. Construct the first two when the feature is prepared, not inside a button action or a body evaluation. Specialization is device-specific work; putting it on first tap makes the UI look like inference is slow when the runtime is still compiling.
Load Core AI models behind a Swift boundary keeps tensor names out of views. Building an on-device vocab app shows the same rule in a camera flow: one model, one job, typed output, specialization as a product state.
Keep transformer history in Core AI states
If each generation step resubmits the full token history, the cost grows with conversation length. Mutable Core AI model states exist so a transformer can reuse a key-value cache instead of recomputing it. Confirm that history is actually the cost before adding state, then give each conversation its own state object.
Keep transformer inference fast with Core AI states is the implementation note for that loop.
Debug Python and Xcode as one workflow
When outputs diverge, start with the smallest failing tensor, not with a UI screenshot. Core AI Debugger maps values back toward Python. The Xcode gauge separates load, specialize, and run. Instruments tells you whether the hot path is inference or everything around it.
Debug Core AI models across Python and Xcode is the closed loop. Use it before changing the Swift adapter a second time.
Local agents are a different runtime
A Mac-only agent that talks to a local server is not the same product as a specialized .aimodel inside an iOS app. MLX-LM can be the right endpoint for that Mac loop. Do not copy that architecture into the on-device app path just because both run on Apple silicon.
Running local agents on a Mac with MLX stays on the server side of that split.
Production default
Start from the feature’s contract: input shapes, failure modes, and whether the model must ship in the app. If Core AI is the runtime, convert with tests, load outside the interaction, keep views on application values, and debug Python and Xcode as one system. The rest of the notes in this cluster are the pieces of that default, not a menu of optional extras.