Run Custom Models On Device With Core AI

June 8, 2026 |

Core AI might be the most interesting developer announcement from WWDC26 because it fills a gap that a lot of app teams have been awkwardly working around.

Foundation Models is great when Apple’s system model is the right model. Core ML is still a solid fit for many classic machine learning features. Core AI sits in a different spot: it is the new Apple framework for running custom, optimized AI models on device, including popular open-source models that have been converted into Core AI’s .aimodel format.

That changes the design conversation. Instead of asking “Can I afford to send this to a server?” or “Can the system model do this exact task?”, you can ask:

Is there a smaller model that is good at this specific job?
Can it fit within the memory and storage budget for the devices I support?
Can I make the first run feel intentional instead of surprising?
Does the user’s data need to stay on device?

For a lot of app features, that is a much better set of questions.

A good Core AI feature is decomposed

Apple’s WWDC26 Core AI demo is a language-learning app. The user points the camera at an object, the app segments the object out of the image, then a small language model creates a vocabulary card.

The important design choice is that this is not one giant model doing everything. It is two focused models:

A vision model that does text-prompted segmentation.
A language model that turns the object label into structured vocabulary data.

That decomposition is exactly how I would think about Core AI in a production app. A recipe app might use one model for image classification and another for ingredient cleanup. A fitness app might use a pose or motion model, then a language model to write a coaching note. A notes app might use an embedding model for clustering, then a language model for summaries.

Small, task-shaped models are easier to test, easier to replace, and easier to budget.

Load a model from your app assets

After conversion, a Core AI export gives you either a standalone .aimodel file or a resource folder that contains one or more .aimodel files plus anything else the runtime needs, like a tokenizer. Xcode 27 can inspect those assets so you can see the model size, target platform, functions, input tensor shapes, output tensor shapes, and metadata.

When you are using a low-level model directly, that inspection step matters. A model function is not magic Swift API. It has exact inputs and outputs. You need the right preprocessing before inference and the right postprocessing after inference.

For popular model families, Apple is also providing a Swift package in the Core AI models repository with runtime libraries for language models, segmentation, diffusion, and object detection. The public Core AI framework itself centers on lower-level types such as AIModel, InferenceFunction, and NDArray; the package libraries are model-family wrappers that hide preprocessing and postprocessing when they fit your use case. In the WWDC26 demo, the segmentation library hides the tensor plumbing behind a clean Swift API:

import CoreAIImageSegmenter
import CoreGraphics

struct ObjectSegmenter {
    private let segmenter: ImageSegmenter

    init(resourcesPath: String) async throws {
        segmenter = try await ImageSegmenter(resourcesAt: resourcesPath)
    }

    func segment(image: CGImage, prompt: String) async throws {
        let response = try await segmenter.segment(image: image, prompt: prompt)

        guard let mask = response.segments.first?.mask else {
            return
        }

        // Save or render the mask in your app-specific image pipeline.
        print(mask)
    }
}

That is the right shape for app code. Keep the model-specific API behind a small type that the rest of the app understands. Your SwiftUI view should not know where the .aimodel lives, what the model’s tensor shapes are, or whether the first call needs preparation.

Use Foundation Models ergonomics with your own language model

The other piece of the demo is more surprising. Once you load a custom language model through CoreAILanguageModel, you can use it with LanguageModelSession, the same session type you use with Apple’s system model.

That means structured output still feels familiar:

import CoreAILanguageModels
import FoundationModels

@Generable
struct VocabCard {
    let targetWord: String
    let englishMeaning: String
    let exampleSentence: String
}

struct VocabCardGenerator {
    private let session: LanguageModelSession

    init(resourcesPath: String) async throws {
        let model = try await CoreAILanguageModel(resourcesAt: resourcesPath)
        session = LanguageModelSession(model: model)
    }

    func makeCard(for objectName: String, language: String) async throws -> VocabCard {
        let response = try await session.respond(
            to: """
            Create a vocabulary card for "\(objectName)" in \(language).
            Keep the example sentence short and natural.
            """,
            generating: VocabCard.self
        )

        return response.content
    }
}

That API design is a big deal. You can swap the model underneath the session without rewriting your app around a different prompting, streaming, or guided-generation abstraction.

It also gives you a nice production boundary. A feature model can decide which model resources to load. A view model can decide when to run the generation. The SwiftUI view can stay focused on progress, results, and errors.

Do not surprise users with model preparation

The rough edge is first-run latency. Core AI specializes a model for the current device hardware and OS version before inference can begin. For large models, that preparation can take long enough to be visible.

This is where a lot of AI demos accidentally become bad products. The user taps a shiny button, then stares at a spinner while the app quietly prepares a model it could have prepared earlier.

I would treat model setup as a feature state:

enum LocalAIState: Equatable {
    case notInstalled
    case downloading(progress: Double)
    case preparing
    case ready
    case unavailable(message: String)
}

Then build a real first-run experience around that state. Explain that the feature runs privately on device. Download large models only when the user opts in. Prepare the model before the user enters the tight interaction loop. Let Core AI cache the specialized result, and treat that cache as a performance win rather than permanent storage.

Apple specifically called out Background Assets for keeping huge model files out of the base app download. That is worth doing if the model is optional. Nobody wants a 1 GB update for a feature they never open.

Compile ahead of time when model size hurts

Core AI can also compile models ahead of time with coreai-build:

xcrun coreai-build compile MyModel.aimodel --platform iOS --min-deployment-version 27.0 --output compiled/

Ahead-of-time compilation turns the portable .aimodel into architecture-specific .aimodelc assets. Use AIModel.deviceArchitectureName to select the right compiled asset for the current device architecture. AOT compilation does not remove all device-specific work, but it moves the expensive compilation step to your build machine so the model has less to do when it first reaches a user’s device. For a large optional feature, that changes the feel from “this app froze when I tapped a button” to “this app prepared the feature while I was opting in.”

My checklist for shipping a Core AI model would look like this:

Keep the model optional unless it is central to the app.
Inspect the .aimodel in Xcode before writing integration code.
Prefer model-specific package libraries when they hide real preprocessing and postprocessing complexity.
Move model download and first specialization out of the hot path.
Use Instruments to measure model loading, specialization, and inference time on real devices.
Keep a smaller fallback or disable the feature gracefully on unsupported devices.

Core AI vs Foundation Models vs Core ML

I do not think Core AI replaces the existing choices. It makes the decision tree clearer.

Use Foundation Models when Apple’s model is a good fit, when you want the system availability story, and when prompts, tool calls, streaming, or structured generation are the core of the feature.

Use Core AI when you need a specific model, want a popular open-source model on device, or need to combine several specialized AI models under your app’s control.

Use Core ML when the model is a classic ML model, when you already have a Core ML pipeline, or when the Core ML ecosystem is enough for the task.

The exciting part is that these are starting to compose. A Core AI language model can back a LanguageModelSession. A Core AI vision model can produce structured context for a Foundation Models prompt. A feature can use the smallest tool that works for each step.

That is the shape I like: private by default, explicit about model cost, and boring enough that the rest of the app can trust it.