Introduction

Artificial Intelligence has become one of the most influential technologies in modern software development. From chatbots and recommendation systems to sentiment analysis and intelligent search, machine learning models are now expected features in many mobile applications.

For several years, integrating AI into iOS applications almost always meant sending user data to cloud services. APIs such as OpenAI, Anthropic Claude, and Google Gemini allowed developers to leverage state-of-the-art language models without worrying about infrastructure or hardware limitations. While this approach is simple, it also introduces several challenges including network latency, API costs, internet dependency, and privacy concerns.

Fortunately, the landscape has changed dramatically. Today's Apple devices contain incredibly powerful hardware, including the Apple Neural Engine (ANE), powerful GPUs, and highly optimized CPUs capable of running sophisticated machine learning models directly on the device.

This shift has made on-device AI more practical than ever. Instead of relying entirely on cloud services, developers can now deploy transformer models directly within their applications, enabling offline functionality, lower latency, improved privacy, and reduced operational costs.

In this article, we'll explore how to integrate an ONNX-based transformer model into an iOS application using Swift. We'll load a model, execute inference with ONNX Runtime, and prepare the necessary transformer inputs for models such as DistilBERT.

A Brief History of LLM Integration in Swift Projects

Large Language Models were originally designed to run on powerful cloud infrastructure because of their immense computational requirements. Training these models required thousands of GPUs, and even inference demanded hardware far beyond what smartphones could provide at the time.

Because of these limitations, early Swift applications integrated AI almost exclusively through cloud APIs. User prompts were transmitted to remote servers where the model generated a response before sending the results back to the application.

Although this architecture worked well, it also introduced unavoidable drawbacks:

- Internet connectivity became mandatory.

- Responses depended on network latency.

- User data had to leave the device.

- API usage generated recurring operational costs.

Meanwhile, Apple continued investing heavily in machine learning acceleration.

The introduction of the Apple Neural Engine in 2017 marked a turning point. Every generation of iPhone, iPad, and Mac became increasingly capable of executing neural networks efficiently. At the same time, Apple expanded Core ML, Metal Performance Shaders, and hardware acceleration APIs that allowed developers to run increasingly sophisticated models locally.

The open-source AI community accelerated this transition even further.

Frameworks such as llama.cpp, MLX, MLC LLM, and ONNX Runtime made it possible to execute optimized transformer models directly on Apple devices. Developers could now deploy popular open-source models including Llama, Mistral, Phi, Gemma, and Qwen without requiring any cloud infrastructure.

Apple later introduced Foundation Models as part of Apple Intelligence, further demonstrating the industry's movement toward local AI processing.

Today, Swift developers have more choices than ever before. Depending on the application, developers can choose between cloud-hosted models, hybrid cloud/local inference, or fully offline on-device inference.

For many applications, including text classification, semantic search, recommendation engines, and lightweight AI assistants, on-device inference has become the preferred solution.

Why ONNX?

Before diving into the implementation, it's worth understanding why ONNX has become one of the most popular deployment formats for machine learning models.

ONNX (Open Neural Network Exchange) is an open standard for representing machine learning models. Instead of locking your project into a specific framework such as TensorFlow or PyTorch, ONNX provides a portable format that can be executed across many different platforms.

This portability offers several advantages. A model trained in Python using PyTorch can be exported as an .onnx file and later executed inside an iOS application without rewriting the model itself.

Likewise, the exact same model can often be shared between iOS, Android, Windows, Linux, and macOS. This dramatically simplifies deployment across multiple platforms.

Microsoft maintains ONNX Runtime, a highly optimized inference engine capable of executing ONNX models efficiently across different hardware accelerators.

For Swift developers, this means we only need to load the ONNX model, provide the expected inputs, and retrieve the outputs generated by the runtime.

Loading an ONNX Model

Let's assume we've already trained our transformer model and exported it to ONNX.

Our project now contains a file named MoodClassifier.onnx.

The model can either be added directly to the application bundle or packaged as a Swift Package resource.

The first step is locating the model inside the application.

let modelPath = Bundle.main.path(forResource: "MoodClassifier", ofType: "onnx")

If modelPath is not nil, the application has successfully located the model.

If it returns nil, verify the following:

- The model has been added to the target.

- The filename matches exactly.

- The resource exists inside the application bundle.

- The file extension is correct.

Successfully locating the model is the first indication that everything has been configured correctly.

Installing ONNX Runtime

Executing an ONNX model requires an inference engine.

Fortunately, Microsoft provides an official Swift Package for ONNX Runtime that can be added using Swift Package Manager.

.package(url: "https://github.com/microsoft/onnxruntime-swift-package-manager",from: "1.24.2")

After adding the dependency, we're ready to create an inference session.

Running Inference

Running inference simply means executing a trained machine learning model using new input data.

Unlike training, inference does not modify the model. It only computes predictions.

Creating an ONNX Runtime session requires three primary components:

- ORTEnv

- ORTSessionOptions

- ORTSession

The environment configures runtime behavior and logging.

The session options allow developers to customize execution behavior.

Finally, the session loads the model into memory and prepares it for inference.

let env = try ORTEnv(loggingLevel: .warning)
let options = try ORTSessionOptions()
let modelPath = Bundle.main.path(forResource: "MoodClassifier", ofType: "onnx")! 
let session = try ORTSession(env: env, modelPath: modelPath, sessionOptions: options)
let outputs = try session.run(withInputs: [:], outputNames: ["logits"], runOptions: nil)

In this example, the model returns a tensor called logits.

Depending on how the model was exported, your output tensor may have a different name. Always inspect the exported model to determine the available output names.

Preparing Inputs for Transformer Models

Most transformer models, including DistilBERT, BERT, and RoBERTa, cannot process raw text directly.

Instead, they expect numerical tensors representing the input sentence.

This process is called tokenization.

Tokenization converts natural language into token IDs that correspond to entries within the model's vocabulary.

Alongside the token IDs, transformer models also require an attention mask.

The attention mask tells the model which tokens belong to the original sentence and which tokens are merely padding added to maintain a fixed sequence length.

Using the correct tokenizer is extremely important.

The tokenizer used during inference must be identical to the tokenizer used while training the model. Even small differences in vocabulary or preprocessing rules can generate completely different token IDs, resulting in poor predictions despite using the correct model.

Using Swift Transformers

Hugging Face provides an excellent package called Swift Transformers that simplifies tokenization directly within Swift.

The package can be added using Swift Package Manager.

.package(url: "https://github.com/huggingface/swift-transformers", from: "1.3.3" )

Once installed, you can load the tokenizer that matches the model used during training and generate the input_ids and attention_mask required by the transformer.

After generating these arrays, they must be converted into ONNX tensors before inference.

Creating ONNX Input Tensors

The generated token arrays must be wrapped inside ORTValue tensors.

The following example converts both arrays into tensors before executing the model.

let env = try ORTEnv(loggingLevel: .warning)
let options = try ORTSessionOptions() 
let modelPath = Bundle.main.path(forResource: "MoodClassifier", ofType: "onnx")! 
let session = try ORTSession(env: env, modelPath: modelPath, sessionOptions: options)
let inputData = NSMutableData(bytes: &inputIDs, length: inputIDs.count * MemoryLayout<Int64>.size)
let inputTensor = try ORTValue(tensorData: inputData, elementType: .int64, shape: [1, inputIDs.count] as [NSNumber])
let attentionData = NSMutableData(bytes: &attentionMask, length: attentionMask.count * MemoryLayout<Int64>.size)
let attentionTensor = try ORTValue(tensorData: attentionData, elementType: .int64, shape: [1, attentionMask.count] as [NSNumber])
let outputs = try session.run(withInputs: ["input_ids": inputTensor, "attention_mask": attentionTensor], outputNames: ["logits"], runOptions: nil)

In this example, two tensors are created: input_ids, which contains the numerical representation of the input text, and attention_mask, which tells the model which tokens should participate in the attention mechanism. These tensors are then passed into the ONNX Runtime session, which executes the model and returns the requested outputs.

Conclusion

On-device AI is no longer a niche capability reserved for flagship applications. Thanks to frameworks such as ONNX Runtime and Swift Transformers, integrating transformer models into iOS projects has become both accessible and practical.

In this article, we explored how to load an ONNX model, execute inference using Microsoft's ONNX Runtime, and prepare the input_ids and attention_mask tensors required by transformer models. These components form the foundation for deploying a wide range of AI-powered features directly within Swift applications.

As Apple's hardware continues to evolve and transformer models become increasingly efficient, local inference will play an even greater role in the future of mobile development. Whether you're building a sentiment analyzer, semantic search engine, recommendation system, or lightweight AI assistant, ONNX Runtime provides a robust and portable solution for bringing modern machine learning to iOS.