LearnComputing FoundationsMPS & Metal for ML on Mac

⚡EasyFine-Tuning & Training

MPS & Metal for ML on Mac

Train the same PyTorch ticket classifier on Apple silicon: select the `mps` device, keep tensors together, handle unsupported ops, time queued work honestly, and respond to unified-memory pressure.

15 min read

Learning path

Step 6 of 178 in the full curriculum

CUDA for ML Training Data Structures for AI

Personalize this lesson

Adapt explanations and teaching visuals to your background and preferred voice.

Platform path

If you train on Linux or Windows with an NVIDIA GPU, start with CUDA for ML Training. If you train on a Mac with Apple silicon, use this matching backend path after that lesson's placement and timing contracts.

This builds on the CUDA placement and timing contracts. Four ticket descriptions still need a device tag, a matching model, and a timer that waits for queued work. The teaching batch is (B, T, D) = (4, 8, 16): four tickets, eight token positions, sixteen features. On an Apple silicon Mac the tag is mps instead of cuda, and the memory hardware underneath is not a second VRAM pool.^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/}^{[2]Reference 2MPS backend.https://docs.pytorch.org/docs/stable/notes/mps}

Python still runs on the CPU. PyTorch launches GPU kernels only for tensors that already live on the selected device. The CUDA lesson used an NVIDIA device string. Here the same placement and timing contracts apply to Metal through PyTorch's MPS backend.

Metal, MPS, and the `mps` device

Metal is Apple's graphics and compute API. PyTorch doesn't ask you to write Metal shaders for ordinary training. It uses a Metal Performance Shaders (MPS) backend that maps tensor ops onto MPS Graph and tuned MPS kernels.^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/}^{[2]Reference 2MPS backend.https://docs.pytorch.org/docs/stable/notes/mps}

Three names sit on different layers:

Metal: Apple's GPU programming framework.
MPS: Metal Performance Shaders, the backend PyTorch talks to.
mps: the PyTorch device string you pass to tensors and modules.

So the Mac question isn't "CUDA or GPU?" It's "which backend does this machine expose to PyTorch?"

Backend status

Apple's PyTorch setup page still labels the MPS backend beta. Operator coverage changes across PyTorch releases. An available backend means the GPU is reachable, not that every model runs there or runs faster than CPU.^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/}^{[3]Reference 3MPS Environment Variables.https://docs.pytorch.org/docs/stable/mps_environment_variables.html}

One memory pool, two PyTorch devices

CUDA's picture was two physical pools: host RAM and discrete GPU video memory, with copies across an interconnect. Apple silicon uses a unified memory architecture: CPU and GPU share system memory instead of splitting storage into CPU RAM plus a separate VRAM card.^{[4]Reference 4MLX: An array framework for Apple siliconhttps://github.com/ml-explore/mlx}

Keep the contrast tight:

On a discrete NVIDIA GPU, tensor.to("cuda") places storage in the GPU's own device memory.
On Apple silicon, tensor.to("mps") doesn't cross that same RAM-to-VRAM boundary. It still returns an mps tensor and selects MPS-backed execution. Don't treat the framework move as free or as always zero-copy. Apple's MLX arrays are designed around shared memory; PyTorch still keeps cpu and mps as separate device targets.^{[4]Reference 4MLX: An array framework for Apple siliconhttps://github.com/ml-explore/mlx}^{[2]Reference 2MPS backend.https://docs.pytorch.org/docs/stable/notes/mps}

Side-by-side memory topology for the same (4, 8, 16) ticket batch. On a discrete NVIDIA GPU the batch starts in host RAM as a cpu tensor, then .to("cuda") copies it into a separate GPU VRAM pool as cuda:0. On Apple silicon the batch stays in one unified physical memory pool while .to("mps") still retags it as mps:0. Pool count changes; PyTorch still requires the model and batch to share one device tag. — The stacked bars are topology, not capacity: CUDA has two pools with a copy between them, Apple silicon has one shared pool. Either way the `(4, 8, 16)` batch and the classifier must share one PyTorch device tag before the forward pass.

Two practical consequences follow:

There isn't a dedicated VRAM budget for the integrated GPU. The GPU shares physical memory with the CPU, macOS, and other apps. The pool is still limited. macOS and PyTorch enforce working-set ceilings (recommended_max_memory, high watermark), so unified memory doesn't remove GPU memory pressure.
Placement is still explicit. You still write cpu and mps, keep model and batch on compatible devices, and avoid needless host-visible scalar reads in the hot path.

Unified memory changes the hardware boundary. It doesn't let you skip device discipline.

The CUDA lesson kept (4, 8, 16) small enough for any card. That tiny float32 batch is only 2 KiB of raw features. A more realistic classifier step, 32 tickets with 128 tokens and 768 features, is what makes the memory bill visible:

ticket_batch_bytes.py

tiny = 4 * 8 * 16 * 4
realistic = 32 * 128 * 768 * 4
mib = 1024 ** 2

print(f"CUDA teaching batch (4, 8, 16): {tiny / 1024:.1f} KiB")
print(f"larger step (32, 128, 768): {realistic / mib:.1f} MiB")
print("training also keeps weights, activations, gradients, and optimizer state")

Output

CUDA teaching batch (4, 8, 16): 2.0 KiB
larger step (32, 128, 768): 12.0 MiB
training also keeps weights, activations, gradients, and optimizer state

The expected output is the KiB and MiB pair for those two shapes. The 12.0 MiB figure is only the input features. Training still stores weights, activations, gradients, optimizer state, and temporary workspaces. On a Mac those tensors compete for the same system memory as the rest of the laptop.

Install on a supported Mac

Apple's setup page documents a wheel path for an Apple silicon Mac with macOS 14.0 or later, Python 3.10 or later, and Xcode command-line tools.^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/} That page names a specific PyTorch release. The PyTorch install selector names its own current stable build. Both numbers move, so treat Apple's page as the Mac requirements floor, then confirm the wheel for your Python and macOS versions.^{[5]Reference 5Get Started.https://pytorch.org/get-started/locally/}

Check the machine before changing model code. If xcode-select -p reports that developer tools are missing, run xcode-select --install once.

terminal

xcode-select -p
python3 --version
sw_vers

Create a virtual environment so this lesson doesn't replace packages used by another project. Then install the current torch wheel:

terminal

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install torch

Prove what the installed backend can do

Start with one tiny script that distinguishes three states:

This PyTorch binary was not built with MPS support.
The binary knows about MPS, but this machine or OS can't use it right now.
MPS is available, so you can move model and tensors onto mps.

PyTorch's own MPS note uses that is_built() / is_available() split. If the backend is missing from the wheel, it says the install was not built with MPS. If the wheel has MPS but the runtime can't use it, the current docs point at macOS below 14.0 or a machine without an MPS-capable device.^{[2]Reference 2MPS backend.https://docs.pytorch.org/docs/stable/notes/mps}

Diagram showing is_built()?, False, No MPS in this wheel, and True. — is_built()?, False, No MPS in this wheel, and True.

mps_sanity_check.py

import torch

has_mps_backend = hasattr(torch.backends, "mps")
mps_built = bool(has_mps_backend and torch.backends.mps.is_built())
mps_available = bool(has_mps_backend and torch.backends.mps.is_available())

device = torch.device("mps") if mps_available else torch.device("cpu")

x = torch.arange(6, dtype=torch.float32).reshape(2, 3).to(device)
model = torch.nn.Linear(3, 2).to(device)
y = model(x)

print(f"mps built: {mps_built}")
print(f"mps available: {mps_available}")
print(f"selected device: {device}")
print(f"output shape: {tuple(y.shape)}")

Output

mps built: True
mps available: True
selected device: mps
output shape: (2, 2)

The printed mps built / mps available lines above are an example from a compatible Apple silicon Mac. On non-Mac hosts, older macOS, or a CPU-only wheel, expect False and selected device: cpu. The output shape stays (2, 2) either way.

built = False means this PyTorch binary lacks MPS support. That's expected off macOS. On a supported Mac, check the wheel and Python architecture.
built = True, available = False usually means the backend exists in the package, but OS, hardware, or runtime access is missing.
available = True means you can use torch.device("mps").

is_built() answers "does this wheel even know about MPS?" is_available() answers "can this specific machine use it right now?" Keep those two questions separate.

The same four tickets on `mps`

Reuse the CUDA lesson's batch. Four tickets, eight token positions, sixteen features. Average those token positions into one vector per ticket, then produce three logits per ticket. Labels stay [0, 2, 1, 0].

On a Mac training run, one small step usually looks like this:

Step	CPU side	`mps` side	Why it matters
batch assembly	tokenizer, collator, padding, labels	nothing yet	data still starts on host
device move	Python asks for `.to("mps")`	batch becomes an `mps` tensor	placement is explicit
forward pass	host launches ops	Metal kernels run the math	most heavy arithmetic lives here
loss read	maybe host asks for a scalar	device may need to finish queued work first	logging can stall the loop
backward pass	autograd schedules gradient work	gradient kernels run on `mps`	memory now includes activations and grads
optimizer step	host calls `step()`	parameter updates happen on `mps`	model stays on device across steps

Use the table as a debugging trace: for each row, name tensor location, kernel location, synchronization point, and likely failure.

Unified memory tempts people into "one laptop, one pool, one device." The hardware shares memory. PyTorch still doesn't. cpu and mps are separate device targets, and the forward pass still fails if model and batch land on different devices.^{[2]Reference 2MPS backend.https://docs.pytorch.org/docs/stable/notes/mps}

mps_device_placement.py

import torch
import torch.nn as nn

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model = nn.Linear(16, 3).to(device)
ticket_batch = torch.randn(4, 8, 16).to(device)

ticket_vectors = ticket_batch.mean(dim=1)
logits = model(ticket_vectors)
devices_match = next(model.parameters()).device == ticket_batch.device == logits.device
print("model and batch agree:", devices_match)
print("ticket vectors:", tuple(ticket_vectors.shape))
print("logits shape:", tuple(logits.shape))

Output

model and batch agree: True
ticket vectors: (4, 16)
logits shape: (4, 3)

That's the CUDA shape contract with a different device string: ticket vectors (4, 16), logits (4, 3). Continue from the placed classifier with one optimizer step. It uses MPS when available and still runs on a non-Mac machine:

one_mps_ticket_step.py

import math

import torch.nn.functional as F

optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
labels = torch.tensor([0, 2, 1, 0], device=device)
weights_before = model.weight.detach().clone()

optimizer.zero_grad()
loss = F.cross_entropy(model(ticket_vectors), labels)
loss.backward()
optimizer.step()
logged_loss = loss.detach().cpu().item()

print("weights changed:", not torch.equal(weights_before, model.weight.detach()))
print("finite loss:", math.isfinite(logged_loss))

Output

weights changed: True
finite loss: True

The .cpu().item() call sits outside gradient computation. It marks the boundary where a scalar returns to the host for reporting.

If model and batch devices don't agree, fix placement before hunting deeper bugs:

catch_mps_mismatch.py

import torch

def require_same_device(model_device: torch.device, batch: torch.Tensor) -> None:
    if batch.device != model_device:
        raise RuntimeError(f"batch device does not match model device {model_device}")

batch = torch.randn(4, 8, 16)
try:
    require_same_device(torch.device("mps"), batch)
except RuntimeError as error:
    print("caught:", error)

Output

caught: batch device does not match model device mps

Same rule as CUDA. The Mac path is not an exemption from device consistency.

One Mac GPU isn't a CUDA cluster

PyTorch exposes torch.mps.device_count() for the number of available MPS devices.^{[6]Reference 6torch.mps.https://docs.pytorch.org/docs/stable/mps.html} This lesson uses the default mps device, which Apple's verification example prints as mps:0.^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/} Don't translate an Apple GPU core count into mps:0, mps:1, and so on. Core count and PyTorch device count answer different questions.

That differs from a CUDA server with several discrete cards. If a job needs multi-device or multi-node training, verify the target framework and communication backend on that cluster instead of translating CUDA distributed settings onto the Mac. The local MPS path here is for one-device development and smaller training runs.

Unsupported operations and CPU fallback

An operation without an MPS implementation can stop an otherwise valid training loop. PyTorch exposes PYTORCH_ENABLE_MPS_FALLBACK=1 so unsupported MPS operations can run on CPU instead of failing immediately.^{[3]Reference 3MPS Environment Variables.https://docs.pytorch.org/docs/stable/mps_environment_variables.html} Treat that flag as a temporary compatibility aid, not a promise that every unsupported operation will work.

terminal

PYTORCH_ENABLE_MPS_FALLBACK=1 python train.py

Set the variable in the process environment before PyTorch loads. Putting os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1" after import torch is often a no-op, because the C++ backend reads that state at initialization.

Fallback is a debugging tool

Fallback keeps debugging unblocked, but it can hide a CPU detour inside the accelerator path. That detour may add synchronization, device conversion, and copy work even though CPU and GPU share physical memory. If one hot operation keeps falling back, throughput can collapse while the script still looks correct.

CPU work isn't automatically fallback. Tokenization and batch assembly normally happen on CPU before .to("mps"). MPS fallback means a PyTorch operation on the accelerator path has no MPS implementation and runs on CPU instead.

One training-step bounce: the (4, 8, 16) ticket batch is tokenized on CPU, tagged mps, and mean runs as an MPS kernel. An unsupported operator then drops to CPU with a host sync before Linear(16, 3) continues on mps. Tokenization before .to("mps") is ordinary host prep, not this mid-step fallback. — Follow the dip: host prep on CPU is expected, then `.to("mps")` tags the batch. A missing MPS kernel is the expensive bounce. Linear continues on `mps` only after that sync.

Use fallback to identify the blocking operation. Then remove the flag and choose deliberately: rewrite that operation, try a newer supported PyTorch release, keep the full workload on CPU, or accept the measured detour.

Keep precision changes measurable

The examples so far use float32. Keep that as the correctness baseline before trying lower precision. A blanket model.half() changes every floating-point parameter, including operations that may need more range, and makes numerical failures harder to localize.

PyTorch's automatic mixed precision (AMP) API chooses dtypes per operation. Its documentation recommends leaving model and inputs in their normal dtype, wrapping the forward pass and loss with torch.autocast, and using gradient scaling when float16 training needs it.^{[7]Reference 7Automatic Mixed Precision package - torch.amphttps://docs.pytorch.org/docs/stable/amp.html} The official AMP examples are written for CUDA and CPU. On Mac, ask the installed build whether autocast is available for mps before copying a CUDA AMP snippet:

mps_precision_probe.py

from contextlib import nullcontext

import torch

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model = torch.nn.Linear(8, 3).to(device)
features = torch.randn(4, 8, device=device)

use_mps_amp = (
    device.type == "mps"
    and torch.amp.autocast_mode.is_autocast_available("mps")
)
precision_context = (
    torch.autocast(device_type="mps", dtype=torch.float16)
    if use_mps_amp
    else nullcontext()
)

with precision_context:
    logits = model(features)
    loss = logits.square().mean()

print("device:", device.type)
print("MPS autocast enabled:", use_mps_amp)
print("logits dtype:", logits.dtype)
print("finite loss:", bool(torch.isfinite(loss).item()))

On a compatible Mac, this probe tells you whether the installed release enables MPS autocast and which dtype reached the logits. It doesn't prove the full training job is stable. Compare loss curves or evaluation metrics with the float32 baseline, then compare synchronized step time and peak memory. If results become non-finite or accuracy shifts, return to float32 before changing anything else. The later Mixed Precision Training lesson develops autocast and gradient scaling in depth.

Time and profile MPS work

The Mac timing trap looks like the CUDA timing trap. Kernel launches and queued device work can make naive timers lie. Warm up the operation first, then synchronize before and after the measured block. PyTorch exposes torch.mps.synchronize() for that boundary.^{[6]Reference 6torch.mps.https://docs.pytorch.org/docs/stable/mps.html}

mps_timing.py

import time
import torch

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
x = torch.randn(256, 256, device=device)
w = torch.randn(256, 256, device=device)

for _ in range(3):
    y = x @ w
if device.type == "mps":
    torch.mps.synchronize()

start = time.perf_counter()
for _ in range(10):
    y = x @ w
if device.type == "mps":
    torch.mps.synchronize()
elapsed_ms = (time.perf_counter() - start) * 1000

print("timed matmuls:", 10)
print("result shape:", tuple(y.shape))
print("elapsed is nonnegative:", elapsed_ms >= 0)

Output

timed matmuls: 10
result shape: (256, 256)
elapsed is nonnegative: True

Same hidden sync points still matter:

loss.item() when loss is an MPS tensor
tensor.cpu(), including tensor.cpu().numpy() for NumPy analysis
printing values that must come back to host memory

Calling .numpy() directly on an MPS tensor isn't the route back to NumPy: move the data to CPU first.

mps_logging_boundary.py

import torch

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
loss = torch.tensor(2.5, device=device)
reported_loss = loss.detach().cpu().item()

print(f"reported loss: {reported_loss:.1f}")

Output

reported loss: 2.5

Don't trust timing claims until you know whether the host waited for the device. Also benchmark CPU and MPS on the same Mac, with the same shapes, dtype, warmup, and number of steps. An available GPU isn't proof that a tiny (4, 8, 16) step is faster there.

When a timer says the step is slow

A synchronized timer tells you whether the step is slow. A trace helps explain where time went. PyTorch's MPS profiler emits operating-system signposts that Xcode Instruments can record and display.^{[6]Reference 6torch.mps.https://docs.pytorch.org/docs/stable/mps.html}

After opening Instruments with an OS Signpost or Logging trace, wrap only the steady-state region you want to inspect:

profile_mps_step.py

import torch

if not torch.backends.mps.is_available():
    raise SystemExit("Run this profiler example on an MPS-capable Mac")

x = torch.randn(256, 256, device="mps")
w = torch.randn(256, 256, device="mps")

for _ in range(3):
    y = x @ w
torch.mps.synchronize()

with torch.mps.profiler.profile(
    mode="interval",
    wait_until_completed=False,
):
    for _ in range(10):
        y = x @ w
torch.mps.synchronize()
print("profiled matmuls:", 10)

Output

profiled matmuls: 10

Keep wait_until_completed=False for a representative trace. Setting it to True waits after each encoded operation, which can make individual dispatches easier to inspect but changes the performance you're trying to understand. Look for long gaps between operations, many tiny dispatches, and CPU detours before guessing at a fix.^{[6]Reference 6torch.mps.https://docs.pytorch.org/docs/stable/mps.html}

Memory pressure on Apple GPUs

The first memory lesson is the same as CUDA: weights aren't the whole bill. Activations, gradients, optimizer state, and temporary workspaces matter too.

Unified memory adds one twist. There isn't a separate video-memory pool: a large run competes for system memory with macOS and every other app. PyTorch exposes current_allocated_memory() for bytes occupied by live tensors, driver_allocated_memory() for total memory allocated by Metal for the process (including cached allocator blocks and MPS/MPSGraph allocations), and empty_cache() to release unoccupied cached memory.^{[6]Reference 6torch.mps.https://docs.pytorch.org/docs/stable/mps.html} empty_cache() doesn't free tensors that are still alive, so it can't repair a workload whose real peak exceeds the limit.

Inspect those counters only after MPS is available:

mps_allocator_check.py

import torch

if torch.backends.mps.is_available():
    before = torch.mps.current_allocated_memory()
    tensor = torch.ones(1024, 1024, device="mps")
    after = torch.mps.current_allocated_memory()
    del tensor
    torch.mps.empty_cache()
    print("live tensor allocation increased:", after > before)
    print("recommended limit reported:", torch.mps.recommended_max_memory() > 0)
else:
    print("MPS allocator counters need an available mps device")

When memory gets tight, fix order should stay boring:

Symptom	First question	First fix
OOM on first real batch	Is batch or sequence length too large?	shrink batch size first
Step time swings wildly	Are unsupported ops or sync points bouncing work back to CPU?	check fallback and logging paths
MPS allocator errors	Are you near working-set limits?	reduce workload before touching allocator env vars
macOS starts swapping	Is training competing with other memory-heavy apps?	stop the run, reduce workload, then close unneeded apps

PyTorch also exposes MPS-specific allocator controls such as PYTORCH_MPS_HIGH_WATERMARK_RATIO and PYTORCH_MPS_LOW_WATERMARK_RATIO. The current docs default the high watermark to 1.7 and use different low-watermark defaults for unified versus discrete GPUs.^{[3]Reference 3MPS Environment Variables.https://docs.pytorch.org/docs/stable/mps_environment_variables.html} Those are advanced tuning knobs, not first response. Disabling the high watermark (0.0) can cause system failure under system-wide out-of-memory conditions. Start by shrinking work.

For the larger (32, 128, 768) budget, common first fixes stay boring on purpose:

lower per-step batch size before touching allocator ratios
shorten sequence length if the task allows it
remove needless .cpu() calls before blaming Metal
confirm fallback isn't firing inside the hot path

mps_workload_reduction.py

batch_size = 32
sequence_length = 128
baseline_positions = batch_size * sequence_length

for label, batch, tokens in [
    ("baseline", 32, 128),
    ("half batch", 16, 128),
    ("half length", 32, 64),
]:
    share = (batch * tokens) / baseline_positions
    print(f"{label:11s}: {share:.0%} of token positions")

Output

baseline   : 100% of token positions
half batch : 50% of token positions
half length: 50% of token positions

Memory lever: Both changes halve token positions and many activation tensors. Shorter sequences can reduce attention score tensors faster because attention has two sequence axes.

Diagnose by symptom, not by backend name

Use the smallest failing batch and preserve its shape, dtype, device, and PyTorch version. Those four facts separate placement, operator coverage, precision, timing, and capacity failures that otherwise all look like "MPS is slow."

Observed symptom	Most likely boundary to check first	Evidence to collect
`is_built()` or `is_available()` is false	package, hardware, or macOS	both booleans, `torch.__version__`, `sw_vers`
forward pass reports mixed devices	placement	model parameter device and every batch tensor device
run needs the fallback flag	operator coverage	exact unsupported operator and PyTorch version
loss becomes `NaN` after precision change	numerical range	dtype, first non-finite step, float32 baseline
host timer looks impossibly fast	asynchronous execution	warmup plus synchronized timing
allocator OOMs or macOS swaps	live workload and system pressure	batch shape, both MPS memory counters, open-app pressure

What to take onto the next lesson

Metal is Apple's GPU stack. mps is PyTorch's device name for using it.
Apple silicon uses unified memory: CPU and GPU share system memory, but .to("mps") still selects MPS execution and doesn't promise a cost-free move. Apple currently labels the MPS backend beta.^{[4]Reference 4MLX: An array framework for Apple siliconhttps://github.com/ml-explore/mlx}^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/}
Mac training still needs explicit device placement. The CUDA (4, 8, 16) ticket step is the same contract with a different tag.
is_built() and is_available() answer different setup questions.
PYTORCH_ENABLE_MPS_FALLBACK=1 is useful, but it can hide slow CPU detours.
Keep float32 as the baseline; test autocast against accuracy, memory, and synchronized timing.
Honest timing needs synchronization, while MPS signpost traces explain slow dispatch paths.

If you also work on NVIDIA servers, keep CUDA for ML Training nearby. Same placement and timing habits, different backend details.

Next Step

Continue to Data Structures for AI

You can now place a training batch on `cuda` or `mps`, time queued kernels honestly, and shrink work when memory pressure shows up. The next lesson leaves the accelerator and asks a host-side question: once those tensors feed real pipelines, which in-memory structures make lookups, queues, caches, and <span data-glossary="top-k">top-k</span> cheap enough to keep the GPU busy.

PreviousCUDA for ML Training

Share this article

X Facebook LinkedIn Bluesky Reddit Hacker News Email

References

Accelerated PyTorch training on Mac.

Apple · 2026

https://developer.apple.com/metal/pytorch/

MPS backend.

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/notes/mps

MPS Environment Variables.

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/mps_environment_variables.html

MLX: An array framework for Apple silicon

Apple (ml-explore) · 2026

https://github.com/ml-explore/mlx

Get Started.

PyTorch Contributors · 2026

https://pytorch.org/get-started/locally/

torch.mps.

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/mps.html

Automatic Mixed Precision package - torch.amp

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/amp.html

Discussion

Questions and insights from fellow learners.

Discussion loads when you reach this section.

Back to Topics

LearnComputing FoundationsMPS & Metal for ML on Mac

⚡EasyFine-Tuning & Training

MPS & Metal for ML on Mac

Train the same PyTorch ticket classifier on Apple silicon: select the `mps` device, keep tensors together, handle unsupported ops, time queued work honestly, and respond to unified-memory pressure.

15 min read

Learning path

Step 6 of 178 in the full curriculum

CUDA for ML Training Data Structures for AI

Personalize this lesson

Adapt explanations and teaching visuals to your background and preferred voice.

Platform path

Metal, MPS, and the `mps` device

Three names sit on different layers:

Metal: Apple's GPU programming framework.
MPS: Metal Performance Shaders, the backend PyTorch talks to.
mps: the PyTorch device string you pass to tensors and modules.

So the Mac question isn't "CUDA or GPU?" It's "which backend does this machine expose to PyTorch?"

Backend status

One memory pool, two PyTorch devices

Keep the contrast tight:

On a discrete NVIDIA GPU, tensor.to("cuda") places storage in the GPU's own device memory.
On Apple silicon, tensor.to("mps") doesn't cross that same RAM-to-VRAM boundary. It still returns an mps tensor and selects MPS-backed execution. Don't treat the framework move as free or as always zero-copy. Apple's MLX arrays are designed around shared memory; PyTorch still keeps cpu and mps as separate device targets.^{[4]Reference 4MLX: An array framework for Apple siliconhttps://github.com/ml-explore/mlx}^{[2]Reference 2MPS backend.https://docs.pytorch.org/docs/stable/notes/mps}

Two practical consequences follow:

There isn't a dedicated VRAM budget for the integrated GPU. The GPU shares physical memory with the CPU, macOS, and other apps. The pool is still limited. macOS and PyTorch enforce working-set ceilings (recommended_max_memory, high watermark), so unified memory doesn't remove GPU memory pressure.
Placement is still explicit. You still write cpu and mps, keep model and batch on compatible devices, and avoid needless host-visible scalar reads in the hot path.

Unified memory changes the hardware boundary. It doesn't let you skip device discipline.

ticket_batch_bytes.py

tiny = 4 * 8 * 16 * 4
realistic = 32 * 128 * 768 * 4
mib = 1024 ** 2

print(f"CUDA teaching batch (4, 8, 16): {tiny / 1024:.1f} KiB")
print(f"larger step (32, 128, 768): {realistic / mib:.1f} MiB")
print("training also keeps weights, activations, gradients, and optimizer state")

Output

CUDA teaching batch (4, 8, 16): 2.0 KiB
larger step (32, 128, 768): 12.0 MiB
training also keeps weights, activations, gradients, and optimizer state

Install on a supported Mac

Check the machine before changing model code. If xcode-select -p reports that developer tools are missing, run xcode-select --install once.

terminal

xcode-select -p
python3 --version
sw_vers

Create a virtual environment so this lesson doesn't replace packages used by another project. Then install the current torch wheel:

terminal

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install torch

Prove what the installed backend can do

Start with one tiny script that distinguishes three states:

This PyTorch binary was not built with MPS support.
The binary knows about MPS, but this machine or OS can't use it right now.
MPS is available, so you can move model and tensors onto mps.

mps_sanity_check.py

import torch

has_mps_backend = hasattr(torch.backends, "mps")
mps_built = bool(has_mps_backend and torch.backends.mps.is_built())
mps_available = bool(has_mps_backend and torch.backends.mps.is_available())

device = torch.device("mps") if mps_available else torch.device("cpu")

x = torch.arange(6, dtype=torch.float32).reshape(2, 3).to(device)
model = torch.nn.Linear(3, 2).to(device)
y = model(x)

print(f"mps built: {mps_built}")
print(f"mps available: {mps_available}")
print(f"selected device: {device}")
print(f"output shape: {tuple(y.shape)}")

Output

mps built: True
mps available: True
selected device: mps
output shape: (2, 2)

built = False means this PyTorch binary lacks MPS support. That's expected off macOS. On a supported Mac, check the wheel and Python architecture.
built = True, available = False usually means the backend exists in the package, but OS, hardware, or runtime access is missing.
available = True means you can use torch.device("mps").

is_built() answers "does this wheel even know about MPS?" is_available() answers "can this specific machine use it right now?" Keep those two questions separate.

The same four tickets on `mps`

On a Mac training run, one small step usually looks like this:

Step	CPU side	`mps` side	Why it matters
batch assembly	tokenizer, collator, padding, labels	nothing yet	data still starts on host
device move	Python asks for `.to("mps")`	batch becomes an `mps` tensor	placement is explicit
forward pass	host launches ops	Metal kernels run the math	most heavy arithmetic lives here
loss read	maybe host asks for a scalar	device may need to finish queued work first	logging can stall the loop
backward pass	autograd schedules gradient work	gradient kernels run on `mps`	memory now includes activations and grads
optimizer step	host calls `step()`	parameter updates happen on `mps`	model stays on device across steps

Use the table as a debugging trace: for each row, name tensor location, kernel location, synchronization point, and likely failure.

mps_device_placement.py

import torch
import torch.nn as nn

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model = nn.Linear(16, 3).to(device)
ticket_batch = torch.randn(4, 8, 16).to(device)

ticket_vectors = ticket_batch.mean(dim=1)
logits = model(ticket_vectors)
devices_match = next(model.parameters()).device == ticket_batch.device == logits.device
print("model and batch agree:", devices_match)
print("ticket vectors:", tuple(ticket_vectors.shape))
print("logits shape:", tuple(logits.shape))

Output

model and batch agree: True
ticket vectors: (4, 16)
logits shape: (4, 3)

one_mps_ticket_step.py

import math

import torch.nn.functional as F

optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
labels = torch.tensor([0, 2, 1, 0], device=device)
weights_before = model.weight.detach().clone()

optimizer.zero_grad()
loss = F.cross_entropy(model(ticket_vectors), labels)
loss.backward()
optimizer.step()
logged_loss = loss.detach().cpu().item()

print("weights changed:", not torch.equal(weights_before, model.weight.detach()))
print("finite loss:", math.isfinite(logged_loss))

Output

weights changed: True
finite loss: True

The .cpu().item() call sits outside gradient computation. It marks the boundary where a scalar returns to the host for reporting.

If model and batch devices don't agree, fix placement before hunting deeper bugs:

catch_mps_mismatch.py

import torch

def require_same_device(model_device: torch.device, batch: torch.Tensor) -> None:
    if batch.device != model_device:
        raise RuntimeError(f"batch device does not match model device {model_device}")

batch = torch.randn(4, 8, 16)
try:
    require_same_device(torch.device("mps"), batch)
except RuntimeError as error:
    print("caught:", error)

Output

caught: batch device does not match model device mps

Same rule as CUDA. The Mac path is not an exemption from device consistency.

One Mac GPU isn't a CUDA cluster

Unsupported operations and CPU fallback

terminal

PYTORCH_ENABLE_MPS_FALLBACK=1 python train.py

Fallback is a debugging tool

Keep precision changes measurable

mps_precision_probe.py

from contextlib import nullcontext

import torch

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model = torch.nn.Linear(8, 3).to(device)
features = torch.randn(4, 8, device=device)

use_mps_amp = (
    device.type == "mps"
    and torch.amp.autocast_mode.is_autocast_available("mps")
)
precision_context = (
    torch.autocast(device_type="mps", dtype=torch.float16)
    if use_mps_amp
    else nullcontext()
)

with precision_context:
    logits = model(features)
    loss = logits.square().mean()

print("device:", device.type)
print("MPS autocast enabled:", use_mps_amp)
print("logits dtype:", logits.dtype)
print("finite loss:", bool(torch.isfinite(loss).item()))

Time and profile MPS work

mps_timing.py

import time
import torch

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
x = torch.randn(256, 256, device=device)
w = torch.randn(256, 256, device=device)

for _ in range(3):
    y = x @ w
if device.type == "mps":
    torch.mps.synchronize()

start = time.perf_counter()
for _ in range(10):
    y = x @ w
if device.type == "mps":
    torch.mps.synchronize()
elapsed_ms = (time.perf_counter() - start) * 1000

print("timed matmuls:", 10)
print("result shape:", tuple(y.shape))
print("elapsed is nonnegative:", elapsed_ms >= 0)

Output

timed matmuls: 10
result shape: (256, 256)
elapsed is nonnegative: True

Same hidden sync points still matter:

loss.item() when loss is an MPS tensor
tensor.cpu(), including tensor.cpu().numpy() for NumPy analysis
printing values that must come back to host memory

Calling .numpy() directly on an MPS tensor isn't the route back to NumPy: move the data to CPU first.

mps_logging_boundary.py

import torch

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
loss = torch.tensor(2.5, device=device)
reported_loss = loss.detach().cpu().item()

print(f"reported loss: {reported_loss:.1f}")

Output

reported loss: 2.5

When a timer says the step is slow

After opening Instruments with an OS Signpost or Logging trace, wrap only the steady-state region you want to inspect:

profile_mps_step.py

import torch

if not torch.backends.mps.is_available():
    raise SystemExit("Run this profiler example on an MPS-capable Mac")

x = torch.randn(256, 256, device="mps")
w = torch.randn(256, 256, device="mps")

for _ in range(3):
    y = x @ w
torch.mps.synchronize()

with torch.mps.profiler.profile(
    mode="interval",
    wait_until_completed=False,
):
    for _ in range(10):
        y = x @ w
torch.mps.synchronize()
print("profiled matmuls:", 10)

Output

profiled matmuls: 10

Memory pressure on Apple GPUs

The first memory lesson is the same as CUDA: weights aren't the whole bill. Activations, gradients, optimizer state, and temporary workspaces matter too.

Inspect those counters only after MPS is available:

mps_allocator_check.py

import torch

if torch.backends.mps.is_available():
    before = torch.mps.current_allocated_memory()
    tensor = torch.ones(1024, 1024, device="mps")
    after = torch.mps.current_allocated_memory()
    del tensor
    torch.mps.empty_cache()
    print("live tensor allocation increased:", after > before)
    print("recommended limit reported:", torch.mps.recommended_max_memory() > 0)
else:
    print("MPS allocator counters need an available mps device")

When memory gets tight, fix order should stay boring:

Symptom	First question	First fix
OOM on first real batch	Is batch or sequence length too large?	shrink batch size first
Step time swings wildly	Are unsupported ops or sync points bouncing work back to CPU?	check fallback and logging paths
MPS allocator errors	Are you near working-set limits?	reduce workload before touching allocator env vars
macOS starts swapping	Is training competing with other memory-heavy apps?	stop the run, reduce workload, then close unneeded apps

For the larger (32, 128, 768) budget, common first fixes stay boring on purpose:

lower per-step batch size before touching allocator ratios
shorten sequence length if the task allows it
remove needless .cpu() calls before blaming Metal
confirm fallback isn't firing inside the hot path

mps_workload_reduction.py

batch_size = 32
sequence_length = 128
baseline_positions = batch_size * sequence_length

for label, batch, tokens in [
    ("baseline", 32, 128),
    ("half batch", 16, 128),
    ("half length", 32, 64),
]:
    share = (batch * tokens) / baseline_positions
    print(f"{label:11s}: {share:.0%} of token positions")

Output

baseline   : 100% of token positions
half batch : 50% of token positions
half length: 50% of token positions

Memory lever: Both changes halve token positions and many activation tensors. Shorter sequences can reduce attention score tensors faster because attention has two sequence axes.

Diagnose by symptom, not by backend name

Observed symptom	Most likely boundary to check first	Evidence to collect
`is_built()` or `is_available()` is false	package, hardware, or macOS	both booleans, `torch.__version__`, `sw_vers`
forward pass reports mixed devices	placement	model parameter device and every batch tensor device
run needs the fallback flag	operator coverage	exact unsupported operator and PyTorch version
loss becomes `NaN` after precision change	numerical range	dtype, first non-finite step, float32 baseline
host timer looks impossibly fast	asynchronous execution	warmup plus synchronized timing
allocator OOMs or macOS swaps	live workload and system pressure	batch shape, both MPS memory counters, open-app pressure

What to take onto the next lesson

Metal is Apple's GPU stack. mps is PyTorch's device name for using it.
Apple silicon uses unified memory: CPU and GPU share system memory, but .to("mps") still selects MPS execution and doesn't promise a cost-free move. Apple currently labels the MPS backend beta.^{[4]Reference 4MLX: An array framework for Apple siliconhttps://github.com/ml-explore/mlx}^{[1]Reference 1Accelerated PyTorch training on Mac.https://developer.apple.com/metal/pytorch/}
Mac training still needs explicit device placement. The CUDA (4, 8, 16) ticket step is the same contract with a different tag.
is_built() and is_available() answer different setup questions.
PYTORCH_ENABLE_MPS_FALLBACK=1 is useful, but it can hide slow CPU detours.
Keep float32 as the baseline; test autocast against accuracy, memory, and synchronized timing.
Honest timing needs synchronization, while MPS signpost traces explain slow dispatch paths.

If you also work on NVIDIA servers, keep CUDA for ML Training nearby. Same placement and timing habits, different backend details.

Next Step

Continue to Data Structures for AI

PreviousCUDA for ML Training

Share this article

X Facebook LinkedIn Bluesky Reddit Hacker News Email

References

Accelerated PyTorch training on Mac.

Apple · 2026

https://developer.apple.com/metal/pytorch/

MPS backend.

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/notes/mps

MPS Environment Variables.

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/mps_environment_variables.html

MLX: An array framework for Apple silicon

Apple (ml-explore) · 2026

https://github.com/ml-explore/mlx

Get Started.

PyTorch Contributors · 2026

https://pytorch.org/get-started/locally/

torch.mps.

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/mps.html

Automatic Mixed Precision package - torch.amp

PyTorch Contributors · 2026

https://docs.pytorch.org/docs/stable/amp.html

Discussion

Questions and insights from fellow learners.

Discussion loads when you reach this section.

MPS & Metal for ML on Mac

Metal, MPS, and the `mps` device

One memory pool, two PyTorch devices

Install on a supported Mac

Prove what the installed backend can do

The same four tickets on `mps`

One Mac GPU isn't a CUDA cluster

Unsupported operations and CPU fallback

Your tokenizer runs on CPU before `ticket_batch.to("mps")`. Is that MPS fallback?

Keep precision changes measurable

Time and profile MPS work

When a timer says the step is slow

Memory pressure on Apple GPUs

Diagnose by symptom, not by backend name

You are on an Apple silicon Mac. `torch.backends.mps.is_built()` is `True`, but `torch.backends.mps.is_available()` is `False`. What does that tell you first?

Your training loop "works" only with `PYTORCH_ENABLE_MPS_FALLBACK=1`, but step time is awful. What is most likely happening?

What to take onto the next lesson

Mastery Check

Discussion

MPS & Metal for ML on Mac

Metal, MPS, and the `mps` device

One memory pool, two PyTorch devices

Install on a supported Mac

Prove what the installed backend can do

The same four tickets on `mps`

One Mac GPU isn't a CUDA cluster

Unsupported operations and CPU fallback

Your tokenizer runs on CPU before `ticket_batch.to("mps")`. Is that MPS fallback?

Keep precision changes measurable

Time and profile MPS work

When a timer says the step is slow

Memory pressure on Apple GPUs

Diagnose by symptom, not by backend name

You are on an Apple silicon Mac. `torch.backends.mps.is_built()` is `True`, but `torch.backends.mps.is_available()` is `False`. What does that tell you first?

Your training loop "works" only with `PYTORCH_ENABLE_MPS_FALLBACK=1`, but step time is awful. What is most likely happening?

What to take onto the next lesson

Mastery Check

Discussion

Your tokenizer runs on CPU before ticket_batch.to("mps"). Is that MPS fallback?

Your tokenizer runs on CPU before ticket_batch.to("mps"). Is that MPS fallback?

You are on an Apple silicon Mac. torch.backends.mps.is_built() is True, but torch.backends.mps.is_available() is False. What does that tell you first?

You are on an Apple silicon Mac. torch.backends.mps.is_built() is True, but torch.backends.mps.is_available() is False. What does that tell you first?

Your training loop "works" only with PYTORCH_ENABLE_MPS_FALLBACK=1, but step time is awful. What is most likely happening?

Your training loop "works" only with PYTORCH_ENABLE_MPS_FALLBACK=1, but step time is awful. What is most likely happening?

Mastery Check

Discussion

Your tokenizer runs on CPU before ticket_batch.to("mps"). Is that MPS fallback?

Your tokenizer runs on CPU before ticket_batch.to("mps"). Is that MPS fallback?

You are on an Apple silicon Mac. torch.backends.mps.is_built() is True, but torch.backends.mps.is_available() is False. What does that tell you first?

You are on an Apple silicon Mac. torch.backends.mps.is_built() is True, but torch.backends.mps.is_available() is False. What does that tell you first?

Your training loop "works" only with PYTORCH_ENABLE_MPS_FALLBACK=1, but step time is awful. What is most likely happening?

Your training loop "works" only with PYTORCH_ENABLE_MPS_FALLBACK=1, but step time is awful. What is most likely happening?

Mastery Check

Discussion

Your tokenizer runs on CPU before `ticket_batch.to("mps")`. Is that MPS fallback?

You are on an Apple silicon Mac. `torch.backends.mps.is_built()` is `True`, but `torch.backends.mps.is_available()` is `False`. What does that tell you first?

Your training loop "works" only with `PYTORCH_ENABLE_MPS_FALLBACK=1`, but step time is awful. What is most likely happening?

Your tokenizer runs on CPU before `ticket_batch.to("mps")`. Is that MPS fallback?

You are on an Apple silicon Mac. `torch.backends.mps.is_built()` is `True`, but `torch.backends.mps.is_available()` is `False`. What does that tell you first?

Your training loop "works" only with `PYTORCH_ENABLE_MPS_FALLBACK=1`, but step time is awful. What is most likely happening?