Skip to content

Add Apple Silicon (MPS) support #264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Add Apple Silicon (MPS) support #264

wants to merge 9 commits into from

Conversation

@provos
Copy link

@provos provos commented Nov 30, 2025

Adds MPS device support for both image and video predictors on Apple Silicon.

Changes:

  • Add get_default_device() utility that detects MPS availability
  • Fix device mismatches (coords cache, freqs_cis cache)
  • Add MPS workaround for complex tensor repeat() in RoPE
  • Make torch._assert_async conditional on CUDA
  • Fix MPS memory leak in video predictor via synchronization points

Performance of the Video predictor:

  • ~3x faster than CPU
  • Runs with ~38GB peak memory. This is due to the way that MPS caches graphs. Before adding the synchronization points, running the video predictor would consume all available memory.

@provos
chore: change to decord2 as plugin replacement for decord
3c30532 
this has prebuilt wheels for apple silicon. bump numpy from 1.26 to 1.26.4 to meet dependency requirements for decord2
@provos
fix: update numpy dependency to allow for newer versions
9b4b199
@provos
feat: add device management for CUDA and CPU support across models
351ce7b 
Allows systems without CUDA to fallback to CPU.
@provos
feat: implement tensor_to_device utility for efficient tensor transfers
51bc2f6 
The pin_memory() optimization is only available for CUDA backends.
@provos
fix: handle CPU fallback for mask interpolation in PostProcessImage a…
52a29b6 
…nd PostProcessAPIVideo
@provos
fix: handle empty batch case
8893f05 
CUDA handles this internally but we need to handle it directly for CPU
@provos
feat: implement get_default_device utility to pick between cuda, mps …
5b1b686 
…or cpu

introduce workarounds for torch operations not available on mps like repeats of complex tensors
@provos
fix: prevent excessive memory leakage on mps
9f6ddae 
forcefully flush pending operations with synchronize and empty the cache.
@provos
Merge remote-tracking branch 'origin/main' into apple-silicon-support-v2
28574df
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@provos