Apple Machine Learning Research engineering blog

Loading
Split-and-Scale Latent 3D Representations
Presents split-and-scale, a method that increases latent capacity of pretrained 3D tokenizers at inference by applying the same tokenizer to smaller sub-regions of a scene, since 3D scenes are represented as continuous functions of coordinates. The authors ablate compute, memory, latent capacity, and reconstruction quality, and report competitive results against larger models exceeding single-GPU memory, plus an autoregressive 3D generator built on the approach.
Theoretical Limits of Language Model Alignment
This Apple Machine Learning Research paper derives information-theoretic limits on reward improvement for KL-regularized language model alignment, showing that a Jeffreys divergence term governs the optimal gain. It proposes a covariance-based estimator that predicts achievable gains from base model samples, analyzes reward hacking under proxy rewards, and shows that reward ensembling mitigates it. Empirically, best-of-N nearly reaches the theoretical limit on safety and summarization tasks,...
Weighted State Souping for Efficient Long-Context Retrieval
Presents weighted state souping, a method that learns query-dependent weights for composing cached state space model encodings of document chunks to improve long-context retrieval. The authors report gains on document question answering, needle-in-a-haystack retrieval, variable tracking, and tool use, plus a sparse variant with near-constant encoding cost. Theoretical arguments are offered to explain when souping works and when it may fail.
Understanding Private Evolution as Learning-Augmented Clustering
This Apple Machine Learning Research paper recasts Private Evolution, a differentially private synthetic data generation algorithm, as generative model-augmented Wasserstein learning. The authors show that accounting for a generator capturing the true distribution yields much better bounds, such as sample complexity depending on intrinsic rather than ambient dimension. They also identify convergence failures in standard PE variants on well-clustered instances and propose a geometry-aware...
Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment
This Apple Machine Learning Research paper evaluates how well LLM judges assess personalized writing style. It compares human and judge-LLM pairwise annotations across four generator LLMs and three long-form writing tasks. The authors find that judge quality correlates weakly with MMLU, that paired comparisons are more consistent than independent scoring, and that the strongest LLMs do not most reliably reproduce human judgments.
On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models
This Apple Machine Learning Research paper, accepted to NeurIPS, studies why State Space Models lag Transformers on in-context learning and precise retrieval. The authors show through theory and experiments that the gating mechanism causes SSMs to first learn an in-weights memorization solution, delaying or preventing convergence to in-context learning, while gating often aids length generalization.
Toward a Theory of Evaluability
This Apple Machine Learning Research paper by Kunal Talwar formalizes evaluability for black-box models whose test samples may have leaked into training. It characterizes the distributions where accurate output-only evaluation is feasible, gives tight sample complexity bounds, and shows that limited query access can strengthen evaluators, drawing connections to self-correctors from program testing.
Faster Retrieval for Better, More Efficient RAG
Apple researchers present amortized maximum inner product search, training small neural networks offline to route queries to their best-matching keys faster than centroid-based IVF routing. SupportNet learns per-cluster support functions for better routing, while KeyNet acts as a query rewriter in front of FAISS IVF indexes, reaching 99% recall with up to roughly 8x lower latency on Natural Questions. The post details the convex-analysis basis, the tradeoffs between the two models, and measured...
Attention to Mamba: A Recipe for Cross-Architecture Distillation
This Apple Machine Learning Research paper proposes a two-stage recipe for distilling a Transformer into a Mamba-style state space model. It first distills the teacher into a linearized attention model using a kernel-trick adaptation, then distills that into an attention-free Mamba. The distilled Pythia-1B student reaches perplexity 14.01 against the teacher's 13.86, with gaps on in-context recall tasks.
GHARP: Real-Time Gaussian Head Animation from Large-Scale Reconstruction Prior
The authors present GHARP, a method that animates 3D human heads in real time from a few input images and a driving expression signal. It splits the work into an offline identity stage and a lightweight runtime animation network optimized for mobile devices, and adds a body alignment network to reduce blur and temporal flicker. The paper reports state-of-the-art quality on the Ava-256 benchmark, up to 13x faster runtime on an A100 GPU, and 8x fewer Gaussians.
Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
Apple's machine learning team benchmarks running very large open-weight LLMs on Mac Studio clusters using MLX. It compares M3 and M5 Ultra generations and evaluates pipeline versus tensor parallelization over RDMA-over-Thunderbolt, reporting TTFT, decode throughput, and power at 8K and 64K context. The post explains the trade-offs between the two parallelization schemes and where communication overhead dominates.
Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
The paper introduces Selection-based Structured Reasoning (SSR), which replaces freeform reasoning in small multimodal search agents with selection among pre-specified reusable reasoning candidates scored by likelihood. Parallel scoring over a shared KV cache is used to speed up inference. Across seven multimodal search benchmarks with 2B and 4B models, SSR reportedly cuts per-turn reasoning latency by over 90% and total inference latency by 28–54% while remaining competitive on success rate.
Plan or Retrieve First? A Scale Study of Tool Discovery Architectures
This Apple Machine Learning Research paper compares Retrieve-then-Plan and Plan-then-Retrieve architectures for selecting APIs in tool-augmented LLM agents on ToolBench's 46,984 endpoints. It finds that Plan-then-Retrieve outperforms on multi-tool queries while Retrieve-then-Plan wins on single-tool queries, and it analyzes stub hallucination, token overhead, and complexity-adaptive routing.
Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models
The post presents a lightweight method for detecting code-switching directly from speech, using language-selective hidden-state dimensions in pretrained speech encoders rather than transcription. Whisper-small's encoder reaches a mean F1 of 0.798 across 14 language pairs on CS-FLEURS, ahead of a CNN and a HuBERT linear baseline. The authors argue that pretrained encoders already encode the information needed to identify code-switched utterances.
The Design Space of Tri-Modal Masked Diffusion Models
Apple researchers introduce the first tri-modal Masked Diffusion Model, pretrained from scratch on text, image-text, and audio-text data. The paper analyzes multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects, and proposes an SDE-based reparameterization that decouples physical from logical batch size. A 3B-parameter model trained on 6.4T tokens shows results across text generation, text-to-image, and text-to-speech tasks.
Mix, Don't Tune: Bilingual Pre-Training Outperforms Hyperparameter Search in Data-Constrained Settings
This research paper compares hyperparameter tuning (such as high weight decay) against mixing auxiliary high-resource language data for pre-training language models in data-constrained settings. Using Arabic as the low-resource target and English as auxiliary across roughly 1000 runs and model scales from 150M to 1.43B parameters, the authors find mixing yields larger gains on validation loss and downstream accuracy, and that target-language validation loss underestimates mixing's value....
Self-Compacting Language Model Agents
The paper introduces SELFCOMPACT, an inference-time scaffold that lets language-model agents decide when and how to summarize long tool-use trajectories. Across six benchmarks and seven models, it reduces token costs by 30–70% while improving performance over unsummarized agents and matching or exceeding fixed-interval compaction.
Normalizing Trajectory Models
Apple researchers introduce Normalizing Trajectory Models, which represent each coarse diffusion reverse step as an expressive conditional normalizing flow with exact likelihood training. The approach supports self-distillation and produces competitive text-to-image samples in four steps while retaining likelihood over the full generative trajectory.