Presents split-and-scale, a method that increases latent capacity of pretrained 3D tokenizers at inference by applying the same tokenizer to smaller sub-regions of a scene, since 3D scenes are represented as continuous functions of coordinates. The authors ablate compute, memory, latent capacity, and reconstruction quality, and report competitive results against larger models exceeding single-GPU memory, plus an autoregressive 3D generator built on the approach.
Apple Machine Learning Research engineering blog
Loading
This Apple Machine Learning Research paper derives information-theoretic limits on reward improvement for KL-regularized language model alignment, showing that a Jeffreys divergence term governs the optimal gain. It proposes a covariance-based estimator that predicts achievable gains from base model samples, analyzes reward hacking under proxy rewards, and shows that reward ensembling mitigates it. Empirically, best-of-N nearly reaches the theoretical limit on safety and summarization tasks,...
Presents weighted state souping, a method that learns query-dependent weights for composing cached state space model encodings of document chunks to improve long-context retrieval. The authors report gains on document question answering, needle-in-a-haystack retrieval, variable tracking, and tool use, plus a sparse variant with near-constant encoding cost. Theoretical arguments are offered to explain when souping works and when it may fail.
This Apple Machine Learning Research paper recasts Private Evolution, a differentially private synthetic data generation algorithm, as generative model-augmented Wasserstein learning. The authors show that accounting for a generator capturing the true distribution yields much better bounds, such as sample complexity depending on intrinsic rather than ambient dimension. They also identify convergence failures in standard PE variants on well-clustered instances and propose a geometry-aware...
This Apple Machine Learning Research paper evaluates how well LLM judges assess personalized writing style. It compares human and judge-LLM pairwise annotations across four generator LLMs and three long-form writing tasks. The authors find that judge quality correlates weakly with MMLU, that paired comparisons are more consistent than independent scoring, and that the strongest LLMs do not most reliably reproduce human judgments.
This Apple Machine Learning Research paper, accepted to NeurIPS, studies why State Space Models lag Transformers on in-context learning and precise retrieval. The authors show through theory and experiments that the gating mechanism causes SSMs to first learn an in-weights memorization solution, delaying or preventing convergence to in-context learning, while gating often aids length generalization.
This Apple Machine Learning Research paper by Kunal Talwar formalizes evaluability for black-box models whose test samples may have leaked into training. It characterizes the distributions where accurate output-only evaluation is feasible, gives tight sample complexity bounds, and shows that limited query access can strengthen evaluators, drawing connections to self-correctors from program testing.
Apple researchers present amortized maximum inner product search, training small neural networks offline to route queries to their best-matching keys faster than centroid-based IVF routing. SupportNet learns per-cluster support functions for better routing, while KeyNet acts as a query rewriter in front of FAISS IVF indexes, reaching 99% recall with up to roughly 8x lower latency on Natural Questions. The post details the convex-analysis basis, the tradeoffs between the two models, and measured...
This Apple Machine Learning Research paper proposes a two-stage recipe for distilling a Transformer into a Mamba-style state space model. It first distills the teacher into a linearized attention model using a kernel-trick adaptation, then distills that into an attention-free Mamba. The distilled Pythia-1B student reaches perplexity 14.01 against the teacher's 13.86, with gaps on in-context recall tasks.
The authors present GHARP, a method that animates 3D human heads in real time from a few input images and a driving expression signal. It splits the work into an offline identity stage and a lightweight runtime animation network optimized for mobile devices, and adds a body alignment network to reduce blur and temporal flicker. The paper reports state-of-the-art quality on the Ava-256 benchmark, up to 13x faster runtime on an A100 GPU, and 8x fewer Gaussians.
Apple's machine learning team benchmarks running very large open-weight LLMs on Mac Studio clusters using MLX. It compares M3 and M5 Ultra generations and evaluates pipeline versus tensor parallelization over RDMA-over-Thunderbolt, reporting TTFT, decode throughput, and power at 8K and 64K context. The post explains the trade-offs between the two parallelization schemes and where communication overhead dominates.
The paper introduces Selection-based Structured Reasoning (SSR), which replaces freeform reasoning in small multimodal search agents with selection among pre-specified reusable reasoning candidates scored by likelihood. Parallel scoring over a shared KV cache is used to speed up inference. Across seven multimodal search benchmarks with 2B and 4B models, SSR reportedly cuts per-turn reasoning latency by over 90% and total inference latency by 28–54% while remaining competitive on success rate.
This Apple Machine Learning Research paper compares Retrieve-then-Plan and Plan-then-Retrieve architectures for selecting APIs in tool-augmented LLM agents on ToolBench's 46,984 endpoints. It finds that Plan-then-Retrieve outperforms on multi-tool queries while Retrieve-then-Plan wins on single-tool queries, and it analyzes stub hallucination, token overhead, and complexity-adaptive routing.
The post presents a lightweight method for detecting code-switching directly from speech, using language-selective hidden-state dimensions in pretrained speech encoders rather than transcription. Whisper-small's encoder reaches a mean F1 of 0.798 across 14 language pairs on CS-FLEURS, ahead of a CNN and a HuBERT linear baseline. The authors argue that pretrained encoders already encode the information needed to identify code-switched utterances.
Apple researchers introduce the first tri-modal Masked Diffusion Model, pretrained from scratch on text, image-text, and audio-text data. The paper analyzes multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects, and proposes an SDE-based reparameterization that decouples physical from logical batch size. A 3B-parameter model trained on 6.4T tokens shows results across text generation, text-to-image, and text-to-speech tasks.
This research paper compares hyperparameter tuning (such as high weight decay) against mixing auxiliary high-resource language data for pre-training language models in data-constrained settings. Using Arabic as the low-resource target and English as auxiliary across roughly 1000 runs and model scales from 150M to 1.43B parameters, the authors find mixing yields larger gains on validation loss and downstream accuracy, and that target-language validation loss underestimates mixing's value....
Minions presents a distributed virtual GPU cloud that aggregates idle Apple Silicon devices to run machine-learning inference workloads. The system addresses centralized GPU cost and capacity constraints through dynamic task execution, fault tolerance, and improved utilization of existing macOS hardware.
Apple researchers evaluate large language models against human annotators for machine translation quality assessment using MQM and ESA across 70 language pairs and multiple domains. The study finds substantial reliability challenges, identifies where humans and LLMs struggle, and discusses collaborative annotation pipelines.
The paper introduces SELFCOMPACT, an inference-time scaffold that lets language-model agents decide when and how to summarize long tool-use trajectories. Across six benchmarks and seven models, it reduces token costs by 30–70% while improving performance over unsummarized agents and matching or exceeding fixed-interval compaction.
Apple researchers introduce Normalizing Trajectory Models, which represent each coarse diffusion reverse step as an expressive conditional normalizing flow with exact likelihood training. The approach supports self-distillation and produces competitive text-to-image samples in four steps while retaining likelihood over the full generative trajectory.