Selected AI research, with notes on how each method works and what its experiments do—and do not—establish.
Archive
2025–202612 papers with reading notes
Paper index
12 papers
No papers match these filters.
Video & diffusionarXiv:2610.05608
Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
Abstract
The central question is how to add audio to a pretrained video model without sacrificing quality when either modality is considered on its own. [12]
How it works
The authors train an audio stream first, then train jointly on paired data. CrossDiT connects it to the pretrained video stream with bidirectional cross-attention. Later stages include supervised fine-tuning, reinforcement learning, distillation, and super-resolution. [12]
What to look for
Human preferences describe the evaluated samples, not all inputs. Check dialogue, ambience, and lip sync independently, and do not extrapolate short-clip results to long, multi-shot narratives.
Full author list · 88 credited names
Kandinsky Team; Julia Agafonova; Bulat Akhmatov; Mikhail Aksyutin; Grigorii Alekseenko; Anastasia Aliaskina; Olga Androsova; Vladimir Arkhipkin; Anna Averchenkova; Alexander Belykh; Serafima Bocharova; Sofiya Bogakovskaya; Anton Bukashkin; Mark Bulygin; Kirill Buzygin; Irina Cheremnykh; Kirill Chernyshev; Mikhail Chernyshov; Vladimir Chernyy; David Chikovani; Georgy Daniltsev; Denis Dimitrov; Anna Dmitrienko; Vladimir Dokholyan; Sergey Emelyanov; Dmitry Ermilov; Georgii Fedorov; Polina Gavrilova; Nikolai Gerasimenko; Aleksandr Gordeev; Andrey Inozemtsev; Andrei Ivaniuta; Alexander Ivanov; Mikhail Karaev; Anastasiia Kargapoltseva; Ivan Kirillov; Nikita Kiselev; Valeria Kobenko; Yury Kolabushin; Denis Koposov; Anatoly Korobov; Vladimir Korviakov; Kirill Kozlov; Denis Krzhivokolskiy; Konstantin Kuklev; Alexander Kunitsyn; Sergey Kuzin; Vladislav Lakhtionov; Alexey Letunovskiy; Maxim Litvinov; Alexander Lyulkov; Georgy Makarov; Kirill Malakhov; Egor Malykh; Mikhail Mamaev; Dmitrii Mikhailov; Polina Mikhailova; Ivan Mikheev; Elizaveta Muromtseva; Nikolai Nazarkin; Tatiana Nikulina; Lev Novitskiy; Stanislav Onuchin; Nikita Osterov; Denis Parkhomenko; Anatoliy Parpara; Vladimir Polovnikov; Konstantin Reznikov; Azat Saginbaev; Nikita Samsonov; Alexander Sentsov; Nikita Shaimov; Artem Sherstyuk; Andrey Shutkin; Egor Silvestrov; Bulat Suleimanov; Matvey Suprunov; Sergey Taranov; Irina Tolstykh; Tatiana Trofimuk; Ilya Trushkin; Aleksandra Tsybina; Olga Varlashina; Viacheslav Vasilev; Ilya Vasiliev; Eugeny Vilisov; Sergey Yakubson; Konstantin Zakharov
Source [12]arXiv:2610.05608v1 · 2026-10-04 · Find the original on arXiv by ID
Kandinsky Team, Julia Agafonova, Bulat Akhmatov and 85 others
TL;DR Kandinsky 6.0 Video uses dual-stream diffusion to generate roughly five seconds of synchronized audio and video from text or a starting image. [12]
CrossDiTAudio-video synchronization
MoE & architecturearXiv:2606.19348
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Abstract
Accepting a long input is only part of the problem. The paper considers retaining useful history, controlling the KV cache, and keeping decode costs manageable. [11]
How it works
The models combine Compressed Sparse Attention and Heavily Compressed Attention, alongside mHC and Muon. At the million-token setting, the authors report lower single-token inference FLOPs and KV-cache use for V4-Pro than for V3.2. [11]
What to look for
Single-token FLOPs and cache size do not directly measure user-perceived latency. Evaluate long-input prefill, sustained decoding, and retrieval accuracy separately.
Source [11]arXiv:2606.19348v1 · 2026-04-26 · Find the original on arXiv by ID
DeepSeek-AI, Anyi Xu, Bangcai Lin and 316 others
TL;DR Both DeepSeek-V4 variants support a million-token context. The report focuses on reducing the attention and cache costs that make such lengths difficult to serve. [11]
MoELong context
Agent systemsarXiv:2603.27490
AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents
Abstract
The paper separates two failure modes: exhausting the budget before finding enough evidence, and finishing within budget but returning an incorrect answer. [10]
How it works
At a trigger point, the agent branches from its current trajectory using different context treatments. A short lookahead ranks the continuations. The authors compare this routing against fixed strategies across deep-search benchmarks and model backbones. [10]
What to look for
Fewer interaction turns do not necessarily mean lower cost, because each parallel branch can consume tokens or tool calls. Compare total calls, elapsed time, and answer accuracy together.
Source [10]arXiv:2603.27490v1 · 2026-03-29 · Find the original on arXiv by ID
Zhaopeng Feng, Liangcai Su, Zhen Zhang and 16 others
TL;DR AgentSwing tries several context-management strategies in parallel at selected points in a long search, then continues from the most promising branch. [10]
Context managementParallel search
Agent systemsarXiv:2601.11868
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Abstract
The benchmark evaluates the state an agent leaves behind. A plausible-looking transcript is not enough if the environment never reaches the requested outcome. [9]
How it works
Each task includes its own environment, a human-written solution, and tests. The evaluation harness and failure analysis help readers distinguish reasoning errors from tool use and verification mistakes. [9]
What to look for
A score reflects the model, harness, and resource limits together. Compare models under the same time and retry budgets, and ask whether the tests capture the user's actual goal.
Full author list · 85 credited names
Mike A. Merrill; Alexander G. Shaw; Nicholas Carlini; Boxuan Li; Harsh Raj; Ivan Bercovich; Lin Shi; Jeong Yeon Shin; Thomas Walshe; E. Kelly Buchanan; Junhong Shen; Guanghao Ye; Haowei Lin; Jason Poulos; Maoyu Wang; Marianna Nezhurina; Jenia Jitsev; Di Lu; Orfeas Menis Mastromichalakis; Zhiwei Xu; Zizhao Chen; Yue Liu; Robert Zhang; Leon Liangyu Chen; Anurag Kashyap; Jan-Lucas Uslu; Jeffrey Li; Jianbo Wu; Minghao Yan; Song Bian; Vedang Sharma; Ke Sun; Steven Dillmann; Akshay Anand; Andrew Lanpouthakoun; Bardia Koopah; Changran Hu; Etash Guha; Gabriel H. S. Dreiman; Jiacheng Zhu; Karl Krauth; Li Zhong; Niklas Muennighoff; Robert Amanfu; Shangyin Tan; Shreyas Pimpalgaonkar; Tushar Aggarwal; Xiangning Lin; Xin Lan; Xuandong Zhao; Yiqing Liang; Yuanli Wang; Zilong Wang; Changzhi Zhou; David Heineman; Hange Liu; Harsh Trivedi; John Yang; Junhong Lin; Manish Shetty; Michael Yang; Nabil Omi; Negin Raoof; Shanda Li; Terry Yue Zhuo; Wuwei Lin; Yiwei Dai; Yuxin Wang; Wenhao Chai; Shang Zhou; Dariush Wahdany; Ziyu She; Jiaming Hu; Zhikang Dong; Yuxuan Zhu; Sasha Cui; Ahson Saiyed; Arinbjörn Kolbeinsson; Jesse Hu; Christopher Michael Rytting; Ryan Marten; Yixin Wang; Alex Dimakis; Andy Konwinski; Ludwig Schmidt
Source [9]arXiv:2601.11868v1 · 2026-01-17 · Find the original on arXiv by ID
Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini and 82 others
TL;DR Terminal-Bench 2.0 evaluates agents on 89 tasks inspired by real workflows, using isolated terminal environments and tests rather than judging command text alone. [9]
Terminal agentsExecutable evaluation
MoE & architecturearXiv:2601.07372
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Abstract
At matched model size and compute, the paper asks how much sparse capacity should go to expert computation and how much should go to lookup memory. [8]
How it works
Engram modernizes n-gram embeddings for constant-time retrieval. Predictable addresses enable prefetching from host memory. The experiments compare different allocations against a MoE baseline with the same parameter and FLOP budgets. [8]
What to look for
Deterministic lookup is not semantic retrieval over documents and does not add new knowledge at inference time. Examine factual, reasoning, and long-context results separately, and account for serving memory costs.
Source [8]arXiv:2601.07372v1 · 2026-01-12 · Find the original on arXiv by ID
Xin Cheng, Wangding Zeng, Damai Dai and 11 others
TL;DR Engram adds conditional lookup alongside MoE computation, using deterministic n-gram memory for some local patterns. [8]
EngramConditional memory
Video & diffusionarXiv:2601.03233
LTX-2: Efficient Joint Audio-Visual Foundation Model
Abstract
The model treats the timing between movement, dialogue, and scene sounds as part of generation rather than a separate production step. [7]
How it works
A 14B video stream and a 5B audio stream communicate through bidirectional cross-attention, temporal position embeddings, and shared timestep conditioning. Modality-aware classifier-free guidance controls how the two streams follow the prompt. [7]
What to look for
Check lip sync, action-to-sound timing, and continuity of background audio separately. An overall preference score can hide failures on any one of these dimensions.
Full author list · 29 credited names
Yoav HaCohen; Benny Brazowski; Nisan Chiprut; Yaki Bitterman; Andrew Kvochko; Avishai Berkowitz; Daniel Shalem; Daphna Lifschitz; Dudu Moshe; Eitan Porat; Eitan Richardson; Guy Shiran; Itay Chachy; Jonathan Chetboun; Michael Finkelson; Michael Kupchick; Nir Zabari; Nitzan Guetta; Noa Kotler; Ofir Bibi; Ori Gordon; Poriya Panet; Roi Benita; Shahar Armon; Victor Kulikov; Yaron Inger; Yonatan Shiftan; Zeev Melumian; Zeev Farbman
Source [7]arXiv:2601.03233v1 · 2026-01-06 · Find the original on arXiv by ID
Yoav HaCohen, Benny Brazowski, Nisan Chiprut and 26 others
TL;DR LTX-2 generates picture and sound together using a dual-stream Transformer rather than adding audio after the video is complete. [7]
Audio-video generationDual stream
MoE & architecturearXiv:2507.20534
Kimi K2: Open Agentic Intelligence
Abstract
The report connects training stability with downstream agent performance, explaining both how the model was trained and how it was evaluated in tool-using settings. [6]
How it works
Pretraining uses MuonClip, a Muon-based optimizer with QK clipping to address instability. Post-training combines large-scale synthetic agent data with joint reinforcement learning on real and synthetic environments. [6]
What to look for
Total parameters and activated parameters describe different costs; neither alone predicts throughput. For tool-use results, check the context budget, environment, and agent harness.
Source [6]arXiv:2507.20534v1 · 2025-07-28 · Find the original on arXiv by ID
Kimi Team, Yifan Bai, Yiping Bao and 166 others
TL;DR Kimi K2 pairs a trillion-parameter MoE with 32B activated parameters and agent-oriented post-training. The report emphasizes tool use without extended reasoning. [6]
MoETool use
Agent systemsarXiv:2504.12516
BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
Abstract
The benchmark makes searching difficult but scoring straightforward. It measures persistence in information seeking rather than the ability to write a complete research report. [5]
How it works
Questions are designed to resist one-shot keyword lookup. A browser agent may need to reformulate queries and follow evidence across pages, then submit a short answer that can be checked against a reference. [5]
What to look for
Record the retriever, web access, and action budget when comparing runs. A change to the search interface can change the end-to-end score without any change to the model.
Full author list · 10 credited names
Jason Wei; Zhiqing Sun; Spencer Papay; Scott McKinney; Jeffrey Han; Isa Fulford; Hyung Won Chung; Alex Tachard Passos; William Fedus; Amelia Glaese
Source [5]arXiv:2504.12516v1 · 2025-04-16 · Find the original on arXiv by ID
Jason Wei, Zhiqing Sun, Spencer Papay and 7 others
TL;DR BrowseComp uses 1,266 questions with short, checkable answers to test whether browsing agents can persistently find and connect hard-to-locate clues. [5]
Web agentsEvaluation
Video & diffusionarXiv:2503.20314
Wan: Open and Advanced Large-Scale Video Generative Models
Abstract
The report treats video generation as a system: the latent representation, data, generative backbone, and evaluation all affect the final result. [4]
How it works
A video encoder compresses frames into a latent representation. The diffusion model generates in that space before a decoder reconstructs pixels. The family also covers conditioned tasks such as image-to-video generation and editing. [4]
What to look for
When comparing variants, hold resolution, frame count, sampling steps, and hardware fixed. A handful of polished clips cannot establish temporal consistency or a low failure rate.
Source [4]arXiv:2503.20314v1 · 2025-03-26 · Find the original on arXiv by ID
WanTeam, Ang Wang, Baole Ai and 59 others
TL;DR Wan combines a video VAE, a diffusion Transformer, and a large-scale data pipeline, with 1.3B and 14B models targeting different resource budgets. [4]
DiffusionVideo VAE
Post-training & RLarXiv:2503.14476
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Abstract
Starting from Qwen2.5-32B, the authors train for mathematical reasoning and examine which practical choices make reward-based training work at scale. [3]
How it works
The recipe separates upper and lower clipping thresholds, dynamically resamples groups, aggregates policy gradients at the token level, and shapes rewards for overlong responses. Resampling helps replace all-correct or all-incorrect groups, which provide little relative learning signal. [3]
What to look for
Read accuracy alongside response length and truncation rate. Component ablations help distinguish a more stable update rule from gains that simply follow from generating longer answers.
Source [3]arXiv:2503.14476v1 · 2025-03-18 · Find the original on arXiv by ID
Qiying Yu, Zheng Zhang, Ruofei Zhu and 32 others
TL;DR DAPO publishes a large-scale reinforcement learning recipe, training code, and data so readers can inspect how sampling and policy updates affect learning. [3]
Verifiable rewardsTraining stability
LLM reasoningarXiv:2501.19393
s1: Simple test-time scaling
Abstract
The paper isolates a practical question about test-time scaling: how much can a fixed model gain when only its reasoning budget changes? [2]
How it works
The authors select s1K examples for difficulty, diversity, and quality. At inference time, they either end the reasoning trace early or append the word ‘Wait’ when the model is about to stop, giving it a chance to revisit its work. [2]
What to look for
Compare methods at the same token budget. Extra reasoning may correct a mistake, but it can also reinforce one; gains on competition math do not automatically transfer to open-ended tasks.
Full author list · 10 credited names
Niklas Muennighoff; Zitong Yang; Weijia Shi; Xiang Lisa Li; Li Fei-Fei; Hannaneh Hajishirzi; Luke Zettlemoyer; Percy Liang; Emmanuel Candès; Tatsunori Hashimoto
Source [2]arXiv:2501.19393v1 · 2025-01-31 · Find the original on arXiv by ID
Niklas Muennighoff, Zitong Yang, Weijia Shi and 7 others
TL;DR s1 fine-tunes a model on 1,000 reasoning examples and uses budget forcing to decide when generation should stop or continue. [2]
Inference budgetSupervised fine-tuning
LLM reasoningarXiv:2501.12948
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Abstract
The report contrasts two routes to reasoning: train a base model directly from rewards, or first establish output habits with a small set of high-quality reasoning traces and then optimize for problem solving. [1]
How it works
R1-Zero applies reinforcement learning directly to the base model. R1 adds cold-start supervision, reasoning-focused reinforcement learning, and later training stages. The authors also distill R1 reasoning traces into smaller dense models based on Qwen and Llama. [1]
What to look for
Compare readability and language mixing alongside benchmark scores. R1 and R1-Zero differ in multiple ways, so their comparison alone cannot isolate the contribution of any one stage.
Source [1]arXiv:2501.12948v1 · 2025-01-22 · Find the original on arXiv by ID
DeepSeek-AI, Daya Guo, Dejian Yang and 197 others
TL;DR R1-Zero develops behaviors such as checking and revising its reasoning through reinforcement learning without an initial supervised fine-tuning stage. R1 adds cold-start examples and a multi-stage training recipe to make its outputs easier to read. [1]