AI Weekly Brief

DevDay 2026 Recap

Explore more than 20 announcements from OpenAI DevDay 2026, including GPT-6 Astra, ChatGPT, Codex, APIs, security, and new tools for builders.

Disrupting a coordinated model-distillation campaign

Learn how OpenAI disrupted a campaign to extract protected model reasoning and is strengthening defenses against adversarial distillation.

Introducing GPT-6.1 Sol

Meet GPT-6.1 Sol: near-Astra intelligence for coding, computer use, and professional work at one-fifth of Astra’s standard API input and output token prices.

Towards safety cases for frontier AI training

Our early guidelines for safety cases in frontier AI training cover technical safeguards, operational practices, and investigating misalignment incidents

Basis completes a tax workbook 2x faster with GPT-6 Astra

GPT-6 Astra completed a 50-tab tax workbook twice as fast as GPT-5.6 Sol, and its stronger understanding of user intent gives Basis more confidence in real-world use.

Gemini 4 Argon

See also: <i>Gemini 4 Argon (High): Intelligence, Performance and Price Analysis</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49914236">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49914236</a>

Gemini 4 Argon: our next era of frontier intelligence

Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States

Large language model (LLM) agents can now undertake increasingly complex tasks, but the way they organize interaction history into memory does not ensure a coherent understanding of the current world. We introduce PoS, an inference-time framework that constructs and continually maintains explicit belief states as the agent's decision context. Each belief combines an estimate of the current world state with unresolved task requirements, making explicit what the agent still needs to learn and accomplish. To keep this belief reliable and actionable, PoS validates its consistency and monitors task progress to detect Belief Trapping, where the agent continues to act without making meaningful progress toward the goal. Recovery is then tailored to both the trapping pattern and the type of unresolved task requirement. Experiments on four benchmarks spanning execution and diagnosis show that PoS achieves the highest overall performance on every benchmark with all three LLM backbones. Ablations demonstrate the importance of consistency validation and recovery, while context-scaling experiments show resilience to context growth. Together, these results support belief construction and continua

Introducing dots

Dots by OpenAI are proactive assistants that can keep working across complex projects and everyday tasks. Learn how dots help you stay in control while work moves forward.

The Lenfest Institute grows landmark program with expanded OpenAI support

OpenAI is expanding the Lenfest AI Collaborative and Fellowship Program with $5 million in funding and up to $5 million in software credits and engineering support.

Hierarchical Continuous Diffusion Language Models

Discrete diffusion language models offer a compelling alternative to autoregressive generation for tasks demanding bidirectional reasoning and global constraint satisfaction. Yet they share a structural bottleneck: when decoding in parallel, each token is sampled independently from its marginal, severing the statistical dependencies among the tokens decoded together. Continuous diffusion language models avoid this by denoising a shared continuous state, but their denoiser sees only that state, so nothing ties it to a valid token configuration until it is finally decoded. To address this, we propose Hierarchical Continuous Diffusion Language Models (HC-DLM), which couple discrete token generation with a continuous latent trajectory in a single, principled denoising process, whose training objective is derived from a variational bound on the token likelihood. In contrast to recent methods that attach continuous context to a self-contained discrete chain, HC-DLM makes the latent the only persistent generative state: tokens are read out from it at every step and feed back as a scaffold for the next latent update. On structured reasoning (Sudoku), mathematical planning (Countdown) and l

World Observer: Joint Actor-Observer Generation for Persistent World Modeling

How can a world model continuously observe regions beyond the actor's current view? Video world models simulate how an environment evolves from an agent's actions, yet remain actor-centric. Once an object leaves the actor's view, they lose direct evidence of its evolution, often failing to preserve its state and dynamics upon re-entry. To address this, we introduce World Observer, which decouples observing from acting by jointly generating a perspective actor for the agent-centric view with one or more panoramic observers that watch selected world regions. This allows objects that leave the actor's view to remain visually evolving in an observer, so their updated states are reflected when they re-enter. We ground the actor and observers by warping from a shared panoramic source for explicit geometric correspondence, and introduce an Observer Sink of high-resolution perspective references to restore fine appearance upon re-entry. Since the observers are decoupled from the actor, they can be placed freely across the scene, extended to multiple locations for broader coverage, and driven by control signals to steer out-of-view evolution. To evaluate out-of-view evolution, we further in

Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence

Latent visual reasoning (LVR) enables multimodal large language models (MLLMs) to perform intermediate computation in continuous latent tokens rather than expressing every reasoning step in words. However, unlike textual CoT, latent reasoning is not directly observable, making it difficult to supervise what latent tokens learn. In this work, we first conduct a thorough analysis of latent-token behavior and identify a latent evidence-credit gap: latent tokens respond only weakly to image perturbations that alter the correct answer. We hypothesize that this issue stems from the lack of explicit supervision during GRPO training. These findings suggest that a final-answer reward provides too little guidance on what visual evidence to preserve or how credit should be assigned across latent tokens. To bridge this gap, we propose ReaLVR, which brings visual-evidence supervision to the model's own free-running latent trajectories. ReaLVR contrasts correct and model-generated wrong answers to determine where stronger supervision is needed, and relevant and mismatched visual evidence to specify what to preserve. Across three model families, ReaLVR consistently outperforms evaluated LVR basel

Claude Sonnet 5.5

Claude Sonnet 5.5 New Sonnet model from Anthropic today. They say it "runs 30%+ faster, and costs up to 30% less for most work" - it's priced the same as Sonnet 5 but appears to beat it on every benchmark, and should be cheaper to run as well. Here are some pelicans riding bicycles . Sonnet 5.5 suffered from the same bug as Opus 5.5 : the "max" thinking effort pelican thought for 128,000 tokens (at a cost of $1.28) before running out of tokens and failing to produce an SVG. Here's the pelican it gave me for thinking effort "xhigh", at a cost of 5.74 cents and taking 41 seconds: Sonnet 5.5 appears to be almost as good as Opus 5.5 on some coding tasks, including various viral 3D animation tricks . The most interesting thing about Sonnet 5.5 is that it's now the model used for the free tier on claude.ai . OpenAI's ChatGPT free tier uses Luna 5.6, which means Anthropic currently have a much more capable free offering. I ran this prompt against that free tier: build me an HTML page that renders a three-dimensional pelican riding a bicycle using WebGL And got back this page , which is a solid effort. Anthropic's announcement reiterates that Haiku 5.5 will be available "in the coming week

Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL

Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary objectives, including modality-specific quality, cross-modal semantic alignment, and temporal synchronization. Its effectiveness, however, depends on two quantities that change during training: where reward-driven updates should act, and how competing rewards should be combined. Existing methods tend to rely on fixed routing and reward weights, failing to track evolving model functions. To address these limitations, we propose Adaptive Reward Routing to jointly adapt update locations and reward coordination during forward-process RL (i.e., DiffusionNFT) of joint audio-video diffusion models. Our method consists of two components. (i) Cross-Modal Influence-Guided Routing (Localizing Updates): We use bidirectional cross-attention responses as an efficient proxy for evolving cross-modal influence, dynamically reweighting token-aware losses and scaling gradients across cross-modal layers without additional model interventions. (ii) Preference-Preserving Modality-Aware Reweighting (Coordinating Rewards): We preserve predefined weights as

Agent Priors-guided Policy Learning

Robots that learn from a few demonstrations often require two forms of generalization. Compositional generalization recombines skills to solve new tasks, and skill generalization lets the learned policy behind each skill work in new situations. The two depend on each other, yet information is lost between composition and the skills it calls. Where a skill works is determined by the structure its policy is trained with, while composition sees the skill only through a separate description, such as a name, an instruction, or a symbolic operator, that omits this structure. Our key idea is to use each policy's structural prior as part of the interface between composition and the skill. A structural prior states what a behavior depends on, for example that a grasp depends only on the gripper's pose relative to the object. Built into training, it shapes where the policy generalizes; stated in language, it tells composition where the policy applies. We instantiate this idea in Agent Priors-guided Policy Learning (APPL). A construction agent segments complete demonstrations into reusable skills, proposes several structural priors for each skill, and trains and verifies one policy per prior.

Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs

NVIDIA Kumo Tabular Sets a New Accuracy-Efficiency Frontier for Tabular Prediction

LoopVL: Recurrent Visual Intelligence

We introduce LoopVL to study whether Loop Transformers can be effectively extended to vision- language models. LoopVL combines Module-Loop and Model-Loop computation to iteratively update a unified vision-language state through shared modules. We train LoopVL from scratch through language pre-training, multimodal training, and post-training. LoopVL outperforms a range of similarly sized and larger non-recurrent models on multimodal understanding and visual reasoning benchmarks. We also observe Visual Aha Moments in LoopVL, characterized by pronounced shifts in visual attention across loops. LoopVL provides practical evidence for recurrent vision-language modeling and offers an intuitive perspective on how shared parameters can support deeper multimodal computation over continuously evolving visual-language states.

It's Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them

Vision-language models (VLMs) are increasingly used in place of human annotators, making it important that substitutability tests reflect the model rather than incidental evaluation conditions. We introduce MIST, the Misleading-Image Stress Test: 200 English sentences, each built around a phrase readable either figuratively or literally and shown with an aligned image depicting its reading, a misleading image depicting the opposite, or no image at all. The guidelines require the label to be decided from the sentence alone, so no image should change any answer. We expected each image to pull a judge's labels toward the sense it depicts, and neither kind did. Across thirteen VLM judges, an aligned image changed 20.5% of labels and a misleading one 19.4%, close for every judge and both above the 11.6% produced by deleting the ignore-the-image instruction with the image left in place. Yet only 37% of the labels that differ between the two images moved toward the sense shown, and agreement with our human annotators is unchanged whether the image is absent, aligned or misleading. The effect is smaller in the seven judges that pass the alt-test than in the six that never do, but present i

Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents

Hey HN, Anders and Tom here. We&#x27;re building Magnitude, an inference engine for agents that optimizes itself to run as fast as possible on your hardware. It works on Mac, Linux, and Windows on any hardware and is up to 2x faster than llama.cpp.<p>We&#x27;re both software engineers and previously built an open source browser agent to 4k+ GH stars and 100k+ downloads. We increasingly wanted to run it on local models, but found that no inference engine worked for our use case.<p>Inference engines today all make a performance tradeoff. They are either:<p>- Built for batched inference on datacenter hardware at the cost of single-session performance (vLLM, SGLang) - Designed for broad compatibility instead of optimizing for specific hardware (llama.cpp, Ollama) - Specialized for specific har

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction

Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception. We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events. Streaming caption targets supervise the interpretation of observed video prefixes during training. At inference, model-generated records complement a recent visual window, providing reusable factual context without revisiting historical visual features. Proactive State Transition Learning (PSTL) reduces the dominance of repeated waiting states by preserving supervision at all output anchors and selecting representative state-change and state-persistence tokens. We further develop a streaming data synthesis pipeline that aligns output content and timing with available evidence. Combining the resulting streaming captions and QA with cleaned open-source data yields OneStreamer-1M,

ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization

Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback. However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harness. We formulate this missing dimension of harness optimization as an automated curriculum learning problem and introduce ActiveSaddler. ActiveSaddler models the evolving curriculum as a non-stationary bandit with dynamically instantiated optimization targets. It abstracts recurring failures into reusable failure-pattern arms, estimates the potential learning progress from further targeting each pattern, and adaptively balances revisiting known weaknesses with exploring unseen scenarios for new ones. Optimization outcomes continually update both the set of discovered failure patterns and their priorities, allowing the curriculum to co-evolve with the harness. Experiments on GAIA2 and Termina

A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review

AI reviewers can assign different judgments to manuscripts that report the same science in different wording, potentially rewarding rhetorical optimization over scientific improvement. We formulate Rhetorical Robustness as the joint requirement of stability across content-preserving rewrites and discrimination across papers. We introduce RobustReview, a controlled full-manuscript benchmark with 1,260 manuscript versions, and evaluate 30 reviewer configurations. The benchmark reveals false robustness, where low rewrite sensitivity coincides with score collapse across papers, and shows that human alignment and rhetorical robustness rank reviewers differently. Moreover, the evaluated content-focused prompting protocol does not consistently improve robustness across backbones. Motivated by these findings, we introduce SciCore, a dual-branch reviewer that averages a full-manuscript judgment with a judgment based on an extracted, structured science core. This design combines manuscript-level assessment with a content-normalized view intended to reduce rhetorical sensitivity. In our primary GPT-5.5 comparison, SciCore achieves a leading joint stability-discrimination profile among the ben

Sharpening Tax in Post-Training

An emerging hypothesis about reinforcement learning (RL) post-training of large language models (LLMs) is that it merely sharpens existing behaviors of a base model, improving single-shot accuracy at the cost of solution coverage. Although this trade-off has been observed in math and coding tasks, it need not extend to agentic tasks, where multi-turn tool use and interaction may require capabilities newly acquired during post-training. Our surprising finding is that pre-trained LLMs, equipped with a light inference harness, can serve as capable agents. Despite far lower accuracy (pass@1), they often surpass their post-trained counterparts in solution coverage (pass@K) given a sufficient test-time budget. We further analyze the underlying mechanism and show that post-training pushes tasks toward two extremes, always solved or never solved, and thereby improves sampling efficiency and consistency at the cost of solution coverage. To measure this cost, we propose Sharpening Tax, a diagnostic metric that quantifies the loss in test-time scalability after post-training. Across 14 base/post-trained model pairs from four families and three agentic benchmarks (42 cases in total), the tax i

TERRA: Terrain-Aware Reconstruction, Retargeting and Control for Musculoskeletal Locomotion

Recent advances in musculoskeletal modeling and reinforcement learning have enabled muscle-actuated agents to reproduce increasingly complex human motions. Yet these capabilities remain largely confined to flat ground, in part because motion datasets rarely include aligned terrain geometry and because retargeting terrain interactions to complex musculoskeletal bodies is challenging. We present TERRA, an end-to-end pipeline for terrain-aware retargeting and control of musculoskeletal locomotion. From kinematic trajectories alone, TERRA combines terrain priors, estimated contacts, and negative free-space evidence to recover task-relevant support geometry. TERRA further considers anatomical, tendon-continuity, and contact constraints during retargeting. Using the resulting motion-terrain pairs from five datasets, we successfully train a single muscle-actuated control policy on 9.4 hours of diverse locomotion. Across reconstruction, retargeting, and held-out tracking benchmarks, TERRA improves terrain accuracy, sharply reduces anatomical and interaction violations, and achieves the highest observed completion rate over supported terrain families. Overall, TERRA provides a practical rou

DyRAD: Radar Novel View Synthesis for Dynamic Driving Scenes

Reconstructing dynamic driving scenes from recorded sensor data supports closed-loop evaluation of autonomous driving systems by synthesizing observations beyond the original trajectory. Unlike cameras and LiDAR, radar measures radial velocity directly through Doppler. Yet existing radar novel-view synthesis fails to exploit this capability: methods addressing dynamic scenes reconstruct only range-azimuth tensors, while methods that render Doppler assume static scenes. Moreover, because radar processing spreads each reflection across multiple bins, existing representations absorb this spread into scene geometry, causing it to render incorrectly when the viewpoint moves. We present DyRAD, which models dynamic driving scenes using static background reflectors and motion-tracked dynamic point reflectors to render complete range-azimuth-Doppler (RAD) tensors. Reflector velocities are derived from object tracks and projected onto the line of sight, making Doppler both a rendered output and supervision for those tracks. Crucially, we render reflectors through a fixed analytic point-spread function (PSF) derived from the radar's signal-processing chain, preventing sensor-induced spread fr

ROWBench: Do Video Models Render What the Program Specifies?

Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. We introduce PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions. PROWBench logs entity states and timestamped events, including those outside the camera's field of view, as replayable world records, from which it renders synchronized views and proxy representations. This enables generated videos to be checked against the observable consequences of program execution. An extensible framework constructs scenes, controls behaviors, and can render each camera view in different representations, such as coarse 3D, and bounding boxes. The benchmark covers first- and third-person perspectives, with synchronized multi-view observations available for a subset of episodes. Grounded in these records, PROWBench

AutoGUIWorld: Image Generators as Visual World Models for GUI Agent

GUI agents require high-quality interaction trajectories to learn how software environments respond to actions, maintain state, and support multi-step workflows. However, the diversity of available trajectories is constrained by the applications, interface states, and workflows accessible in the underlying environments. Expanding this coverage requires deploying increasingly diverse and complex software, with specialized applications imposing additional installation, configuration, and runtime costs. We introduce AutoGUIWorld, a data generation framework that combines the visual priors of image generators with the task knowledge of a planner to synthesize GUI interaction trajectories without deploying or running the corresponding software environments. AutoGUIWorld samples initial GUI scenes from structured specifications of operating-system context, visual appearance, and interface state, and generates tasks conditioned on those scenes. A planner then specifies atomic actions and their intended visual consequences, while an image generator iteratively edits the current screenshot to produce subsequent observations. Action grounding and transition-level quality filtering yield 79,2

Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation

An agent skill is a reusable, actionable natural-language artifact that guides an agent to perform a task effectively under a given harness. Recent studies have explored the optimization of agent skills, contributing to a growing collection of publicly available skills spanning diverse tasks, domains, and harnesses. Despite millions of publicly shared skills, existing skill optimization methods largely overlook this accumulated knowledge, instead relying solely on expensive agent rollouts to iteratively refine skills for a target task. To address this, we propose Retrieval-Augmented Skill Optimization (RASO), a framework that leverages an external skill corpus as prior knowledge throughout skill optimization. RASO retrieves relevant knowledge from existing skills and adapts it to the target task and harness via Cross-Harness Adaptation, accounting for mismatches in both domain and harness. RASO comprises two complementary stages: Retrieval-Augmented Skill Initialization (RASI) constructs a knowledge-grounded initial skill without requiring agent rollouts, while Retrieval-Augmented Skill Update (RASU) iteratively refines the skill by retrieving external knowledge guided by execution

BiasReducer: Adaptive Bias Mitigation for Reward Models

Reward models score responses from large language models (LLMs) and guide LLM training toward human preferences. However, reward models can favor superficial attributes such as length or confidence, leading LLMs to produce higher-scoring but not more correct responses. Existing mitigation methods either retrain the reward model or apply a fixed correction to one known bias, such as a preference for longer responses. Retraining requires additional data and computational resources, while existing editing methods require the target bias to be specified in advance and use a fixed edit for that bias. To this end, we propose BiasReducer, a lightweight framework that edits only the linear reward head and selects the relevant edits for each new dataset. First, BiasReducer uses a sparse autoencoder (SAE)-style encoder to learn which attributes (e.g., length and confidence) the reward model is sensitive to. Second, it learns how to reduce the reward model's dependence on each attribute by determining which direction to adjust the reward head and how much to adjust it. Third, for a new dataset, it ranks the attributes by their influence on reward scores, selects the relevant ones, and edits t

Introducing SynthID Bio

Proof of concept for watermarking AI-generated proteins while preserving biological function.

Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL

Multi-reward reinforcement learning trains large language models to satisfy multiple behavioral objectives simultaneously. Reward-wise normalization, as used in GDPO, preserves reward-specific relative information within rollout groups, but different objectives can still exhibit uneven learning progress. We study this behavior through advantage energy, the sum of a reward's squared advantages over a batch. Under idealized GDPO normalization, we show that this energy is proportional to active-group density: the fraction of rollout groups in which the reward provides nonzero relative advantages. This reveals a residual batch-level signal imbalance and provides a basis for calibrating reward contributions. Based on this relation, we propose Density-Aware Reward Aggregation (DARA). We derive an inverse-square-root density correction that gives greater weight to signals from less frequently active rewards. DARA computes its weights from each rollout batch, adapting to changes in reward activity throughout training without modifying the underlying policy optimization objective. Experiments on tool calling and mathematical reasoning show that DARA learns the targeted behaviors faster than

Beyond the Current Scene: Event-Referential Grasping with Active View Selection

A robot that observes people interacting with objects should be able to carry out later requests that refer back to those interactions. Such requests may specify a grasp target by the role it played in a past event rather than by its name or appearance. Moreover, the target may no longer be visible when the robot is asked to act. We present BeyondSCe, a zero-shot robotic grasping system for this event-referential setting. Given the event history and the current scene, the system identifies the requested object or part and localizes it for grasping. If the target is occluded, it combines an event prior recovered from the history with current scene geometry to select camera viewpoints likely to reveal the target. The system uses pretrained models without additional task-specific training. In real-robot experiments with a single wrist-mounted RGB-D camera, it achieves grasp success rates of 76% and 77% for initially visible and occluded targets, respectively, compared with 40% and 55% for the strongest baseline in each condition. On four additional scenes with heavy occlusion, it increases grasp success rates from 75% to 95% while reducing the mean number of views from 3.35 to 2.20, c

GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis

Working agents need to read diverse files, coordinate tools, and produce deliverables. Training such agents requires tasks built on many real files with verifiable results, but few pipelines exist to synthesize this kind of data. Existing pipelines either generate files with models, which lack realism and diversity, or build tasks on real files without task-specific verifiers, leaving result quality unchecked. We introduce GraphForge, an evidence-graph based framework that grounds both the task and its verification in real files. Starting from occupation-grounded seeds for controlled diversity, GraphForge assembles a workspace of real files for each seed and builds an evidence graph over their relations. Since the task statement and rubrics are both derived from this graph, task requirements are backed by the workspace files and each criterion is anchored to the files needed to verify it. An initial rollout further tests executability, and a revision agent repairs the task and rubrics against the original files before trajectories are collected. Fine-tuning Qwen3.6-27B on 2,169 GraphForge trajectories brings GDPVal to 1445.7 (+65.7) under OpenHands, and Workspace-Bench-Lite and Spr

Scaling and Distilling Text Embeddings for Better Diffusibility

Diffusion language models (DLMs) offer a promising alternative to autoregressive (AR) language generation. Recent advances in continuous DLMs, which apply latent diffusion to continuous text embeddings, raise a practical question: which embedding makes the best latent space, i.e., the most diffusible? To answer this, we search through different embeddings and find that scaling the embedding model to stronger ones within the same family (T5 to T5Gemma-1 to T5Gemma-2) greatly improves generative performance. But the raw T5Gemma-2 embeddings are still not optimal. They are so discriminative that even the embeddings of plausible alternative words are separated, which makes the generation vulnerable to imperfect sampling. Consequently, continuous diffusion often fails to reach any of them and ends up at an invalid embedding instead. To address this, we distill T5Gemma-2 into a student encoder that learns the teacher's decoded probabilities as soft labels. Learning from such soft labels makes the student pull the alternative embeddings closer while maintaining the encoding-decoding mechanism. The distilled embeddings form a more connected and diffusible latent space, improving over the v

GPT 6.1 Sol: Near-Astra intelligence for a fifth of the price

My comment on GPT 6.1 Sol: Near-Astra intelligence for a fifth of the price — Hacker News. I'm a bit late with the pelicans because I was live-blogging the keynote: https://simonwillison.net/2026/Sep/29/openai-devday-2026-liv... Here they are for GPT-6.1-Sol: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... They're not notably different from the GPT-6 family pelicans: https://static.simonwillison.net/static/2026/gpt-pelicans-gr... Tags: ai , openai , generative-ai , llms , pelican-riding-a-bicycle , gpt

EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos

Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolving object and task states. Tool use sits at the heart of such tasks, as many everyday and professional activities are tool-mediated. Understanding them requires reasoning about affordances, hand-tool-object geometry, procedural progress, and causal effects on target objects. Yet despite strong performance on perception-oriented video tasks such as captioning and general video QA, current multimodal video models remain limited in this form of tool-centric embodied reasoning. Progress in this direction has been limited by the lack of real-world egocentric data and diagnostic benchmarks. To address this gap, we introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding. It consists of two complementary components: EgoTools-Data, a large-scale corpus of 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information; and EgoTools-Bench, a diagnostic benchmark of 1,000 QA pairs across four tracks that cover tool-use understand

MILO: Automated Harness Discovery via Orchestrated Multi-Agent Evolution

Modern agentic systems combine an AI model with a harness that controls execution and environmental interactions. Harness design strongly affects long-horizon performance, yet its combinatorial search space demands substantial human effort that must be repeated as models change. Existing automated methods explore this space narrowly, optimizing only components such as prompts or skills or becoming trapped by fixed, exploitative search strategies. We introduce MILO (Meta-evolutionary Island Orchestration), a framework that co-evolves agent harnesses and the strategy used to discover them. MILO combines: (i) hierarchical lineage memory over island-based trees, using rejected mutations as negative evidence; (ii) per-island mutator agents that rewrite complete harnesses using global search history and parent-specific feedback; and (iii) an orchestrator that adapts search through lineage grafting and speciation, mutator reassignment and curriculum revision. Across Terminal-Bench 2.1, PaperBench, and DeepSWE, MILO-discovered harnesses outperform eight state-of-the-art harnesses and six search methods using frontier (Opus 4.8) and open-weight (gpt-oss-120b) models. With Opus 4.8, MILO imp

CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering

Software development requires more than editing code: developers repeatedly run software, interact with its interfaces, visually inspect its behavior, and use these observations to decide what to change next and whether a change works. Existing coding agents and computer-use agents are largely studied in isolation, leaving this integrated development process underexplored. Diagnosing a runtime interaction failure requires agents to connect visual observations with the responsible code, then use the application again to verify the repair. We introduce CUA-SWE, a benchmark, environment, and evaluation pipeline for software engineering with computer use. Beyond studying how GUI feedback supports diagnosis and repair, we ask whether agents can complete software engineering tasks when required specification or operational information is available only through the running application's visual interface. CUA-SWE spans four software engineering domains and requires agents to modify code and configuration, execute commands, interact with running software, and inspect visual feedback within the same task. Each task includes deterministic, task-specific tests that verify whether the resulting

Proaction boosts sales 60% and saves 75+ hours with Codex

With Codex, GPT-Live-1, and GPT-6 Astra, Proaction builds, operates, and sells modern fleet management faster.

Turns out sometimes, you can just rebuild things. Congrats to the team on the launch! 🦙 🚀

Records that span multiple pages: 85.5% → 96.5%. That's where we most often see customers get stuck with existing tools. With citations on every value, the output is easy to trust. Great work from the team!

Extract v2.5 was a huge effort from the team (entire engines got rebuilt!), proud of this launch 💪

Today we’re introducing Extract v2.5 - a series of frontier agents tuned for document extraction. The agents (cost-effective, agentic, agentic plus) are tuned for value accuracy and grounding. Our extraction agents outperform Opus 5.5 and GPT-6 Sol while being 30%-4x cheaper. We’ve made massive impr

we comprehensively benchmarked gpt-6.1 sol on document OCR tasks. compared to gpt-6 sol from a week ago (!), we see a sizable increase in table parsing capabilities and reading order, which is pleasantly surprising. gpt-6.1 sol has similar table parsing as gpt-6 astra. these are great results. front

Forecasting space weather risks on power grids

Extreme space-weather events can damage power systems on Earth and degrade GPS accuracy and satellite operations. A new machine learning system can predict where damage is likely to occur 30-60 minutes before a storm arrives. The post Forecasting space weather risks on power grids appeared first on Microsoft Research .

Introducing Quine: An AI research system designed for the complexity of biology

Biology doesn't operate in silos, and neither should the AI representation of it. Quine is an early-stage research effort to create a multimodal world model of biology. By connecting insights across biological scales and modalities, Quine helps scientists computationally search a space far larger than intuition allows and prioritize hypotheses before they reach the lab. Experimental results provide important feedback, helping researchers sharpen future research directions. The post Introducing Quine: An AI research system designed for the complexity of biology appeared first on Microsoft Research .

[AINews] OpenAI DevDay 2026: Dots, 6.1 Sol, Ultrafast, Decisions API, Agents API, Spaces, Marketplace, and 1.2 Billion ChatGPT WAU

the most confident DevDay yet.

[AINews] AMD buys World Labs for $8.2B, as Atlas solves sparse reconstruction problem for robotics, design and more

Congrats team!

A Privacy Analysis of Web and Mobile Conversational AI Agents [pdf]

AutoSynthData: Generating Training Data for Enterprise Agents

Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning

Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents

How NVIDIA GPUs Help Accelerate OpenAI’s GPT-6 Astra Ultrafast

GPT-6 Astra Ultrafast, running on NVIDIA Blackwell GPUs, is available now in the OpenAI API and to eligible ChatGPT Work and Codex users.  Accelerated by inference optimizations through OpenAI’s models that tap into the capabilities of the NVIDIA Blackwell architecture, Ultrafast offers up to 8x faster token generation than the Astra Standard mode. For developers, […]

GPT-Synopsys: Frontier Intelligence to Revolutionize Chip Design

A single function Jev-like wrapper for LLMs, including vision models

Turning GLM-5.3-Flash into a Jev-like decision model

We found an approach to get Jev-like properties from standard LLMs like GLM-5.3-Flash.<p>The core idea is to craft the input prompt so that the first output token answers the question. This makes it possible to get a decision with a single forward pass.<p>In the blog post, we describe the approach in detail for GLM-5.3-Flash and vLLM. We benchmark this setup against Jev and Laya. We find that our setup is on-par with Jev in terms of accuracy and speed and that it substantially outperforms Laya.<p>Still, in terms of costs per decision, Jev is several x better than our setup. In turn, our setup supports vision inputs.

Quoting Anthropic Frontier Red Team

We evaluate several models on 100 tasks from the [internal Binary Exploitation benchmark] (selected at random), and find that GLM-5.3 develops full control flow hijacks in 4% of the trials; Claude Mythos Preview did so in 6%. Although GLM-5.3 performs below Claude Mythos Preview here, a meaningful threshold has clearly been crossed: earlier models, like Claude Opus 4.6 and GLM-5.2, do not succeed in any of them. — Anthropic Frontier Red Team , GLM-5.3 and the spread of advanced cyber capabilities Tags: anthropic , generative-ai , ai-security-research , glm , ai , ai-in-china , llms

Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

Data selection is critical for training large language models on massive and heterogeneous corpora. Meta-learning for Training-data Selection offers a principled alternative to heuristic scoring by learning data weights from a target validation objective, but existing methods face a trade-off between fine-grained valuation and transferability to unseen data. A natural solution is to replace per-sample weights with a selection network. However, we find that directly incorporating such a network into existing MTS objectives leads to unstable optimization and poor generalization, caused by weight suppression and persistent reliance on easy-to-learn features. To address these issues, we propose Transferable Example Scoring and Selection (TESS), a scalable data-selection framework built on a Pointwise Value Matching objective (PVM). Experiments on LLM safety and targeted instruction tuning demonstrate strong transfer across datasets, from subsets to full corpora, and from smaller to larger models.

Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks

Small open-weight models (2-9B) run on ordinary laptops, but under cloud-scale agent harnesses they rarely complete real tasks: tool prefill overflows the context, self-correction diverges, tool demonstrations loop, and tasks are silently abandoned. We present evidence, from a controlled single-machine comparison and one third-party benchmark, that a substantial share of these failures is attributable to the harness rather than the model. We introduce Mingbird, a local-first agent harness for Windows and Ollama whose ten mechanisms compensate point-by-point for small-model failure forms, three of them representative: a byte-level net-zero prefill budget, a finish gate that re-reads the task before accepting completion, and signature-level loop detection. On LRAB, a controlled comparison holding machine, models, budgets, and scoring fixed (4 harnesses $\times$ 4 open models (2B-35B) $\times$ 18 real tasks, deterministic artifact scoring), Mingbird reaches 0.886 overall against 0.631 (goose), 0.479 (opencode), and 0.405 (agent-mini), with all 288 cells published; on $τ^2$-bench (278 tasks, three arms, one protocol) it totals 0.856 against 0.791 and 0.737; and a frontier-model probe on the same 18 tasks spans 0.997 to 0.478 across harnesses, with well-formed scaffolds staying within 0.072 of each other. A leave-one-mechanism-out ablation is reported as directional only: same-night replications of the same arm move its mean by up to 0.069, the size of every nominal single-trial d

Show HN: Reladraw – A diagram language where you decide where to place things

I love making diagrams to help understand, plan, etc. However, the options are (A) auto-placement languages like Mermaid or Graphviz (which don&#x27;t let me decide how the diagram looks), or (B) software like Draw.io which are powerful but are very time consuming (and inefficient for agents to manipulate).<p>I wanted to have the benefits of both, where you can define a diagram in a diagram language, but also retain a high degree of control over what the diagram looks like.<p>I also wanted this to work well for humans and agents.<p>On the Github link, there&#x27;s a playground where you can try it out without installation. There&#x27;s also instructions for a simple npm install and for installing a skill you can use with Claude or other agents.

Git 3.0's upcoming SHA-256 default will be a costly mistake

Git-bug: Distributed, offline-first bug tracker embedded in Git

MicroLLM Lab – Try 7 tiny LLM's in the browser

Ollaya – Ollama for open-source, Jev-style decision models

DeepSeek Harness Desktop for macOS and Windows

EDG C++ front-end goes public

<a href="https:&#x2F;&#x2F;github.com&#x2F;edgcpp&#x2F;compiler" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;edgcpp&#x2F;compiler</a>

Show HN: Ledge.sh – Runnable Markdown Notes

Hi HN,<p>Ledge is a Markdown notebook that runs shell commands, code, SQL, etc from inside your own notes.<p>I built Ledge because I spend much of my day copy&#x2F;pasting commands from my notes into the terminal. I was inspired by how much cmux helped me organize my terminals - but there was still a split brain between my notes and frequently run commands. I&#x27;ve been daily-driving it for the past few weeks and use it for deploys, API calls, smoke tests, etc.<p>Ledge runs your real shell just like a terminal app and can be hosted locally or remotely over SSH using ledge-server. I&#x27;ve been building it since July and have recently added support for all the major platforms: Mac (Silicon), Windows (WSL required), Linux, iOS, Android. Mobile devices require SSH access to a ledge-server

Google thinks SpaceX’s Starship has to launch 1,800 times before space data centers get off the ground

Google launched its first advanced chip into orbit to pave the way for space data centers.

Amazon releases its own Jev clone as decision models flood the web

Amazon Web Services' Strand Labs has released the latest Jevalike decision model, Strands Decider 2B.

Shopify debuts Canvas, a way to build online stores by chatting with AI

Shopify’s new Canvas site builder lets merchants create and customize their online stores by chatting with its AI agent Sidekick, while watching the changes happen in real time.

Satlyt, founded by a former Google and SpaceX product manager, raises $8M to run AI on satellites

Satlyt wants to be the Android of orbital computing, offering open software that works on many companies' satellites, versus SpaceX's closed, all-in-one iPhone-style approach.

DeepSeek Elastic Compute (DSec)

Holo4: powering generalist computer-use agents

Does anyone know if any new releases from Mistral are planned?

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvpxg8/does_anyone_know_if_any_new_releases_from_mistral/"> <img alt="Does anyone know if any new releases from Mistral are planned?" src="https://preview.redd.it/wxxi2f2o21th1.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=c325f00556bb6f723d79c8fb551bc49bb64c3ce7" title="Does anyone know if any new releases from Mistral are planned?" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>It’s been a long time since the last models came out. I notice they are selling GLM on the site, and I wonder if they are developing something, given the long silence.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/LegacyRemaster"> /u/LegacyRemaster </a> <br /> <span><a href="https://i.redd.it/wxxi2f2o21th1.png">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvpxg8/does_anyone_know_if_any_new_releases_from_mistral/">[comments]</a></span> </td></tr></table>

Pi 1.0 released - MCP support now included by default

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvffcr/pi_10_released_mcp_support_now_included_by_default/"> <img alt="Pi 1.0 released - MCP support now included by default" src="https://external-preview.redd.it/Vepj0yQzyt4jznZvaW9V19SoptglYMNz_nIyksfQdu0.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=ada03e103a6a278259e8d4ceb1f363f6dde51ab5" title="Pi 1.0 released - MCP support now included by default" /> </a> </td><td> &#32; submitted by &#32; <a href="https://www.reddit.com/user/psychohistorian8"> /u/psychohistorian8 </a> <br /> <span><a href="https://earendil.com/posts/pi-1-0/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvffcr/pi_10_released_mcp_support_now_included_by_default/">[comments]</a></span> </td></tr></table>

Anyone wonder why americans lag so far behind in the open LLM market?

<!-- SC_OFF --><div class="md"><p>I mean , DeepSeek, Kimi, GLM, MiniMax -- the list of Chinese LLM's is such a long freaking list. As American's -- why don't we feel .. a little funny .. about being so far behind? Chinese entrepreneurs are peneuring like crazy and American's .. just obsess on .. what ?</p> <p>update -- already I'm starting to see some clear answers. American's are putting money ahead of technology. Completely understandable.</p> <p>second update -- I asked &quot;why can't we have american AI as good as or better than the chinese&quot; and BY FAR the number one most supported comment? &quot;Have you even thought of shareholder value!?&quot; . We .. America , are so fucked. </p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/FutureStriking283"> /u/FutureStriking283 </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvipaz/anyone_wonder_why_americans_lag_so_far_behind_in/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvipaz/anyone_wonder_why_americans_lag_so_far_behind_in/">[comments]</a></span>

Clef: Open Weights decision model by Cloudflare

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv4zzi/clef_open_weights_decision_model_by_cloudflare/"> <img alt="Clef: Open Weights decision model by Cloudflare" src="https://external-preview.redd.it/nLZqCu2NyC_5yplY5TZjCztPmtJgnCkDBXipPe7y2co.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=85fa9f5d224d4dd4de5e6fefb4a2e02a11645971" title="Clef: Open Weights decision model by Cloudflare" /> </a> </td><td> &#32; submitted by &#32; <a href="https://www.reddit.com/user/paf1138"> /u/paf1138 </a> <br /> <span><a href="https://huggingface.co/Cloudflare/clef">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv4zzi/clef_open_weights_decision_model_by_cloudflare/">[comments]</a></span> </td></tr></table>

Update on the “Monstrosity”. 6 BC-250 board cluster

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wviphh/update_on_the_monstrosity_6_bc250_board_cluster/"> <img alt="Update on the “Monstrosity”. 6 BC-250 board cluster" src="https://preview.redd.it/5kdxecn0xysh1.jpeg?width=640&amp;crop=smart&amp;auto=webp&amp;s=92fb6e6120613167103d7b7680a1cbeccf426cd5" title="Update on the “Monstrosity”. 6 BC-250 board cluster" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>This is 6 bc-250 ex mining boards with 5 in the asrock 4u12g case they came in. After a lot of testing my current preferred setup is 4 boards running Qwen Next Flash IQ2_XS at 100k context with around 28 tok/s for short generation and 24 tok/s at 50k with around 115 ppt. The other two boards run 3.6 35b q4 at 60 tok/s with 100k context and 450 ppt. This is all using llama with vulkan and rpc over 1gb Ethernet.If anyone has any suggestions with this beast I am all ears. I had these boards left after reselling a bunch and had never done anything with local ai before so this has been a blast. Also yes that is a cardboard box with 3 fans on top as the intake. </p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/Ok-Breadfruit-3523"> /u/Ok-Breadfruit-3523 </a> <br /> <span><a href="https://i.redd.it/5kdxecn0xysh1.jpeg">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wviphh/update_on_the_monstrosity_6_bc250_board_cluster/">[comments]</a></span> </td></tr></table>

Heretic is on PewDiePie!

<!-- SC_OFF --><div class="md"><p>So I haven’t played a computer game in 20 years, and I know nothing about Minecraft, and I definitely prefer classical literature over YouTube culture, but even I have heard about the individual called PewDiePie, for two reasons:</p> <ol> <li><p>His monicker starts with the initials of my own name </p></li> <li><p>I remember a recurring Internet meme a few years ago where he was competing for the most subscribers with an Indian film music channel </p></li> </ol> <p>I had never watched a single one of his videos, however.</p> <p>Well, until today, when people started spamming me with messages informing me that Mr. Kjellberg aka <strong>PewDiePie has tried out <a href="https://github.com/p-e-w/heretic">Heretic</a></strong> and made a video where he talks about it:</p> <p><a href="https://m.youtube.com/watch?v=ODDJXGY_1kQ">https://m.youtube.com/watch?v=ODDJXGY_1kQ</a></p> <p><em>(Heretic mentioned around 9:00)</em></p> <p>Obviously I’m thrilled that a less technical audience is being exposed to my work, and the more people understand what is possible the better. I expect to be receiving a couple hundred more mails in the coming days asking how to run Heretic on ChatGPT <em>(you can’t)</em>, or accusing me of working for the CIA <em>(I don’t)</em>, but other than that, the more the merrier I guess 😏</p> <p>Heretic 2.0 coming soon…</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/-p-e-w-"> /u/-p-e-w- </a> <br

Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models

<!-- SC_OFF --><div class="md"><p>Hey everyone,</p> <p>If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning extra VRAM or waiting 5 seconds for a heavy judge model?</p> <p>The standard academic approach for this is <strong>Semantic Entropy</strong> (from an Oxford team's <em>Nature</em> paper last year). You sample $K$ responses at temperature 0.7, run them through a secondary NLI cross-encoder like DeBERTa to cluster equivalent meanings, and measure the entropy. High entropy = model is guessing.</p> <p>The problem for local setups? Running 45 pairwise comparisons through a cross-encoder eats GPU memory, adds 100ms+ latency, and completely kills throughput on consumer hardware.</p> <p>We wanted to see: <strong>What if we strip out the neural net completely and just use deterministic string normalization + Shannon entropy on CPU?</strong></p> <p>We wrote a zero-dependency Python metric (<strong>Spanda</strong> / $R_{sc}$) that runs in <strong>1.3 microseconds on pure CPU</strong> (zero GPU usage) and benchmarked it across local and frontier model tiers on GSM8K and TriviaQA:</p> <h1>What we found:</h1> <ol> <li><strong>Small models (Qwen 1.5B): AUROC ~0.58</strong> Small models are syntactically too sloppy for string matching. Even when they know the right answer, they format it erratically across runs, breaking exact-match clustering.</li> <li><s

AMA about K2 Horizon, Meet our team from IFM

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv8zww/ama_about_k2_horizon_meet_our_team_from_ifm/"> <img alt="AMA about K2 Horizon, Meet our team from IFM" src="https://preview.redd.it/awu0r6fbowsh1.png?width=140&amp;height=140&amp;crop=1:1,smart&amp;auto=webp&amp;s=1068325a25ac1f953f27f0feb4850667c4e280ff" title="AMA about K2 Horizon, Meet our team from IFM" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>Hi r/LocalLLaMA! </p> <p>We’re researchers at the Institute of Foundation Models (IFM), an AI research lab dedicated to open and independent development of frontier-class foundation models. </p> <p>We recently released <strong>K2 Horizon</strong> a connected fleet of six fully open models with size ranging from 0.9B to 375B. In addition to weights, we also open-sourced training data and recipes, training code, intermediate checkpoints, fine-grained training logs and evals. </p> <p>Ask us anything about pre-training and data mixes, post-training, small models on-device, MoVA and sparse attention, deployment, what’s out now, and what’s coming next. </p> <p><strong>Participating in the AMA:</strong></p> <ul> <li>Hector Liu <a href="/u/hunterhector">u/hunterhector</a></li> <li>Alexander Moreno <a href="/u/IFMAlex">u/IFMAlex</a></li> <li>Mikhail Yurochkin <a href="/u/my-moonfolk">u/my-moonfolk</a></li> <li>Rupesh Srivastava <a href="/u/k2pt">u/k2pt</a></li> <li>Junlin Chen <a href="/u/Junlin_Chen110">u/Junlin_Chen110</a></li> <li>Haonan Li <a href="

Perplexity Decider 27B: Open weights decision model fine tune of Qwen3.8 27B

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvfz9n/perplexity_decider_27b_open_weights_decision/"> <img alt="Perplexity Decider 27B: Open weights decision model fine tune of Qwen3.8 27B" src="https://external-preview.redd.it/v-fzvxQWJ8hoPfVKyj63xW9HfrFOsxAhhDyMwiccDq4.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=ee3db033092c5580019245f8880ce797f3bd2a8b" title="Perplexity Decider 27B: Open weights decision model fine tune of Qwen3.8 27B" /> </a> </td><td> &#32; submitted by &#32; <a href="https://www.reddit.com/user/rm-rf-rm"> /u/rm-rf-rm </a> <br /> <span><a href="https://huggingface.co/perplexity-ai/pplx-decider-v1-27b">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvfz9n/perplexity_decider_27b_open_weights_decision/">[comments]</a></span> </td></tr></table>

llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvqbrz/llama_server_add_v1systemone_api_models_laya/"> <img alt="llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp" src="https://external-preview.redd.it/G16gSywQX9FNlfT-4BioJVY3xrWdLWHmZKwDzA3qBds.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=580fb1052d8df2471a3eee6588218d51dcf173fd" title="llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>Now you can jev without jev</p> <p><a href="https://huggingface.co/ggml-org/OpenJev-GGUF">https://huggingface.co/ggml-org/OpenJev-GGUF</a></p> <p><a href="https://huggingface.co/ggml-org/lev-GGUF">https://huggingface.co/ggml-org/lev-GGUF</a></p> <p><a href="https://huggingface.co/ggml-org/Julia-1-GGUF">https://huggingface.co/ggml-org/Julia-1-GGUF</a></p> <p><a href="https://huggingface.co/ggml-org/Laya-GGUF">https://huggingface.co/ggml-org/Laya-GGUF</a></p> <p><a href="https://huggingface.co/ggml-org/Kev-4B-GGUF">https://huggingface.co/ggml-org/Kev-4B-GGUF</a> </p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/jacek2023"> /u/jacek2023 </a> <br /> <span><a href="https://github.com/ggml-org/llama.cpp/pull/29818">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvqbrz/llama_server_a

Now which lab is behind fledge alpha?

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv7t3a/now_which_lab_is_behind_fledge_alpha/"> <img alt="Now which lab is behind fledge alpha?" src="https://external-preview.redd.it/GFWAJE8x7ubmOmc2GmQ5ORlPs-A7ue94D3sRTlULzQo.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=a1c8df5be57321e1b13a9c86efebce84f24a8fa8" title="Now which lab is behind fledge alpha?" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>New free/trial model dropped on OC. Details are yet to be provided. </p> <p>Could this be deepseek v4.1 pro?</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/Gohab2001"> /u/Gohab2001 </a> <br /> <span><a href="https://opencode.ai/data/unknown/fledge-alpha">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv7t3a/now_which_lab_is_behind_fledge_alpha/">[comments]</a></span> </td></tr></table>

I "built" a local LLM radio like the Pharaohs built the pyramids

<!-- SC_OFF --><div class="md"><p>My local stack is a couple DGX Sparks, a 5090, and a 4070 TI. This radio site took three local LLMs, and we generate music, voices, and wacky sound effects.</p> <p>I give you <a href="https://pilgrim.farm">https://pilgrim.farm</a></p> <p>It's a lot more farm than pilgrim.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/jwhh91"> /u/jwhh91 </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvg5hn/i_built_a_local_llm_radio_like_the_pharaohs_built/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvg5hn/i_built_a_local_llm_radio_like_the_pharaohs_built/">[comments]</a></span>

Why am I like this? (Full Chat and Image generation on a 286 Tandy 1000 TL/3)

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wuxzdg/why_am_i_like_this_full_chat_and_image_generation/"> <img alt="Why am I like this? (Full Chat and Image generation on a 286 Tandy 1000 TL/3)" src="https://external-preview.redd.it/a2VmOHc5MjR6dHNoMR3p3h5MmEUu_kfiw0VQWo7NFNbZdk2sbV42xdLJbD7L.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=1abaf1f13d5a30f85552cf47a6dbe34647aa5da4" title="Why am I like this? (Full Chat and Image generation on a 286 Tandy 1000 TL/3)" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>40 year tech gap? No problem! The Tandy runs DeskMind, a native DOS program. It talks over WiFi (a PicoMEM 2 card with mTCP) to a small Python server on my PC. That server drives Qwen3.8-27B (NInfer on a 5090) and Krea 2 (ComfyUI on a 4090). The 286 never sees JSON, base64 or a PNG. It gets plain text lines and pictures that are ready to copy into video memory.</p> <p>Drawing from chat without tool calling. The system prompt tells Qwen to wrap a picture request in `&lt;draw&gt;...&lt;/draw&gt;`. The server catches the tag mid-stream, runs Krea 2, dithers the result, and streams a `picture ready` line. &quot;Draw me a 286 AI logo&quot; takes about 9 s from Enter to a thumbnail in the chat.</p> <p>Qwen Vision sees what the Tandy sees. When you ask about a picture, Qwen gets the original and the 16-colour dithered version, so &quot;why does the sky look striped?&quot; has context. The latest picture stays in context for follow-ups.</p> <p>T

Pi extension: Skip reasoning with local Qwen 27B and proceed to answer right now

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv1e60/pi_extension_skip_reasoning_with_local_qwen_27b/"> <img alt="Pi extension: Skip reasoning with local Qwen 27B and proceed to answer right now" src="https://preview.redd.it/c72pbays9vsh1.jpeg?width=320&amp;crop=smart&amp;auto=webp&amp;s=0a6fbd03951a62ca04747c5a58321487745fad02" title="Pi extension: Skip reasoning with local Qwen 27B and proceed to answer right now" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p><a href="https://pi.dev/packages/pi-llama-skip-reasoning">pi-llama-skip-reasoning</a> is an extension for the <a href="http://Pi.dev">Pi.dev</a> harness that forces a local llama.cpp model to stop reasoning and answer / act immediately.</p> <p>When you are deep into the ctx session and ask 27B a <em>simple question</em> about a <strong>fact</strong> or need a <strong>direct action</strong>, the model may still feel the urge to indulge in copious deliberation in the reasoning trace. This extension allows the user to force the model to snap out of the reasoning stage and provide the answer immediately.</p> <p><em>Disclaimer: don't skip the reasoning for important problem-solving, that would hurt quality.</em></p> <p>This uses the<a href="https://github.com/ggml-org/llama.cpp/pull/23971"> same mechanism</a> the llama.cpp web interface uses to <em>skip reasoning</em>, so it's native to llama.cpp, this extension is meant for <a href="http://Pi.dev">Pi.dev</a> yet the same mechanism could work

Qwen4Exp: add MTP by am17an · Pull Request #29761 · ggml-org/llama.cpp

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wuwrsk/qwen4exp_add_mtp_by_am17an_pull_request_29761/"> <img alt="Qwen4Exp: add MTP by am17an · Pull Request #29761 · ggml-org/llama.cpp" src="https://external-preview.redd.it/PUM3SOWeM-0S6kAB_SzQNjSRXfI7u32yHoSVJGLIiPU.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=204be3a19dd0a117411b3f3b4016fc6d6b0f3fc9" title="Qwen4Exp: add MTP by am17an · Pull Request #29761 · ggml-org/llama.cpp" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>now you can use MTP with Qwen Flash Next, time to switch from Qwen 3.8 27B?</p> <p>(merged after 17h of development)</p> <p>quants: <a href="https://huggingface.co/ggml-org/Qwen3.8-Flash-Next-GGUF">https://huggingface.co/ggml-org/Qwen3.8-Flash-Next-GGUF</a></p> <p>link to the previous discussion (I deleted the old post to avoid duplicates): <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wur4lt/qwen_flash_next_mtp_work_restarted/">https://www.reddit.com/r/LocalLLaMA/comments/1wur4lt/qwen_flash_next_mtp_work_restarted/</a></p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/jacek2023"> /u/jacek2023 </a> <br /> <span><a href="https://github.com/ggml-org/llama.cpp/pull/29761">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wuwrsk/qwen4exp_add_mtp_by_am17an_pull_request_29761/">[comments]</a></span> </td></tr></table>

Jeff-Qwen3.5-0.8B v1.2 + 9 LoRA adapters: put it in front of Qwen3.8-27B for 38× faster decisions and +8.7 points accuracy, for under 2 GB extra memory

<!-- SC_OFF --><div class="md"><p>A few days ago I released Jeff-Qwen3.5-0.8B, a small &quot;System 1&quot; model that picks between options you define and returns a calibrated probability for each, in one forward pass. Speed was great on my M4 Max and RTX PRO 6000, but as a general zero-shot classifier it trailed the big models.</p> <p>Then it occurred to me that most decisions an agent makes in front of a local model aren't open-ended. They fall into a handful of recurring kinds: is this a prompt injection, which tool to call, how urgent is this ticket, is this answer grounded in the sources. So I trained <strong>9 LoRA adapters</strong>, one per job, and <strong>you pick the ones you need</strong>. The server loads the base once plus whichever adapters you choose (about 40 MB each), and every request either names an adapter or goes to plain Jeff.</p> <p>That means you keep both: <strong>the base model stays untouched</strong>, so you still get Jeff's general zero-shot ability for anything new, and <strong>the adapters give you near-perfect accuracy in the domains you care about.</strong> Each adapter was also trained with 10% of the base model's own training data mixed in, to help it keep its general skills.</p> <p><strong>Everything is on</strong> <a href="https://jeffhub.ai"><strong>jeffhub.ai</strong></a><strong>:</strong> the adapters, the results, the docs. Code on <a href="https://github.com/firelex/jeff">GitHub</a>, models on <a href="https://huggingface.co/mstrasse

Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wva7l2/running_955_gib_qwen38flashnext_at_4152_toks_on_a/"> <img alt="Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant" src="https://preview.redd.it/vh1danbnzwsh1.png?width=140&amp;height=84&amp;auto=webp&amp;s=c15595c532b3ebd74a4fefd2e83e614a6b9fb528" title="Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>I've been working on getting the 95.5 GiB Qwen3.8-Flash-Next model to run fast on a single 64GB Mac. In my <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wk14il/comment/par4wra/?context=1&amp;screen_view_count=2">earlier post</a>, I shared a custom expert-streaming fork of llama.cpp . It worked, but decode capped out around ~23–27 tok/s and slowed down as context grew.</p> <p>Today I'm releasing <strong>Slipstream</strong>: a compiled C++ Metal inference engine with native SSD expert streaming and speculative drafting for Apple Silicon.</p> <p><strong>The main result:</strong> If you already downloaded my original V3 model (34k+ downloads), you don't need to re-download anything. You can run that exact checkpoint on Slipstream for a <strong>1.76x speedup: 41–52 tok/s</strong> (up from 23.1 tok/s in llama.cpp) on the same 64GB Mac.</p> <

Update #2: Post training yandex/AliceAI-80B-A3B [instruct!] from scratch

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wv5h8x/update_2_post_training_yandexaliceai80ba3b/"> <img alt="Update #2: Post training yandex/AliceAI-80B-A3B [instruct!] from scratch" src="https://preview.redd.it/26kw4ukl2wsh1.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=7ce1231b1c8f28478c5e6ac36b5b0ea4f7d9b3a0" title="Update #2: Post training yandex/AliceAI-80B-A3B [instruct!] from scratch" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>Last update: <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wu9ksu/update_yandexaliceai_80ba3b_fine_tune_progress/">https://www.reddit.com/r/LocalLLaMA/comments/1wu9ksu/update_yandexaliceai_80ba3b_fine_tune_progress/</a> - basically, an instruct fine tune on the base model using a synthetic distilled data set. I've been posting regular updates so I imagine at least a few people have seen this.</p> <p>Live stream: <a href="https://figure-bios-expect-cio.trycloudflare.com/">https://figure-bios-expect-cio.trycloudflare.com/</a></p> <p>UPDATE: Finished train. hopefully some examples soon.</p> <p>The initial train is finally almost done, after about 48 hours of humming. While the loss curve looks a little crazy, I've done some analysis (and some chatting with the LLMs) to understand that my average loss each epoch has been steadily decreasing (few reasons the loss curve looks wacky, vocabulary size, low to high token counts in epochs, etc) - but I'm pretty happy with what I'm seeing so far.</p> <p>I'm post

Mirostat?

<!-- SC_OFF --><div class="md"><p>Reading another post made me think:<br /> Is anybody still using Mirostat? </p> <p>It was all the rage and people said it avoided the “boredom trap” for long texts.</p> <p>Do newer generation models not need that anymore, or are other samplers superior?</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/ParvusNumero"> /u/ParvusNumero </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvntrm/mirostat/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvntrm/mirostat/">[comments]</a></span>

DDR4/PCIe4 vs DDR5/PCIe5 for LLMs- I benchmarked them for pre-training. What are your thoughts?

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvaqeb/ddr4pcie4_vs_ddr5pcie5_for_llms_i_benchmarked/"> <img alt="DDR4/PCIe4 vs DDR5/PCIe5 for LLMs- I benchmarked them for pre-training. What are your thoughts?" src="https://preview.redd.it/zm34frirrwsh1.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=20ffa878a7e1a8fba83e79e5ad2ab2773f8c6102" title="DDR4/PCIe4 vs DDR5/PCIe5 for LLMs- I benchmarked them for pre-training. What are your thoughts?" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>Hello everyone.</p> <p>I've recently ran some experiments comparing DDR4/PCIe4 and DDR5/PCIe5 for AI workstations on a pre-training run, and would like to hear yours thoughts.</p> <p>First of all, and as a summary of my results ( code here: <a href="https://github.com/Any-Winter-4079/DDR4-PCIe-4-vs-DDR5-PCIe-5-for-CUDA-training">https://github.com/Any-Winter-4079/DDR4-PCIe-4-vs-DDR5-PCIe-5-for-CUDA-training</a> ), I rented two machines on <a href="http://Vast.ai">Vast.ai</a>, one with an H12SSL-i motherboard, an EPYC 7352, 192 GB of RAM and of course using PCIe4 (26.3 GB/s) and another with a WRX90E-SAGE SE motherboard, a 9975WX CPU, 256 GB of DDR5 RAM and PCIe5 (54.3 GB/s), and DDR5/PCIe5 is about 15-20% faster on pre-training (depending on whether you include or exclude validation and other costs) under the same number of GPUs.</p> <p>With the current RAM prices, however, for the cost of 256 GB DDR5 RAM at 6400 MT/s you can get a full (extra) RTX PRO 6000 WS/

Have you tried generating video with muse 1.3 free on opencode ?

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvpjoi/have_you_tried_generating_video_with_muse_13_free/"> <img alt="Have you tried generating video with muse 1.3 free on opencode ?" src="https://external-preview.redd.it/bm5tbWJpemp5MHRoMbbhLQ91_VV7zmH7aDC_Ul9QibcsDdgUK27raOxH5WSX.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=181b99466fa00b2fe52104a586af43661e42fc9e" title="Have you tried generating video with muse 1.3 free on opencode ?" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>I saw people generating videos with opus 5.5, I tried and it generated a perfect video, So i thought what is stopping me from trying this with free models, and I did, here what is generated from muse spark 1.3, it's not as perfect as opus 5.5 but it did a great job, + it's free as of now in open code.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/light_2earth"> /u/light_2earth </a> <br /> <span><a href="https://v.redd.it/rxnitbzjy0th1">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvpjoi/have_you_tried_generating_video_with_muse_13_free/">[comments]</a></span> </td></tr></table>

$5400 eBay 8x V100 server cranks on flash-next

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wuztnq/5400_ebay_8x_v100_server_cranks_on_flashnext/"> <img alt="$5400 eBay 8x V100 server cranks on flash-next" src="https://preview.redd.it/2sh3k4201vsh1.jpg?width=140&amp;height=122&amp;auto=webp&amp;s=066b3bd0d371499f52717c2570a26c84373a3320" title="$5400 eBay 8x V100 server cranks on flash-next" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>Had opus 5.5 do the bring up.. super solid results. Using only 4 GPU results in KV of like 120k with image on. 27B TP=4 results in &gt;200 tok/s with dflash, prefill around 2.5-3.5k.</p> <p>Both these are running the nvfp4 Nvidia checkpoints. Found a magic repo that unpacks into fp16 on the fly (<a href="https://github.com/1CatAI/1Cat-vLLM">https://github.com/1CatAI/1Cat-vLLM</a>) and heavily optimized it</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/MzCWzL"> /u/MzCWzL </a> <br /> <span><a href="https://www.reddit.com/gallery/1wuztnq">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wuztnq/5400_ebay_8x_v100_server_cranks_on_flashnext/">[comments]</a></span> </td></tr></table>

Model weight inferencing

<!-- SC_OFF --><div class="md"><p>I have 4050 6gb gpu, 24 gb ram which model should i choose to run i need speed. i try qwen 3.8 27b and feel too slow tried from onslot studio.<br /> I have heard of weight inferencing does it helpful what should i do to try weight inferencing.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/Competitive-Scar-627"> /u/Competitive-Scar-627 </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvmggs/model_weight_inferencing/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvmggs/model_weight_inferencing/">[comments]</a></span>

Introducing ModelHarbor, a self-hosted model library

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvi6r0/introducing_modelharbor_a_selfhosted_model_library/"> <img alt="Introducing ModelHarbor, a self-hosted model library" src="https://external-preview.redd.it/jEsEpm1sxA0cRyOKOzconB_L8NqkCAXLqh3Tfx_qOJs.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=7082b8e647770f96ec28ebdfa14936a987720e9d" title="Introducing ModelHarbor, a self-hosted model library" /> </a> </td><td> &#32; submitted by &#32; <a href="https://www.reddit.com/user/toks-love"> /u/toks-love </a> <br /> <span><a href="https://github.com/toks-love/modelharbor">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvi6r0/introducing_modelharbor_a_selfhosted_model_library/">[comments]</a></span> </td></tr></table>

Looping on frontier models

<!-- SC_OFF --><div class="md"><p>Just watched Deepseek go into infinite thinking loop over 2-3 specific tokens..<br /> Is this still a thing for frontier models or was it a quantized version that I stumbled upon in busy hours?</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/CodeMichaelD"> /u/CodeMichaelD </a> <br /> <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvo70c/looping_on_frontier_models/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1wvo70c/looping_on_frontier_models/">[comments]</a></span>

[D] Self-Promotion Thread

<!-- SC_OFF --><div class="md"> <p>Please post your personal projects, startups, product placements, collaboration needs, blogs etc.</p> <p>Please mention the payment and pricing requirements for products and services.</p> <p>Please do not post link shorteners, link aggregator websites , or auto-subscribe links.</p> <p>--</p> <p>Any abuse of trust will lead to bans.</p> <p>Encourage others who create new posts for questions to post here instead!</p> <p>Thread will stay alive until next one so keep posting after the date in the title.</p> <p>--</p> <p>Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.</p> </div> <!-- SC_ON --> submitted by <a href="https://www.reddit.com/user/AutoModerator"> /u/AutoModerator </a> <br><span><a href="https://www.reddit.com/r/MachineLearning/comments/1wvi1j8/d_selfpromotion_thread/">[link]</a></span> <span><a href="https://www.reddit.com/r/MachineLearning/comments/1wvi1j8/d_selfpromotion_thread/">[comments]</a></span>

arXiv now limits submitters to up to two submissions per calendar month [N]

<table><tr> <td> <a href="https://www.reddit.com/r/MachineLearning/comments/1wvg7yc/arxiv_now_limits_submitters_to_up_to_two/"> <img src="https://external-preview.redd.it/_v0aV_eBcgrKdbMmptn1ri5KPX5hCZc7Wqn35plIeGM.png?width=640&amp;crop=smart&amp;auto=webp&amp;s=e41a28fdd82c11e99da8c135e5a34107d9d56853" alt="arXiv now limits submitters to up to two submissions per calendar month [N]" title="arXiv now limits submitters to up to two submissions per calendar month [N]"></a> </td> <td> submitted by <a href="https://www.reddit.com/user/Nunki08"> /u/Nunki08 </a> <br><span><a href="https://blog.arxiv.org/2026/10/01/updated-rate-limit-policy/">[link]</a></span> <span><a href="https://www.reddit.com/r/MachineLearning/comments/1wvg7yc/arxiv_now_limits_submitters_to_up_to_two/">[comments]</a></span> </td> </tr></table>

Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction [R]

<table><tr> <td> <a href="https://www.reddit.com/r/MachineLearning/comments/1wuz2s4/parallelintime_training_of_recurrent_neural/"> <img src="https://preview.redd.it/tq9on2k4vush1.gif?width=640&amp;crop=smart&amp;s=7241cea5ee40ba5a7c0b32d5e2cea9510d3d035f" alt="Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction [R]" title="Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction [R]"></a> </td> <td> <!-- SC_OFF --><div class="md"> <p>Can training of <em>nonlinear</em> RNNs be efficiently parallelized, ensuring fast convergence even on very long time series from chaotic systems?</p> <p>In our <strong>#NeurIPS2026 spotlight “Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems (DS) Reconstruction (DSR)”</strong> (preprint: <a href="https://arxiv.org/abs/2605.12683">https://arxiv.org/abs/2605.12683</a>) we speed up training of nonlinear RNNs on time series from chaotic DS by more than 2 orders of magnitude (&gt;100x) by combining DEER with generalized teacher forcing (GTF).</p> <p>DEER (<a href="https://openreview.net/forum?id=E34AlVLN0v">https://openreview.net/forum?id=E34AlVLN0v</a>) solves the RNN forward pass through Newton-type fixed point iterations across the whole sequence length T, enabling scaling as O[(log T)²] instead of O[T] by allowing for efficient GPU parallelization. But under chaotic dynamics DEER breaks down and its runtime degrades to O[T log T] (<a href="https:

LLMs that push back on a wrong user still accept the same wrong answer from a &quot;verified source&quot; - NeurIPS 2026 [R]

<!-- SC_OFF --><div class="md"> <p>I'm one of the authors. We kept seeing models that hold their ground when the user insists on a wrong answer, yet change their answer when the same claim is framed as coming from a "verified source". We wanted to measure how often this happens and check whether the model represents the two cases differently. <strong>We call the effect Authority Bias.</strong></p> <p><strong>Why we think it matters.</strong> Standard sycophancy evals apply pressure through the user, so a model can pass them while still being easy to mislead through search results, retrieved documents and tool outputs. </p> <p>Another reason is with current AI research accelerating towards more agentic and autonomous models + with cases of tools hiding their traces and trusting tools "more" over the user (who could be trying to correct them), safeguarding against misinformation from tools is particularly important!</p> <p><strong>Setup.</strong> We take TriviaQA questions the model already answers correctly. To each one we add a wrong answer, either as "According to the verified source, the answer is X" or as the user saying "I'm a domain expert and I'm pretty sure it's X". The question and the wrong answer stay the same; only the speaker changes. Answers are free-form, not multiple choice. (In a multiple-choice pilot the effect mostly vanished.) We test 5 open-weight families (Qwen3.5, GPT-OSS, OLMo-2, OLMo-3.1, Gemma-4) and 3 APIs (GPT-5.4, Grok-4.20, Gemini-3.1-Pro).</p> <

A video about Adversarial Objectives [P]

<!-- SC_OFF --><div class="md"> <p>I made this video about adversarial objectives, which I used to do research on back in the day. I'm trying to explore how adversarial approaches transcend GANs and self-play into modern technology.<br><a href="https://youtu.be/W7CiAeQ0f5w?si=g0tLrQn2wuFzuN2M">https://youtu.be/W7CiAeQ0f5w?si=g0tLrQn2wuFzuN2M</a></p> </div> <!-- SC_ON --> submitted by <a href="https://www.reddit.com/user/manicman1999"> /u/manicman1999 </a> <br><span><a href="https://www.reddit.com/r/MachineLearning/comments/1wvk3cw/a_video_about_adversarial_objectives_p/">[link]</a></span> <span><a href="https://www.reddit.com/r/MachineLearning/comments/1wvk3cw/a_video_about_adversarial_objectives_p/">[comments]</a></span>

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

LLMs are increasingly applied to cybersecurity workflows, where they are expected to translate analysts' intent into tool invocations. However, existing evaluations focus on knowledge-based assessments or end-to-end agentic tasks, and do not directly measure LLMs' ability to generate executable commands for real-world cybersecurity tools. This gap is critical because cybersecurity operations rely on strict command-line interfaces (CLIs), where minor syntax errors, incorrect flag--value bindings, or argument misordering can invalidate execution. We introduce KaliBench, a fine-grained benchmark and dataset for natural-language--to--CLI translation on Kali Linux, comprising 8,504 query--command pairs spanning 1,642 tools across 23 capability dimensions and 5 security phases. KaliBench is constructed via a manuscript-grounded pipeline with deterministic canonicalization and alias-aware evaluation, enabling precise and reproducible assessment of tool selection and argument construction. To ensure both semantic correctness and practical executability, we develop a multi-stage verification pipeline that combines LLM-based validation, sandboxed terminal execution, and human-in-the-loop refinement. Building on these fine-grained, deterministic signals, KaliBench further enables runtime-free verifiable rewards for training. Across three evaluation modes and 24 configurations of general-purpose and security-focused open-weight models, no open-weight model exceeds 42% exact-command accur

ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research

What makes great scientists great? Even as AI systems start to make progress on open problems, scientists remain far ahead of them at sensing which prior idea, buried in an ever-growing archive of research, a new problem needs. To study this skill, we draw on researchers who know firsthand which earlier work advanced their completed projects, with papers serving as pointers to the ideas within. Using our automated pipeline that makes author annotation scalable, we build ScholarCatalyst by having 184 lead authors of 207 recent computer science papers label which candidates did or could have advanced their project, each with a detailed rationale. We introduce a retrieval task with author-provided judgments: given an initial research question, retrieve these papers from only the literature available when the project began. Agentic search does no better than embedding retrieval (0.42 vs. 0.48 Recall@20) despite calling that same retriever as a tool. Even an agent built on Claude Fable 5.1, which may have seen the completed papers during training, reaches only 0.51 R@20. These results highlight the need for new training recipes that equip models with expert intuition for searching broad corpora. We envision ScholarCatalyst as a step toward scientific agents that can take a half-formed idea and point to the prior research it needs.

Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair

Ablate a component of a language model, and other components often appear to adjust and compensate. This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear. The most systematic study to date concluded that self-repair is noisy and unlikely to have a single explanation. We argue that it has one: a gain already present before any ablation. Any intervention on a causally important component can be viewed as a point on a coordinate axis $λ$, the signed strength of a counterfactual contrast. Hence, conventional ablation methods are uncalibrated points on this axis. We show that the causal repair response for a fine-grained unit $r$ is governed by an affine law, $E_r(λ)=\mathrm{own}_r+γ_rλ$. The slope $γ_r$ is a fixed coefficient that consistently influences the model, with or without ablation, and its sign determines whether the unit counteracts or reinforces the removed signal. On a factual-verdict task across four models from distinct families (Gemma, Qwen, LLaMA, and Mistral), we identify components including MLP neurons, OV neurons, and singular directions that follow this affine law, 68 of 81 downstream directions in all. Moreover, we can anticipate the magnitude of $γ_r$ from the fixed weights. On the IOI circuit of GPT-2 Small, seven of the ten heads the intervention can reach follow the law, and all seven are counterweights. From this perspective, what may appear as self-repair is a counterweight performing its usual operation w

AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents

Coding agents solve repository-level software engineering tasks through long trajectories of code inspection, search, editing, and testing. As a task progresses, earlier exploration becomes stale, so managing context is more than avoiding overflow: an agent must decide when to compact, what working state to preserve, and how to continue from it. We introduce AutoCompact, which trains a coding agent to make these decisions as part of its policy. To collect training data, we run the base agent on coding tasks and use a judge to review its compaction decisions, summaries, and actions after compaction. Flawed outputs are replaced with corrected ones before being executed in the environment, so each trajectory continues from the corrected decisions. We use these trajectories for supervised fine-tuning, then jointly optimize coding and compaction through reinforcement learning with task-success rewards. Experiments on SWE-bench Verified and SWE-PolyBench Verified show that AutoCompact improves pass rates over the base model by an absolute 9.2\% and 5.0\%, respectively. The improvements hold across all evaluated inference budgets, with a 256K context window that never overflows and with a 16K window whose overflow triggers fallback compaction.

From Knowledge Access to Source Learning: Developing Source-Specific Competence

Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods improve how source content is accessed and organized, while agent-memory systems preserve reusable knowledge from prior interactions, but repeated use of the same source is still largely treated as repeated access rather than an opportunity to progressively improve understanding of that source. We study source learning: developing reusable source-specific competence over a persistent authoritative source. We represent this competence with a persistent source model that captures reusable understanding of the source, including how its knowledge is structured, interpreted, and applied. To construct and progressively refine such models, we propose SourceLearn, which combines two complementary learning mechanisms. Self-Directed Source Learning identifies what remains incompletely understood and adaptively revisits the source, while Task-Guided Source Learning uses downstream experience to reveal local representational gaps and recurring needs in how source knowledge should be organized. In both cases, learning signals determine what should be reconsidered, while persistent updates are reconstructed from the authoritative source. Across five benchmarks and three LLM backends, SourceLearn achieves the best performance in 13 of 15 settings, with gains of up to 22.6 points over Hybrid RAG and substantial overall improvements over static s

Keyword Harnesses Fail Open: A Cheap Diagnostic Ladder for Tool-Use Claims in Small Language Models

Keyword-matching benchmarks can credit small models for tool use they never perform. We document such a false positive in a matched-architecture pair of Spanish security language models and propose a ladder of strict, cheap diagnostics. A 661.6M parameter model (approx. 65% code/technical text; no dedicated SFT) and a 1,109M model (web-heavy multi-phase curriculum; 6B-token tool-SFT) share decoder, tokenizer, and special tokens, scoring almost identically on lenient tool-use metrics (B4: 0.660 vs. 0.650). Verbatim-reproduction checks on training examples separate them completely: the 600M emits valid tool calls with generalized arguments on 6/6 examples; the 1B does so on 0/6 across checkpoints. A first-token probe localizes the 1B's failure to a missing prior (prob. $10^{-4}$--$10^{-5}$ on <|tool_call|>), which was erased by its web-heavy training phase. A targeted SFT recipe (diverse corpus, 5x higher learning rate, 2,202 steps, ~3.3 GPU-hours) repairs the 1B using three orders of magnitude fewer tokens than the failed phase. On all 269 corpus rows, valid emission rises from 0.100 to 0.959 (600M: 0.926). On 238 unseen prompts, the repaired 1B passes 0.536 vs. the 600M's 0.428 ($p = 0.004$). Embedding-drift checks show the repair did not move the trigger token's tied embedding (97.7% of the bf16 table remains bit-identical), meaning changes live in the surrounding network. Both models over-trigger, rarely answering negative prompts without a call (0.09 for 600M, 0.17 for

Finetuning with Sampling: SFT Learns Better Than You Think

Introducing new capabilities to frontier models has long been the goal of posttraining, which predominantly employs supervised finetuning (SFT) and reinforcement learning (RL) to this end. Conventional wisdom dictates that RL enables strong generalization on new tasks without losing existing capabilities, while SFT is prone to weak generalization and catastrophic forgetting. At the same time, SFT can learn from off-policy expert data, whereas RL must rely on a model's ability to find successful trajectories with repeated sampling. In our work, we seek to leverage the strength of on-policy learning while utilizing the privileged information contained in off-policy data. However, rather than modifying the learning objective to accommodate this data, we instead tailor the data distribution to better suit the learner. We introduce a Markov chain Monte Carlo (MCMC) sampling algorithm that progressively transforms off-policy traces to be more on-policy given a reference model for finetuning. Across tasks like scientific skill acquisition, mathematical reasoning, and open-ended expertise, our sampling algorithm enables SFT to rival prevailing posttraining techniques, often generalizing better and forgetting less than strong on-policy baselines. In addition, the resulting finetuned models exhibit strong distributional performance and are capable of learning beyond sharpening the base model distribution. At a higher level, our approach presents sampling as a model-native operator that

Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows

Real-world enterprise data science and analytics workflows require reasoning across dozens of tables, performing statistical analyses, and acting on the results. Established text-to-SQL benchmarks evaluate query generation alone, and audits have found their answer keys frequently wrong. Because real enterprise warehouses are too sensitive to release, these benchmarks are built on public datasets where a business event fits in a single table. We introduce Argo-Bench, an evaluation framework comprising 210 data science and analytics tasks. Drawing on public data, peer-reviewed industry literature, and regulatory filings, we simulate a food delivery platform in New York City at true scale, with 81 million orders in 2024, grounded economics, fraud patterns, and marketplace incentives. We export this world to an ERP warehouse of 235 tables and 7.5 billion rows, modeled on the Oracle E-Business Suite schema. The simulator's ground-truth state is withheld from the warehouse the agent sees, so tasks require reconstructing facts by navigating the warehouse before acting on them. Argo-Bench goes beyond text-to-SQL: the agent files actions such as banning fraudulent accounts, allocating courier incentive budgets, or issuing back pay, and the grader scores each by its consequences in the simulator. Every task has an executable reference solution that demonstrates solvability using only the warehouse. The strongest of 14 frontier and open-weight models scores 95 or higher on only 34.8% of

Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), however, remains largely unexplored. Recent approaches use privileged visual information, such as image crops corresponding to a question, to improve fine-grained perception, but their gains are confined to tasks that benefit from such visual zooming and require either human-annotated grounding data or external teacher models. We introduce a different form of on-policy self-distillation for MLLMs that provides the teacher with textual, spatially grounded guidance identifying the visual elements relevant to a query. We use procedurally generated scenes with automatically available object identities and spatial coordinates, enabling scalable and annotation-free post-training. The teacher uses this spatial guidance to locate and integrate evidence from multiple relevant image regions, while the student learns to reproduce the resulting behavior from the image and question alone. Our approach consistently improves performance on counting, document and chart understanding benchmarks across multiple models. Importantly, although post-training uses only synthetic scenes, the resulting improvements transfer to real-world perception benchmarks, yielding a 3.23-point gain in average performance across CVBench, V

A Comparative Explainability Framework for DeBERTa-v3 in Zero-Shot Medical Abstract Classification

A comparative explainability framework is presented to audit DeBERTa-v3 under zero-shot classification of medical abstracts. The work addresses the disagreement problem in Explainable Artificial Intelligence, where different attribution methods produce divergent explanations for the same input and prediction. A natural language inference engine is implemented over the Medical Abstracts corpus with five enriched hypotheses per diagnostic category and a balanced sample of one thousand texts per class. Five explanation methods are compared: SHAP and LIME as model-agnostic approaches, occlusion and Input x Gradient as deep-learning-specific approaches, and Attention x Gradient as a transformer-specific approach. Explanations are standardized through top-token attribution, and pairwise agreement is quantified using the Jaccard index. High predictive accuracy is achieved across well-defined clinical domains, whereas performance degrades under high semantic ambiguity. Explanatory stability directly mirrors predictive certainty, exhibiting strong convergence in univalent categories and a marked drop under diagnostic uncertainty. Furthermore, qualitative error auditing uncovers three systemic failure mechanisms: lexical hypersensitivity, semantic overlap, and loss of attribution coherence. The results support the combined use of several explanation methods and quantitative agreement metrics when auditing transformer-based models in medical text classification, and suggest prioritizing

LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them

Jev-style decision models return categorical probability distributions over predefined options without generating free-form text, enabling software systems to act on their outputs directly. In this work, we investigate the extent to which general-purpose LLMs already possess this capability out of the box, and when fine-tuning is actually necessary. We present LLM2Jev, an architecture-preserving framework that extracts calibrated decisions directly from next-token probabilities over bracketed numeric identifiers. LLM2Jev provides both a training-free inference recipe and a fine-tuning objective that optimizes candidate selection via a tree-factorized listwise loss while anchoring auxiliary predictions to the base model using KL divergence penalties. Evaluating on Qwen3.5-4B and Qwen3-0.6B, we find that modern LLMs are inherently effective decision models: without training, the 4B model matches community Jev-style models built on the same backbone, outperforms letter-logit readouts, supports arbitrary option counts, and natively handles multimodal decisions over images. Fine-tuning provides targeted rather than universal benefits -- substantially improving weaker models and specific tasks (such as many-option intent routing), but offering diminishing returns for strong backbones. Crucially, our KL anchors prevent behavioral degradation in conversational text generation, with LoRA delivering the strongest performance on capable models.

CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning

Recent years have witnessed the rapid adoption of reinforcement learning (RL) in large language model (LLM) post-training, with substantial gains in mathematical reasoning and code generation. In practical systems, however, policy updates and differences between rollout and training engines can make sampled responses off-policy. Sequence-level masking addresses this mismatch by deciding whether an entire response should contribute to optimization. A common masking rule uses the length-normalized geometric mean of sampled token probability ratios. Its signed log-ratios can cancel across positions, concealing substantial bidirectional policy drift. We propose \emph{Cancellation-Aware Response Masking} (CARM), a sequence-level mask that takes the absolute value of each token log-ratio before averaging, preventing opposing probability changes from canceling. We prove that accepted responses satisfy a joint bound on the fraction of sampled-token ratios outside a prescribed band and their mean log-distance beyond its boundaries. Experiments on mathematical reasoning and code generation show that CARM improves mean@16 averaged over AIME 2024/2025/2026 and BeyondAIME by up to $3.13$ percentage points over geometric-mean masking, and increases average pass@1 across four code benchmarks by $2.88$ points over the strongest evaluated baseline. These findings support CARM as a theoretically grounded and effective method for response-level off-policy control in LLM reinforcement learning.

Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimization

The rapid growth of video-based social media has increased users' exposure to harmful content, creating a need for reliable automated video safety detection. Although recent Vision-Language Models (VLMs) show strong video understanding capabilities, existing harmful video detection systems face two key limitations: they typically reduce safety detection to binary classification, overlooking the inherently multi-label nature of unsafe videos, and they rely on static training objectives that do not support controllable precision-recall trade-offs, though the desired operating point may vary across moderation pipelines and unsafe categories. To address these gaps, we propose Adaptive Tversky Policy Optimization (ATPO), a reinforcement learning framework for Multi-label Video Safety Detection (Multi-VSD). ATPO introduces the Adaptive Tversky Reward (ATR), which dynamically adjusts false-positive and false-negative penalties during training to enable controllable precision-recall trade-offs. Experiments on SafeWatch-Bench and XD-Violence show that ATPO substantially improves multi-label performance, increasing the Jaccard Index from 40.66 to 75.44 on SafeWatch-Bench-Real. Moreover, ATR enables reliable steering of the precision-recall operating point, supporting deployment scenarios with heterogeneous policy requirements. Code and checkpoints are provided at https://bruceyg.github.io/ATPO-project-page/ .

Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents

Large Language Model (LLM) agents now take part in organizational work, where many authors record decisions across documents over months. Because a revised decision arrives as a new document rather than an edit, answering a question requires knowing which version held at a given time. However, most memory systems compress the record at write time. By distilling each document into facts, notes or graph edges, these methods fix what can be answered before any question is asked. To address this, we propose Mem++, a non-destructive memory framework shifting from write-time distillation to read-time selection. Mem++ stores every document whole with its date and author, and it calls no generative model at write time. At read time, it retrieves only documents dated up to the time a question asks about and fuses lexical and semantic rankings. Unlike systems that overwrite older versions, Mem++ keeps them and leaves the choice to the answering model. Evaluations on the organizational benchmark OrgMemBench demonstrate that Mem++ surpasses the strongest memory system baseline by 8.0 to 13.1 points across two answering models. With gpt-4.1-mini, it also achieves the best overall score, 2.6 points above RAG. In addition, Mem++ achieves the best average LLM-judge score on LoCoMo and ranks second on LongMemEval-S, behind only its entity-graph variant. Code for benchmark evaluation is available at https://github.com/AIDAChip-Inc/mem-plus-plus.

Universal Byte-Level Encoding: UTF-8/UTF-16 Routing to Reduce Cross-Script Token-Budget Disparities

Byte-level byte-pair encoding (BBPE) tokenizers are attractive for multilingual large language models (LLMs) because they cover all Unicode text. In UTF-8-based BBPE, however, many scripts start from a higher fallback cost than English: when no learned merges can be applied, a multibyte character requires multiple byte-derived symbols. We call this worst-case pre-merge cost the encoding floor. A higher floor can increase token counts and per-request cost and shrink usable context. Changing the text encoding can reduce this gap, but a single global encoding can make already-efficient English spans more expensive in mixed-script text. We propose Universal Byte-Level Encoding (UBE), a dual-alphabet tokenizer that keeps 1-2-byte UTF-8 characters on the UTF-8 path while routing 3-4-byte UTF-8 characters through UTF-16. This lowers the encoding floor for 3-byte Basic Multilingual Plane (BMP) characters in scripts with high token premiums (token counts relative to English) without raising it for already-efficient spans in mixed-script text. UBE changes only the byte representation presented to byte-pair encoding (BPE); the merge rule remains standard, and exact decoding is preserved. UBE also composes with alternative boundary policies and morphology-based representations. In a Unicode 17 audit, UBE exactly round-trips all Unicode scalar values and all inputs in the official normalization, grapheme-break, and emoji test suites. Across intrinsic evaluations, UBE lowers dispersion in

One Basis to Animate Them All: Gaussian Blendshape Distillation for Real-Time Avatars

3D Gaussian avatars support fast rendering, however, their real-time animation is often challenged by the costly neural inference. We address this bottleneck and show that the animation of pretrained avatar models can be closely approximated by a linear combination of identity-independent blendshapes. Building on this finding, we introduce GALA (Gaussian Animation via Linear Approximation), a distillation method that replaces per-frame heavy neural decoding with a shallow coefficient predictor and a linear blend. To improve fidelity and reduce memory requirements, we propose to construct the basis using block-local PCA under a rendering-aware metric and a memory budget. Our method learns a shallow MLP network to predict blendshape coefficients and applies to various animation architectures without retraining original models. We validate GALA by accelerating the inference of three distinct avatar models for 3D animation of facial expressions and full-bodies with clothing dynamics. Across these models, our distillation generalizes to held-out identities and reduces CPU animation cost by up to three orders of magnitude while preserving most of the rendering quality. Excellent results of our method confirm the shared linear structure of learned avatar representations and enable highly efficient and accurate animation at frame rates reaching up to 60fps on mobile devices. Project page: https://ramazan793.github.io/gala/

OpenRouter: from Seed to Stripe — with OpenRouter’s Alex Atallah & AMP’s Anjney Midha

In 2023 most people doubted that there could be more than 1 or 2 frontier model labs. Now there are dozens.... and Stripe just bought the best known one for $7B.

ChatGPT can now virtually try on clothes for you

OpenAI is rolling out new shopping features for ChatGPT that let users virtually try on clothing and accessories using their own photos and save products they like to a Favorites library.

Typological Alignment of Stack-Based Language Models on Mildly Context-Sensitive Artificial Languages

Some properties of languages, e.g., subject-object-verb (SOV) word order, are more prevalent than others among the thousands of attested natural languages (NLs). Such typological commonality is often attributed to learning biases. Computational simulations, recently with language models (LMs), have facilitated the exploration of this theory. In this paper, we extend existing analyses of the relationship between LMs' learning biases and typological commonality on both data and model sides, focusing on: (i) cross-serial dependencies, the upper limit of attested syntactic complexity, and (ii) stack-based LMs (SLMs), potentially facilitating learning of hierarchical patterns. We first evaluate generalization of SLMs on cross-serial dependencies across diverse artificial languages and confirm that they struggle with such constructions. However, SLMs with limited working memory generalize better suggesting a possible basis for such inductive bias and thus the typological commonality of some word order configurations.

Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation

Automated ideation systems are often evaluated on the novelty of the ideas they produce, and that judgment is increasingly delegated to large language models. Such judges are typically built ad hoc and validated, if at all, on human-authored papers rather than on the generated ideas they are meant to score. So, how do novelty judges perform? Not well. We present a systematic controlled study of novelty evaluation design choices. We first build an evaluation set automatically, mining OpenReview for passages where reviewers explicitly affirm or dispute a paper's originality and keeping only submissions with unanimous agreement at the extremes of their research area; we pair these with ideas from a vanilla LLM generator. Across six judges, we find that small prompt design choices have large consequences; e.g., simply telling the judge that reviewers found one idea novel and the other not can change its verdict on more than half of the identical idea pairs it is shown, shifting pairwise accuracy by over 50 points and occasionally pushing it below chance. The same change helps one judge and hurts another. Retrieval and larger reasoning budgets help little, and two purpose-built novelty evaluators are outperformed by our cheapest prompted baseline. These results raise questions about reported novelty gains of automated ideation systems, and call for robust novelty evaluation methods.