TL;DR: AI written reviews, and AI-writing in papers

I was going to give this a title "An Obituary of the Review Process", but in the interest of not sounding too defeatist, I will just say "Critical Condition" because of AI writing, both on the author side and the reviewer side.

I am a reviewer for AAAI 2027 and ICLR 2027. Since Phase 1 and Phase 2 was completed, I can now see other reviews. I can bet good money that about half of the reviews in the papers I'm seeing are AI-generated completely, or edited after AI-generation. This is subjective opinion and I cannot prove it but I will try to convince you - at least convince you that it's a problem that needs much much more attention than is being currently given.

I have been reviewing for top conferences (NeurIPS, ICML, AAAI, ICLR) since 2022, in total about 10 times. I have not felt the same way about reviews, or even about papers, ever, except until recently.

Author side: There was only one paper in the 3 ICLR submissions that felt 100% human-written (now if that isn't either - that'll make me very sad). The others had some elements that smell like AI but you can't prove it e.g. excessive use of very descriptive headings with colourful, marketing-like language e.g. a heading called "The theory predicts performance, and empirical evaluations confirm it", or very dense long sentences where you lose track of the thought where instead of already used terms their synonyms are used and then later even more synonyms, and so on.

Reviewer Side, talking about the reviews I suspect: There is very little variety in the language, between the reviews, and within the same review. One review puts a field "questions for authors" in the weaknesses section, which I suspect is because the field name is changed this time around to "specific points for rebuttal". They've put "N/A" in the new field. The paragraph lengths are almost exactly the same. As if an LLM was told to give 5 strengths, 5 weaknesses, 5 questions. There are also signs of extra effort that a human rarely does e.g. instead of just saying "provide peer-reviewed references instead of arXiv", it also adds the arXiv number; or another review which not only puts doi link (this much is fine), but then in an extra piece just writes "(doi.org)" which seems like an attempt by an LLM to give a source. Reviewers are usually pressed for time, they usually try to communicate so the authors understand, not that everything is exact. These reviews are written as if for public consumption. Then there are smart quotes - which one can think are there because maybe Word or something was used to write them; but seen in light of everything else, now these quotes also smell like AI to me. (I used to not see smart quotes this often in earlier years. Word made quotes smart back then as well.) Unicode symbols instead of LaTeX for things such as \tilde{d}, dashes that are not hyphens - in one review even for a minus sign, and so on.

There are at least two AAAI submissions where there are only two reviews and I strongly suspect the other review to be AI-written.

Everything is broken. I am sorry I have no concrete ideas how to fix this, so far. Do you?

Except one maybe, disallow AI usage even for polishing styling and tone? Just completely disallow AI writing. The author feels it's a change in tone and style only but the reader can sometimes very quickly _feel_ it. Before these powerful LLMs we were not judging papers based on language anyway. I know it's an honor system, but I think that's okay. Saying "you can use AI to polish" perhaps also creates an expectation to polish using AI.

I’ve uploaded a minimal, self-contained implementation of manual autograd for a transformer-based classifier in PyTorch. It can help build intuition for what autograd is doing under the hood and is a useful hands-on reference for low-level differentiation in Transformer models, such as writing custom backward passes and tracing how gradients flow through attention blocks.

🐙 GitHub:

https://github.com/ifiaposto/transformer_custom_autograd/tree/main

📓 Colab:

https://colab.research.google.com/drive/1Lt7JDYG44p7YHJ76eRH_8QFOPkkoIwhn

Papers with Code is now open-source with an Apache 2.0-license.

The goal is to collaboratively create the best paper interface with the community!

Over the past couple of months, I've been working with my buddies Claude and Codex on creating a website that tracks trending AI research, benchmarks and state-of-the-art (SOTA). It's a revival of the original Papers with Code website, which Meta acquired and later shut down.

The website is pretty straightforward:
- Python FastAPI backend
- React frontend

The app itself is hosted on a Hetzner Virtual Private Server (VPS). The code is also hosted there as a single monorepo.

The frontend mostly calls various API endpoints to visualize papers. The API itself is also publicly available, as is an accompanying MCP server.

I've been getting cool feature requests lately, so rather than just working on it myself, I thought it would make a lot more sense to share the codebase openly so anyone can contribute.

Code: https://github.com/huggingface/paperswithcode

Chain-of-thought transformed AI reasoning, but it is limited to words.

What's the physical-world analogue?

Introducing Ψ-0.6, one model that reasons in *streams of thought*: words, depth, motion, camera pose, and imagined scenes. And it learns when to use which!

• Ψ-0.6 surpasses leading open-weight models on complex spatial reasoning (MindCube, zero-shot).
• On FlowBench, it outperforms other world models at predicting how objects move. Trained on simulated rooms, it transfers to real robot tabletops.
• One model as perceiver, VLM, world model, reasoner, and policy. RL learns which representation to reason through.

Thread (with video): https://x.com/khai_loong_aw/status/2115099953433600011

• Paper: https://arxiv.org/abs/2610.32177
• Website: https://neuroailab.github.io/psi-website/blog/psi-0-6.html
• Code: https://github.com/neuroailab/psi-0.6

Hi everyone,

I wanted to share an open-source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.

Link: https://github.com/Quantix-eng/ncm-experiment-notes

This is an independent project documenting empirical experiments on how transformers process and edit internal features. No performance breakthroughs here—just honest documentation of observations alongside rigorous controls.

The repo covers NCM on structured tasks (like Sudoku), mechanistic interpretability in GPT-2, and toy parallel models. I've included all source code, saved results, and interpretations for every run.

For full transparency, I’ve categorized every outcome, including the mixed and completely negative results. AI tools were also heavily used during the planning and coding phases.

Would love to get your feedback on the setups or interpretation limits.

I am a 3rd year PhD student in a small lab working on ML in healthcare. My PI is new (we have no PhD graduates yet - lab is 4 years old).

I want to publish in a top ML conference (I know its necessary for an ML researcher job).

And my PI is saying you need to do method research and I don't know how to do it?

We don't have the amount of data to do foundational models. Our data is very small.

I feel like my PI had me jumping around topics he was excited about for the 1st two years of my PhD and nothing paned out or I had no idea what I was doing.

I started thinking of a certain idea but I needed guidance on how to implement it... My PI was no help and tried pushing me to a new idea. He is usually very "attacky" to my ideas, I always get the feeling I need to prepare to really really defend any idea I have to him.

It's either too incremental, too applied, or not impactfull enough. I tried asking what exactly is a method research? His example was the transformers paper, and his own paper (which was incremental according to his current criteria).

I now have no idea what I am doing, where I am going or what I am going to do...

I still have zero idea what my PhD research question is...

Do you guys have any advice?

* this is my 1st time posting so please be gentle 😅

Hi guys,

Self-distillation methods, see Self-Distillation Fine Tuning SDFT, usually use an EMA (Exponential Moving Average) teacher to get synced and stable targets. I wanted to ask the community for research papers/ works that consider different types of teachers that also hold these properties (they eventually also improve along with student) but are stable (prevent student from drifting).

any links to repos/ papers would be appreciated.

thanks!

Self-distillation methods usually rely on an Exponential Moving Average (EMA) teachers to get synced and stable critics. However, this doesn't scale well for large models since holding both teacher and student (trained) on GPU increases latency, zero3 communication, etc.

We can mitigate this with LoRA finetuning: the backbone is shared & frozen, and we need to maintain two light adapters: one for the student (trained) and one for the teacher (EMA of the student adapter).

Getting this to work cleanly under TRL and ZeRO-3 took some fixes, so I packaged it as a plug-and-play repo. It supports:

-> robust SDFT implementation that runs under ZeRO-3

-> ZeRO-3 + LoRA integration with adapter switching for teacher/student passes

-> Saving and resuming EMA teacher checkpoint

-> working deepspeed configs that fix memory spikes when GPU memory is tight

I tested on Qwen3.5 and ToolAlpaca.

Repo: https://github.com/ifiaposto/sdft_zero3

Feedback/ more examples/ issues welcome!

Recently I've been getting bombarded with reach outs from companies that build RL environments and sell to frontier labs like Anthropic, OAI, etc. I'm not really understanding how and why these companies exist.

Lets say you somehow create an environment that no other model is able to perform well on. Your second criteria would be that this environment is suitable for RLVR, perhaps you have some metric of gradient information per example in the dataset. So you have to prove to the lab you are selling this on, that 1) their model cannot do this task currently and 2) this task is solvable via RL.

At that point, why couldn't you just approach a VC (most of these companies are startups with a good amount of funding anyways), show them scaling laws + your current RL env and just raise money to train your own model? We already see open source models tend to lag behind frontier ones by ~3 months or so, it sort of implies the frontier is mostly new RL environments and stuff (or distillation methods are really good). I'm kinda operating under the impression that pre-training/architecture/infra for large models is largely much easier nowadays, but maybe this is just not the case?

I guess an answer to my question would be either 1) I am totally wrong regarding my assumptions on the difficulty of pre-training + infra (which from my experience in industry does seem to be much easier) or 2) actually collecting all of these other, past, RL environments is still hard and only certain labs have access to all of them or 3) all of these companies are or eventually will train their own models?

Hey! We published a set of 6-step LoRAs, one without the upsampling step: https://huggingface.co/PrunaAI/P-Q-Image-2.1

- Generate or edit images in just 6 steps instead of 40. Use 6-steps for preserving inputs when editing and 6-steps-upsampled for improved visual quality and shorter end-to-end generation.
- 6-step distillation provides improved quality with few generation steps. 6-steps-upsampled is the first distillation that removes the need for a separate prompt-upsampling step. This means 41 to 80 seconds saved, depending on your prompt style.
- The LoRA adapters load directly on top of Qwen-Image-2.1, so the pipeline stays unchanged. Few-step distillation cuts the diffusion cost, and the upsampled adapter also removes the prompt-upsampling step before image generation.

For my master’s thesis I built an accelerator for diffusion-based action generation in robotics policies. Offline, it looks great: the median absolute deviation between the accelerated output and the original model’s output is extremely small, so I concluded it reproduces the model’s behavior faithfully.

When I deployed it on a real robot for a task (picking up a book and placing it on a shelf), it did not work as well as the original model. The offline numbers did not predict the real-world result.

I want to submit this to MLSys. My offline results are strong, but the deployment result is negative. My questions:

1. Is it acceptable to submit with only the offline results, or should I include the failed deployment?
2. If I include it, how do I frame it so it adds to the paper instead of sinking it?
3. Why might a tiny median deviation still lead to task failure? My guesses are tail errors and error compounding in closed loop, but I would like other views.
4. What additional metrics would you report (percentiles, per-step error, closed-loop success rate in simulation)?

Any experience from people who have submitted systems papers with negative real-world results is very welcome.

Hey, I've been working on a PyTorch optimizer called ApexW and figured I'd share it here to get some feedback from people who know more about this stuff than I do.

The main goal is to reduce optimizer-state memory usage and see how much faster I can make optimizer steps on larger GPU workloads. I've been experimenting with compact optimizer states and a Triton-based execution path.

In my current tests, the Triton + compact-state path was around 1.5x faster than PyTorch's fused AdamW on workloads with a few large tensors, and the optimizer-state memory usage was roughly cut in half.

The results aren't all good, though. Those numbers are from optimizer-only benchmarks, not full model training. The automatic execution mode doesn't consistently get the same speedup yet, and workloads with lots of small tensors are much slower than PyTorch's fused optimizer. So I'm definitely not claiming this is a better AdamW replacement for every workload.

I want to test it on more realistic models and different GPUs, and I'd appreciate a second pair of eyes on both the implementation and the benchmarks. the Reality Check (The Trade-Offs)

I want to be completely realistic here—this is an engineering trade-off, not magic:

1. Python Loop Overhead on Small Tensors: When testing a layout with many tiny tensors (many-small test), ApexW’s Triton kernels suffer heavily from Python's launch latency. For example, at 128M parameters spread across many small tensors, PyTorch native was 16.1 ms/step while ApexW compact was 76.0 ms/step (~0.21x speedup). If your model is composed of many tiny layers, standard PyTorch is significantly better.
2. End-to-End Impact: While the optimizer step itself is up to 1.5x faster, this step only represents a fraction of a full training step (which is dominated by forward and backward passes). In my end-to-end MLP test, PyTorch still runs slightly faster overall because of C++ deep integration.

Why/When should you care?

If you are training larger deep learning models (like Transformers or LLM fine-tuning) where you are constantly hitting Out-Of-Memory (OOM) barriers,**,**cutting your optimizer state memory strictly in half lets you increase your batch size or scale up your model size without upgrading your hardware.

GitHub: https://github.com/omen42515-prog/apexw/tree/main

If you've worked on PyTorch optimizers, Triton kernels, or training larger models, I'd be interested to hear what you think. Especially curious if there's an obvious issue with my benchmark setup or an important test I'm missing.

For the past year I have been studying using a fast K-dim hermite spline manifold algorithm, called K-Splanifolds or KS, for general purpose fast regression. In this video I show the third version, KS3, compared to MLP and older KS versions fitting a 4D mandelbulb via gradient descent on random samples. KS3 converges faster and has nearly identical latency on a GPU as MLP.

KS blocks are suitable for a wide range of regression problems and I have found KS3 ranges from considerably faster convergence as MLP (noise removal, geometric fit, financial prediction) to neck and neck (LLMs). You can view code for all three versions of the algorithm and a paper for KS1 here: https://github.com/curvedinf/k-splanifolds

I am an independent and if anyone would like to explore this with me, let me know. Also if anyone would like to donate access to 8x H100, I would like to attempt a world record nanogpt speedrun with KS3 using configurations I have found beat the WR on proxy hardware.

I'm working through a free series on GPU kernel engineering (my own channel; I posted the roadmap here earlier: https://www.reddit.com/r/MachineLearning/comments/1xa4h2g/open_gpu_kernel_engineering_series_naive_gemm_to/ ) . Session 1 is a setup session. While preparing it I collected a few things that cost me time and that I rarely see in one place, so I'm sharing them here directly. The video and code are linked at the end.

1. Check compute capability, not just "has an NVIDIA GPU"

nvidia-smi --query-gpu=name,compute_cap --format=csv

7.5 is Turing, 8.0/8.6 Ampere, 8.9 Ada, 9.0 Hopper. This decides what you can do:

- CUDA 13 dropped Maxwell, Pascal and Volta. For those cards, install a CUDA 12 toolkit.

- GTX 16-series is Turing but has no tensor cores.

- Tensor core work needs RTX 20-series or newer (or a T4); Ampere-specific instructions need Ampere or newer.

- FP8 needs Ada or Hopper. Hopper-specific features (TMA, WGMMA) need an H100, which most people will have to rent.

1. Driver, toolkit and PyTorch's runtime are three different things

The CUDA version in the nvidia-smi header is the newest version your driver supports, not what's installed. pip-installed PyTorch bundles its own CUDA runtime, so torch.cuda.is_available() returning True says nothing about whether nvcc exists. On WSL 2, don't install a Linux driver inside the distro; use the WSL-Ubuntu toolkit-only installer, because the Windows driver is used.

1. GPU launches are asynchronous, so naive timing measures queueing

A Python timer around x @ y mostly times the launch, not the work. Warm up, synchronize, use CUDA events, average over several runs:

def bench(fn, iters=20):

for _ in range(3):

fn()

torch.cuda.synchronize()

s, e = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)

s.record()

for _ in range(iters):

fn()

e.record()

torch.cuda.synchronize()

return s.elapsed_time(e) / iters # ms

For a matmul, throughput is about 2*M*N*K FLOPs divided by time. Record the dtype next to the number, since FP32 and BF16 baselines differ a lot.

1. Profiler counters: ERR_NVGPUCTRPERM

By default, non-admin users can't read GPU performance counters, so Nsight Compute fails with this error. Fixes: run ncu with sudo (use the full path, since sudo usually drops your PATH), or have an admin enable counter access for all users. On Windows/WSL it's enabled from the Windows side. If you rent GPUs, test this before committing to a provider, because managed container platforms can block counter access.

1. Smoke test before anything else

A tiny add-one kernel with an error check on every CUDA call, built with nvcc -arch=native, then nsys and ncu run on it once. If that works, the driver, toolkit, GPU and both profilers are all fine.

The session also covers the prerequisites (pointers, flat matrix indexing, matmul and its FLOP count, PyTorch basics) and compares ways to get a GPU. It runs about 40 to 45 minutes.

Video: https://youtu.be/h_zWngMWHBE

Code: https://github.com/prasoonmhwr/gpu_kernel_engineering (folder: session-01-prerequisites-setup)

Caveats: the commands are written for Ubuntu and I haven't tested every distro.

Questions for people who do this daily:

- What setup trap cost you the most time? I'm sure I've missed some (Secure Boot and driver loading, container toolkits, mismatched host compilers).

- Is anything above wrong or out of date for your setup?

This is a "survey of current practices" type of research for a niche area, nothing earth-shattering. I’ve written a 2-part Medium series (free, no paywall) on scoring Intelligent Document Processing (IDP). It focuses on Key Information Extraction (KIE) from structured documents, especially forms. I lean heavy on metrics derived from the confusion matrix: different ways to map items such as blanks or misreads to the four possible values, how to count them, when to use each metric.

• Scoring IDP, Part 1: Confidence Is Not Correctness
• Scoring IDP, Part 2: Approaches to Evaluating Form Extraction

Topics I include, but not limited to: recording raw outcomes before mapping them to the confusion matrix (correct, misread, missed, invented, correct blank); confusion matrix scoring for text fields and checkboxes; misread conventions and their effect on scores (FN-only, FP-only; FN & FP); how blanks, NULLs, unreported checkboxes, and unreadable values are counted; F1 vs. Recall vs. Accuracy, including how correctly identified blanks inflate Accuracy; Jaccard Index (IoU) as Accuracy without True Negatives, and its conversion to F1; Precision, Specificity, and False Positive Rate for LLM hallucination control; micro, macro, and weighted F1; choosing a headline score and stating the convention behind it; confidence vs. correctness and calibration (ECE, MCE); and Key Information Extraction (KIE) vs. OCR metrics (CER, WER).

I believe these would fall under "machine learning scoring techniques" applied to Intelligent Document Processing (IDP)--how to transparently and fairly measure (score) rather than how to implement. It may also help folks understand how to read results coming from different benchmarks.

I try not to recommend one opinionated way of doing things and instead cover the variations I’ve run into and the tradeoffs between them. I’m not endorsing any vendor; the series is about scoring itself.

I ran Population Based Training over learning rate and weight decay on grokking (modular addition mod 97, 35% train split, small MLP, 20 members, 20k steps, full batch).

Results

• vs a grid (equal compute, 3 seeds): the best fixed cell reached 95% val acc in 317 ± 29 steps; PBT took 2667 ± 2244. PBT was slower in every seed, but the grid’s corner already contained the answer.
• Dead-zone start (lr 1e-4 to 1e-3, wd 0.01 to 0.08, where no fixed setting groks): fixed 0/3, slow PBT 0/3, fast PBT (every 100 steps, ×0.5/×2) 3/3 at about 6850 steps.
• Its path: lr drops to the floor, then wd climbs to the ceiling, then lr rises. Val loss rewards low lr × wd early on, which is the wrong direction for grokking, then flips sign later.
• Ablations: a random donor still escapes, just later. With no selection, members grok and then the population loses it.
• 10 seeds: wide random search hits 95% at a median of 300 steps, vs 6650 for PBT from the dead zone. PBT also sometimes collapses into the lr-floor/wd-ceiling corner and never leaves.
• Replay: the winning lr/wd schedule with no PBT, on fresh inits: 55/60 reach 95%. Constant knobs at the schedule’s geometric mean: 0/60. The schedule carries the result, not the weight copying.

Takeaway: PBT is a rescue tool for a bad range, not a speed-up.

Caveats: one task, one model, one split. 3 to 10 seeds per test, with p-values mostly above 0.05. The replayed schedules are post-hoc winners. I haven’t tested a range with no grokking region at all, which is where PBT should beat random search.

Write-up with interactive figures: https://shauryasharma.tech/pbt-grokking.html
Code and all results: https://github.com/Shaurya-34/grok-pbt

Thank you for your efforts in protecting the community from AI slop.

Nanbeige released Nanbeige4.3-33B as the teacher for its planned smaller agents, including an upcoming 3B model. the weights are Apache 2.0, and the model card lists a 512K context window.

the team reports 80.9 on Terminal-Bench 2.1 versus 73.0 for Qwen3.8-27B, using the same harness and inference settings within each benchmark. it does not win everywhere. on SWE-Bench Pro, the reported scores are 60.3 versus 61.7. these are the releasing team's evaluations, not independent replication.

one practical detail for reproducing the agent results: the evaluations retain earlier reasoning with preserve_thinking=true. the card recommends that setting for multi-turn tool use, but not general chat. comparing task success and total generated tokens with that setting held fixed would be more useful than treating the headline scores as a deployment ranking.

source: https://huggingface.co/Nanbeige/Nanbeige4.3-33B

been digesting the details behind memt-1.0's memory-native architecture recently, and the engineering tradeoffs between full context kv caching and internalized parametric memory are pretty wild.

instead of using append-only prompt stacking or external rag pipelines, memt-1.0 bakes memory directly into the model computation layer using gated deltanet-2 with a stable token-level memory update mechanism within each turn, combined with a mixture-of-storage (mos) layer to scale capacity.

a few quick technical takeaways from the setup:

internalizing procedural experience: standard agents retrieve past execution logs as raw text, which bloats the context window. memt-1.0 continuously internalizes retrieved procedural experience into model weights/states during multi-turn agentic tasks.

efficiency at 128k context: at a history length of 128k tokens, memt-1.0-nano delivers roughly 1.86x end-to-end inference speedup and a 66x reduction in persistent storage compared to a full-context kv cache setup.

general capability preservation: training parametric memory layers usually degrades base llm capabilities, but their two-stage training pipeline (continued pre-training + post-training across 34b tokens) keeps general reasoning competitive while outperforming existing parametric memory baselines.

for those building long-horizon agents, is internalizing state via architecture-level update layers (like gated deltanet / mos) the endgame to escape kv cache memory bottlenecks?

What do you all think about this? It seems increasingly common for undergraduate students to conduct research and publish at A* conferences without a supervisor. As a graduate student, I find it both impressive and intimidating, and it sometimes makes me question my own progress.

I'm interested in how people build datasets for fine-tuning open-weight models, especially when it is about making a model handling edge cases than just perform well on common examples.

I'm curious about the data side of the process.

• Finding edge cases: How do you identify edge cases that are rare or unusual. Also how do you make edge cases on an existing dataset to make it more specialized for what you need.
• Generating examples: Do you write them by yourself, or use a good model to generate synthetic data, mine real data, etc?
• Quality control: How do you catch incorrect, repetitive, or unrealistic examples?
• Coverage: How do you know when your dataset covers enough of the failure cases that matter?
• Measuring impact: Have you had cases where adding more edge cases actually made the model worse or failed to improve it?

everything here ran on my own hardware, an M5 Max MacBook Pro with 64GB. blind test of self-hosted AI agents, every one on Qwen 3.8 27B uncensored Q4_K_M in llama.cpp, same 5 everyday tasks like counting log errors and renaming a function. names hidden, results below, take a guess

The tasks

1. Merge two contact lists, messy emails, two different Alex Kims. All 5 got it right (46 people).
2. Rename a function across 3 files with messy spacing. All 5 passed.
3. Fix a bug, with hidden tests. 4 passed. C turned "Q3/Q4" into "q3q4" instead of "q3-q4".
4. Count the errors in 3 log files. All 5 got 43.
5. Turn a receipt PDF into a spreadsheet. 4 passed. A and B installed a missing Python library without asking. C stopped and asked "can I install this?", nobody answered, so it failed. In a test with nobody watching, that counts as a fail. On my actual hardware, that's the behavior I'd want.

Setup

One agent at a time, auto-approve on, a script decides pass or fail. Agents in alphabetical order, NOT the results order: OpenCode, goose, Atomic Agent, Hermes, OpenClaw

Which agent is A, B, C, D and E? Would you rather run one that installs stuff on its own and finishes, or one that asks first? Drop your guesses below, I'll share the full answer key in a day or two.

TL;DR

We prove a mathematical equivalence between collective decision-making via local imitation (such as honey bees choosing a nest site via the waggle dance) and single-agent Reinforcement Learning (RL) in a multi-armed bandit setting. Even when individual members have no learning capability and merely blindly imitate peers, the population distribution pi updates according to a macroscopic bandit RL algorithm that we call Maynard-Cross Learning (MCL), where N individuals act as N parallel environment rollouts with an effective per-sample step size of alpha = 1/N.

Main intuition

In nature, optimal decision-making emerges through two seemingly distinct paradigms:

1. Individual trial-and-error (RL): A single agent interacts with its environment and updates its policy based on reward signals.
2. Collective imitation (Swarm Intelligence / Evolutionary Game Theory): Simple individuals copy neighbors' strategies (e.g., best-of-n collective decision-making).

Biologists have long described insect colonies as "superorganisms" or "hive minds" with parallels to vertebrate brains. We wanted to see if this metaphor holds up mathematically under the lens of RL: Does a swarm of non-learning imitators literally execute a single-agent RL update at the macro level?

Core development

The paper's core theoretical development establishes a mathematical equivalence between decentralized social imitation in large populations and single-agent reinforcement learning (RL) in multi-armed bandits. By treating the proportion of individuals committed to each option as the action-probability policy of a single macro-agent, we use evolutionary replicator dynamics to bridge local individual interactions and global policy updates.

First, we show that a population following the classic "imitation of success" protocol (where individuals copy neighbors with a probability proportional to those neighbors' payoffs) evolves identically to the expected update of the Cross Learning bandit algorithm. Extending this approach to the "weighted voter" model of the honey bee waggle dance (where scouts advertise nesting sites for a duration proportional to site quality and peers blindly copy the first advertisement they encounter without comparing payoffs) we derive a new macro-level RL algorithm that we call Maynard-Cross Learning. Under this duality, each non-learning individual acts as a single parallel environment sample whose fractional influence on the group mirrors an RL learning rate equal to the inverse of the swarm size.

In practice, this development demonstrates how blind local imitation aggregates into a coherent, parallelized reinforcement learner at the collective level.

Impact

By proving a mathematical equivalence between local social imitation and single-agent reinforcement learning, this work provides a formal mechanism explaining how "hive mind" intelligence and trial-and-error learning emerge at the group level from simple, non-learning individuals. Beyond substantiating evolutionary theories of collective cognition in biology, this framework offers new tools for analyzing imitative dynamics in social and economic systems and for designing scalable, decentralized swarm robotics algorithms.

Links

Paper: https://arxiv.org/abs/2410.17517
Project page: https://hivemind.yannbouteiller.com/
Code/reproducibility: https://github.com/MISTLab/HiveMindRL