忆阻器交叉阵列修复方法的34分领先在调优基线后消失
A 34-point win for my memristor-crossbar repair method came from an unfair baseline; tuned, step-matched baselines removed it [R]
作者为存在卡死单元的忆阻器交叉阵列构建修复控制器，初稿中比对手方法高34.0分，调优对手学习率并匹配重训练步数后降至4.2分和1.06分，8种子复现实验均值为-0.005分（95% CI -1.7 至 +1.7）。作者称至少97%的原始领先来自未调优的学习率与不对等算力，按预先设定的规则判定该复现失败。实验仅在CrossSim仿真中进行，故障由作者注入，无器件实测，论文为未经同行评审的预印本。
TL;DR (a confession): I was ready to celebrate a big win. Then I tuned the rival. I built a repair controller for memristor crossbars with stuck cells. At first it led a rival by 34.0 points. After I tuned the rival's learning rate and matched retraining steps, the lead fell to 4.2, then 1.06. A pre-specified eight-seed replication gave -0.005 points (95% CI -1.7 to +1.7). This is a preprint. It is not peer reviewed yet.
Setting. A memristor stores a number as resistance. A crossbar is a grid of them that runs a neural network. Over time, some cells get stuck. I study repair controllers in simulation. A pool of spare cells exists. Faults arrive in waves. The controller can remap cells and run short retraining bursts. The network is ResNet-9 on CIFAR-10. Spares are 15%. There are three waves of 10% stuck cells.
What happened. My first draft reported my controller (D) at 83.3 and the rival (C') at 49.3. That is a 34.0-point lead. The lead fell in steps:
| Step | Lead of D over rival (points) |
|---|---|
| Old rate, unmatched | 34.0 |
| Tuned rate | 4.2 |
| Tuned and step-matched (5 seeds) | 1.06 |
| 8-seed replication | -0.005 (95% CI -1.7 to +1.7) |
At least 97% of the original lead came from an untuned learning rate and unequal compute. The rival rose from 49.30 to 84.31 when tuned.
Replication. I wrote the success and failure rules before I saw the new seeds. Failure meant a mean below +0.5 or at most 5 of 8 seeds ahead. Result: mean -0.005, D ahead on 6 of 8 seeds, Holm-adjusted p = 1.000. By my rules, this is a failure. I do not call the +1.06 replicated.
What still holds.
- At equal burst steps, D beats retraining-only by 3.78 points (5 of 5 seeds).
- That retraining-only lead was 44.7 points before tuning.
- A forward-only BatchNorm refit stays near chance once spares run out. It is 76.06 points below D, on 0 of 8 seeds.
- Living spares match frozen-reference spares at a tuned rate (+0.03 points). An earlier +2.48 came from a barely working learning rate.
Limits.
- Simulation only (CrossSim). I injected the faults. No device measurement.
- D has no measured margin over the matched rival on the default setting. Effects up to about 1.7 points either way are not excluded. I ran no equivalence test.
- D lost to the matched rival on 2 of 8 seeds. I call this a gate-drift failure. The diagnosis is post-hoc.
- The retraining-only comparison rests on 5 seeds.
- Learning rates were picked on test data of seed 42. That is optimistic.
- Seeds were added after first looks (sequential design).
- Compute matching counts burst steps only. D's extra passes are not charged.
- The paper is a case study, not a new method. It does not measure other papers. Prior ideas (importance-ranked remap, BatchNorm refit, plateau stop) are credited to earlier work.
- Code, runs and analysis will be released on acceptance.
Preprint: https://doi.org/10.5281/zenodo.23474217
I would like critique. Where is my matching still unfair? Which baseline should I add?
submitted by /u/Connect_Agency_3975
[link] [留言]
来源：r/MachineLearning · reddit.com