稀疏预测层级（SPH）在国际象棋任务上的实证评估：非反向传播学习负结果
Empirical Evaluation of Sparse Predictive Hierarchies (SPH) & Non-Backprop Learning on Chess (Lc0 Baseline) Negative Result [R]
一项实证研究测试了基于 Ogma 白皮书的稀疏预测层级（SPH）架构能否在不使用端到端反向传播的情况下完成国际象棋策略/价值预测，结果为负。约 44.9M 参数的 SPH 模型在 Lc0 test80 数据子集上严重落后于仅 3.8M 参数的标准 CNN 基线，策略准确率与价值损失均明显更差。作者认为约 99% 的极端稀疏性阻碍了战术信息传播，局部学习规则难以将全局胜负信号回传至早期感受野。
Hi everyone,
I wanted to share an empirical study testing whether biologically inspired architectures with local learning rules, specifically Sparse Predictive Hierarchies (SPH) based on Ogma's whitepapers (AOgmaNeo) can handle complex policy/value prediction tasks without end-to-end backpropagation.
Chess was chosen as the testbed because it requires both local pattern recognition and long-range spatial dependencies.
- GitHub Repo: https://github.com/P1X3R/sparse-chess
- Control Group Repo: https://github.com/P1X3R/sparse-chess-control-group
1. Motivation & Core Hypothesis
Standard deep learning models (ResNets, Transformers) rely on global backpropagation. I wanted to test if extreme weight/activation sparsity (~99%) combined with local prediction updates could enable scalable, compute-efficient policy/value prediction on modest hardware.
2. Architecture & Optimizations
- Model Size: ~44.9M parameters.
- Encoders: Fuzzy ART-based encoders mapping local receptive fields to columns.
- Body: Dendritic Neural Layers (DNNs) with Leaky ReLU activations acting as non-backprop feature extractors.
- Heads: Specialized policy and value heads matching Lc0 formats (WDL output + Lc0 move probability distributions).
- Quantization: Byte weights with stochastic rounding for optimized SIMD throughput.
3. Methodology & Baseline Control
- Dataset: Chunks of the Lc0
test80dataset (10k training games, 2k validation games). - Control Baseline: A tiny, standard CNN (32 channels, ~3.8M parameters—less than 1/10th the parameter size of the SPH model) trained via standard backpropagation on the exact same dataset splits.
4. Results & Key Findings (Negative Result)
The SPH architecture failed catastrophically when compared against standard end-to-end backpropagation.
- Failure to Converge on Policy: While the standard 3.8M CNN quickly learned board geometry, tactical threats, and basic move distributions, the 44.9M SPH model severely underperformed on policy accuracy and value loss.
- Extreme Sparsity Penalty: Enforcing ~99% sparsity choked the flow of tactical information. In chess, "empty" squares and long-range piece interactions carry high dense information content that local sparse representations struggle to propagate without global credit assignment.
- Local Learning vs. Global Dependencies: Local learning rules excel at unsupervised visual feature extraction, but they fail to pass gradient signals back from a global board outcome (Win/Draw/Loss) to early receptive fields.
5. Discussion & Questions for the Community
- Global Credit Assignment: Has anyone successfully adapted local/biological learning rules (e.g., Target Propagation, Predictive Coding, HTM/SPH) to tasks with strict, long-range global credit assignment like games or logic reasoning?
- Dense Domains & Sparsity: Is ~99% sparsity fundamentally incompatible with dense symbolic/grid representations like chess, or is the limitation purely in my architecture (or maybe a nasty bug I couldn't catch)?
Would love to hear thoughts from anyone working on non-backprop architectures, neuromorphic computing, or biological AI abstractions!
submitted by /u/Severe-Heat-518
[link] [留言]
来源：r/MachineLearning · reddit.com