用名词-动词-连词三元组为神经网络构建“思维语言”的构想
Noun-verb-conjunction triads [D]
一位 Reddit 用户提出让神经网络的每个注意力头持有一个“名词+动词+连词”三元组，用结构化内部语言组合想法。该方案还可在注意力头中加入冠词、时态、情态动词等修饰项，并支持把“because mouse poisoned”这类结论追加回输入、迭代推理。作者设想整本书可表示为由三元组构成的巨大图，但网络按句子等单元分批处理，而非同时处理全图。
The neural network would consist of multiple layers with at least one layer consisting of multiple attention heads. Each attention head would hold one triad consisting of one noun + one verb + one conjunction.
The siamese cat ate the mouse then died:
- (Null) Cat ate
- also Mouse eaten
- and cat is-siamese
- Then cat died
"Also" means the two events are the same event. (Not in english).
A possible conclusion would be "because mouse poisoned" which could then be appended to the input and the process repeated.
In addition a few other ideas can be added to each attention head: * the, a, same, this, that, other, next... * Past, present, future * Could, would, should, might, * Try to * Question mark
Hence * (Null) a-Cat ate * also a-Mouse eaten * and the-cat is-siamese * Then the-cat died * therefore the-mouse might-have-been-poisoned
The goal is to give a neural network a simple, structured internal language in which it can form and combine thoughts. The language of thought.
In principle, an entire book could be represented as one enormous graph of interconnected noun–verb–conjunction triads. But that doesn’t mean the neural network needs to process the entire graph simultaneously.
Humans don’t normally process a book that way either. We divide it into manageable units, such as sentences, understand those units, and gradually build up our understanding of the larger story.
I’m proposing something similar: groups of triads would be processed together, with the network carrying forward relevant information and generating additional triads as needed.
submitted by /u/Comfortable_World_69
[link] [留言]
来源：r/MachineLearning · reddit.com