小规模时间序列预测器的合成训练数据：增益卡在约 0.2%，缺了什么？
Synthetic training data for a small time-series forecaster: stuck at ~0.2% gains. What are we missing? [D]
一个小型 Transformer 预测器在固定预训练 checkpoint 和固定短训练预算下，仅靠合成数据生成器调优，最佳手段是在约 6% 的记录上增加一个复用现有内容的堆叠通道，6 个种子平均仅带来约 0.2% 提升，而目标需要约 1%。
Setup (the data is our only lever)
A small transformer forecaster starts from a fixed pretrained checkpoint and gets a short, fixed training budget on series produced by our synthetic data generator. We can't change the architecture, optimizer or step count. Only the generator is ours.
It's scored on fresh real-world multivariate sensor series, and the eval set rolls over every few days. The score is the geometric mean of normalized error across short, medium and long horizons, relative to a reference generator. We run paired seeds; seed-to-seed noise is about the size of our best effect.
What we've seen
- The best lever so far: on ~6% of records, add one extra stacked channel that re-uses existing content. That gives about +0.2% over 6 seeds. We need ~1%.
- Adding more such records hurts. What goes in the channel barely matters; the structure does.
- The gain doesn't carry over to a different starting checkpoint.
- The reference generator's edge is concentrated at the longest horizon, and it fades as the eval data rolls forward.
Questions
- With a warm-started small forecaster and a tiny budget, which data-side changes have moved error by more than ~1% for you? For example: ordering/curriculum, mixture weights, targeting by horizon or frequency, augmentations.
- Are there synthetic priors for TS foundation models, beyond KernelSynth / ForecastPFN-style generators, where the gains survive distribution shift?
- How would you quickly find which slices of a rolling eval set a reference model is weak on?
- Is a data-only lever with a short fixed budget fundamentally capped near where we are?
submitted by /u/tatjr13
[link] [留言]
来源：r/MachineLearning · reddit.com