开发者训练 31251 参数 encoder-only Transformer 预测自身血糖，零样本迁移到真实 CGM 数据

This is related to my previous post where I shared an encoder-only transformer model trained on ohiot1dm + shanghait1dm + azt1d datasets. This time I trained the model on the outputs of my T1DM patient simulator and then measured its zero-shot performance on my real-world blood glucose traces.

The above model has 31,251 parameters (16 layers, 1 attention head per layer, and a hidden dimension size of 16). Training took <60 minutes on nvidia dgx spark. It's an encoder-only transformer that predicts the next 2 hours, and can be used autoregressively for long-horizon predictions (e.g. 8-hour nocturnal predictions). I have trained it specifically to have counterfactual reasoning capabilities. The model has only been trained on synthetic data, and hasn't seen my blood glucose readings prior to testing. I use LoRA adapters on my app for light fine-tuning on my actual CGM traces, but the figures & tables you see above are from the base model without any LoRA adapter attached. The app was used to test the model on the traces of three different CGM models: Libre 3 plus, Anytime CT5, and Linx sensor data spanning the past 30 days.

The testing itself was done on my android app with ExecuTorch backend.

Model source code: github.com/0xdeadf1sh/T1DMAI

Simulator source code: github.com/0xdeadf1sh/T1DMSIM

Android app source code: github.com/0xdeadf1sh/T1DMDROID