Teppei Nakano — Blog https://kekeke29341.github.io/blog/en.html Notes on medical ML, local LLMs, and air-gapped AI en Split medical ML by patient, not by row https://kekeke29341.github.io/blog/post.html?slug=patient-level-split&lang=en https://kekeke29341.github.io/blog/post.html?slug=patient-level-split&lang=en Sat, 15 Aug 2026 12:00:00 +0000 The same person’s stays in train and test memorize the person, not the disease. Group by patient id; cut by time when you can. Do not mix future notes into an EHR prediction https://kekeke29341.github.io/blog/post.html?slug=ehr-temporal-leakage&lang=en https://kekeke29341.github.io/blog/post.html?slug=ehr-temporal-leakage&lang=en Fri, 14 Aug 2026 12:00:00 +0000 A discharge summary and post-outcome labs are not prediction. Freeze time t; keep only what was observed by then. Do not lead a rare-event paper with AUROC alone https://kekeke29341.github.io/blog/post.html?slug=auroc-rare-events&lang=en https://kekeke29341.github.io/blog/post.html?slug=auroc-rare-events&lang=en Thu, 13 Aug 2026 12:00:00 +0000 At low prevalence, high discrimination still pages mostly misses. Put precision and alert counts on the same table. The dangerous guideline RAG error is a cited hallucination https://kekeke29341.github.io/blog/post.html?slug=guideline-rag-clinical&lang=en https://kekeke29341.github.io/blog/post.html?slug=guideline-rag-clinical&lang=en Wed, 12 Aug 2026 12:00:00 +0000 Keep exceptions in the same chunk. Do not emit withdrawn doses. Score faithfulness to the document, not “correctness.” A hospital LLM is about not sending, first https://kekeke29341.github.io/blog/post.html?slug=phi-hospital-llm&lang=en https://kekeke29341.github.io/blog/post.html?slug=phi-hospital-llm&lang=en Tue, 11 Aug 2026 12:00:00 +0000 Regex de-id still re-identifies. Do not SFT on raw notes. Identifiers live on RAG metadata. Talk-to-note is not a BLEU task https://kekeke29341.github.io/blog/post.html?slug=clinical-text-from-talk&lang=en https://kekeke29341.github.io/blog/post.html?slug=clinical-text-from-talk&lang=en Mon, 10 Aug 2026 12:00:00 +0000 Fluency hides added and dropped facts. The teacher is the progress note right after the talk. Do not fluently “fix” ASR. Do not mean-impute the chart https://kekeke29341.github.io/blog/post.html?slug=ehr-missingness&lang=en https://kekeke29341.github.io/blog/post.html?slug=ehr-missingness&lang=en Sun, 09 Aug 2026 12:00:00 +0000 Why it was not measured can be the severity. Keep a mask and time since last observation. ICD-supervised “diagnosis” often reproduces billing https://kekeke29341.github.io/blog/post.html?slug=icd-label-noise&lang=en https://kekeke29341.github.io/blog/post.html?slug=icd-label-noise&lang=en Sat, 08 Aug 2026 12:00:00 +0000 Suspected, history, and primary are allowed to differ. Name weak labels; do not hire if human spans drop. Radiology-report NLP is negation and time https://kekeke29341.github.io/blog/post.html?slug=radiology-report-nlp&lang=en https://kekeke29341.github.io/blog/post.html?slug=radiology-report-nlp&lang=en Fri, 07 Aug 2026 12:00:00 +0000 Half the findings are absences. If text enters an imaging task, acquisition time and report time must agree. Medical-image scores die on the next scanner https://kekeke29341.github.io/blog/post.html?slug=scanner-shift-imaging&lang=en https://kekeke29341.github.io/blog/post.html?slug=scanner-shift-imaging&lang=en Thu, 06 Aug 2026 12:00:00 +0000 Suspect reconstruction and protocol before disease shift. An embedding that predicts site will be used by the disease model. Is your RNA-seq model predicting the kit? https://kekeke29341.github.io/blog/post.html?slug=rnaseq-batch-effects&lang=en https://kekeke29341.github.io/blog/post.html?slug=rnaseq-batch-effects&lang=en Wed, 05 Aug 2026 12:00:00 +0000 Batch can outrun biology. Put correction order and a cross-study split before the learner. How much heterogeneity can one biopsy support? https://kekeke29341.github.io/blog/post.html?slug=cancer-heterogeneity-ml&lang=en https://kekeke29341.github.io/blog/post.html?slug=cancer-heterogeneity-ml&lang=en Tue, 04 Aug 2026 12:00:00 +0000 The observation is partial. Diversity inside the sample and unseen regions need different teachers. Do not split across a patient’s other sites. MoE compute is sparse; memory is not https://kekeke29341.github.io/blog/post.html?slug=moe-routing&lang=en https://kekeke29341.github.io/blog/post.html?slug=moe-routing&lang=en Mon, 03 Aug 2026 12:00:00 +0000 A router picks experts per token. Capacity is large; collapse kills the sparsity. Do not borrow a dense sizing equation. FlashAttention is I/O, not an approximation https://kekeke29341.github.io/blog/post.html?slug=flashattention-io&lang=en https://kekeke29341.github.io/blog/post.html?slug=flashattention-io&lang=en Sun, 02 Aug 2026 12:00:00 +0000 Do not materialize the O(T²) score matrix. Same softmax, finished in SRAM. The win is long prefill, not one-token decode. KV cache fills VRAM before the weights do https://kekeke29341.github.io/blog/post.html?slug=kv-cache-gqa&lang=en https://kekeke29341.github.io/blog/post.html?slug=kv-cache-gqa&lang=en Sat, 01 Aug 2026 12:00:00 +0000 Under concurrency and long context, weights are not what runs out. What MHA / MQA / GQA actually reduce, from the memory equation. What quantization kills first https://kekeke29341.github.io/blog/post.html?slug=quantization-llm&lang=en https://kekeke29341.github.io/blog/post.html?slug=quantization-llm&lang=en Fri, 31 Jul 2026 12:00:00 +0000 GPTQ, AWQ, and FP8 all throw information away. They differ in which information, and in how calibration drift shows up on names and long context. Why DPO comes after SFT https://kekeke29341.github.io/blog/post.html?slug=sft-then-dpo&lang=en https://kekeke29341.github.io/blog/post.html?slug=sft-then-dpo&lang=en Thu, 30 Jul 2026 12:00:00 +0000 SFT moves a distribution; DPO orders two outputs. Skip a step and you keep honorifics and lose refusal. Cut RAG “accuracy” into four layers https://kekeke29341.github.io/blog/post.html?slug=rag-evaluation&lang=en https://kekeke29341.github.io/blog/post.html?slug=rag-evaluation&lang=en Wed, 29 Jul 2026 12:00:00 +0000 Do not fold retrieval, use, answer, and refusal into one score. Split recall@k from faithfulness, and keep a refusal set as large as the answer set. Half of RAG misses are chunk boundaries https://kekeke29341.github.io/blog/post.html?slug=rag-chunking&lang=en https://kekeke29341.github.io/blog/post.html?slug=rag-chunking&lang=en Tue, 28 Jul 2026 12:00:00 +0000 Too big averages; too small cuts the referent. Metadata is a filter; the vector is the body. RoPE in LLMs, from the rotation up https://kekeke29341.github.io/blog/post.html?slug=rope&lang=en https://kekeke29341.github.io/blog/post.html?slug=rope&lang=en Mon, 27 Jul 2026 12:00:00 +0000 Do not add a position vector — rotate Q and K. Why the relative offset survives the inner product, and what breaks when you stretch context. Windows and attention sinks throw the past away https://kekeke29341.github.io/blog/post.html?slug=attention-sink-window&lang=en https://kekeke29341.github.io/blog/post.html?slug=attention-sink-window&lang=en Sun, 26 Jul 2026 12:00:00 +0000 Not the same as stretching RoPE. Policy outside the window is politely violated. Sink or re-inject. Speculative decoding is an acceptance-rate game https://kekeke29341.github.io/blog/post.html?slug=speculative-decoding&lang=en https://kekeke29341.github.io/blog/post.html?slug=speculative-decoding&lang=en Sat, 25 Jul 2026 12:00:00 +0000 A draft model writes ahead; the large model verifies without changing its distribution. Low acceptance and you are only hosting two models. Temperature is not a sampling policy https://kekeke29341.github.io/blog/post.html?slug=sampling-beyond-temperature&lang=en https://kekeke29341.github.io/blog/post.html?slug=sampling-beyond-temperature&lang=en Fri, 24 Jul 2026 12:00:00 +0000 Temperature is softmax sharpness; top-k and top-p decide the cut. Do not eval at 0.8 and serve at 0.2. Quote T, k, and p together. Pick LoRA layers before you pick rank https://kekeke29341.github.io/blog/post.html?slug=lora-what-rank-fits&lang=en https://kekeke29341.github.io/blog/post.html?slug=lora-what-rank-fits&lang=en Thu, 23 Jul 2026 12:00:00 +0000 Rank is the dimension of the update subspace. Start with q,v, then the MLP. “It’s LoRA, so it won’t forget” is false. RMSNorm and pre-norm are why deep LLMs stand up https://kekeke29341.github.io/blog/post.html?slug=rmsnorm-prenorm&lang=en https://kekeke29341.github.io/blog/post.html?slug=rmsnorm-prenorm&lang=en Wed, 22 Jul 2026 12:00:00 +0000 From why BatchNorm fails on sequences, to RMSNorm without a mean, pre-norm residuals, and SwiGLU. Why decoder-only won the local box https://kekeke29341.github.io/blog/post.html?slug=decoder-only&lang=en https://kekeke29341.github.io/blog/post.html?slug=decoder-only&lang=en Tue, 21 Jul 2026 12:00:00 +0000 Match train and serve, keep one objective, keep one box. Bidirectional still wins some understand-only jobs. Distillation copies a distribution, not an answer key https://kekeke29341.github.io/blog/post.html?slug=distillation-logits&lang=en https://kekeke29341.github.io/blog/post.html?slug=distillation-logits&lang=en Mon, 20 Jul 2026 12:00:00 +0000 Pass teacher logits via KL. Keep hard refusals. For draft models, measure acceptance, not loss. BPE cuts quietly break SKUs and Japanese https://kekeke29341.github.io/blog/post.html?slug=tokenizer-bpe-traps&lang=en https://kekeke29341.github.io/blog/post.html?slug=tokenizer-bpe-traps&lang=en Sun, 19 Jul 2026 12:00:00 +0000 The vocabulary is not words. Read digit, full-width, and proper-name splits before you talk parameters. InfoNCE is the common language of embeddings https://kekeke29341.github.io/blog/post.html?slug=contrastive-infonce&lang=en https://kekeke29341.github.io/blog/post.html?slug=contrastive-infonce&lang=en Sat, 18 Jul 2026 12:00:00 +0000 Dense retrieval and CLIP are a classify-the-positive loss. Negatives set the representation. Do not use a generator’s hidden states as a retriever unchanged. 0.9 is not nine times in ten https://kekeke29341.github.io/blog/post.html?slug=calibration-ece&lang=en https://kekeke29341.github.io/blog/post.html?slug=calibration-ece&lang=en Fri, 17 Jul 2026 12:00:00 +0000 Confidence is not calibration. ECE and temperature scaling. Do not treat cosine or post-DPO logits as probabilities. A public bench score is not a synonym for smart https://kekeke29341.github.io/blog/post.html?slug=eval-contamination&lang=en https://kekeke29341.github.io/blog/post.html?slug=eval-contamination&lang=en Thu, 16 Jul 2026 12:00:00 +0000 Contamination and shift are different. Hire from an in-house set plus overlap with the train corpus. Why Transformers still train with AdamW https://kekeke29341.github.io/blog/post.html?slug=adamw-transformers&lang=en https://kekeke29341.github.io/blog/post.html?slug=adamw-transformers&lang=en Wed, 15 Jul 2026 12:00:00 +0000 Adaptive second moment, and weight decay decoupled from the gradient. Leave Norm γ out of decay. What to decide first for an air-gapped local LLM https://kekeke29341.github.io/blog/post.html?slug=air-gapped-llm-checklist&lang=en https://kekeke29341.github.io/blog/post.html?slug=air-gapped-llm-checklist&lang=en Tue, 14 Jul 2026 12:00:00 +0000 A checklist of decisions to lock down before picking a model, when the stack cannot call any external API. Opening this blog https://kekeke29341.github.io/blog/post.html?slug=welcome&lang=en https://kekeke29341.github.io/blog/post.html?slug=welcome&lang=en Mon, 13 Jul 2026 12:00:00 +0000 I added a Markdown-powered blog to this portfolio. Notes on air-gapped LLMs and machine learning will live here. How to add a post https://kekeke29341.github.io/blog/post.html?slug=how-to-write&lang=en https://kekeke29341.github.io/blog/post.html?slug=how-to-write&lang=en Sun, 12 Jul 2026 12:00:00 +0000 Drop one Markdown file and add one entry to posts.json. No build step.