All pull requests

Pull requests

• [gemma4]: support E2B L1-L2 validation and E4B single-GPU text training

  #6520·richardoo-707 opened Nov 11, 2026Collaborator·3 of 4 tasks·

  ·

• [example] recompute the Cosmos3 transformer blocks in the backward

  #6519·JazyJiang opened Nov 10, 2026Collaborator·4 of 7 tasks·

  ·

• [example] align the Cosmos3 packed input sequence with cosmos-framework

  #6518·JazyJiang opened Nov 10, 2026Collaborator·4 of 7 tasks·

  ·

• [ci] Validate Attention #6508 with upstream Layer 2

  #6517·b1ack-j opened Nov 10, 2026Collaborator·

  ·

• [example] align the Cosmos3 DROID SFT pipeline with cosmos-framework

  #6516·JazyJiang opened Nov 10, 2026Collaborator·4 of 7 tasks·

  ·

• [moe] integrate EP checkpoints, numerics, gradient sync, and Qwen3-MoE

  #6515·refinedheart opened Nov 9, 2026Collaborator·

  ·

• [moe] support EP expert checkpoints in EPMoELayer

  #6514·refinedheart opened Nov 9, 2026Collaborator·

  ·

• [shardformer] Add explicit attention backends and packed varlen support

  #6508·b1ack-j opened Nov 4, 2026Collaborator·

  ·

• feat(shardformer): add HF5 adapters for LLaDA and Ouro

  #6507·kiseki-zj opened Nov 4, 2026Collaborator·11 tasks·

  ·

• [doc] define Gemma 2C2 requirements for shared numerics

  #6489·richardoo-707 opened Nov 1, 2026Collaborator·3 of 4 tasks·

  ·
  1

• [workflow] improve the bug report issue template and add a regression template

  #6484·JazyJiang opened Oct 26, 2026Collaborator·

  ·
  1

• [booster] use the accelerator API for the TorchFSDP device id

  module: booster/plugin
  #6482·JazyJiang opened Oct 26, 2026Collaborator·

  ·
  1

• Restrict unpickler in async optimizer checkpoint loading (fixes #6471)

  #6478·08raviyadav opened Oct 22, 2026·1 of 11 tasks·

  ·

• [moe] add EPMoELayer with a dispatcher / experts data contract

  module: moe
  #6477·JazyJiang opened Oct 22, 2026Collaborator·

  ·

• [shardformer]: fix pooled token for left-padded inputs in pipeline sequence classification

  #6458·Arthur031221 opened Sep 29, 2026·7 of 11 tasks·

  ·
  2

• fix: use the accelerator API instead of hard-coded cuda/cpu device selection

  #6455·li-lizhe opened Sep 26, 2026·

  ·
  1

  2

• [shardformer]: fix llama causal mask with sequence parallelism and no flash attention

  #6452·LiRunGuo opened Sep 22, 2026·5 of 11 tasks·

  ·
  1

• [shardformer]: fix qwen2/qwen3 causal mask with pipeline + sequence parallelism and no flash attention

  #6451·LiRunGuo opened Sep 22, 2026·5 of 11 tasks·

  ·
  2

• [shardformer]: support Qwen3-MoE with expert parallelism

  #6450·LiRunGuo opened Sep 22, 2026·8 of 11 tasks·

  ·
  1

• [checkpoint]: unpad padded parameters when saving sharded MoE checkpoints

  #6449·LiRunGuo opened Sep 22, 2026·5 of 11 tasks·

  ·
  1

• [tensor]: fix saving checkpoints of padded parameters with tp_size=1 (keep detach/clone of padded tensors unpaddable)

  #6448·LiRunGuo opened Sep 22, 2026·8 of 11 tasks·

  ·
  1

• [gemini]: fix loading checkpoints of models with tied parameters

  #6447·LiRunGuo opened Sep 21, 2026·8 of 11 tasks·

  ·
  1

• [shardformer]: fall back to native RMSNorm when apex is not installed

  #6446·LiRunGuo opened Sep 21, 2026·8 of 11 tasks·

  ·
  1

• [example]: fix Pytorch_DDP/Pytorch_ZeRO plans crashing in gpt gemini demo

  #6445·LiRunGuo opened Sep 21, 2026·3 of 11 tasks·

  ·

• [kernel]: fix Q/K cross-wiring and out-of-bounds accesses in fused_rotary_emb triton kernel

  #6444·LiRunGuo opened Sep 21, 2026·8 of 11 tasks·

  ·
  1

  1