Qwen3.5 model: When data is not padding, an error is reported, indicating that the shape does not match.

[Vectorwh]

commit id：fc9137225880a9d03f130634c20f9dbe36a7b8bf
Qwen3_5 Whether the position_ids input when the text model invokes the decoder_layer is text_position_ids

[vvvdwbvvv]

It seems like related to issue #44322

[pjgao]

In my test, change this line

transformers/src/transformers/models/qwen3_5/modular_qwen3_5.py

Line 681 in 8468dbf

position_ids=position_ids,

position_ids to text_position_ids can fix.

In Qwen3VLTextModel, it is text_position_ids too.

transformers/src/transformers/models/qwen3_vl/modular_qwen3_vl.py

Line 819 in 8468dbf

position_ids=text_position_ids,

, Now why using position_ids instead of text_position_ids in Qwen3_5TextModel?

[zucchini-nlp]

Not sure what is the error @Vectorwh is reporting, a small reproducer would be nice

@pjgao huh indeed, must have been a typo. It might cause some issues in FA2, do you want to open a PR to fix?

[OiPunk]

Opened #44399 to fix the position_ids → text_position_ids typo in Qwen3_5TextModel.forward, as identified by @pjgao and confirmed by @zucchini-nlp.

[JakobJBauer]

I ran into the same issue here:

from datasets import load_dataset
from trl import SFTConfig, SFTTrainer
from peft import LoraConfig

dataset = load_dataset("json", data_files=str(data_path), split="train")

training_args = SFTConfig(
    output_dir=args.output_dir,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    num_train_epochs=3,
    learning_rate=2e-5,
)

peft_config = LoraConfig(
    task_type="CAUSAL_LM",
    r=8,
    lora_alpha=64,
    lora_dropout=0.05,
    bias="none",
    target_modules=["q_proj", "k_proj"],
)

trainer = SFTTrainer(
    model=args.model,
    args=training_args,
    train_dataset=dataset,
    peft_config=peft_config,
)

trainer.train()
trainer.save_model(args.output_dir)

This gives the error:

Traceback (most recent call last):
  File "/workspace/writeable/axiom-guided-structured-reasoning/scripts/train_sft_codebook_qa.py", line 97, in <module>
    main()
  File "/workspace/writeable/axiom-guided-structured-reasoning/scripts/train_sft_codebook_qa.py", line 92, in main
    trainer.train()
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/trainer.py", line 1412, in train
    return inner_training_loop(
           ^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/trainer.py", line 1742, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/trl/trainer/sft_trainer.py", line 1320, in training_step
    return super().training_step(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/trainer.py", line 1951, in training_step
    loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/trl/trainer/sft_trainer.py", line 1216, in compute_loss
    (loss, outputs) = super().compute_loss(
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/trainer.py", line 2022, in compute_loss
    outputs = model(**inputs)
              ^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/accelerate/utils/operations.py", line 819, in forward
    return model_forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/accelerate/utils/operations.py", line 807, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/amp/autocast_mode.py", line 44, in decorate_autocast
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/peft/peft_model.py", line 1923, in forward
    return self.base_model(
           ^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/peft/tuners/tuners_utils.py", line 311, in forward
    return self.model.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 841, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1938, in forward
    outputs = self.model(
              ^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 841, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1694, in forward
    outputs = self.language_model(
              ^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 915, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/utils/output_capturing.py", line 253, in wrapper
    outputs = func(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1354, in forward
    causal_mask = create_causal_mask(
                  ^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/utils/deprecation.py", line 171, in wrapped_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/masking_utils.py", line 933, in create_causal_mask
    causal_mask = mask_interface(
                  ^^^^^^^^^^^^^^^
  File "/workspace/writeable/axiom-guided-structured-reasoning/venv/lib/python3.12/site-packages/transformers/masking_utils.py", line 509, in sdpa_mask
    attention_mask = attention_mask.expand(batch_size, -1, q_length, kv_length)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expand(torch.cuda.LongTensor{[4, 1, 1, 2075, 2075]}, size=[4, -1, 1, 2075]): the number of sizes provided (4) must be greater or equal to the number of dimensions in the tensor (5)

Environment:

- python: 3.12.3 (main, Jan 22 2026, 20:57:42) [GCC 13.3.0]
- platform: Ubuntu 24.04 (Linux-6.8.0-55-generic-x86_64-with-glibc2.39)
- GPU: NVIDIA A100-SXM4-40GB (1x), CUDA available: True
- torch: 2.10.0+cu128
- transformers: 5.2.0
- trl: 0.29.0
- peft: 0.18.1
- accelerate: 1.12.0
- datasets: 4.6.1
- wandb: 0.25.0
- Installation: pip (inside virtualenv)

I also tried installing the Qwen3.5 position_ids fix from PR #44399, but this did not change the behavior; the error in sdpa_mask remains.

[zucchini-nlp]

Hmm, interesting, why would the mask be 5D in forward. Might be an issue with peft or trl, I'll check it out tomorrow