Skip to content

Add AutoModelForSequenceClassification support for Qwen3.5 (Qwen3_5Config) #44405

@medhakimbedhief

Description

@medhakimbedhief

Feature request

What happens

When trying to load a Qwen3.5 model for sequence classification:

from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    "Qwen/Qwen3.5-0.8B",
    num_labels=2,
    trust_remote_code=True,
)

Transformers raises:

ValueError: Unrecognized configuration class <class 'transformers.models.qwen3_5.configuration_qwen3_5.Qwen3_5Config'>
for this kind of AutoModel: AutoModelForSequenceClassification.

This error occurs because Qwen3_5Config is not registered in the internal mapping of configuration classes to sequence-classification model classes. Sequence classification mappings currently include Qwen3Config and related, but not Qwen3_5Config. This prevents classification from loading via the auto class.

Expected Behavior

AutoModelForSequenceClassification.from_pretrained("Qwen/Qwen3.5-0.8B", ...) should:

  • Recognize Qwen3_5Config
  • Instantiate a Qwen3_5ForSequenceClassification class (or equivalent)
  • Allow fine-tuning and inference on classification tasks without manual patching

Metadata

Libraries / Versions

  • transformers >= 5.2.0
  • torch >= 2.x
  • Qwen3.5 model on HF Hub

Task

  • sequence classification

Motivation

Many users are adopting Qwen3.5 for fine-tuning tasks beyond generation (e.g., classification, routing), and the auto model infrastructure currently does not support classification out of the box. Addressing this will improve usability for downstream tasks.

Your contribution

I am working on implementing it and will open a PR shortly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions