Qwen

/

Qwen3-Reranker-4B

like 130

Follow

Qwen 80.7k

Hello!

Pull Request overview

• Integrate this model using a Sentence Transformers CrossEncoder

Details

This PR adds the configuration files needed to load this model directly as a CrossEncoder via Sentence Transformers. The model uses a text-generation Transformer with a LogitScore head that computes the logit difference between the "yes" and "no" tokens, i.e. the model's confidence that a document is relevant to a query.

A custom chat_template.jinja maps Sentence Transformers' structured messages (with "query" and "document" roles) to the model's expected format with the <Instruct>, <Query>, and <Document> fields, including the <think>\n\n</think> suffix. The template includes a default instruction ("Given a web search query, retrieve relevant passages that answer the query") as a fallback when no prompt is provided.

Added files:

• modules.json: pipeline: Transformer & LogitScore
• sentence_bert_config.json: text-generation task, flat message format
• config_sentence_transformers.json: default prompt, Identity activation
• chat_template.jinja: custom template for the reranker format
• 1_LogitScore/config.json: yes/no token IDs

Once the Sentence Transformers v5.4 release it out, the model can be used immediately like so:

from sentence_transformers import CrossEncoder

model = CrossEncoder("Qwen/Qwen3-Reranker-4B", revision="refs/pr/11")

query = "What is the capital of China?"
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun.",
]

pairs = [(query, doc) for doc in documents]
scores = model.predict(pairs)
print(scores)
# [  6.4375 -14.375 ]

rankings = model.rank(query, documents)
print(rankings)
# [{'corpus_id': 0, 'score': 6.4375}, {'corpus_id': 1, 'score': -14.375}]

And after merging, the revision argument can be dropped. These scores match the transformers code.

Note that none of the old behaviour is affected/changed. It only adds an additional way to run this model in a familiar and common format.
if you are able to merge this before tomorrow's Sentence Transformers v5.4 release, then I will be able to include this in my blogpost and documentation as a release model without revision.

• Tom Aarsen