README.md

---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:1000
- loss:MultipleNegativesRankingLoss
base_model: google/embeddinggemma-300m
widget:
- source_sentence: Qu'est-ce qui a motivé le retour de Claude LeBouthilier au Nouveau-Brunswick?
  sentences:
  - The driver of a vehicle that is approaching a railway crossing at which a stop
    sign has been erected shall stop the vehicle within fifteen metres, but not less
    than five metres, from the nearest rail of the railway.
  - Je suis revenu vivre au Nouveau-Brunswick parce que je ne pouvais plus dissocier
    mon écriture de mon lieu d’origine et de mon existence quotidienne.
  - Quelles sont les procédures pour obtenir un passeport canadien?
- source_sentence: Quels sont les moyens de dépistage du cancer du col de l'utérus?
  sentences:
  - Comprendre les différences entre le test Pap et le test VPH.
  - Employed and self-employed Nova Scotians who are not receiving Employment Insurance
    (EI) and those who had or are in an EI waiting period may qualify for this relief
    grant.
  - Quelles sont les conditions pour obtenir une allocation familiale?
- source_sentence: What are the responsibilities of crew members regarding surface
    contamination?
  sentences:
  - What are the requirements for obtaining a Canadian passport?
  - Crew members are responsible to report suspected surface contamination to the
    pilot-in-command as soon as it is discovered.
  - Plant breeders receive legal protection for up to 25 years for trees and vines,
    and 20 years for other plant varieties.
- source_sentence: Do oil and gas field workers have the same rights to consecutive
    hours off as other employees in BC?
  sentences:
  - The provision of the Act which provides for 32 consecutive hours free from work
    each week does not apply to employees referred to in section 37.6 of this regulation.
  - Les nouveaux bureaux internationaux offriront des services pour faciliter l'investissement
    dans la Saskatchewan et améliorer les exportations vers l'Asie.
  - What are the requirements for registering a new business in British Columbia?
- source_sentence: What is the purpose of the funding provided by the Government of
    Canada to the Federation of Black Canadians?
  sentences:
  - Ghana is an attractive market for industries such as Agriculture, Professional
    Training, Technical and vocational education and training (TVET), Clean technologies,
    Infrastructure, Mining, and Oil and gas.
  - What are the eligibility requirements for the Canada Pension Plan?
  - This investment through the Black Entrepreneurship Program (BEP) Ecosystem Fund
    will allow the FBC to provide tools and resources to 170 Black youth entrepreneurs
    across multiple regions, supporting them to successfully launch and grow their
    businesses.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---

# SentenceTransformer based on google/embeddinggemma-300m

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [google/embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [google/embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m) <!-- at revision 671e8c118e27f9061355bce059ee2d1d86d048df -->
- **Maximum Sequence Length:** 2048 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Neelkumar/my-embedding-gemma-1000")
# Run inference
queries = [
    "What is the purpose of the funding provided by the Government of Canada to the Federation of Black Canadians?",
]
documents = [
    'This investment through the Black Entrepreneurship Program (BEP) Ecosystem Fund will allow the FBC to provide tools and resources to 170 Black youth entrepreneurs across multiple regions, supporting them to successfully launch and grow their businesses.',
    'What are the eligibility requirements for the Canada Pension Plan?',
    'Ghana is an attractive market for industries such as Agriculture, Professional Training, Technical and vocational education and training (TVET), Clean technologies, Infrastructure, Mining, and Oil and gas.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.9830, -0.5013,  0.8960]])
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 1,000 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                           | positive                                                                           | negative                                                                           |
  |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                           | string                                                                             | string                                                                             |
  | details | <ul><li>min: 6 tokens</li><li>mean: 15.8 tokens</li><li>max: 35 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 32.04 tokens</li><li>max: 130 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 15.01 tokens</li><li>max: 42 tokens</li></ul> |
* Samples:
  | anchor                                                                                                  | positive                                                                                                                                      | negative                                                                        |
  |:--------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|
  | <code>Quelles mesures les propriétaires peuvent-ils prendre pour éliminer les punaises de lit?</code>   | <code>Les propriétaires peuvent instaurer différentes mesures pour prévenir et éliminer les punaises des lits.</code>                         | <code>Quelles sont les conditions pour obtenir une assurance automobile?</code> |
  | <code>Comment les pages web du gouvernement de la Saskatchewan sont-elles traduites en français?</code> | <code>Un certain nombre de pages sur le site web du gouvernement de la Saskatchewan ont été traduites professionnellement en français.</code> | <code>Quelles sont les exigences pour obtenir un permis de conduire?</code>     |
  | <code>How long do plant breeders' rights last in Canada?</code>                                         | <code>Plant breeders receive legal protection for up to 25 years for trees and vines, and 20 years for other plant varieties.</code>          | <code>What are the requirements for importing a pet into Canada?</code>         |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 1
- `learning_rate`: 2e-05
- `num_train_epochs`: 5
- `warmup_ratio`: 0.1
- `prompts`: task: sentence similarity | query: 

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 1
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: task: sentence similarity | query: 
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}

</details>

### Training Logs
| Epoch | Step | Training Loss |
|:-----:|:----:|:-------------:|
| 1.0   | 1000 | 0.1065        |
| 2.0   | 2000 | 0.368         |
| 3.0   | 3000 | 0.2343        |
| 4.0   | 4000 | 0.1016        |
| 5.0   | 5000 | 0.0154        |


### Framework Versions
- Python: 3.11.13
- Sentence Transformers: 5.1.1
- Transformers: 4.57.0.dev0
- PyTorch: 2.6.0+cu124
- Accelerate: 1.8.1
- Datasets: 3.6.0
- Tokenizers: 0.22.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->