The Trump administration has been getting absolutely pummeled for its refusal to acknowledge climate change, even as it becomes more obvious in our surroundings.On Monday, CNN s Jim Acosta tore Trump s Homeland Security advisor, Tom Bossert, apart in front of everyone for the administration s ignorance. Acknowledging the fact that several massive hurricanes have been ravaging the United States, Acosta cornered Bossert, asking him if Trump and his administration  saw a connection between climate change and homeland security and that the frequency and intensity of powerful storms like Harvey and Irma can pose a problem for future administrations  and if the storms were  giving the administration some pause when it comes to the issue of climate change and homeland security. Bossert was almost beside himself as he said: Causality s something outside of my ability to analyze right now. We continue to take seriously the climate change, not the cause of it, but the things that we observe. Then, Acosta nailed him and the entire Trump administration for their stupidity and ignorance: When you see three category 4 hurricanes all on the same map at the same time, does the thought occur to you,  Jeez, you know, maybe there is something to this climate change thing and its connection to powerful hurricanes?  Or do you just separate the two and say,  Boy, these are a lot of hurricanes coming our way? Bossert, nearly crumbling in front of the entire room of reporters, responded: There is a cyclical nature to the hurricane seasons and I thank the scientists for their forecast on this particular one. They were dead on that this would be a stronger and more powerful hurricane season with slightly more than average large storms making landfall  So we ll have to do a larger trend analysis at a later date. Trump s administration is being challenged by the obvious evidence for climate change, and now they re being called out for it. No one can ignore what is currently happening in our environment, and the enormous impact it will have if Trump s administration doesn t wake up. You can watch Acosta destroy Bossert below:Featured image via video screen capture


NAIROBI (Reuters) - Kenyan police shot dead two people and wounded a third when a crowd tried to storm a police station during an opposition protest against a looming presidential vote re-run, a senior official said.  Around 20 youths attempted to take over the station during an otherwise peaceful rally in the town of Bondo in the southwest county of Siaya, forcing officers to open fire, County Commissioner Josephine Onunga said by phone.  Police also used teargas to break up small demonstrations in Kenya s three main cities - Kisumu, the capital and the port of Mombasa - defying a government ban on rallies in city centers. Hospital authorities said 20 people were injured in Kisumu, an opposition stronghold.  Kenya s supreme court nullified the presidential election in August, citing procedural irregularities and voiding the victory of President Uhuru Kenyatta. The ruling opening the way to a repeat vote on Oct. 26.  But opposition leader Raila Odinga has refused to take part, saying the re-run should not happen until wide-ranging reforms are brought in to prevent another failed vote. The election board has said the polls will go ahead anyway, pitting Kenyatta against six other candidates, none of whom polled more than 1 percent in August.  Odinga told Reuters that he might consider returning to the Supreme Court for clarification on whether the Oct. 26 poll was legal.   As far as we are concerned, that (original Supreme court) ruling is still valid,  Odinga said during a visit to London.  What we are demanding is that the electoral commission should respect the Supreme Court and carry out elections in accordance with the ruling.  He supported protesters  right to demonstrate, he said, but has so far stopped short of calling people onto the streets himself.  In Nairobi, opposition legislators were repeatedly teargassed as they tried to drive into the city center.  We are asking Uhuru s government to ... let us protest in peace and truth and we are saying we want change and we do not want Uhuru and his people,  said protester Mellen Kirumbo.  In Kisumu, protesters reacted angrily when police turned water cannon on them to prevent them from entering the city center.   Our demonstrations have a (valid) basis and are peaceful,  said Odinga supporter Hezron Tirus Aloyo.  We condemn the directive ... on the limitation of our rights to demonstrate.   The Jaramogi Oginga Odinga hospital in Kisumu, named after Odinga s father, said it had admitted 20 patients injured in the demonstrations. Three had gunshot wounds, said superintendent Juliana Otieno.  Martin Kimani, the Director of the Kenyan government s National Counter Terrorism Centre, said the authorities were trying to maintain law and order.   A lot of those provocations or demonstrations taking place are an attempt to create the appearance of a crisis,  he said.


PC is killing our country. This is called fighting back While counterculture crackpots are working themselves into a lather over the prospect that, in some places, men who think that they re women can go tinkle in the same restroom where a 6-year-old girl is pulling down her panties, one state legislator has decided that it might be time to push back.Indiana State Sen. Jim Tomes, who probably accepts basic biology and believes that people born with a penis are male, proposed a bill that would make it a Class A misdemeanor for trannies to use a bathroom that doesn t correspond to their birth gender.Last week, Tomes uttered the following line that s sure to ignite tempers among the social justice mob:  If you were born a man, then you are obliged to use the males  restroom. What s next? A hereditary monarchy?Anyone convicted of a Class A misdemeanor could face a year in prison and as much as a $5,000 fine.It should be noted, though, that the language in the legislation makes exceptions for janitors, first aid providers, and parents accompanying children under the age of 8.The bill does, however, cover schools. So if it passes and Little Boy Twinkletoes decides that he needs to use the girls  locker room after gym class, then he s facing charges.Indiana saw some action in the culture war last year when the state passed a religious freedom law. That law was widely opposed by American liberals, who hate religious freedom.Via: DownTrend


Paul Ryan just gave another one of his notoriously lukewarm endorsements of Donald Trump. Intent on walking the thin line between backing his party s nominee and dumping Trump, the speaker of the house has tried to set himself up to come out smelling like a rose regardless of the outcome of tomorrow s election.During a radio interview on Monday, Ryan conceded that, yes, Trump won the Republican nomination  fair and square.  However, when conservative host Charlie Sykes asked if the GOP was now Trump s party, Ryan made it as clear as he could that no, the Republican party does not belong to Donald Trump. It is no one person s party. Donald Trump won the primary fair and square,  Ryan said. As a party leader, as the highest elected official in the party, I have always felt a duty to the process, to democracy, to the primary voter who must be respected. And he won this fair and square,  Ryan added.  But no one person controls this party. This is a bottom-up, organic grassroots party based on conservative principles. What a ringing endorsement to give on the eve of the election. (Insert sarcasm here.)Ryan was slow to endorse Trump and has repeatedly condemned the bombastic billionaire s offensive actions. Ryan denounced the Muslim ban as unconstitutional and declared Trump s attacks on the Mexican-American judge presiding over a Trump University lawsuit as the  textbook definition  of racism. But Ryan just doesn t have the cajones to stand up and walk away from Trump.Sykes asked the speaker about the long list of women who have come forward to accuse Trump of sexually assaulting them after the Access Hollywood tape was leaked, in which Trump bragged about committing sexual assault. Suddenly, Ryan leaped back onto the Trump Train. What do you think helps [Sen.] Ron Johnson and Mike Gallagher and all our candidates across the country more in the closing two days of an election: having party discord and having party leaders snub each other, or unifying the Republican Party and focusing and prosecuting our case against Hillary Clinton?  Ryan responded. What helps Republicans more: infighting or unifying, focusing on Clinton or focusing on our differences as Republicans?  the Speaker continued.  I don t want to harm our team going into the election. I want to unify our team going into the election so as many of our candidates as possible can win this election. Either Ryan can t make up his mind or he is trying to play both sides. Ummm, weasel much?Featured image via Alex Wong/Getty Images and Mark Wilson/Getty Images


If you weren t convinced of the fabulousness of First Lady Melania Trump before her 9-day foreign trip, you re certainly aware now. Not only did she dress impeccably but it was obvious that this wise woman put so much thought into what she would wear on this overseas trip. Who wasn t proud to see our beautiful and smart First Lady in appropriate and stylish outfits every day. She won us over in a big way and made the designers who refused to dress her look like schmucks. She needs no one to hold her hand to put together a wardrobe If anything, First Lady Melania Trump proved to the naysayers that you can be beautiful AND smart Well done!Richard Johnson of the New York Post: Melania Trump is proving to the fashion designers who said they wouldn t help her that dressing well is the best revenge.The first lady s trip to Saudi Arabia, the Vatican, Belgium and Sicily   wearing mostly Dolce & Gabbana   was a fashion tour de force that has forced many naysayers in the rag trade to rethink.The list of designers who said they wouldn t dress Melania is long, led by Tom Ford, Marc Jacobs, Zac Posen, Christian Siriano and Sophie Theallet.But Melania has managed to look smashing in dozens of different ensembles, from her white Ralph Lauren jumpsuit on election night to the D&G 3-D floral coat she wore in Sicily on Friday.Defending the former model against critics who harped on the garment s $51,500 cost, the Washington Post s Robin Givhan opined,  Frankly, the floral coat is beautiful. Givhan gets that Melania Trump is one smart woman who coordinated her fashion according to the place she was visiting:Givhan reported:  Clothes can be deeply symbolic. And Trump s choice of Dolce & Gabbana   an Italian brand that has been deeply inspired by Sicilian culture   for a trip to Sicily makes sense. Gabbana also has been quite vocal and enthusiastic in his willingness to associate his brand with the first lady There is a softening, a melting,  stylist Phillip Bloch told me.  Fashion people are fickle and fake. They are starting to see she is a beautiful woman who is married to the president, and it is an honor to dress her. Deliveries to Trump Tower have picked up.  Most days, the lobby is brimming with wardrobe boxes delivered for Melania,  a source told me.  Once she tries on the outfits and decides what she will keep, the boxes come back downstairs. Not since Jacqueline Kennedy has there been a first lady who needs less help. She doesn t need couture. She can buy off the rack, and it looks beautiful,  Bloch said.  She knows her size, and she knows what works on her. She luxuriates in minimal. Read more NYP


If there were a contest to describe who the most grotesque, misogynistic and outright awful human beings on the planet are, Donald Trump and those who he surrounds himself with would likely top the list. Not only is Trump wildly sexist and racist, but he comes to the defense of those who abuse others in his name. He has said it at his campaign rallies for those who go after protesters, and now he s come out in full defense of his campaign manager, Corey Lewandowski, who was caught on video brutally grabbing a reporter.Lewandowski, while at a campaign event on March 8th at Trump National Golf Club, grabbed then Breitbart News Network reporter Michelle Fields. She filed charges, and subsequently Lewandowski has now turned himself into the police.You can see here, via security footage, that Lewandowski, without a doubt, grabbed Fields:Newly released security footage of the Corey Lewandowski incident: https://t.co/O5tTflBb5C  Frank Luntz (@FrankLuntz) March 29, 2016Yet, Trump is still jumping to the defense of his abusive campaign manager. Shortly after Lewandowski turned himself in, Trump tweeted out:Wow, Corey Lewandowski, my campaign manager and a very decent man, was just charged with assaulting a reporter. Look at tapes-nothing there!  Donald J. Trump (@realDonaldTrump) March 29, 2016And then he blames the victim like the good misogynist he is:Why aren't people looking at this reporters earliest statement as to what happened, that is before she found out the episode was on tape?  Donald J. Trump (@realDonaldTrump) March 29, 2016THEN he tries to make up an excuse as to why she was being grabbed, turning it back on her:Victory press conference was over. Why is she allowed to grab me and shout questions? Can I press charges? pic.twitter.com/qbW2RjkINX  Donald J. Trump (@realDonaldTrump) March 29, 2016Why is this reporter touching me as I leave news conference? What is in her hand?? pic.twitter.com/HQB8dl0fhn  Donald J. Trump (@realDonaldTrump) March 29, 2016His tweets are literally going from bad to worse.The right thing to do would be to acknowledge the abuse and swiftly fire Lewandowski, but instead he s choosing to victim-shame and back up his brownshirt flunky.It s beyond belief that people still support Donald Trump. Either they are racist and/or misogynistic themselves, hate themselves, or are simply too stupid and likely cannot even tie their own shoe laces. We need to make sure Trump stays as far away from the Oval Office as possible. No matter who, we need to vote blue. Don t let a vote of pure conscience and ego let this country slip into the hands of a man who will most definitely have his hands on nuclear codes. Wake up, people. This race is bigger than ourselves and even our nation. It affects the whole world.Featured Photo by Joe Raedle/Getty Images   Twitter


SWABI, Pakistan (Reuters) - Three police officers stand daily guard at the tomb of Pakistani student Mashal Khan to prevent religious hardliners from fulfilling threats to blow up the grave of the 23-year-old beaten to death over rumors he blasphemed against Islam. His grieving family, now also under police protection, say they have little hope the shocking campus killing will prompt a re-examination of blasphemy laws that carry a death penalty, or action against the mob justice that often erupts in such cases.  On Friday, there was more evidence the opposite is happening.  A new political party that has made punishing blasphemers its main rallying cry won a surprisingly strong 7.6 percent of the vote in a by-election in Peshawar, 60 km (36 miles) from where Mashal Khan was killed six months ago.  Death to blasphemers! Death to blasphemers!  was a common chant of supporters of the Tehrik-e-Labaik Pakistan party at its campaign rallies in the conservative northwestern city.  The party s relatively strong showing - and a separate outcry over a proposed change to an election law that outraged the religious right - has elevated blasphemy into a potent political issue in the run-up to a general election in 2018. While Tehrik-e-Labaik (Movement of the Prophet s Followers) is unlikely to break out of single digits in coming votes, its rapid rise, along with another ultra-religious party, could create an additional challenge for the ruling Pakistani Muslim League-Nawaz (PML-N).  The PML-N party s leader, Nawaz Sharif, was ousted as prime minister in July by the Supreme Court, and opposition leader Imran Khan - who spearheaded the legal case that removed him over unreported income - is seeking to press the advantage. In this week s Peshawar by-election, former cricket star Imran s Pakistan Tehreek-e-Insaf (PTI) party swept to a comfortable victory to retain the parliamentary seat, winning 34.8 percent of the vote. Sharif s PML-N had 18.9 percent, narrowly coming in third to the regionally strong Awami National Party that won just 40 more votes. But the gains by the Labaik party - formed just last year -  have grabbed attention. Labaik draws most of its support from the Barelvi branch of Sunni Islam, the largest sect in Pakistan that is traditionally considered moderate. Though the party does not publicly talk about its funding, the Barelvis have a network of mosques and madrassa religious schools that collect donations. The party emerged out of a protest movement against the state s execution of Mumtaz Qadri, a bodyguard of the governor of Punjab province who gunned down his boss in 2011 over his call to reform Pakistan s blasphemy laws, among the world s harshest, to prevent abuses. Qadri is considered a hero by the party, and its candidate in Peshawar, Muhammad Shafiq Ameeni, was equally supportive of Mashal Khan s killers, although the student s death was not a main feature at campaign rallies.  It was state s responsibility to punish a blasphemer, no two opinions, but when state doesn t do its job and someone does kill, he shouldn t be punished as a murderer,  Amini said, referring to the 57 people who face trial over Mashal Khan s death. In Pakistan, allegiance to Islam is the official line of most major parties, but ultra-religious parties have so far remained on the fringes.  Labaik is one of two new ultra-religious parties formed in roughly the past year. Together, Labaik and the Milli Muslim League (MML) gained about 11 percent of the vote in last month s by-election in Lahore and 10.4 percent in Peshawar, whereas the established religious parties, such as Jamaat-e-Islami and Jamaat Ulema-e-Islam, combined had 5.3 percent in the 2013 national election. Blasphemy is such an effective wedge issue in Pakistan because there is almost no defense against an accusation. For that reason, say critics, blasphemy laws are often invoked to settle personal scores and to intimidate liberal journalists, lawyers and politicians. Dozens of Pakistanis are sitting on death row after being convicted of insulting Islam s prophet, a specific charge that carries a mandatory death sentence, though no executions have been carried out in recent decades. Now, political parties may be in danger of facing blasphemy accusations themselves.  Earlier in October, the PML-N found itself in the middle of a firestorm when it voted through seemingly small changes to the nation s electoral law. The changes, among other things, turned a religious oath in the electoral laws stating that Mohammad was the last prophet of Muslims into a declaration using the words  I declare .  The alterations prompted accusations of blasphemy from the religious right and the government quickly retreated, terming the change a  clerical  mistake and apologizing in parliament. Labaik has vowed to hold a mass rally on Nov. 6 to demand the lawmakers responsible be prosecuted for blasphemy. Even before the Labaik party s political debut, politicians found promising swift action against blasphemers an easy way to appeal to conservative voters. In March, then-prime minister Sharif issued a public order to prosecute anyone posting blasphemous content online. The next month, Mashal Khan was accused of online blasphemy and beaten to death by fellow students and religious activists  as onlookers filmed the scene. Sharif said he was  shocked and saddened  by the  senseless display of mob justice . At least 67 people have been killed over unproven blasphemy allegations since 1990, according human rights groups. Mashal Khan s father, Iqbal, said his son was the victim of false rumors. The family has received death threats from right-wingers and Mashal s sisters had to drop out of school.  The snakes our country nurtured are now biting us,  the father said, two days before the Peshawar by-election, standing beside his son s gave strewn with flowers, lace and poetry.  Learning of the Labaik party s gains a few days later only made him more pessimistic about the government s ability to stop abuse of blasphemy accusations.  I know very well, I m not going to get my son back,  he said.  But this only adds to my pain.


If any other American company did this, Donald Trump would be attacking them on Twitter.But Trump has a double standard when it comes to his own business and the businesses owned by his family members.As you may recall, Trump has openly attacked Ford, General Motors, and a slew of other companies for building factories overseas instead of building them here and hiring American workers to do the jobs.Trump even attacked a Japanese automaker for building a factory in Mexico instead of in the United States.The hypocrisy here is that while Trump is attacking these companies for not hiring Americans, his own son is begging the Labor Department to let him hire foreign workers instead of giving jobs to Americans.Eric Trump owns Trump Vineyards, and for the second time he is asking the Labor Department to let him hire foreign workers.According to the petition filed, the workers would be paid over $11 an hour to perform manual labor in cold weather. Workers would also be exposed to chemicals and would have to lift up to 60 pounds of grapes onto a truck.Here s a screenshot of the specific section.Again, this is hypocritical. Donald Trump is constantly promising Americans that he will make sure they have jobs but his own son wants to hire foreign workers.This is yet another example of Trump not following the rules he s setting for everyone else.A real leader leads by example and if Trump refuses to tell his own son to hire American workers why should companies like Ford comply with his demands?Keep in mind that Trump is ordering raids to round up undocumented immigrants because Trump claims they take jobs away from Americans. The reality, however, is that these immigrants take jobs that most Americans do not want to perform, such as managing and harvesting crops. That doesn t mean there aren t American out there who wouldn t take them, and that means Eric Trump is taking jobs away from Americans by trying to hire foreign workers instead. That s shameful, especially since his dad promised to put America first.Featured image via Joe Raedle/Getty Images


Question: Who is at risk for infection with SARS-CoV-2, the virus that causes COVID-19?

Answer: Currently, those at greatest risk of infection are persons who have had prolonged, unprotected close contact (i.e., within 6 feet for 15 minutes or longer) with a patient with confirmed SARS-CoV-2 infection, regardless of whether the patient has symptoms. Persons frequently in congregate settings (e.g., homeless shelters, assisted living facilities, college or university dormitories) are at increased risk of acquiring infection because of the increased likelihood of close contact. Those who live in or have recently been to areas with sustained transmission may also be at higher risk of infection. All persons can reduce the risk to themselves and others by wearing a mask, practicing physical distancing, washing their hands often, and taking other prevention measures. For more information, see Risk Assessment and Your Health.


Question: Who is at risk for severe COVID-19?

Answer: Among adults, the risk for severe illness from COVID-19 increases with age, with older adults at highest risk. Severe illness means that the person with COVID-19 may require hospitalization, intensive care, or a ventilator to help them breathe, or they may even die. People of any age with certain underlying medical conditions are also at increased risk for severe illness from SARS-CoV-2 infection.


Question: What is multisystem inflammatory syndrome in children (MIS-C) and who is at risk?

Answer: CDC continues to investigate multisystem inflammatory syndrome in children (MIS-C) associated with COVID-19. Children and adolescents with MIS-C have presented with a persistent fever and a variety of signs and symptoms including involvement of multiple organs (e.g., cardiac, gastrointestinal, renal, hematologic, dermatologic, neurologic) and elevated inflammatory markers. CDC is collaborating with domestic and international partners to better understand this new syndrome, including how common it is and its risk factors. For more information, including a full case definition and how to report MIS-C to your health department, visit MIS-C Information for Healthcare Providers.


Question: What should visitors use for source control (masks or respirators) when visiting healthcare facilities?

Answer: CDC recommends that people visiting healthcare facilities use the most protective form of source control (masks or respirators) that fits well and will be worn consistently. Healthcare facilities may choose to offer well-fitting facemasks as a source control option for visitors but should allow the use of a clean mask or respirator with higher level protection by people who chose that option based on their individual preference.  Masks and respirators used for source control should be changed if they become visibly soiled, damaged, or hard to breathe through. Learn more about the types of masks and respirators and infection control recommendations for healthcare personnel.


Question: Should facilities that perform elective medical pre-procedural testing of asymptomatic patients modify their testing based on a patient's report of a positive viral test for SARS-CoV-2 in the prior 90 days?

Answer: In general, viral testing is not necessary for asymptomatic patients who have recovered from SARS-CoV-2 infection in the prior 90 days.  However, if a facility is performing medical pre-procedural viral testing and the patient cannot provide documentation of their prior positive viral test, the safest approach is for the facility to retest using their current protocol.  In this situation, antigen testing is preferred. As viral tests for SARS-CoV-2 can remain positive for extended periods, even among patients who are no longer infectious, this approach could result in unnecessary isolation of the patient or delay of an elective medical procedure. Facilities may choose to accept self-test results to determine when to proceed with elective medical procedures. Such an approach assumes that the patient self-administered the viral test properly and interpreted the viral test results correctly.  If there is uncertainty about the validity of reported self-test results, facilities might attempt to acquire documentation of the test results or elect to proceed with their own pre-procedure viral testing.


Question: How should isolation and quarantine recommendations be applied to outpatients and visitors in healthcare settings?

Answer: Healthcare facilities should generally refer to the isolation and quarantine recommendations for patients when making decisions about visitation, non-urgent in-person appointments, and the recommended precautions that should be implemented to accommodate visits. Outpatients and visitors who are infected with SARS-CoV-2. It is safest to defer non-urgent, in-person care and visitation until the outpatient/visitor has met the same criteria to discontinue Transmission-based Precautions that are applied to patients in a healthcare setting. If visitation or an in-person outpatient visit is allowed for a person who has not met criteria to discontinue Transmission-based Precautions, then facilities should follow healthcare infection prevention and control recommendations for people with suspected or confirmed SARS-CoV-2 infection. Outpatients and visitors who have had close-contact with someone with SARS-CoV-2 infection and are not up to date with all recommended COVID-19 vaccine doses. It is safest to defer non-urgent, in-person care and visitation until the outpatient/visitor has met the same criteria to discontinue empiric Transmission-Based Precautions (quarantine) that are applied to patients in a healthcare setting. Although CDC does provide an option for inpatients who remain asymptomatic to discontinue their quarantine after 7 days, provided they have a negative test within 48 hours, outpatients and visitors should consider postponing their non-urgent, in-person appointment or visit until 10 days have passed since their close contact. If visitation or an in-person outpatient visit is allowed for a person who has not met these criteria, then facilities should ensure visitors use well-fitting source control and (to the extent possible) physically distance from others while in the facility and that healthcare personnel care for patients using precautions recommended for patients with suspected or confirmed SARS-CoV-2 infection. Outpatients and visitors who have had close-contact with someone with SARS-CoV-2 infection and are up to date with all recommended COVID-19 vaccine doses or who have recovered from SARS-CoV-2 infection in the prior 90 days. In general, CDC does not currently recommend quarantine following close contact with someone with SARS-CoV-2 infection if someone is up to date with all recommended COVID-19 vaccine doses or has recovered from SARS-CoV-2 infection in the prior 90 days. However, it is recommended that these people avoid contact with people who are at high risk for severe outcomes for the 10 days following their exposure. As it may not be possible to avoid contact with people at high risk for severe outcomes in a healthcare setting, it is safest to defer non-urgent in-person appointments and visitation until the outpatient/visitor has met criteria to discontinue quarantine in a healthcare setting as described above. If visitation or an in-person outpatient visit is allowed for a person who has not met these criteria, then facilities should ensure these individuals wear well-fitting source control and (to the extent possible) physically distance from others while in the facility.


Question: Can self-tests (also known as over-the-counter or home tests) be used when determining when healthcare personnel (HCP) with SARS-CoV-2 infection or higher-risk exposures may return to work?

Answer: Viral testing is recommended following a higher-risk exposure and might also be used to inform when HCP with SARS-CoV-2 infection may return to work. Using laboratory-based or point-of-care tests are generally preferred in these situations; these tests help ensure the test was administered correctly by a trained provider and allow for official results to be shared with occupational health services. Use of self-tests might be considered in some situations. However, such an approach may inappropriately transfer the cost of testing to HCP, Requires trust that HCP self-administered and interpreted the test results correctly, and Requires that HCP report their own results to occupational health services. If self-tests are used, consideration should be given to performing a second test at least 24 hours following an initial negative test; if negative, this increases the confidence the HCP are not infectious. Facilities could also consider having HCP present for a proctored test to assist with ensuring appropriate collection and interpretation.  CDC has resources to assist individuals with self-testing.


A bicycle replica with a clock as the front wheel.


The bike has a clock as a tire.


A black metal bicycle with a clock inside the front wheel.


A bicycle figurine in which the front wheel is replaced with a clock


A clock with the appearance of the wheel of a bicycle


What is the clock saying the time is? 10:10.


Is it possible to ride the bicycle? No.


What is the front wheel of the bike? Clock.


What color do you think the house has? White.


Is there a bike to the left of the vehicle in the top of the image? No.


---
language: en
tags: deberta-v1
thumbnail: https://huggingface.co/front/thumbnails/microsoft.png
license: mit
---

## DeBERTa: Decoding-enhanced BERT with Disentangled Attention

[DeBERTa](https://arxiv.org/abs/2006.03654) improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. It outperforms BERT and RoBERTa on  majority of NLU tasks with 80GB training data.

Please check the [official repository](https://github.com/microsoft/DeBERTa) for more details and updates.

#### Fine-tuning on NLU tasks

We present the dev results on SQuAD 1.1/2.0 and MNLI tasks.

| Model             | SQuAD 1.1 | SQuAD 2.0 | MNLI-m |
|-------------------|-----------|-----------|--------|
| RoBERTa-base      | 91.5/84.6 | 83.7/80.5 | 87.6   |
| XLNet-Large       | -/-       | -/80.2    | 86.8   |
| **DeBERTa-base**  | 93.1/87.2 | 86.2/83.1 | 88.8   |

### Citation

If you find DeBERTa useful for your work, please cite the following paper:

``` latex
@inproceedings{
he2021deberta,
title={DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION},
author={Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=XPZIaotutsD}
}
```


---
language: en
tags:
- exbert
license: apache-2.0
datasets:
- bookcorpus
- wikipedia
---

# BERT base model (uncased)

Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in
[this paper](https://arxiv.org/abs/1810.04805) and first released in
[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference
between english and English.

Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by
the Hugging Face team.

## Model description

BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it
was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of
publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it
was pretrained with two objectives:

- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run
  the entire masked sentence through the model and has to predict the masked words. This is different from traditional
  recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like
  GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the
  sentence.
- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes
  they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to
  predict if the two sentences were following each other or not.

This way, the model learns an inner representation of the English language that can then be used to extract features
useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard
classifier using the features produced by the BERT model as inputs.

## Intended uses & limitations

You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to
be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for
fine-tuned versions on a task that interests you.

Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)
to make decisions, such as sequence classification, token classification or question answering. For tasks such as text
generation you should look at model like GPT2.

### How to use

You can use this model directly with a pipeline for masked language modeling:

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-base-uncased')
>>> unmasker("Hello I'm a [MASK] model.")

[{'sequence': "[CLS] hello i'm a fashion model. [SEP]",
  'score': 0.1073106899857521,
  'token': 4827,
  'token_str': 'fashion'},
 {'sequence': "[CLS] hello i'm a role model. [SEP]",
  'score': 0.08774490654468536,
  'token': 2535,
  'token_str': 'role'},
 {'sequence': "[CLS] hello i'm a new model. [SEP]",
  'score': 0.05338378623127937,
  'token': 2047,
  'token_str': 'new'},
 {'sequence': "[CLS] hello i'm a super model. [SEP]",
  'score': 0.04667217284440994,
  'token': 3565,
  'token_str': 'super'},
 {'sequence': "[CLS] hello i'm a fine model. [SEP]",
  'score': 0.027095865458250046,
  'token': 2986,
  'token_str': 'fine'}]
```

Here is how to use this model to get the features of a given text in PyTorch:

```python
from transformers import BertTokenizer, BertModel
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained("bert-base-uncased")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```

and in TensorFlow:

```python
from transformers import BertTokenizer, TFBertModel
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = TFBertModel.from_pretrained("bert-base-uncased")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='tf')
output = model(encoded_input)
```

### Limitations and bias

Even if the training data used for this model could be characterized as fairly neutral, this model can have biased
predictions:

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-base-uncased')
>>> unmasker("The man worked as a [MASK].")

[{'sequence': '[CLS] the man worked as a carpenter. [SEP]',
  'score': 0.09747550636529922,
  'token': 10533,
  'token_str': 'carpenter'},
 {'sequence': '[CLS] the man worked as a waiter. [SEP]',
  'score': 0.0523831807076931,
  'token': 15610,
  'token_str': 'waiter'},
 {'sequence': '[CLS] the man worked as a barber. [SEP]',
  'score': 0.04962705448269844,
  'token': 13362,
  'token_str': 'barber'},
 {'sequence': '[CLS] the man worked as a mechanic. [SEP]',
  'score': 0.03788609802722931,
  'token': 15893,
  'token_str': 'mechanic'},
 {'sequence': '[CLS] the man worked as a salesman. [SEP]',
  'score': 0.037680890411138535,
  'token': 18968,
  'token_str': 'salesman'}]

>>> unmasker("The woman worked as a [MASK].")

[{'sequence': '[CLS] the woman worked as a nurse. [SEP]',
  'score': 0.21981462836265564,
  'token': 6821,
  'token_str': 'nurse'},
 {'sequence': '[CLS] the woman worked as a waitress. [SEP]',
  'score': 0.1597415804862976,
  'token': 13877,
  'token_str': 'waitress'},
 {'sequence': '[CLS] the woman worked as a maid. [SEP]',
  'score': 0.1154729500412941,
  'token': 10850,
  'token_str': 'maid'},
 {'sequence': '[CLS] the woman worked as a prostitute. [SEP]',
  'score': 0.037968918681144714,
  'token': 19215,
  'token_str': 'prostitute'},
 {'sequence': '[CLS] the woman worked as a cook. [SEP]',
  'score': 0.03042375110089779,
  'token': 5660,
  'token_str': 'cook'}]
```

This bias will also affect all fine-tuned versions of this model.

## Training data

The BERT model was pretrained on [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038
unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and
headers).

## Training procedure

### Preprocessing

The texts are lowercased and tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are
then of the form:

```
[CLS] Sentence A [SEP] Sentence B [SEP]
```

With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in
the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a
consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two
"sentences" has a combined length of less than 512 tokens.

The details of the masking procedure for each sentence are the following:
- 15% of the tokens are masked.
- In 80% of the cases, the masked tokens are replaced by `[MASK]`.
- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace.
- In the 10% remaining cases, the masked tokens are left as is.

### Pretraining

The model was trained on 4 cloud TPUs in Pod configuration (16 TPU chips total) for one million steps with a batch size
of 256. The sequence length was limited to 128 tokens for 90% of the steps and 512 for the remaining 10%. The optimizer
used is Adam with a learning rate of 1e-4, \\(\beta_{1} = 0.9\\) and \\(\beta_{2} = 0.999\\), a weight decay of 0.01,
learning rate warmup for 10,000 steps and linear decay of the learning rate after.

## Evaluation results

When fine-tuned on downstream tasks, this model achieves the following results:

Glue test results:

| Task | MNLI-(m/mm) | QQP  | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE  | Average |
|:----:|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|:-------:|
|      | 84.6/83.4   | 71.2 | 90.5 | 93.5  | 52.1 | 85.8  | 88.9 | 66.4 | 79.6    |

### BibTeX entry and citation info

```bibtex
@article{DBLP:journals/corr/abs-1810-04805,
  author    = {Jacob Devlin and
               Ming{-}Wei Chang and
               Kenton Lee and
               Kristina Toutanova},
  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
               Understanding},
  journal   = {CoRR},
  volume    = {abs/1810.04805},
  year      = {2018},
  url       = {http://arxiv.org/abs/1810.04805},
  archivePrefix = {arXiv},
  eprint    = {1810.04805},
  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


---
language: en
tags:
- exbert
license: apache-2.0
datasets:
- bookcorpus
- wikipedia
---

# DistilBERT base model (uncased)

This model is a distilled version of the [BERT base model](https://huggingface.co/bert-base-uncased). It was
introduced in [this paper](https://arxiv.org/abs/1910.01108). The code for the distillation process can be found
[here](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation). This model is uncased: it does
not make a difference between english and English.

## Model description

DistilBERT is a transformers model, smaller and faster than BERT, which was pretrained on the same corpus in a
self-supervised fashion, using the BERT base model as a teacher. This means it was pretrained on the raw texts only,
with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic
process to generate inputs and labels from those texts using the BERT base model. More precisely, it was pretrained
with three objectives:

- Distillation loss: the model was trained to return the same probabilities as the BERT base model.
- Masked language modeling (MLM): this is part of the original training loss of the BERT base model. When taking a
  sentence, the model randomly masks 15% of the words in the input then run the entire masked sentence through the
  model and has to predict the masked words. This is different from traditional recurrent neural networks (RNNs) that
  usually see the words one after the other, or from autoregressive models like GPT which internally mask the future
  tokens. It allows the model to learn a bidirectional representation of the sentence.
- Cosine embedding loss: the model was also trained to generate hidden states as close as possible as the BERT base
  model.

This way, the model learns the same inner representation of the English language than its teacher model, while being
faster for inference or downstream tasks.

## Intended uses & limitations

You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to
be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=distilbert) to look for
fine-tuned versions on a task that interests you.

Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)
to make decisions, such as sequence classification, token classification or question answering. For tasks such as text
generation you should look at model like GPT2.

### How to use

You can use this model directly with a pipeline for masked language modeling:

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='distilbert-base-uncased')
>>> unmasker("Hello I'm a [MASK] model.")

[{'sequence': "[CLS] hello i'm a role model. [SEP]",
  'score': 0.05292855575680733,
  'token': 2535,
  'token_str': 'role'},
 {'sequence': "[CLS] hello i'm a fashion model. [SEP]",
  'score': 0.03968575969338417,
  'token': 4827,
  'token_str': 'fashion'},
 {'sequence': "[CLS] hello i'm a business model. [SEP]",
  'score': 0.034743521362543106,
  'token': 2449,
  'token_str': 'business'},
 {'sequence': "[CLS] hello i'm a model model. [SEP]",
  'score': 0.03462274372577667,
  'token': 2944,
  'token_str': 'model'},
 {'sequence': "[CLS] hello i'm a modeling model. [SEP]",
  'score': 0.018145186826586723,
  'token': 11643,
  'token_str': 'modeling'}]
```

Here is how to use this model to get the features of a given text in PyTorch:

```python
from transformers import DistilBertTokenizer, DistilBertModel
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertModel.from_pretrained("distilbert-base-uncased")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```

and in TensorFlow:

```python
from transformers import DistilBertTokenizer, TFDistilBertModel
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = TFDistilBertModel.from_pretrained("distilbert-base-uncased")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='tf')
output = model(encoded_input)
```

### Limitations and bias

Even if the training data used for this model could be characterized as fairly neutral, this model can have biased
predictions. It also inherits some of
[the bias of its teacher model](https://huggingface.co/bert-base-uncased#limitations-and-bias).

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='distilbert-base-uncased')
>>> unmasker("The White man worked as a [MASK].")

[{'sequence': '[CLS] the white man worked as a blacksmith. [SEP]',
  'score': 0.1235365942120552,
  'token': 20987,
  'token_str': 'blacksmith'},
 {'sequence': '[CLS] the white man worked as a carpenter. [SEP]',
  'score': 0.10142576694488525,
  'token': 10533,
  'token_str': 'carpenter'},
 {'sequence': '[CLS] the white man worked as a farmer. [SEP]',
  'score': 0.04985016956925392,
  'token': 7500,
  'token_str': 'farmer'},
 {'sequence': '[CLS] the white man worked as a miner. [SEP]',
  'score': 0.03932540491223335,
  'token': 18594,
  'token_str': 'miner'},
 {'sequence': '[CLS] the white man worked as a butcher. [SEP]',
  'score': 0.03351764753460884,
  'token': 14998,
  'token_str': 'butcher'}]

>>> unmasker("The Black woman worked as a [MASK].")

[{'sequence': '[CLS] the black woman worked as a waitress. [SEP]',
  'score': 0.13283951580524445,
  'token': 13877,
  'token_str': 'waitress'},
 {'sequence': '[CLS] the black woman worked as a nurse. [SEP]',
  'score': 0.12586183845996857,
  'token': 6821,
  'token_str': 'nurse'},
 {'sequence': '[CLS] the black woman worked as a maid. [SEP]',
  'score': 0.11708822101354599,
  'token': 10850,
  'token_str': 'maid'},
 {'sequence': '[CLS] the black woman worked as a prostitute. [SEP]',
  'score': 0.11499975621700287,
  'token': 19215,
  'token_str': 'prostitute'},
 {'sequence': '[CLS] the black woman worked as a housekeeper. [SEP]',
  'score': 0.04722772538661957,
  'token': 22583,
  'token_str': 'housekeeper'}]
```

This bias will also affect all fine-tuned versions of this model.

## Training data

DistilBERT pretrained on the same data as BERT, which is [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset
consisting of 11,038 unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia)
(excluding lists, tables and headers).

## Training procedure

### Preprocessing

The texts are lowercased and tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are
then of the form:

```
[CLS] Sentence A [SEP] Sentence B [SEP]
```

With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in
the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a
consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two
"sentences" has a combined length of less than 512 tokens.

The details of the masking procedure for each sentence are the following:
- 15% of the tokens are masked.
- In 80% of the cases, the masked tokens are replaced by `[MASK]`.
- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace.
- In the 10% remaining cases, the masked tokens are left as is.

### Pretraining

The model was trained on 8 16 GB V100 for 90 hours. See the
[training code](https://github.com/huggingface/transformers/tree/master/examples/distillation) for all hyperparameters
details.

## Evaluation results

When fine-tuned on downstream tasks, this model achieves the following results:

Glue test results:

| Task | MNLI | QQP  | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE  |
|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|
|      | 82.2 | 88.5 | 89.2 | 91.3  | 51.3 | 85.8  | 87.5 | 59.9 |

### BibTeX entry and citation info

```bibtex
@article{Sanh2019DistilBERTAD,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},
  journal={ArXiv},
  year={2019},
  volume={abs/1910.01108}
}
```


---
language: en
tags:
- exbert
license: apache-2.0
datasets:
- bookcorpus
- wikipedia
---

# BERT base model (cased)

Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in
[this paper](https://arxiv.org/abs/1810.04805) and first released in
[this repository](https://github.com/google-research/bert). This model is case-sensitive: it makes a difference between
english and English.

Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by
the Hugging Face team.

## Model description

BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it
was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of
publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it
was pretrained with two objectives:

- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run
  the entire masked sentence through the model and has to predict the masked words. This is different from traditional
  recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like
  GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the
  sentence.
- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes
  they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to
  predict if the two sentences were following each other or not.

This way, the model learns an inner representation of the English language that can then be used to extract features
useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard
classifier using the features produced by the BERT model as inputs.

## Intended uses & limitations

You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to
be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for
fine-tuned versions on a task that interests you.

Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)
to make decisions, such as sequence classification, token classification or question answering. For tasks such as text
generation you should look at model like GPT2.

### How to use

You can use this model directly with a pipeline for masked language modeling:

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-base-cased')
>>> unmasker("Hello I'm a [MASK] model.")

[{'sequence': "[CLS] Hello I'm a fashion model. [SEP]",
  'score': 0.09019174426794052,
  'token': 4633,
  'token_str': 'fashion'},
 {'sequence': "[CLS] Hello I'm a new model. [SEP]",
  'score': 0.06349995732307434,
  'token': 1207,
  'token_str': 'new'},
 {'sequence': "[CLS] Hello I'm a male model. [SEP]",
  'score': 0.06228214129805565,
  'token': 2581,
  'token_str': 'male'},
 {'sequence': "[CLS] Hello I'm a professional model. [SEP]",
  'score': 0.0441727414727211,
  'token': 1848,
  'token_str': 'professional'},
 {'sequence': "[CLS] Hello I'm a super model. [SEP]",
  'score': 0.03326151892542839,
  'token': 7688,
  'token_str': 'super'}]
```

Here is how to use this model to get the features of a given text in PyTorch:

```python
from transformers import BertTokenizer, BertModel
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
model = BertModel.from_pretrained("bert-base-cased")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```

and in TensorFlow:

```python
from transformers import BertTokenizer, TFBertModel
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
model = TFBertModel.from_pretrained("bert-base-cased")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='tf')
output = model(encoded_input)
```

### Limitations and bias

Even if the training data used for this model could be characterized as fairly neutral, this model can have biased
predictions:

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-base-cased')
>>> unmasker("The man worked as a [MASK].")

[{'sequence': '[CLS] The man worked as a lawyer. [SEP]',
  'score': 0.04804691672325134,
  'token': 4545,
  'token_str': 'lawyer'},
 {'sequence': '[CLS] The man worked as a waiter. [SEP]',
  'score': 0.037494491785764694,
  'token': 17989,
  'token_str': 'waiter'},
 {'sequence': '[CLS] The man worked as a cop. [SEP]',
  'score': 0.035512614995241165,
  'token': 9947,
  'token_str': 'cop'},
 {'sequence': '[CLS] The man worked as a detective. [SEP]',
  'score': 0.031271643936634064,
  'token': 9140,
  'token_str': 'detective'},
 {'sequence': '[CLS] The man worked as a doctor. [SEP]',
  'score': 0.027423162013292313,
  'token': 3995,
  'token_str': 'doctor'}]

>>> unmasker("The woman worked as a [MASK].")

[{'sequence': '[CLS] The woman worked as a nurse. [SEP]',
  'score': 0.16927455365657806,
  'token': 7439,
  'token_str': 'nurse'},
 {'sequence': '[CLS] The woman worked as a waitress. [SEP]',
  'score': 0.1501094549894333,
  'token': 15098,
  'token_str': 'waitress'},
 {'sequence': '[CLS] The woman worked as a maid. [SEP]',
  'score': 0.05600163713097572,
  'token': 13487,
  'token_str': 'maid'},
 {'sequence': '[CLS] The woman worked as a housekeeper. [SEP]',
  'score': 0.04838843643665314,
  'token': 26458,
  'token_str': 'housekeeper'},
 {'sequence': '[CLS] The woman worked as a cook. [SEP]',
  'score': 0.029980547726154327,
  'token': 9834,
  'token_str': 'cook'}]
```

This bias will also affect all fine-tuned versions of this model.

## Training data

The BERT model was pretrained on [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038
unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and
headers).

## Training procedure

### Preprocessing

The texts are tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are then of the form:

```
[CLS] Sentence A [SEP] Sentence B [SEP]
```

With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in
the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a
consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two
"sentences" has a combined length of less than 512 tokens.

The details of the masking procedure for each sentence are the following:
- 15% of the tokens are masked.
- In 80% of the cases, the masked tokens are replaced by `[MASK]`.
- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace.
- In the 10% remaining cases, the masked tokens are left as is.

### Pretraining

The model was trained on 4 cloud TPUs in Pod configuration (16 TPU chips total) for one million steps with a batch size
of 256. The sequence length was limited to 128 tokens for 90% of the steps and 512 for the remaining 10%. The optimizer
used is Adam with a learning rate of 1e-4, \\(\beta_{1} = 0.9\\) and \\(\beta_{2} = 0.999\\), a weight decay of 0.01,
learning rate warmup for 10,000 steps and linear decay of the learning rate after.

## Evaluation results

When fine-tuned on downstream tasks, this model achieves the following results:

Glue test results:

| Task | MNLI-(m/mm) | QQP  | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE  | Average |
|:----:|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|:-------:|
|      | 84.6/83.4   | 71.2 | 90.5 | 93.5  | 52.1 | 85.8  | 88.9 | 66.4 | 79.6    |

### BibTeX entry and citation info

```bibtex
@article{DBLP:journals/corr/abs-1810-04805,
  author    = {Jacob Devlin and
               Ming{-}Wei Chang and
               Kenton Lee and
               Kristina Toutanova},
  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
               Understanding},
  journal   = {CoRR},
  volume    = {abs/1810.04805},
  year      = {2018},
  url       = {http://arxiv.org/abs/1810.04805},
  archivePrefix = {arXiv},
  eprint    = {1810.04805},
  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


---
tags:
- exbert
language:
- multilingual
- af
- am
- ar
- as
- az
- be
- bg
- bn
- br
- bs
- ca
- cs
- cy
- da
- de
- el
- en
- eo
- es
- et
- eu
- fa
- fi
- fr
- fy
- ga
- gd
- gl
- gu
- ha
- he
- hi
- hr
- hu
- hy
- id
- is
- it
- ja
- jv
- ka
- kk
- km
- kn
- ko
- ku
- ky
- la
- lo
- lt
- lv
- mg
- mk
- ml
- mn
- mr
- ms
- my
- ne
- nl
- no
- om
- or
- pa
- pl
- ps
- pt
- ro
- ru
- sa
- sd
- si
- sk
- sl
- so
- sq
- sr
- su
- sv
- sw
- ta
- te
- th
- tl
- tr
- ug
- uk
- ur
- uz
- vi
- xh
- yi
- zh
license: mit
---

# XLM-RoBERTa (base-sized model)

XLM-RoBERTa model pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. It was introduced in the paper [Unsupervised Cross-lingual Representation Learning at Scale](https://arxiv.org/abs/1911.02116) by Conneau et al. and first released in [this repository](https://github.com/pytorch/fairseq/tree/master/examples/xlmr).

Disclaimer: The team releasing XLM-RoBERTa did not write a model card for this model so this model card has been written by the Hugging Face team.

## Model description

XLM-RoBERTa is a multilingual version of RoBERTa. It is pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages.

RoBERTa is a transformers model pretrained on a large corpus in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts.

More precisely, it was pretrained with the Masked language modeling (MLM) objective. Taking a sentence, the model randomly masks 15% of the words in the input then run the entire masked sentence through the model and has to predict the masked words. This is different from traditional recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the sentence.

This way, the model learns an inner representation of 100 languages that can then be used to extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard classifier using the features produced by the XLM-RoBERTa model as inputs.

## Intended uses & limitations

You can use the raw model for masked language modeling, but it's mostly intended to be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?search=xlm-roberta) to look for fine-tuned versions on a task that interests you.

Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at models like GPT2.

## Usage

You can use this model directly with a pipeline for masked language modeling:

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='xlm-roberta-base')
>>> unmasker("Hello I'm a <mask> model.")

[{'score': 0.10563907772302628,
  'sequence': "Hello I'm a fashion model.",
  'token': 54543,
  'token_str': 'fashion'},
 {'score': 0.08015287667512894,
  'sequence': "Hello I'm a new model.",
  'token': 3525,
  'token_str': 'new'},
 {'score': 0.033413201570510864,
  'sequence': "Hello I'm a model model.",
  'token': 3299,
  'token_str': 'model'},
 {'score': 0.030217764899134636,
  'sequence': "Hello I'm a French model.",
  'token': 92265,
  'token_str': 'French'},
 {'score': 0.026436051353812218,
  'sequence': "Hello I'm a sexy model.",
  'token': 17473,
  'token_str': 'sexy'}]
```

Here is how to use this model to get the features of a given text in PyTorch:

```python
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained('xlm-roberta-base')
model = AutoModelForMaskedLM.from_pretrained("xlm-roberta-base")

# prepare input
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')

# forward pass
output = model(**encoded_input)
```

### BibTeX entry and citation info

```bibtex
@article{DBLP:journals/corr/abs-1911-02116,
  author    = {Alexis Conneau and
               Kartikay Khandelwal and
               Naman Goyal and
               Vishrav Chaudhary and
               Guillaume Wenzek and
               Francisco Guzm{\'{a}}n and
               Edouard Grave and
               Myle Ott and
               Luke Zettlemoyer and
               Veselin Stoyanov},
  title     = {Unsupervised Cross-lingual Representation Learning at Scale},
  journal   = {CoRR},
  volume    = {abs/1911.02116},
  year      = {2019},
  url       = {http://arxiv.org/abs/1911.02116},
  eprinttype = {arXiv},
  eprint    = {1911.02116},
  timestamp = {Mon, 11 Nov 2019 18:38:09 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1911-02116.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


Appellant, a male, was convicted of a crime by a petit jury selected from a venire on which there were no women and which was selected pursuant to a system resulting from Louisiana constitutional and statutory requirements that a woman should not be selected for jury service unless she had previously filed a written declaration of her desire to be subject to jury service. The State Supreme Court affirmed, having rejected appellant's challenge to the constitutionality of the state jury-selection scheme. Held: 1. Appellant had standing to make his constitutional claim, there being no rule that such a claim may be asserted only by defendants who are members of the group excluded from jury service. Peters v. Kiff, . P. 526. 2. The requirement that a petit jury be selected from a representative cross section of the community, which is fundamental to the jury trial guaranteed by the Sixth Amendment, is violated by the systematic exclusion of women from jury panels, which in the judicial district here involved amounted to 53% of the citizens eligible for jury service. Pp. 526-533. 3. No adequate justification was shown here for the challenged jury-selection provisions and the right to a jury selected from a fair cross section of the community cannot be overcome on merely rational grounds. Pp. 533-535. 4. It can no longer be held that women as a class may be excluded from jury service or given automatic exemptions based solely on sex if the consequence is that criminal jury venires are almost all male, and contrary implications of prior cases, e. g., Hoyt v. Florida, , cannot be followed. Pp. 535-537. 282 So.2d 491, reversed and remanded.WHITE, J., delivered the opinion of the Court, in which DOUGLAS, BRENNAN, STEWART, MARSHALL, BLACKMUN, and POWELL, JJ., joined. BURGER, C. J., concurred in the result. REHNQUIST, J., filed a dissenting opinion, post, p. 538.William McM. King argued the cause and filed a brief for appellant. Kendall L. Vick, Assistant Attorney General of Louisiana, argued the cause for appellee. On the brief were William J. Guste, Jr., Attorney General, Walter Smith, and Woodrow W. Erwin.MR. JUSTICE WHITE delivered the opinion of the Court.When this case was tried, Art. VII, 41,1 of the Louisiana Constitution, and Art. 402 of the Louisiana Code of Criminal Procedure2 provided that a woman should not be selected for jury service unless she had previously filed a written declaration of her desire to be subject to jury service. The constitutionality of these provisions is the issue in this case. IAppellant, Billy J. Taylor, was indicted by the grand jury of St. Tammany Parish, in the Twenty-second Judicial District of Louisiana, for aggravated kidnaping. On April 12, 1972, appellant moved the trial court to quash the petit jury venire drawn for the special criminal term beginning with his trial the following day. Appellant alleged that women were systematically excluded from the venire and that he would therefore be deprived of what he claimed to be his federal constitutional right to "a fair trial by jury of a representative segment of the community ... ."The Twenty-second Judicial District comprises the parishes of St. Tammany and Washington. The appellee has stipulated that 53% of the persons eligible for jury service in these parishes were female, and that no more than 10% of the persons on the jury wheel in St. Tammany Parish were women.3 During the period from December 8, 1971, to November 3, 1972, 12 females were among the 1,800 persons drawn to fill petit jury venires in St. Tammany Parish. It was also stipulated that the discrepancy between females eligible for jury service and those actually included in the venire was the result of the operation of La. Const., Art. VII, 41, and La. Code Crim. Proc., Art. 402.4 In the present case, a venire totaling 175 persons was drawn for jury service beginning April 13, 1972. There were no females on the venire.Appellant's motion to quash the venire was denied that same day. After being tried, convicted, and sentenced to death, appellant sought review in the Supreme Court of Louisiana, where he renewed his claim that the petit jury venire should have been quashed. The Supreme Court of Louisiana, recognizing that this claim drew into question the constitutionality of the provisions of the Louisiana Constitution and Code of Criminal Procedure dealing with the service of women on juries, squarely held, one justice dissenting, that these provisions were valid and not unconstitutional under federal law. 282 So.2d 491, 497 (1973).5 Appellant appealed from that decision to this Court. We noted probable jurisdiction, , to consider whether the Louisiana jury-selection system deprived appellant of his Sixth and Fourteenth Amendment right to an impartial jury trial. We hold that it did and that these Amendments were violated in this case by the operation of La. Const., Art. VII, 41, and La. Code Crim. Proc., Art. 402. In consequence, appellant's conviction must be reversed.IIThe Louisiana jury-selection system does not disqualify women from jury service, but in operation its conceded systematic impact is that only a very few women, grossly disproportionate to the number of eligible women in the community, are called for jury service. In this case, no women were on the venire from which the petit jury was drawn. The issue we have, therefore, is whether a jury-selection system which operates to exclude from jury service an identifiable class of citizens constituting 53% of eligible jurors in the community comports with the Sixth and Fourteenth Amendments.The State first insists that Taylor, a male, has no standing to object to the exclusion of women from his jury. But Taylor's claim is that he was constitutionally entitled to a jury drawn from a venire constituting a fair cross section of the community and that the jury that tried him was not such a jury by reason of the exclusion of women. Taylor was not a member of the excluded class; but there is no rule that claims such as Taylor presents may be made only by those defendants who are members of the group excluded from jury service. In Peters v. Kiff, , the defendant, a white man, challenged his conviction on the ground that Negroes had been systematically excluded from jury service. Six Members of the Court agreed that petitioner was entitled to present the issue and concluded that he had been deprived of his federal rights. Taylor, in the case before us, was similarly entitled to tender and have adjudicated the claim that the exclusion of women from jury service deprived him of the kind of factfinder to which he was constitutionally entitled.IIIThe background against which this case must be decided includes our holding in Duncan v. Louisiana, , that the Sixth Amendment's provision for jury trial is made binding on the States by virtue of the Fourteenth Amendment. Our inquiry is whether the presence of a fair cross section of the community on venires, panels, or lists from which petit juries are drawn is essential to the fulfillment of the Sixth Amendment's guarantee of an impartial jury trial in criminal prosecutions.The Court's prior cases are instructive. Both in the course of exercising its supervisory powers over trials in federal courts and in the constitutional context, the Court has unambiguously declared that the American concept of the jury trial contemplates a jury drawn from a fair cross section of the community. A unanimous Court stated in Smith v. Texas, , that "[i]t is part of the established tradition in the use of juries as instruments of public justice that the jury be a body truly representative of the community." To exclude racial groups from jury service was said to be "at war with our basic concepts of a democratic society and a representative government." A state jury system that resulted in systematic exclusion of Negroes as jurors was therefore held to violate the Equal Protection Clause of the Fourteenth Amendment. Glasser v. United States, , in the context of a federal criminal case and the Sixth Amendment's jury trial requirement, stated that "[o]ur notions of what a proper jury is have developed in harmony with our basic concepts of a democratic society and a representative government," and repeated the Court's understanding that the jury "`be a body truly representative of the community' ... and not the organ of any special group or class."A federal conviction by a jury from which women had been excluded, although eligible for service under state law, was reviewed in Ballard v. United States, . Noting the federal statutory "design to make the jury `a cross-section of the community'" and the fact that women had been excluded, the Court exercised its supervisory powers over the federal courts and reversed the conviction. In Brown v. Allen, , the Court declared that "[o]ur duty to protect the federal constitutional rights of all does not mean we must or should impose on states our conception of the proper source of jury lists, so long as the source reasonably reflects a cross-section of the population suitable in character and intelligence for that civic duty."Some years later in Carter v. Jury Comm'n, , the Court observed that the exclusion of Negroes from jury service because of their race "contravenes the very idea of a jury - `a body truly representative of the community'... ." (Quoting from Smith v. Texas, supra.) At about the same time it was contended that the use of six-man juries in noncapital criminal cases violated the Sixth Amendment for failure to provide juries drawn from a cross section of the community, Williams v. Florida, . In the course of rejecting that challenge, we said that the number of persons on the jury should "be large enough to promote group deliberation, free from outside attempts at intimidation, and to provide a fair possibility for obtaining a representative cross-section of the community." Id., at 100. In like vein, in Apodaca v. Oregon, (plurality opinion), it was said that "a jury will come to such a [commonsense] judgment as long as it consists of a group of laymen representative of a cross section of the community who have the duty and the opportunity to deliberate ... on the question of a defendant's guilt." Similarly, three Justices in Peters v. Kiff, 407 U.S., at 500, observed that the Sixth Amendment comprehended a fair possibility for obtaining a jury constituting a representative cross section of the community.The unmistakable import of this Court's opinions, at least since 1940, Smith v. Texas, supra, and not repudiated by intervening decisions, is that the selection of a petit jury from a representative cross section of the community is an essential component of the Sixth Amendment right to a jury trial. Recent federal legislation governing jury selection within the federal court system has a similar thrust. Shortly prior to this Court's decision in Duncan v. Louisiana, supra, the Federal Jury Selection and Service Act of 19686 was enacted. In that Act, Congress stated "the policy of the United States that all litigants in Federal courts entitled to trial by jury shall have the right to grand and petit juries selected at random from a fair cross section of the community in the district or division wherein the court convenes." 28 U.S.C. 1861. In that Act, Congress also established the machinery by which the stated policy was to be implemented. 28 U.S.C. 1862-1866. In passing this legislation, the Committee Reports of both the House7 and the Senate8 recognized that the jury plays a political function in the administration of the law and that the requirement of a jury's being chosen from a fair cross section of the community is fundamental to the American system of justice. Debate on the floors of the House and Senate on the Act invoked the Sixth Amendment,9 the Constitution generally,10 and prior decisions of this Court11 in support of the Act.We accept the fair-cross-section requirement as fundamental to the jury trial guaranteed by the Sixth Amendment and are convinced that the requirement has solid foundation. The purpose of a jury is to guard against the exercise of arbitrary power - to make available the commonsense judgment of the community as a hedge against the overzealous or mistaken prosecutor and in preference to the professional or perhaps over-conditioned or biased response of a judge. Duncan v. Louisiana, 391 U.S., at 155-156. This prophylactic vehicle is not provided if the jury pool is made up of only special segments of the populace or if large, distinctive groups are excluded from the pool. Community participation in the administration of the criminal law, moreover, is not only consistent with our democratic heritage but is also critical to public confidence in the fairness of the criminal justice system. Restricting jury service to only special groups or excluding identifiable segments playing major roles in the community cannot be squared with the constitutional concept of jury trial. "Trial by jury presupposes a jury drawn from a pool broadly representative of the community as well as impartial in a specific case... . [T]he broad representative character of the jury should be maintained, partly as assurance of a diffused impartiality and partly because sharing in the administration of justice is a phase of civic responsibility." Thiel v. Southern Pacific Co., (Frankfurter, J., dissenting).IVWe are also persuaded that the fair-cross-section requirement is violated by the systematic exclusion of women, who in the judicial district involved here amounted to 53% of the citizens eligible for jury service. This conclusion necessarily entails the judgment that women are sufficiently numerous and distinct from men and that if they are systematically eliminated from jury panels, the Sixth Amendment's fair-cross-section requirement cannot be satisfied. This very matter was debated in Ballard v. United States, supra. Positing the fair-cross-section rule - there said to be a statutory one - the Court concluded that the systematic exclusion of women was unacceptable. The dissenting view that an all-male panel drawn from various groups in the community would be as truly representative as if women were included, was firmly rejected: "The thought is that the factors which tend to influence the action of women are the same as those which influence the action of men - personality, background, economic status - and not sex. Yet it is not enough to say that women when sitting as jurors neither act nor tend to act as a class. Men likewise do not act as a class. But, if the shoe were on the other foot, who would claim that a jury was truly representative of the community if all men were intentionally and systematically excluded from the panel? The truth is that the two sexes are not fungible; a community made up exclusively of one is different from a community composed of both; the subtle interplay of influence one on the other is among the imponderables. To insulate the court-room from either may not in a given case make an iota of difference. Yet a flavor, a distinct quality is lost if either sex is excluded. The exclusion of one may indeed make the jury less representative of the community than would be true if an economic or racial group were excluded." 329 U.S., at 193-194.12 In this respect, we agree with the Court in Ballard: If the fair-cross-section rule is to govern the selection of juries, as we have concluded it must, women cannot be systematically excluded from jury panels from which petit juries are drawn. This conclusion is consistent with the current judgment of the country, now evidenced by legislative or constitutional provisions in every State and at the federal level qualifying women for jury service.13 VThere remains the argument that women as a class serve a distinctive role in society and that jury service would so substantially interfere with that function that the State has ample justification for excluding women from service unless they volunteer, even though the result is that almost all jurors are men. It is true that Hoyt v. Florida, , held that such a system14 did not deny due process of law or equal protection of the laws because there was a sufficiently rational basis for such an exemption.15 But Hoyt did not involve a defendant's Sixth Amendment right to a jury drawn from a fair cross section of the community and the prospect of depriving him of that right if women as a class are systematically excluded. The right to a proper jury cannot be overcome on merely rational grounds.16 There must be weightier reasons if a distinctive class representing 53% of the eligible jurors is for all practical purposes to be excluded from jury service. No such basis has been tendered here.The States are free to grant exemptions from jury service to individuals in case of special hardship or incapacity and to those engaged in particular occupations the uninterrupted performance of which is critical to the community's welfare. Rawlins v. Georgia, . It would not appear that such exemptions would pose substantial threats that the remaining pool of jurors would not be representative of the community. A system excluding all women, however, is a wholly different matter. It is untenable to suggest these days that it would be a special hardship for each and every woman to perform jury service or that society cannot spare any women from their present duties.17 This may be the case with many, and it may be burdensome to sort out those who should be exempted from those who should serve. But that task is performed in the case of men, and the administrative convenience in dealing with women as a class is insufficient justification for diluting the quality of community judgment represented by the jury in criminal trials.VIAlthough this judgment may appear a foregone conclusion from the pattern of some of the Court's cases over the past 30 years, as well as from legislative developments at both federal and state levels, it is nevertheless true that until today no case had squarely held that the exclusion of women from jury venires deprives a criminal defendant of his Sixth Amendment right to trial by an impartial jury drawn from a fair cross section of the community. It is apparent that the first Congress did not perceive the Sixth Amendment as requiring women on criminal jury panels; for the direction of the First Judiciary Act of 1789 was that federal jurors were to have the qualifications required by the States in which the federal court was sitting18 and at the time women were disqualified under state law in every State. Necessarily, then, federal juries in criminal cases were all male, and it was not until the Civil Rights Act of 1957, 71 Stat. 638, 28 U.S.C. 1861 (1964 ed.), that Congress itself provided that all citizens, with limited exceptions, were competent to sit on federal juries. Until that time, federal courts were required by statute to exclude women from jury duty in those States where women were disqualified. Utah was the first State to qualify women for juries; it did so in 1898, n. 13, supra. Moreover, Hoyt v. Florida was decided and has stood for the proposition that, even if women as a group could not be constitutionally disqualified from jury service, there was ample reason to treat all women differently from men for the purpose of jury service and to exclude them unless they volunteered.19 Accepting as we do, however, the view that the Sixth Amendment affords the defendant in a criminal trial the opportunity to have the jury drawn from venires representative of the community, we think it is no longer tenable to hold that women as a class may be excluded or given automatic exemptions based solely on sex if the consequence is that criminal jury venires are almost totally male. To this extent we cannot follow the contrary implications of the prior cases, including Hoyt v. Florida. If it was ever the case that women were unqualified to sit on juries or were so situated that none of them should be required to perform jury service, that time has long since passed. If at one time it could be held that Sixth Amendment juries must be drawn from a fair cross section of the community but that this requirement permitted the almost total exclusion of women, this is not the case today. Communities differ at different times and places. What is a fair cross section at one time or place is not necessarily a fair cross section at another time or a different place. Nothing persuasive has been presented to us in this case suggesting that all-male venires in the parishes involved here are fairly representative of the local population otherwise eligible for jury service.VIIOur holding does not augur or authorize the fashioning of detailed jury-selection codes by federal courts. The fair-cross-section principle must have much leeway in application. The States remain free to prescribe relevant qualifications for their jurors and to provide reasonable exemptions so long as it may be fairly said that the jury lists or panels are representative of the community. Carter v. Jury Comm'n, supra, as did Brown v. Allen, supra; Rawlins v. Georgia, supra, and other cases, recognized broad discretion in the States in this respect. We do not depart from the principles enunciated in Carter. But, as we have said, Louisiana's special exemption for women operates to exclude them from petit juries, which in our view is contrary to the command of the Sixth and Fourteenth Amendments.It should also be emphasized that in holding that petit juries must be drawn from a source fairly representative of the community we impose no requirement that petit juries actually chosen must mirror the community and reflect the various distinctive groups in the population. Defendants are not entitled to a jury of any particular composition, Fay v. New York, ; Apodaca v. Oregon, 406 U.S., at 413 (plurality opinion); but the jury wheels, pools of names, panels, or venires from which juries are drawn must not systematically exclude distinctive groups in the community and thereby fail to be reasonably representative thereof.The judgment of the Louisiana Supreme Court is reversed and the case remanded to that court for further proceedings not inconsistent with this opinion. So ordered.MR. CHIEF JUSTICE BURGER concurs in the result.


1. The Trading with the Enemy Act authorizes the vesting of obligations evidenced by negotiable debentures payable to bearer, the obligors of which are within the United States, even though the debentures themselves are not in the possession of the Custodian and are outside the United States. Pp. 331-334. (a) Such obligations are "within the United States" within the meaning of the Executive Order authorizing, pursuant to the Act, the vesting of property "within the United States." P. 332, n. 6; pp. 333-334.2. It is within the constitutional power of Congress to authorize the Custodian to seize an interest represented by a bond or debenture without seizing the instrument itself, where the obligor of the bond or debenture is within the United States. P. 334.3. American obligors who are compelled, under the Trading with the Enemy Act, to make payment to the Custodian on negotiable debentures, payable to bearer, located outside the United States will be entitled under the Fifth Amendment to "just compensation" to the extent of any double liability to which they may be subjected in the event a foreign court holds them liable to a holder in due course; and such cause of action will accrue when, as, and if a foreign court compels them to make payment to a holder in due course. Pp. 334-336. 189 F.2d 744, affirmed. In an action by the Attorney General, as successor to the Alien Property Custodian, to enforce payment of certain negotiable debentures payable to bearer previously vested under the Trading with the Enemy Act, the District Court entered judgment for the defendants. 93 F. Supp. 408. The Court of Appeals reversed. 189 F.2d 744. This Court granted certiorari. . Affirmed, p. 336. Timothy N. Pfeiffer argued the cause for petitioners and was on the brief for the Chase National Bank. With him also on the brief were Theodore N. Johnsen, for the Cities Service Company, and Rebecca M. Cutler, of counsel.George B. Searls argued the cause for respondent. With him on the brief were Solicitor General Perlman, Assistant Attorney General Baynton, James D. Hill and Irwin A. Seibel.MR. JUSTICE CLARK delivered the opinion of the Court.In this suit the Attorney General of the United States as successor to the Alien Property Custodian1 seeks payment by petitioners of two 5% gold debentures of the face value of $1,000 each and payable to bearer. Petitioner Cities Service Company is obligor on the debentures and petitioner Chase National Bank of New York is the indenture trustee. The obligations represented by these debentures had previously been vested, under provisions of the Trading with the Enemy Act,2 upon a finding that the obligations were owned by a resident and national of Germany.3 Neither of the debentures is or ever has been in the possession of respondent. One of the debentures, although not maturing until 1969, was presented for redemption at Chase's offices in New York City on January 5, 1950, subsequent to the date of the vesting order. A legend was then typed on the debenture reciting the issuance of the vesting order and the claims of respondent thereunder. This debenture is at present in the possession of a brokerage house in New York City.4 The other debenture matured in 1950 but has never been presented for payment. Its whereabouts are unknown but it was last reported to be in Berlin in the hands of the Russians.5 The District Court granted summary judgment for petitioners on the ground that the Attorney General, in issuing the vesting order in question, had exceeded his authority to vest property "within the United States."6 93 F. Supp. 408. The Court felt that the obligations represented by the debentures were inseparable from the certificates themselves, which, insofar as is known, were outside this country at the time of vesting. The Court of Appeals reversed and directed summary judgment for respondent, holding that the Act authorized the seizure and enforcement of obligations evidenced by debentures outside the country so long as the obligor is within the United States. 189 F.2d 744. In reaching this result, the Court of Appeals indicated that petitioners would have a "claim against the Treasury for recoupment" in the event of a subsequent recovery against them in a foreign court by a bona fide holder of the debentures. Otherwise, the Court felt, the vesting order would take petitioners' property in violation of the Fifth Amendment. Id., at 747-749. We granted certiorari, .We believe that the Trading with the Enemy Act grants the authority necessary to vest obligations evidenced by domestic negotiable bearer debentures even though the debentures themselves are outside the United States. By 7 (c) of the Act, enacted during World War I, the President is given the authority to seize all enemy property, "including ... choses in action, and rights and claims of every character and description owing or belonging to ... an enemy ... ." At the beginning of World War II, Congress made an even broader grant of authority to the Executive through an amendment to 5 (b), providing that "any property or interest of any foreign country or national thereof shall vest, when, as, and upon the terms, directed by the President ... ." See Markham v. Cabell, ; Silesian-American Corp. v. Clark, ; Clark v. Uebersee Finanz-Korp., . That the obligations represented by negotiable bearer debentures come within these broad terms is beyond question.Petitioners urge, however, that the debentures themselves constitute the debt, and since the debentures were located outside of the United States at the time of vesting, the debts did not have a situs within the United States and therefore were not proper subjects of seizure. To apply this fiction here would not only provide a sanctuary for enemy investments and defeat the recovery of American securities looted by conquering forces; it would also restrict the exercise of the war powers of the United States. Congress did not so intend. The Custodian's authority to reach a debenture or bonded indebtedness without seizure of the instrument itself is explicitly recognized by 9 (n) of the Act, which provides that "[i]n the case of property consisting ... of bonded or other indebtedness ..., evidenced ... by bonds or by other certificates of interest ... or indebtedness ..., where the right, title, and interest in the property (but not the actual ... bond or other certificate of interest or indebtedness) was conveyed, transferred, assigned, delivered, or paid to the Alien Property Custodian, or seized by him ...," then the President may, in proper cases, order return of 80% of the property.7 Moreover, in giving the Custodian this power to seize an interest represented by a bond or debenture without seizure of the actual instrument, Congress transgressed no constitutional limitations on its jurisdiction. As the Court of Appeals pointed out, the obligor, Cities Service Company, is within the United States and the obligation of which the debenture is evidence can be effectively dealt with through the exercise of jurisdiction over that petitioner. See Standard Oil Co. v. New Jersey, .A more serious question is whether application of the seizure provisions of the Act to petitioners will take their property in violation of the Fifth Amendment, unless they have a remedy against the United States in the event a foreign court holds them liable to a holder in due course of the debentures. While petitioners concede that the Act discharges them from liability in any court in the United States,8 they contend that they have extensive properties over the world which subject them to foreign suits from which the Act affords no certain protection. Petitioners readily admit that the court of the country in which suit is brought may apply the laws of the United States and recognize their prior payment to the Attorney General as a complete defense; and that the holder, if qualified, might file a claim under the Act. Nevertheless, they insist, there remains at least the possibility that they will be exposed to liability in a foreign court. While their defense to such litigation seems adequate and final payment by them improbable, we agree that petitioners might suffer judgment the payment of which would effect a double recovery against them. In that event, petitioners will have the right to recoup from the United States, for a "taking" of their property within the meaning of the Fifth Amendment, "just compensation" to the extent of their double liability.9 Such cause of action will accrue when, as, and if a foreign court forces petitioners to pay a holder in due course of the debentures. We agree with the Court of Appeals that only with this assurance against double liability can it fairly be said that the present seizure is not itself an unconstitutional taking of petitioners' property. Affirmed.