Downloading via Apache Pipeline, client cancelled (org.apache.beam.vendor.grpc.v1p43p2.io.grpc.StatusRuntimeException)

Something isn't working

Hi @dan-the-meme-man, thanks for reporting.

We are investigating a similar issue but with Beam+Dataflow (instead of Beam+Flink):
- #4525

In order to go deeper into the root cause, we need as much information as possible: logs from the main process + logs from the workers are very informative.

In the case of the issue with Beam+Dataflow, the logs from the workers report an out of memory issue.

## Describe the bug
When downloading some `wikipedia` languages (in particular, I'm having a hard time with Spanish, Cebuano, and Russian) via FlinkRunner, I encounter the exception in the title. I have been playing with package versions a lot, because unfortunately, the different dependencies required by these packages seem to be incompatible in terms of versions (dill and requests, for instance). It should be noted that the following code runs for several hours without issue, executing the `load_dataset()` function, before the exception occurs.

## Steps to reproduce the bug
```python
# bash commands
!pip install datasets
!pip install apache-beam[interactive]
!pip install mwparserfromhell
!pip install dill==0.3.5.1
!pip install requests==2.23.0

# imports
import os
from datasets import load_dataset
import apache_beam as beam
import mwparserfromhell
from google.colab import drive
import dill
import requests

# mount drive
drive_dir = os.path.join(os.getcwd(), 'drive')
drive.mount(drive_dir)

# confirming the versions of these two packages are the ones that are suggested by the outputs from the bash commands
print(dill.__version__)
print(requests.__version__)

lang = 'es' # or 'ru' or 'ceb' - these are the ones causing the issue
lang_dir = os.path.join(drive_dir, 'path/to/my/folder', lang)

if not os.path.exists(lang_dir):
  x = None
  x = load_dataset('wikipedia', '20220301.' + lang, beam_runner='Flink',
                   split='train')
  x.save_to_disk(lang_dir)
```

## Expected results
Although some warnings are generally produced by this code (run in Colab Notebook), most languages I've tried have been successfully downloaded. It should simply go through without issue, but for these languages, I am continually encountering this error.

## Actual results
Traceback below:
```
Exception in thread run_worker_3-1:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 234, in run
    for work_request in self._control_stub.Control(get_responses()):
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "{"created":"@1655593643.871830638","description":"Error received from peer ipv4:127.0.0.1:44441","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Socket closed","grpc_status":14}"
>

Traceback (most recent call last):
  File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 718, in apache_beam.runners.common.PerWindowInvoker.invoke_process
  File "apache_beam/runners/common.py", line 782, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 426, in __getitem__
    self._cache[target_window] = self._side_input_data.view_fn(raw_view)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 391, in <lambda>
    lambda iterable: from_runtime_iterable(iterable, view_options))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 512, in _from_runtime_iterable
    head = list(itertools.islice(it, 2))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1228, in _lazy_iterator
    self._underlying.get_raw(state_key, continuation_token))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1019, in get_raw
    continuation_token=continuation_token)))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1060, in _blocking_request
    raise RuntimeError(response.error)
RuntimeError: Unknown process bundle instruction id '26'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
    response = task()
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
    lambda: self.create_worker().do_instruction(request), request)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 581, in do_instruction
    getattr(request, request_type), request.instruction_id)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
    bundle_processor.process_bundle(instruction_id))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 996, in process_bundle
    element.data)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
    self.output(decoded_value)
  File "apache_beam/runners/worker/operations.py", line 346, in apache_beam.runners.worker.operations.Operation.output
  File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output
  File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
  File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process
  File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
  File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
  File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 718, in apache_beam.runners.common.PerWindowInvoker.invoke_process
  File "apache_beam/runners/common.py", line 782, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 426, in __getitem__
    self._cache[target_window] = self._side_input_data.view_fn(raw_view)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 391, in <lambda>
    lambda iterable: from_runtime_iterable(iterable, view_options))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 512, in _from_runtime_iterable
    head = list(itertools.islice(it, 2))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1228, in _lazy_iterator
    self._underlying.get_raw(state_key, continuation_token))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1019, in get_raw
    continuation_token=continuation_token)))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1060, in _blocking_request
    raise RuntimeError(response.error)
RuntimeError: Unknown process bundle instruction id '26' [while running 'train/Save to parquet/Write/WriteImpl/WriteBundles']

ERROR:apache_beam.runners.worker.sdk_worker:Error processing instruction 26. Original traceback is
Traceback (most recent call last):
  File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 718, in apache_beam.runners.common.PerWindowInvoker.invoke_process
  File "apache_beam/runners/common.py", line 782, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 426, in __getitem__
    self._cache[target_window] = self._side_input_data.view_fn(raw_view)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 391, in <lambda>
    lambda iterable: from_runtime_iterable(iterable, view_options))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 512, in _from_runtime_iterable
    head = list(itertools.islice(it, 2))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1228, in _lazy_iterator
    self._underlying.get_raw(state_key, continuation_token))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1019, in get_raw
    continuation_token=continuation_token)))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1060, in _blocking_request
    raise RuntimeError(response.error)
RuntimeError: Unknown process bundle instruction id '26'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 267, in _execute
    response = task()
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 340, in <lambda>
    lambda: self.create_worker().do_instruction(request), request)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 581, in do_instruction
    getattr(request, request_type), request.instruction_id)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 618, in process_bundle
    bundle_processor.process_bundle(instruction_id))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 996, in process_bundle
    element.data)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 221, in process_encoded
    self.output(decoded_value)
  File "apache_beam/runners/worker/operations.py", line 346, in apache_beam.runners.worker.operations.Operation.output
  File "apache_beam/runners/worker/operations.py", line 348, in apache_beam.runners.worker.operations.Operation.output
  File "apache_beam/runners/worker/operations.py", line 215, in apache_beam.runners.worker.operations.SingletonConsumerSet.receive
  File "apache_beam/runners/worker/operations.py", line 707, in apache_beam.runners.worker.operations.DoOperation.process
  File "apache_beam/runners/worker/operations.py", line 708, in apache_beam.runners.worker.operations.DoOperation.process
  File "apache_beam/runners/common.py", line 1200, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
  File "apache_beam/runners/common.py", line 1198, in apache_beam.runners.common.DoFnRunner.process
  File "apache_beam/runners/common.py", line 718, in apache_beam.runners.common.PerWindowInvoker.invoke_process
  File "apache_beam/runners/common.py", line 782, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/bundle_processor.py", line 426, in __getitem__
    self._cache[target_window] = self._side_input_data.view_fn(raw_view)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 391, in <lambda>
    lambda iterable: from_runtime_iterable(iterable, view_options))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/pvalue.py", line 512, in _from_runtime_iterable
    head = list(itertools.islice(it, 2))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1228, in _lazy_iterator
    self._underlying.get_raw(state_key, continuation_token))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1019, in get_raw
    continuation_token=continuation_token)))
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/sdk_worker.py", line 1060, in _blocking_request
    raise RuntimeError(response.error)
RuntimeError: Unknown process bundle instruction id '26' [while running 'train/Save to parquet/Write/WriteImpl/WriteBundles']

ERROR:root:org.apache.beam.vendor.grpc.v1p43p2.io.grpc.StatusRuntimeException: CANCELLED: client cancelled
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/data_plane.py", line 634, in _read_inputs
    for elements in elements_iterator:
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.CANCELLED
	details = "Multiplexer hanging up"
	debug_error_string = "{"created":"@1655593654.436885887","description":"Error received from peer ipv4:127.0.0.1:43263","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/data_plane.py", line 651, in <lambda>
    target=lambda: self._read_inputs(elements_iterator),
  File "/usr/local/lib/python3.7/dist-packages/apache_beam/runners/worker/data_plane.py", line 634, in _read_inputs
    for elements in elements_iterator:
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.CANCELLED
	details = "Multiplexer hanging up"
	debug_error_string = "{"created":"@1655593654.436885887","description":"Error received from peer ipv4:127.0.0.1:43263","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[/tmp/ipykernel_219/3869142325.py](https://localhost:8080/#) in <module>
     18   x = None
     19   x = load_dataset('wikipedia', '20220301.' + lang, beam_runner='Flink',
---> 20                    split='train')
     21   x.save_to_disk(lang_dir)

3 frames
[/usr/local/lib/python3.7/dist-packages/apache_beam/runners/portability/portable_runner.py](https://localhost:8080/#) in wait_until_finish(self, duration)
    604
    605     if self._runtime_exception:
--> 606       raise self._runtime_exception
    607
    608     return self._state

RuntimeError: Pipeline BeamApp-root-0618220708-b3b59a0e_d8efcf67-9119-4f76-b013-70de7b29b54d failed in state FAILED: org.apache.beam.vendor.grpc.v1p43p2.io.grpc.StatusRuntimeException: CANCELLED: client cancelled
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.2
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- PyArrow version: 6.0.1
- Pandas version: 1.3.5


Update download url and improve card of `cats_vs_dogs` dataset

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4523). All of your documentation changes will be reflected on that endpoint.

Improve the download URL (reported here: https://huggingface.co/datasets/cats_vs_dogs/discussions/1), remove the `image_file_path` column (not used in Transformers, so it should be safe) and add more info to the card.


Datasets method `.map` not hashing

bug

Something isn't working

Didn't realize it's a bug when I asked the question yesterday! Free free to post an answer if you are sure the cause has been addressed.

https://stackoverflow.com/questions/72664827/can-pickle-dill-foo-but-not-lambda-x-foox

## Describe the bug
Datasets method `.map` not hashing, even with an empty no-op function

## Steps to reproduce the bug
```python
from datasets import load_dataset

# download 9MB dummy dataset
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean")

def prepare_dataset(batch):
     return(batch)

ds = ds.map(
    prepare_dataset,
    num_proc=1,
    desc="preprocess train dataset",
)
```

## Expected results
Hashed and cached dataset preprocessing

## Actual results
Does not hash properly:
```
Parameter 'function'=<function prepare_dataset at 0x7fccb68e9280> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.3.dev0
- Platform: Linux-5.11.0-1028-gcp-x86_64-with-glibc2.31
- Python version: 3.9.12
- PyArrow version: 8.0.0
- Pandas version: 1.4.2

cc @lhoestq


Failure to  hash `dataclasses` - results in functions that cannot be hashed or cached in `.map`

Something isn't working

Dataclasses cannot be hashed. As a result, they cannot be hashed or cached if used in the `.map` method. Dataclasses are used extensively in Transformers examples scripts: (c.f. [CTC example](https://github.com/huggingface/transformers/blob/main/examples/pytorch/speech-recognition/run_speech_recognition_ctc.py)). Since dataclasses cannot be hashed, one has to define separate variables prior to passing dataclass attributes to the `.map` method:
```python
phoneme_language = data_args.phoneme_language
```
in the example https://github.com/huggingface/transformers/blob/3c7e56fbb11f401de2528c1dcf0e282febc031cd/examples/pytorch/speech-recognition/run_speech_recognition_ctc.py#L603-L630

## Steps to reproduce the bug
```python
from dataclasses import dataclass, field
from datasets.fingerprint import Hasher

@dataclass
class DataTrainingArguments:
    """
    Arguments pertaining to what data we are going to input our model for training and eval.
    """

    phoneme_language: str = field(
        default=None, metadata={"help": "The name of the phoneme language to use."}
    )

data_args = DataTrainingArguments(phoneme_language ="foo")

Hasher.hash(data_args)

phoneme_language = data_args.phoneme_language

Hasher.hash(phoneme_language)
```

## Expected results
A hash.
## Actual results
<details>
<summary> Traceback </summary>

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Input In [1], in <cell line: 16>()
     10     phoneme_language: str = field(
     11         default=None, metadata={"help": "The name of the phoneme language to use."}
     12     )
     14 data_args = DataTrainingArguments(phoneme_language ="foo")
---> 16 Hasher.hash(data_args)
     18 phoneme_language = data_args. phoneme_language
     20 Hasher.hash(phoneme_language)

File ~/datasets/src/datasets/fingerprint.py:237, in Hasher.hash(cls, value)
    235     return cls.dispatch[type(value)](cls, value)
    236 else:
--> 237     return cls.hash_default(value)

File ~/datasets/src/datasets/fingerprint.py:230, in Hasher.hash_default(cls, value)
    228 @classmethod
    229 def hash_default(cls, value: Any) -> str:
--> 230     return cls.hash_bytes(dumps(value))

File ~/datasets/src/datasets/utils/py_utils.py:564, in dumps(obj)
    562 file = StringIO()
    563 with _no_cache_fields(obj):
--> 564     dump(obj, file)
    565 return file.getvalue()

File ~/datasets/src/datasets/utils/py_utils.py:539, in dump(obj, file)
    537 def dump(obj, file):
    538     """pickle an object to a file"""
--> 539     Pickler(file, recurse=True).dump(obj)
    540     return

File ~/hf/lib/python3.8/site-packages/dill/_dill.py:620, in Pickler.dump(self, obj)
    618     raise PicklingError(msg)
    619 else:
--> 620     StockPickler.dump(self, obj)
    621 return

File /usr/lib/python3.8/pickle.py:487, in _Pickler.dump(self, obj)
    485 if self.proto >= 4:
    486     self.framer.start_framing()
--> 487 self.save(obj)
    488 self.write(STOP)
    489 self.framer.end_framing()

File /usr/lib/python3.8/pickle.py:603, in _Pickler.save(self, obj, save_persistent_id)
    599     raise PicklingError("Tuple returned by %s must have "
    600                         "two to six elements" % reduce)
    602 # Save the reduce() output and finally memoize the object
--> 603 self.save_reduce(obj=obj, *rv)

File /usr/lib/python3.8/pickle.py:687, in _Pickler.save_reduce(self, func, args, state, listitems, dictitems, state_setter, obj)
    684     raise PicklingError(
    685         "args[0] from __newobj__ args has the wrong class")
    686 args = args[1:]
--> 687 save(cls)
    688 save(args)
    689 write(NEWOBJ)

File /usr/lib/python3.8/pickle.py:560, in _Pickler.save(self, obj, save_persistent_id)
    558 f = self.dispatch.get(t)
    559 if f is not None:
--> 560     f(self, obj)  # Call unbound method with explicit self
    561     return
    563 # Check private dispatch table if any, or else
    564 # copyreg.dispatch_table

File ~/hf/lib/python3.8/site-packages/dill/_dill.py:1838, in save_type(pickler, obj, postproc_list)
   1836             postproc_list = []
   1837         postproc_list.append((setattr, (obj, '__qualname__', obj_name)))
-> 1838     _save_with_postproc(pickler, (_create_type, (
   1839         type(obj), obj.__name__, obj.__bases__, _dict
   1840     )), obj=obj, postproc_list=postproc_list)
   1841     log.info("# %s" % _t)
   1842 else:

File ~/hf/lib/python3.8/site-packages/dill/_dill.py:1140, in _save_with_postproc(pickler, reduction, is_pickler_dill, obj, postproc_list)
   1137     pickler._postproc[id(obj)] = postproc_list
   1139 # TODO: Use state_setter in Python 3.8 to allow for faster cPickle implementations
-> 1140 pickler.save_reduce(*reduction, obj=obj)
   1142 if is_pickler_dill:
   1143     # pickler.x -= 1
   1144     # print(pickler.x*' ', 'pop', obj, id(obj))
   1145     postproc = pickler._postproc.pop(id(obj))

File /usr/lib/python3.8/pickle.py:692, in _Pickler.save_reduce(self, func, args, state, listitems, dictitems, state_setter, obj)
    690 else:
    691     save(func)
--> 692     save(args)
    693     write(REDUCE)
    695 if obj is not None:
    696     # If the object is already in the memo, this means it is
    697     # recursive. In this case, throw away everything we put on the
    698     # stack, and fetch the object back from the memo.

File /usr/lib/python3.8/pickle.py:560, in _Pickler.save(self, obj, save_persistent_id)
    558 f = self.dispatch.get(t)
    559 if f is not None:
--> 560     f(self, obj)  # Call unbound method with explicit self
    561     return
    563 # Check private dispatch table if any, or else
    564 # copyreg.dispatch_table

File /usr/lib/python3.8/pickle.py:901, in _Pickler.save_tuple(self, obj)
    899 write(MARK)
    900 for element in obj:
--> 901     save(element)
    903 if id(obj) in memo:
    904     # Subtle.  d was not in memo when we entered save_tuple(), so
    905     # the process of saving the tuple's elements must have saved
   (...)
    909     # could have been done in the "for element" loop instead, but
    910     # recursive tuples are a rare thing.
    911     get = self.get(memo[id(obj)][0])

File /usr/lib/python3.8/pickle.py:560, in _Pickler.save(self, obj, save_persistent_id)
    558 f = self.dispatch.get(t)
    559 if f is not None:
--> 560     f(self, obj)  # Call unbound method with explicit self
    561     return
    563 # Check private dispatch table if any, or else
    564 # copyreg.dispatch_table

File ~/hf/lib/python3.8/site-packages/dill/_dill.py:1251, in save_module_dict(pickler, obj)
   1248     if is_dill(pickler, child=False) and pickler._session:
   1249         # we only care about session the first pass thru
   1250         pickler._first_pass = False
-> 1251     StockPickler.save_dict(pickler, obj)
   1252     log.info("# D2")
   1253 return

File /usr/lib/python3.8/pickle.py:971, in _Pickler.save_dict(self, obj)
    968     self.write(MARK + DICT)
    970 self.memoize(obj)
--> 971 self._batch_setitems(obj.items())

File /usr/lib/python3.8/pickle.py:997, in _Pickler._batch_setitems(self, items)
    995     for k, v in tmp:
    996         save(k)
--> 997         save(v)
    998     write(SETITEMS)
    999 elif n:

File /usr/lib/python3.8/pickle.py:560, in _Pickler.save(self, obj, save_persistent_id)
    558 f = self.dispatch.get(t)
    559 if f is not None:
--> 560     f(self, obj)  # Call unbound method with explicit self
    561     return
    563 # Check private dispatch table if any, or else
    564 # copyreg.dispatch_table

File ~/datasets/src/datasets/utils/py_utils.py:862, in save_function(pickler, obj)
    859     if state_dict:
    860         state = state, state_dict
--> 862     dill._dill._save_with_postproc(
    863         pickler,
    864         (
    865             dill._dill._create_function,
    866             (obj.__code__, globs, obj.__name__, obj.__defaults__, closure),
    867             state,
    868         ),
    869         obj=obj,
    870         postproc_list=postproc_list,
    871     )
    872 else:
    873     closure = obj.func_closure

File ~/hf/lib/python3.8/site-packages/dill/_dill.py:1153, in _save_with_postproc(pickler, reduction, is_pickler_dill, obj, postproc_list)
   1151 dest, source = reduction[1]
   1152 if source:
-> 1153     pickler.write(pickler.get(pickler.memo[id(dest)][0]))
   1154     pickler._batch_setitems(iter(source.items()))
   1155 else:
   1156     # Updating with an empty dictionary. Same as doing nothing.

KeyError: 140434581781568
```

</details>

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.3.dev0
- Platform: Linux-5.11.0-1028-gcp-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 8.0.0
- Pandas version: 1.4.2

cc @lhoestq


Create new sections for audio and vision in guides

documentation

Improvements or additions to documentation

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4519). All of your documentation changes will be reflected on that endpoint.

This PR creates separate sections in the guides for audio, vision, text, and general usage so it is easier for users to find loading, processing, or sharing guides specific to the dataset type they're working with. It'll also allow us to scale the docs to additional dataset types - like time series, tabular, etc. - while keeping our docs information architecture.

Some other changes include:

- Experimented with decorating text with some CSS to highlight guides specific to each modality. Hopefully, it'll be easier for users to find and realize that these different docs exist!
- Added deprecation warning for Metrics and redirect to Evaluate.
- Updated `set_format` section to recommend using the new `to_tf_dataset` function if you need to convert to a TensorFlow dataset.
- Reorganized `toctree` to nest general usage, audio, vision, and text sections under the how-to guides.
- A quick review and edit to the Load and Process docs for clarity.


Patch tests for hfh v0.8.0

_The documentation is not available anymore as the PR was closed or merged._

This PR patches testing utilities that would otherwise fail with hfh v0.8.0.


choice1: The bully picked up his classmate. choice2: The bully's classmate tripped. premise: The bully stuck his foot out in front of his classmate. question: effect


choice1: The door slammed shut. choice2: The woman thanked me. premise: I held the door for the woman entering behind me. question: effect


choice1: It was pasteurized. choice2: I stored it in the refrigerator. premise: The milk stayed cold. question: cause


choice1: The spy bugged the enemy's phone. choice2: The spy was hired by the government. premise: The spy discovered the enemy's location. question: cause


choice1: He wanted to shut off the alarm clock. choice2: He wanted to iron his pants before work. premise: The man urgently leaped out of bed. question: cause


choice1: The patient's symptoms cleared up. choice2: The patient's family visited him. premise: The hospital sent the patient home. question: cause


choice1: He caught a whiff of her perfume. choice2: He had his back turned to her. premise: The man took notice of the woman. question: cause


choice1: She received a gold medal. choice2: Her team lost the meet. premise: The swimmer won the race. question: effect


choice1: Snowflakes began to fall from the sky. choice2: The moon became visible in the sky. premise: It got dark outside. question: effect


choice1: He checked its price tag. choice2: He decided to buy it. premise: The shopper wondered about the cost of the item. question: effect


SECTION 1. MODIFICATIONS OF REPORTING REQUIREMENTS FOR CERTAIN STATE
              AND LOCAL POLITICAL ORGANIZATIONS.

    (a) Notification.--Paragraph (5) of section 527(i) of the Internal
Revenue Code of 1986 (relating to organizations must notify Secretary
that they are section 527 organizations) is amended by striking ``or''
at the end of subparagraph (A), by striking the period at the end of
subparagraph (B) and inserting ``, or'', and by adding at the end the
following:
                    ``(C) which is a political committee of a State or
                local candidate, or a local committee of a political
                party, as defined by State law.''.
    (b) Exemption for Certain State and Local Political Committees From
Reporting Requirements.--
            (1) In general.--Paragraph (5) of section 527(j) of such
        Code (relating to required disclosures of expenditures and
        contributions) is amended by redesignating subparagraphs (C),
        (D), and (E) as subparagraphs (D), (E), and (F), respectively,
        and by inserting after subparagraph (B) the following new
        subparagraph:
                    ``(C) to any organization which is an exempt State
                or local political organization,''.
            (2) Exempt state or local political organization.--
        Subsection (e) of section 527 of such Code (relating to other
        definitions) is amended by adding at the end the following new
        paragraph:
            ``(5) Exempt state or local political organization.--
                    ``(A) In general.--The term `exempt State or local
                political organization' means a political
                organization--
                            ``(i) which does not engage in any exempt
                        function other than to influence or to attempt
                        to influence the selection, nomination,
                        election, or appointment of any individual to
                        any State or local public office or office in a
                        State or local political organization,
                            ``(ii) which is subject to State or local
                        requirements to submit reports containing
                        information--
                                    ``(I) regarding individual
                                expenditures from and contributions to
                                such organization, and
                                    ``(II) regarding the person who
                                makes such contributions or receives
                                such expenditures,
                        which is substantially similar to the
                        information which would otherwise be required
                        to be reported under this section, and
                            ``(iii) with respect to which the reports
                        referred to in clause (ii) are made public by
                        the agency with which such reports are filed
                        and are publicly available for inspection in a
                        manner similar to that required by section
                        6104(d)(1).
                    ``(B) Participation of federal candidate or office
                holder.--The term `exempt State or local political
                organization' shall not include any organization
                otherwise described in subparagraph (A) if a candidate
                for nomination or election to Federal elective office
                or an individual who holds such office--
                            ``(i) controls or materially participates
                        in the direction of the organization, or
                            ``(ii) directs, in whole or in part,
                        expenditures or fundraising activities of the
                        organization.''.
    (c) Annual Return Requirements.--
            (1) Income tax returns required only where political
        organization taxable income.--Paragraph (6) of section 6012(a)
        of such Code (relating to general rule of persons required to
        make returns of income) is amended by striking ``or which has
        gross receipts of $25,000 or more for the taxable year (other
        than an organization to which section 527 applies solely by
        reason of subsection (f)(1) of such section)''.
            (2) Information returns.--Subsection (g) of section 6033 of
        such Code (relating to returns required by political
        organizations) is amended to read as follows:
    ``(g) Returns Required by Political Organizations.--
            ``(1) In general.--Every political organization (within the
        meaning of section 527(e)(1)), and every fund treated under
        section 527(g) as if it constituted a political organization,
        which has gross receipts of $25,000 or more for the taxable
        year shall file a return--
                    ``(A) containing the information required, and
                complying with the other requirements, under subsection
                (a)(1) for organizations exempt from taxation under
                section 501(a), and
                    ``(B) containing such other information as the
                Secretary deems necessary to carry out the provisions
                of this subsection.
            ``(2) Exceptions from filing.--
                    ``(A) Mandatory exceptions.--Paragraph (1) shall
                not apply to an organization--
                            ``(i) which is an exempt State or local
                        political organization (as defined in section
                        527(e)(5)),
                            ``(ii) which is a State or local committee
                        of a political party, or political committee of
                        a State or local candidate, as defined by State
                        law,
                            ``(iii) which is a caucus or association of
                        State or local elected officials,
                            ``(iv) which is a national association of
                        State or local officials,
                            ``(v) which is an authorized committee (as
                        defined in section 301(6) of the Federal
                        Election Campaign Act of 1971) of a candidate
                        for Federal office,
                            ``(vi) which is a national committee (as
                        defined in section 301(14) of the Federal
                        Election Campaign Act of 1971) of a political
                        party, or
                            ``(vii) to which section 527 applies for
                        the taxable year solely by reason of subsection
                        (f)(1) of such section.
                    ``(B) Discretionary exception.--The Secretary may
                relieve any organization required under paragraph (1)
                to file an information return from filing such a return
                where he determines that such filing is not necessary
                to the efficient administration of the internal revenue
                laws.''.
    (d) Waiver of Penalties.--Section 527 of such Code is amended by
adding at the end the following:
    ``(k) Authority To Waive.--The Secretary may waive all or any
portion of the--
            ``(1) tax assessed on an organization by reason of the
        failure of the organization to give notice under subsection
        (i), or
            ``(2) penalty imposed under subsection (j) for a failure to
        file a report,
on a showing that such failure was due to reasonable cause and not due
to willful neglect.''.
    (e) Effective Date.--The amendments made by this section shall take
effect as if included in the amendments made by Public Law 106-230.

SEC. 2. NOTIFICATION OF INTERACTION OF REPORTING REQUIREMENTS.

    (a) In General.--The Secretary of the Treasury, in consultation
with the Federal Election Commission, shall publicize information on--
            (1) the effect of the amendments made by this Act, and
            (2) the interaction of requirements to file a notification
        or report under section 527 of the Internal Revenue Code of
        1986 and reports under the Federal Election Campaign Act of
        1971.
    (b) Information.--Information provided under subsection (a) shall
be included in any appropriate form, instruction, notice, or other
guidance issued to the public by the Secretary of the Treasury or the
Federal Election Commission regarding reporting requirements of
political organizations (as defined in section 527 of the Internal
Revenue Code of 1986) or reporting requirements under the Federal
Election Campaign Act of 1971.

SEC. 3. TECHNICAL CORRECTIONS TO SECTION 527 ORGANIZATION DISCLOSURE
              PROVISIONS.

    (a) Unsegregated Funds Not To Avoid Tax.--Paragraph (4) of section
527(i) of the Internal Revenue Code of 1986 (relating to failure to
notify) is amended by adding at the end the following new sentence:
``For purposes of the preceding sentence, the term `exempt function
income' means any amount described in a subparagraph of subsection
(c)(3), whether or not segregated for use for an exempt function.''.
    (b) Procedures for Assessment and Collection of Penalty.--Paragraph
(1) of section 527(j) of such Code (relating to required disclosure of
expenditures and contributions) is amended by adding at the end the
following new sentence: ``For purposes of subtitle F, the penalty
imposed by this paragraph shall be assessed and collected in the same
manner as penalties imposed by section 6652(c).''.
    (c) Application of Fraud Penalty.--Section 7207 of such Code
(relating to fraudulent returns, statements, and other documents) is
amended by striking ``pursuant to subsection (b) of section 6047 or
pursuant to subsection (d) of section 6104'' and inserting ``pursuant
to section 6047(b), section 6104(d), or subsection (i) or (j) of
section 527''.
    (d) Duplicate Electronic and Written Filings Not Required.--
            (1) Subparagraph (A) of section 527(i)(1) of such Code is
        amended by striking ``, electronically and in writing,''.
            (2) Subsection (i) of section 527 of such Code is amended
        by adding at the end the following new paragraph:
            ``(7) Electronic filing.--The Secretary shall develop
        procedures for submission in electronic form of notices
        required to be filed under this subsection and reports required
        to be filed under subsection (j).''.
    (e) Effective Dates.--
            (1) Subsections (a) and (b).--The amendments made by
        subsections (a) and (b) shall apply to failures occurring on or
        after the date of the enactment of this Act.
            (2) Subsections (c) and (d).--The amendments made by
        subsections (c) and (d) shall take effect as if included in the
        amendments made by Public Law 106-230.

Amends the Internal Revenue Code to: (1) exempt State and local candidate committees, as well as local committees of political parties, from specified notification requirements; (2) exempt certain State and local political organizations from specified reporting requirements; (3) remove language dictating that certain political organizations with gross receipts of $25,000 or more for a taxable year and specified political newsletter funds with such receipts shall file income tax returns; (4) mandate that, with certain designated exceptions, every political organization with gross receipts of $25,000 or more for a taxable year and specified political newsletter funds with such receipts file information returns with specified information including income, receipts and disbursements, as well as facts deemed necessary by the Secretary of the Treasury; (5) authorize the Secretary to waive certain related penalties; and (6) amend related penalty provisions.

To amend section 527 of the Internal Revenue Code of 1986 to eliminate reporting and return requirements for State and local candidate committees and to avoid duplicate reporting of campaign-related information.


SECTION 1. SHORT TITLE.

    This title may be cited as the ``Temporary Emergency Unemployment
Compensation Act of 2001''.

SEC. 2. FEDERAL-STATE AGREEMENTS.

    (a) In General.--Any State which desires to do so may enter into
and participate in an agreement under this title with the Secretary of
Labor (in this title referred to as the ``Secretary''). Any State which
is a party to an agreement under this title may, upon providing 30 days
written notice to the Secretary, terminate such agreement.
    (b) Provisions of Agreement.--Any agreement under subsection (a)
shall provide that the State agency of the State will make payments of
temporary emergency unemployment compensation to individuals who--
            (1) have exhausted all rights to regular compensation under
        the State law;
            (2) have no rights to compensation (including both regular
        compensation and extended compensation) with respect to a week
        under such law or any other State unemployment compensation law
        or to compensation under any other Federal law (and are not
        paid or entitled to be paid any additional compensation under
        any State or Federal law); and
            (3) are not receiving compensation with respect to such
        week under the unemployment compensation law of Canada.
    (c) Exhaustion of Benefits.--For purposes of subsection (b)(1), an
individual shall be deemed to have exhausted such individual's rights
to regular compensation under a State law when--
            (1) no payments of regular compensation can be made under
        such law because such individual has received all regular
        compensation available to such individual based on employment
        or wages during such individual's base period; or
            (2) such individual's rights to such compensation have been
        terminated by reason of the expiration of the benefit year with
        respect to which such rights existed.
    (d) Weekly Benefit Amount.--For purposes of any agreement under
this title--
            (1) the amount of temporary emergency unemployment
        compensation which shall be payable to any individual for any
        week of total unemployment shall be equal to the amount of the
        regular compensation (including dependents' allowances) payable
        to such individual during such individual's benefit year under
        the State law for a week of total unemployment;
            (2) the terms and conditions of the State law which apply
        to claims for regular compensation and to the payment thereof
        shall apply to claims for temporary emergency unemployment
        compensation and the payment thereof, except where inconsistent
        with the provisions of this title or with the regulations or
        operating instructions of the Secretary promulgated to carry
        out this title; and
            (3) the maximum amount of temporary emergency unemployment
        compensation payable to any individual for whom a temporary
        emergency unemployment compensation account is established
        under section 3 shall not exceed the amount established in such
        account for such individual.
    (e) Election by States.--Notwithstanding any other provision of
Federal law (and if State law permits), the Governor of a State is
authorized and may elect to trigger off an extended compensation period
in order to provide payment of temporary emergency unemployment
compensation to individuals who have exhausted their rights to regular
compensation under State law.

SEC. 3. TEMPORARY EMERGENCY UNEMPLOYMENT COMPENSATION ACCOUNT.

    (a) In General.--Any agreement under this title shall provide that
the State will establish, for each eligible individual who files an
application for temporary emergency unemployment compensation, a
temporary emergency unemployment compensation account with respect to
such individual's benefit year.
    (b) Amount in Account.--
            (1) In general.--The amount established in an account under
        subsection (a) shall be equal to 13 times the individual's
        average weekly benefit amount for the benefit year.
            (2) Reduction for extended benefits.--The amount in an
        account under paragraph (1) shall be reduced (but not below
        zero) by the aggregate amount of extended compensation (if any)
        received by such individual relating to the same benefit year
        under the Federal-State Extended Unemployment Compensation Act
        of 1970 (26 U.S.C. 3304 note).
            (3) Weekly benefit amount.--For purposes of this
        subsection, an individual's weekly benefit amount for any week
        is the amount of regular compensation (including dependents'
        allowances) under the State law payable to such individual for
        such week for total unemployment.

SEC. 4. PAYMENTS TO STATES HAVING AGREEMENTS FOR THE PAYMENT OF
              TEMPORARY EMERGENCY UNEMPLOYMENT COMPENSATION.

    (a) General Rule.--There shall be paid to each State that has
entered into an agreement under this title an amount equal to 100
percent of the temporary emergency unemployment compensation paid to
individuals by the State pursuant to such agreement.
    (b) Treatment of Reimbursable Compensation.--No payment shall be
made to any State under this section in respect of any compensation to
the extent the State is entitled to reimbursement in respect of such
compensation under the provisions of any Federal law other than this
title or chapter 85 of title 5, United States Code. A State shall not
be entitled to any reimbursement under such chapter 85 in respect of
any compensation to the extent the State is entitled to reimbursement
under this title in respect of such compensation.
    (c) Determination of Amount.--Sums payable to any State by reason
of such State having an agreement under this title shall be payable,
either in advance or by way of reimbursement (as may be determined by
the Secretary), in such amounts as the Secretary estimates the State
will be entitled to receive under this title for each calendar month,
reduced or increased, as the case may be, by any amount by which the
Secretary finds that the Secretary's estimates for any prior calendar
month were greater or less than the amounts which should have been paid
to the State. Such estimates may be made on the basis of such
statistical, sampling, or other method as may be agreed upon by the
Secretary and the State agency of the State involved.

SEC. 5. FINANCING PROVISIONS.

    (a) In General.--Funds in the extended unemployment compensation
account (as established by section 905(a) of the Social Security Act
(42 U.S.C. 1105(a)) of the Unemployment Trust Fund (as established by
section 904(a) of such Act (42 U.S.C. 1104(a)) shall be used for the
making of payments to States having agreements entered into under this
title.
    (b) Certification.--The Secretary shall from time to time certify
to the Secretary of the Treasury for payment to each State the sums
payable to such State under this title. The Secretary of the Treasury,
prior to audit or settlement by the General Accounting Office, shall
make payments to the State in accordance with such certification, by
transfers from the extended unemployment compensation account (as so
established) to the account of such State in the Unemployment Trust
Fund (as so established).
    (c) Assistance to States.--There are appropriated, without fiscal
year limitation, such funds as may be necessary for purposes of
assisting States (as provided in title III of the Social Security Act
(42 U.S.C. 501 et seq.) in meeting the costs of administration of
agreements under this title.
    (d) Authorization of Appropriations for Certain Payments.--There
are appropriated from the general fund of the Treasury, without fiscal
year limitation, to the extended unemployment compensation account (as
so established) of the Unemployment Trust Fund (as so established) such
sums as the Secretary estimates to be necessary to make the payments
under this section in respect of--
            (1) compensation payable under chapter 85 of title 5,
        United States Code; and
            (2) compensation payable on the basis of services to which
        section 3309(a)(1) of the Internal Revenue Code of 1986
        applies.
Amounts appropriated pursuant to the preceding sentence shall not be
required to be repaid.

SEC. 6. FRAUD AND OVERPAYMENTS.

    (a) In General.--If an individual knowingly has made, or caused to
be made by another, a false statement or representation of a material
fact, or knowingly has failed, or caused another to fail, to disclose a
material fact, and as a result of such false statement or
representation or of such nondisclosure such individual has received an
amount of temporary emergency unemployment compensation under this
title to which he was not entitled, such individual--
            (1) shall be ineligible for further temporary emergency
        unemployment compensation under this title in accordance with
        the provisions of the applicable State unemployment
        compensation law relating to fraud in connection with a claim
        for unemployment compensation; and
            (2) shall be subject to prosecution under section 1001 of
        title 18, United States Code.
    (b) Repayment.--In the case of individuals who have received
amounts of temporary emergency unemployment compensation under this
title to which they were not entitled, the State shall require such
individuals to repay the amounts of such emergency unemployment
compensation to the State agency, except that the State agency may
waive such repayment if it determines that--
            (1) the payment of such emergency unemployment compensation
        was without fault on the part of any such individual; and
            (2) such repayment would be contrary to equity and good
        conscience.
    (c) Recovery by State Agency.--
            (1) In general.--The State agency may recover the amount to
        be repaid, or any part thereof, by deductions from any
        temporary emergency unemployment compensation payable to such
        individual under this title or from any unemployment
        compensation payable to such individual under any Federal
        unemployment compensation law administered by the State agency
        or under any other Federal law administered by the State agency
        which provides for the payment of any assistance or allowance
        with respect to any week of unemployment, during the 3-year
        period after the date such individuals received the payment of
        the temporary emergency unemployment compensation to which they
        were not entitled, except that no single deduction may exceed
        50 percent of the weekly benefit amount from which such
        deduction is made.
            (2) Opportunity for hearing.--No repayment shall be
        required, and no deduction shall be made, until a determination
        has been made, notice thereof and an opportunity for a fair
        hearing has been given to the individual, and the determination
        has become final.
    (d) Review.--Any determination by a State agency under this section
shall be subject to review in the same manner and to the same extent as
determinations under the State unemployment compensation law, and only
in that manner and to that extent.

SEC. 7. DEFINITIONS.

    In this title, the terms ``compensation'', ``regular
compensation'', ``extended compensation'', ``additional compensation'',
``benefit year'', ``base period'', ``State'', ``State agency'', ``State
law'', and ``week'' have the respective meanings given such terms under
section 205 of the Federal-State Extended Unemployment Compensation Act
of 1970 (26 U.S.C. 3304 note).

SEC. 8. APPLICABILITY.

    An agreement entered into under this Act shall apply to weeks of
unemployment--
            (1) beginning no earlier than the first day of the first
        week beginning after the date on which such agreement is
        entered into; and
            (2) ending before the date that is 18 months after the date
        of enactment of this Act.

SEC. 9. TEMPORARY REDUCTION IN INTEREST RATE APPLICABLE TO REPAYMENTS
              OF ADVANCES TO STATE UNEMPLOYMENT FUNDS.

    With respect to advances made to a State under section 1201 of the
Social Security Act (42 U.S.C. 1321) during the period beginning on the
date of enactment of this Act and ending on the date that is 18 months
after such date of enactment, the rate of interest paid by a State on
such an advance shall be determined under section 1202(b)(4) of the
such Act (42 U.S.C. 1322(b)(4)) by substituting ``5 percent'' for ``10
percent'' in the matter preceding subparagraph (A).

Temporary Emergency Unemployment Compensation Act of 2001 - Provides for a program of temporary emergency unemployment compensation (TEUC).Sets forth TEUC program requirements for Federal-State agreements, formulas for determining amounts in individual TEUC accounts and weekly benefits, payments to States, and financing. Includes among eligibility requirements an individual's not having rights, with respect to a week, to other compensation (including both regular and extended compensation). Reduces an individual TEUC account by the aggregate amount of any extended compensation for the same benefit year.Applies TEUC agreements to weeks of unemployment: (1) beginning on or after the first day of the first week after the date on which such agreement is entered into; and (2) ending before the date that is 18 months after enactment of this Act.Provides for a temporary reduction in the interest rate applicable to repayments of advances to State unemployment funds.

A bill to provide for a program of temporary enhanced unemployment benefits.


SECTION 1. SHORT TITLE.

    This Act may be cited as the ``Combating Climate Change Through
Individual Action Act of 2008''.

SEC. 2. FINDINGS.

    Congress finds the following:
            (1) Agricultural, grassland, and forestry practices play an
        essential role in capturing atmospheric carbon and sequestering
        it as soil organic matter.
            (2) Released carbon can be captured through improved
        grassland management, tree planting, forest preservation, and
        enhanced agronomic and irrigation practices.
            (3) Promoting increased natural carbon sinks could have a
        significant impact on the world's projected carbon emissions
        from the burning of fossil fuels.
            (4) Certain agricultural and forestry practices can reduce
        greenhouse gases: (A) avoiding emissions by maintaining
        existing carbon storage in trees and soils; (B) increasing
        carbon storage by, e.g., tree planting, conversion from
        conventional to conservation tillage practices on agricultural
        lands;
            (5) The large potentials exist through known cropping and
        land management practices such as adoption of no-till, reduced
        fallow and use of cover crops, and conservation set-asides with
        perennial grasses and trees.

SEC. 3. CARBON SEQUESTRATION AND SOIL CONSERVATION CREDIT.

    (a) In General.--Subpart D of part IV of subchapter A of chapter 1
of the Internal Revenue Code of 1986 (relating to business related
credits) is amended by adding at the end the following new section:

``SEC. 45O. CARBON SEQUESTRATION AND SOIL CONSERVATION.

    ``(a) In General.--For purposes of section 38, in the case of a
taxpayer engaged in the business of farming, the credit determined
under this section for the taxable year is an amount equal to 30
percent of the qualified carbon sequestration and soil conservation
expenditures for the taxable year which are paid or incurred with
respect to the land used in such farming.
    ``(b) Limitation.--The credit allowed with respect to a taxpayer
under this section for a taxable year shall not exceed an amount equal
to $10,000, reduced by the sum of the credits allowed with respect to
the taxpayer under subsection (a) for all preceding taxable years.
    ``(c) Qualified Carbon Sequestration and Soil Conservation
Expenses.--For purposes of this section--
            ``(1) In general.--The term `qualified carbon sequestration
        and soil conservation expenditures' means amounts paid or
        incurred to sequester carbon and conserve soil, including--
                    ``(A) expenditures described in section 175(c),
                    ``(B) conservation tillage expenditures,
                    ``(C) cover cropping expenditures,
                    ``(D) amounts paid or incurred to increase the
                nitrogen use efficiency (other than use of nitrogen
                fertilizers) of land used in farming, and
                    ``(E) amounts paid or incurred for multiple year
                rotations, including introduction of a perennial that
                reduces carbon loss and tillage, builds soil tilth, and
                increases carbon capture capacity.
            ``(2) Conservation tillage expenditures.--The term
        `conservation tillage expenditures' means any expenditures paid
        or incurred for a tilling and planting method in which at least
        30 percent of the previous crop residue remains on the soil
        after planting the current crop. Such term includes the
        following tilling practices: no till, ridge till, minimum till,
        and mulch till.
            ``(3) Cover cropping expenditures.--The term `cover
        cropping expenditures' means expenditures paid or incurred for
        the preparation and seeding of land for any grass, legume, or
        small grain--
                    ``(A) which is not the primary crop of the
                taxpayer,
                    ``(B) the primary purpose of which is to achieve
                one or more of the following: reduction in erosion;
                maintenance or improvement in soil fertility, tilth,
                and structure,
                    ``(C) a purpose of which may be interruption of
                pest cycles or conservation of water.
    ``(d) Per Acre Credit Alternative.--
            ``(1) In general.--Not later than 180 days after the date
        of the enactment of this section, the Secretary shall, in
        consultation with the Secretary of Agriculture, establish an
        alternative procedure for determining the credit under
        subsection (a), which, at the election of the taxpayer, shall
        be treated as the amount determined under subsection (a).
            ``(2) Procedure described.--(A) The Secretary shall
        establish credit amounts to apply to land used in farming on a
        per acre basis with respect to each method of carbon
        sequestration and soil conservation described in subsection
        (c)(1).
            ``(B) Such credit amounts shall be based on the efficacy of
        the method in sequestering carbon and preventing soil erosion.
            ``(C) No such credit amount may exceed $15 per acre.
            ``(D) The Secretary shall prescribe rules similar to the
        rules of paragraphs (1) through (4) of subsection (e) to apply
        for purposes of the procedure established under this
        subsection.
            ``(3) Election.--An election to use such alternative method
        shall be made in such form and manner as the Secretary may
        prescribe, and shall apply to the taxable year for which made
        and for all subsequent taxable years.
    ``(e) Definition and Special Rules.--
            ``(1) Land used in farming.--For purposes of this section,
        land shall be treated as used in farming only if such land is
        used (before or simultaneously with the expenditures described
        in subsection (c)(1)) by the taxpayer or his tenant for the
        production of crops, fruits, or other agricultural products or
        for the sustenance of livestock.
            ``(2) Expenditures must be consistent with soil
        conservation plan.--Notwithstanding any other provision of this
        section, subsection (a) shall not apply to any expenditures
        unless such expenditures are consistent with--
                    ``(A) the plan (if any) approved by the Soil
                Conservation Service of the Department of Agriculture
                for the area in which the land is located, or
                    ``(B) if there is no plan described in clause (i),
                any soil conservation plan of a comparable State
                agency.
            ``(3) Basis adjustment.--For purposes of this subtitle, if
        a credit is determined under this section for any expenditure
        with respect to any property, the increase in the basis of such
        property which would (but for this paragraph) result from such
        expenditure shall be reduced by the amount of the credit so
        determined.
            ``(4) Denial of double benefit.--No deduction or other
        credit shall be allowed under this chapter for any amount taken
        into account in determining the credit under this section.
    ``(f) Termination.--This section shall not apply to taxable years
beginning after December 31, 2013.''.
    (b) Credit Made Part of General Business Credit.--Subsection (b) of
section 38 of such Code (relating to current year business credit) is
amended by striking ``plus'' at the end of paragraph (30), by striking
the period at the end of paragraph (31) and inserting ``plus'', and by
adding at the end the following new paragraph:
            ``(32) the carbon sequestration and soil conservation
        credit determined under section 45O(a).''.
    (c) Conforming Amendments.--Subsection (a) of section 1016 of such
Code (relating to adjustments to basis) is amended by striking ``and''
at the end of paragraph (35), by striking the period at the end of
paragraph (36) and inserting ``and'', and by adding at the end the
following new paragraph:
            ``(37) to the extent provided in section 45O(e).''.
    (d) Clerical Amendment.--The table of sections for subpart D of
part IV of subchapter A of chapter 1 of such Code is amended by adding
at the end the following new item:

``Sec. 45O. Carbon sequestration and soil conservation.''.
    (e) Effective Date.--The amendments made by this section shall
apply to expenditures paid or incurred after December 31, 2008.

SEC. 4. QUALIFYING PLANTING EXPENDITURE CREDIT.

    (a) In General.--Subpart B of part IV of subchapter A of chapter 1
of the Internal Revenue Code of 1986 (relating to other credits) is
amended by adding at the end the following new section:

``SEC. 30D. QUALIFIED PLANTING EXPENDITURE CREDIT.

    ``(a) Allowance of Credit.--There shall be allowed as a credit
against the tax imposed by this chapter for the taxable year an amount
equal to 10 percent of the qualified planting expenditures of the
taxpayer for the taxable year.
    ``(b) Limitations.--The amount taken into account under subsection
(a) for any taxable year shall not exceed--
            ``(1) in the case of expenditures paid or incurred by the
        taxpayer with respect to an area which is included under
        section 121 as part of the taxpayer's principal residence,
        $5,000,
            ``(2) in the case of expenditures paid or incurred by the
        taxpayer in the course of, or with respect to, a trade or
        business carried on by the taxpayer, $50,000, and
            ``(3) in any other case, zero.
    ``(c) Qualified Planting Expenditures.--For purposes of this
section--
            ``(1) In general.--The term `qualifying planting
        expenditures' means expenditures paid or incurred--
                    ``(A) for the purchase and planting of any tree,
                plant, shrub, or bush which meets the requirements of
                paragraph (2), and
                    ``(B) for the purchase and installation of a
                vegetated roof system.
        Such term shall not include expenditures relating to any
        property which is held by the taxpayer for use in a trade or
        business or for the production of income, or which is property
        described in section 1221(a)(1) in the hands of the taxpayer.
            ``(2) Trees, plants, shrubs, or bushes.--A tree, plant,
        shrub, or bush satisfies the requirements of the paragraph if
        such tree, plant, shrub, or bush is certified, in accordance
        with guidance prescribed by the Secretary (after consultation
        with the Administrator of the Environmental Protection Agency
        and the Secretary of Agriculture), to be quick-growing,
        appropriate for the region in which it is planted, and
        effective in capturing carbon.
            ``(3) Vegetated roof system.--The term `vegetated roof
        system' means a system by which vegetation growing in a
        substrate is integrated with the roof (or portion thereof) of a
        building owned by the taxpayer.
    ``(d) Application With Other Credits.--The credit allowed under
subsection (a) for any taxable year shall not exceed the excess (if
any) of--
            ``(1) the regular tax liability (as defined in section
        26(b)) reduced by the sum of the credits allowable under
        subpart A and sections 27, 30, 30B, and 30C, over
            ``(2) the tentative minimum tax for the taxable year.
    ``(e) Definition and Special Rules.--For purposes of this section--
            ``(1) Principal residence.--The term `principal residence'
        has the same meaning as when used in section 121, except that
        no ownership requirement shall be imposed.
            ``(2) Joint occupancy, cooperative housing corporations,
        and condominium management associations.--Rules similar to the
        rules of paragraphs (4), (5), and (6) of section 25D(e) shall
        apply.
            ``(3) Expenditures outside united states.--The credit under
        this section shall not be allowed with respect to expenditures
        paid or incurred for areas located outside the United States.
            ``(4) Basis adjustment.--For purposes of this subtitle, if
        a credit is allowed under this section for an expenditure, the
        increase in basis which would result (but for this subsection)
        from such expenditure shall be reduced by the amount of credit
        allowed under this section.
    ``(f) Termination.--This section shall not apply to taxable years
beginning after December 31, 2013.''.
    (b) Conforming Amendments.--
            (1) Subsection (a) of section 1016, as amended by section
        3, is amended by striking ``and'' at the end of paragraph (36),
        by striking the period at the end of paragraph (37) and
        inserting ``and'', and by adding at the end the following new
        paragraph:
            ``(38) to the extent provided in section 30D(e)(4).''.
            (2) The table of sections for subpart B of part IV of
        subchapter A of chapter 1 of such Code is amended by inserting
        after the item relating to section 30C the following new item:

``Sec. 30D. Qualified planting expenditure credit.''.
    (c) Effective Date.--The amendments made by this section shall
apply to expenditures paid or incurred after December 31, 2008.

SEC. 5. GRASSLAND, RANGELAND, AND FOREST CONSERVATION CREDIT.

    (a) In General.--Not later than 180 days after the date of the
enactment of this Act, the Secretary of the Treasury, in consultation
with the Department of Agriculture, shall establish an appropriate tax
credit, with respect to land located in the United States, for--
            (1) the conversion of cropland to pasture for grazing
        purposes or to grassland or rangeland, and
            (2) reforestation and afforestation of land--
                    (A) which is not held by the taxpayer for the
                planting, cultivating, caring for, and cutting of trees
                for sale or use in the commercial production of timber
                products, and
                    (B) with trees which are not held by the taxpayer
                for use in a trade or business or for the production of
                income.
    (b) Other Rules Relating to Credit.--
            (1) Credit to be per acre.--The Secretary shall establish
        credit amounts to apply to land on a per acre basis with
        respect to each method of conservation described in subsection
        (a).
            (2) Pursuant to approved plan.--Such methods must be
        pursuant to a plan submitted by the taxpayer and approved by
        the Secretaries of the Treasury and Agriculture.
            (3) Basis for credit amounts.--Credit amount shall be based
        on--
                    (A) the efficacy of the method in sequestering
                carbon and preventing soil erosion,
                    (B) the expenditures relating to such method, and
                    (C) the number of years the taxpayer certifies to
                the Secretary or ensures (by conservation easement or
                otherwise) that the applicable land will remain subject
                to the approved plan.
            (4) Recapture.--The Secretary shall provide for recapturing
        the benefit of any credit allowed under this section with
        respect to any property that ceases to be used in accordance
        with the approved plan.
            (5) Denial of double benefit and basis adjustment.--The
        Secretary shall provide--
                    (A) an appropriate basis adjustment for property
                with respect to which such credit is allowed, and
                    (B) rules disallowing such deductions and other
                credits as may be appropriate to avoid allowing
                additional tax benefits for the same conservation
                method or expenses.
    (c) Effective Date.--The credit established by the Secretary shall
apply to taxable years beginning after December 31, 2008.

SEC. 6. CARBON SEQUESTRATION CREDIT REPORT.

    (a) In General.--In the case of any substantial change in the
carbon sequestration market (including the enactment into law of a
carbon cap and trade program), the Secretary of the Treasury shall, in
consultation with any appropriate Federal officers, study such change
and any effect of such change on the efficiency of, and need for, the
credits allowed under section 5 of this Act and sections 45O and 30D of
the Internal Revenue Code of 1986.
    (b) Report.--As soon as practicable after sufficient opportunity to
observe the effect of such change in the carbon sequestration market,
the Secretary shall submit a report to Congress containing the results
of the study conducted under subsection (a) and any recommendations of
the Secretary for modifying such credits based on such results.

Combating Climate Change Through Individual Action Act of 2008 - Amends the Internal Revenue Code to allow tax credits for: (1) 30% of carbon sequestration and soil conservation expenditures made by taxpayers engaged in the business of farming; (2) 10% of qualifying planting expenditures, including expenditures for the purchase and planting of any tree, plant, shrub, or bush, and the purchase and installation of a vegetated roof system; (3) the conversion of cropland to pasture for grazing purposes or to grassland or rangeland; and (4) certain types of reforestation and afforestation of land.

To amend the Internal Revenue Code of 1986 to provide incentives for carbon sequestration.


SECTION 1. SHORT TITLE.

    This Act may be cited as the ``Lower Colorado River Multi-Species
Conservation Program Act''.

SEC. 2. DEFINITIONS.

    In this Act:
            (1) Lower colorado river multi-species conservation
        program.--The term ``Lower Colorado River Multi-Species
        Conservation Program'' or ``LCR MSCP'' means the cooperative
        effort on the Lower Colorado River between Federal and non-
        Federal entities in Arizona, California, and Nevada approved by
        the Secretary of the Interior on April 2, 2005.
            (2) Lower colorado river.--The term ``Lower Colorado
        River'' means the Colorado River from Lake Mead to the
        Southerly International Boundary with Mexico, including its
        historic floodplain and its mainstem reservoirs to their full
        pool elevations.
            (3) Program documents.--The term ``Program Documents''
        means the Habitat Conservation Plan, Biological Assessment and
        Biological and Conference Opinion, Environmental Impact
        Statement/Environmental Impact Report, Funding and Management
        Agreement, Implementing Agreement, and Section 10(a)(1)(B)
        Permit issued and, as applicable, executed in connection with
        the LCR MSCP.
            (4) Secretary.--The term ``Secretary'' means the Secretary
        of the Interior.
            (5) State.--The term ``State'' means each of the States of
        Arizona, California, and Nevada.
            (6) Steering committee.--The term ``Steering Committee''
        means the LCR MSCP steering committee established pursuant to
        the Program Documents.

SEC. 3. IMPLEMENTATION AND WATER ACCOUNTING.

    (a) Implementation.--The Secretary shall manage and implement the
LCR MSCP in accordance with the Program Documents.
    (b) Water Accounting.--The Secretary is authorized and directed to
enter into an agreement with the States providing for the use of water
from the Lower Colorado River for habitat creation and maintenance in
accordance with the Program Documents.

SEC. 4. AUTHORIZATION OF APPROPRIATIONS.

    (a) In General.--There is authorized to be appropriated to the
Secretary such sums as may be necessary to meet the obligations of the
Secretary under the Program Documents, to remain available until
expended.
    (b) Investments.--The Secretary is authorized to invest with the
Secretary of the Treasury such portions of appropriations, and any non-
Federal contributions made pursuant to the Program Documents, as are
not, in the judgment of the Secretary, required to meet current
expenditures. Such investments shall be made only in interest-bearing
obligations of the United States. Funds invested under this subsection
and interest on those funds shall be available to the Secretary to meet
the obligations of the Secretary under the Program Documents.
    (c) Non-Reimbursable and Non-Returnable.--All amounts appropriated
to and expended by the Secretary for the LCR MSCP shall be non-
reimbursable and non-returnable.

SEC. 5. APPLICABLE LAW, CONTINUITY OF PROGRAM, ENFORCEABILITY OF
              PROGRAM DOCUMENTS.

    (a) In General.--Nothing in this Act shall impair any right to the
delivery or beneficial consumptive use of Colorado River water under
any compact, treaty, law, decree, or contract in effect on the date of
enactment of this Act.
    (b) Continuity of Program Documents.--No future act of Congress
relating to Public Law 93-205 (16 U.S.C. 1531 et seq.) shall have the
effect of modifying the Program Documents unless expressly made
applicable to the LCR MSCP.
    (c) Enforceability of Program Documents.--Any party to any
agreement entered into with the United States or any agency thereof
pursuant to the LCR MSCP may commence a civil action in United States
district court to enforce the agreement or to declare the rights and
obligations of the parties under the Program Documents. The district
court shall have jurisdiction of such actions and may issue such
orders, judgments, and decrees as are consistent with the court's
exercise of jurisdiction under this section. The United States or any
agency thereof may be named as a defendant in such actions. The
sovereign immunity of the United States is waived for purposes of
actions commenced pursuant to this section. Nothing in this section
waives the sovereign immunity of the United States to claims for money
damages, monetary compensation, the provision of indemnity, or any
claim seeking money from the United States. Any suit pursuant to this
section may be brought in any United States district court in the State
in which any non-Federal party to the suit is situated.
    (d) Applicable Law.--Nothing in this Act affects the enforceability
of the requirement that the Program Documents comply with existing law
as of April 2, 2005, except that the Steering Committee shall not be
subject to the Federal Advisory Committee Act (5 U.S.C. App.).

Lower Colorado River Multi-Species Conservation Program Act - Directs the Secretary of the Interior to manage and implement the Lower Colorado River Multi-Species Conservation Program, and to enter into an agreement with Arizona, California, and Nevada providing for the use of water from the Lower Colorado River for habitat creation and maintenance, in accordance with the Habitat Conservation Plan, Biological Assessment and Biological and Conference Opinion, Environmental Impact Statement/Environmental Impact Report, Funding and Management Agreement, Implementing Agreement (Agreement).

Permits any party to an agreement entered into with the United States pursuant to the Program to commence a civil action in U.S. district court to enforce the agreement or to declare the rights and obligations of the parties under the program documents. Grants the district court jurisdiction over any such action.

A bill to authorize appropriations for the Bureau of Reclamation to carry out the Lower Colorado River Multi-Species Conservation Program in the States of Arizona, California, and Nevada, and for other purposes.