Q:

out of memory when using cupy

When I was using cupy to deal with some big array, the out of memory errer comes out, but when I check the nvidia-smi to see the memeory usage, it didn't reach the limit of my GPU memory, I am using nvidia geforce RTX 2060, and the GPU memory is 6 GB, here is my code:
import cupy as cp

mempool = cp.get_default_memory_pool()
print(mempool.used_bytes())              # 0
print(mempool.total_bytes())             # 0

a = cp.random.randint(0, 256, (10980, 10980)).astype(cp.uint8)
a = a.ravel()
print(a.nbytes)                          # 120560400
print(mempool.used_bytes())              # 120560640
print(mempool.total_bytes())             # 602803712
# when I finish create this array, the nvidia-smi shows like this
#+-----------------------------------------------------------------------------+
 | NVIDIA-SMI 430.86       Driver Version: 430.86       CUDA Version: 10.2     |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name            TCC/WDDM | Bus-Id        Disp.

ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  GeForce RTX 2060   WDDM  | 00000000:01:00.0  On |                  N/A |
  | N/A   45C    P8     9W /  N/A |   5075MiB /  6144MiB |      3%      Default |
  +-------------------------------+----------------------+----------------------+

there seems to have enough space to use, why this error happend, is this because of my GPU doesn't have enough memory,or is it because my code is wrong or I didn't allocate memory correcttly.

A:

Isn't 964,483,584 bigger than your mempool.total_bytes() of 602,803,712?

As said in the comments, you can do it by batches instead of doing the whole computation at once.


Fans News

Munster

Rugby

Relive the action from Saturday night at Irish Independent Park and get the post-match thoughts of man of the match Keith Earls as Munster enjoyed a 22-10 win over Glasgow Warriors in Round 16 of the Guinness PRO12.

The newly developed Irish Independent Park, the home of Munster Rugby in Cork, was officially opened on Saturday night by Minister for Agriculture, Food, the Marine and Defence Simon Coveney T.D. The opening took place ahead of Munster's Guinness PRO12 win over Glasgow Warriors.

Catching up with prop John Ryan on his season to date and Munster's Guinness PRO12 clash with Glasgow Warriors on Saturday, as the province look to right the wrongs of consecutive defeats to the Scotsmen.

It's a big night of action both on and off the pitch at Irish Independent Park as Munster take on top of the table Glasgow Warriors in Round 16 of the Guinness PRO12.

The official opening will also take place before kick-off.

Gates will open at 5.45pm with less than 2,000 tickets now remaining.

Don't miss your chance to cheer on Munster in Cork next weekend at the newly redeveloped Irish Independent Park.

Less than 3,000 tickets remain as Munster get set to host the current Guinness PRO12 table toppers Glasgow Warriors in Round 16 of the competition.

It has been confirmed today that Simon Coveney TD Minister for Agriculture, Food, the Marine and Defence will perform the official opening of the newly developed Irish Independent Park in Cork on Saturday February 28th.

Following the announcement of his retirement from rugby due to injury yesterday, fans, friends and players have taken to social media in their droves to show their appreciation and extend best wishes to the Killaloe-man who made 121 appearances for the province.

Following Saturday's win over Cardiff Blues, Munster will look to maintain home winning ways when they welcome current table toppers Glasgow Warriors to Irish Independent Park for Round 16 of the Guinness PRO12 on February 28th.

Munster and Dolphin centre Cian Bohane give us a walk-through of the brand new player and fan amenities at Irish Independent Park ahead of Munster's first game at the redeveloped stadium tomorrow night.

Munster play Cardiff Blues in their first game at the new Irish Independent Park on Saturday night.

With gates opening at 4:45pm you can check out Ireland v France live on the big screen from 5pm, and take part in the Valentine's fun on our big screen 'Kiss Cam'.

Take a look at our training shots from Irish Independent Park and get the thoughts of second-row Billy Holland as Munster prepare to face Cardiff Blues in Round 14 of the Guinness PRO12 on Saturday night.


Q:

Calling Drupal URL from another server using php

I am trying to call drupal 6 URL from my local system, that URL will generate a xml file.

My system does not have drupal intallation.

I tried file_get_content("www.abc.com") but it does not write my xml file but when I try to run the url normally it runs and xml file gets generated.

Thanks in advance

A:

You can use sockets or just file_get_contents, like
$homepage = file_get_contents('http://www.example.com/');
echo $homepage; // $homepage contains server response without headers


Can't import datasets AttributeError: partially initialized module 'datasets' has no attribute 'utils' (most likely due to a circular import)

bug

Something isn't working

## Describe the bug
A clear and concise description of what the bug is.
Not able to import datasets
## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
import os
os.environ["WANDB_API_KEY"] = "0" ## to silence warning
import numpy as np
import random
import sklearn
import matplotlib.pyplot as plt
import pandas as pd
import sys
import tensorflow as tf
import plotly.express as px
import transformers
import tokenizers
import nlp as nlp
import utils
import datasets
```

## Expected results
A clear and concise description of the expected results.
import should work normal
## Actual results
Specify the actual results or traceback.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-21-b3b5b0b62103> in <module>
     13 import nlp as nlp
     14 import utils
---> 15 import datasets

~\anaconda3\lib\site-packages\datasets\__init__.py in <module>
     44 from .fingerprint import disable_caching, enable_caching, is_caching_enabled, set_caching_enabled
     45 from .info import DatasetInfo, MetricInfo
---> 46 from .inspect import (
     47     get_dataset_config_info,
     48     get_dataset_config_names,

~\anaconda3\lib\site-packages\datasets\inspect.py in <module>
     28 from .download.streaming_download_manager import StreamingDownloadManager
     29 from .info import DatasetInfo
---> 30 from .load import dataset_module_factory, import_main_class, load_dataset_builder, metric_module_factory
     31 from .utils.file_utils import relative_to_absolute_path
     32 from .utils.logging import get_logger

~\anaconda3\lib\site-packages\datasets\load.py in <module>
     53 from .iterable_dataset import IterableDataset
     54 from .metric import Metric
---> 55 from .packaged_modules import (
     56     _EXTENSION_TO_MODULE,
     57     _MODULE_SUPPORTS_METADATA,

~\anaconda3\lib\site-packages\datasets\packaged_modules\__init__.py in <module>
      4 from typing import List
      5
----> 6 from .csv import csv
      7 from .imagefolder import imagefolder
      8 from .json import json

~\anaconda3\lib\site-packages\datasets\packaged_modules\csv\csv.py in <module>
     13
     14
---> 15 logger = datasets.utils.logging.get_logger(__name__)
     16
     17 _PANDAS_READ_CSV_NO_DEFAULT_PARAMETERS = ["names", "prefix"]

AttributeError: partially initialized module 'datasets' has no attribute 'utils' (most likely due to a circular import)

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
Copy-and-paste the text below in your GitHub issue.

- `datasets` version: 2.4.0
- Platform: Windows-10-10.0.22000-SP0
- Python version: 3.8.8
- PyArrow version: 9.0.0
- Pandas version: 1.2.4


[LibriSpeech] Fix dev split local_extracted_archive for 'all' split

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4904). All of your documentation changes will be reflected on that endpoint.

We define the keys for the `_DL_URLS` of the dev split as `dev.clean` and `dev.other`:
https://github.com/huggingface/datasets/blob/2e7142a3c6500b560da45e8d5128e320a09fcbd4/datasets/librispeech_asr/librispeech_asr.py#L60-L61

These keys get forwarded to the `dl_manager` and thus the `local_extracted_archive`.

However, when calling `SplitGenerator` for the dev sets, we query the `local_extracted_archive` keys `validation.clean` and `validation.other`:
https://github.com/huggingface/datasets/blob/2e7142a3c6500b560da45e8d5128e320a09fcbd4/datasets/librispeech_asr/librispeech_asr.py#L212
https://github.com/huggingface/datasets/blob/2e7142a3c6500b560da45e8d5128e320a09fcbd4/datasets/librispeech_asr/librispeech_asr.py#L219

The consequence of this is that the `local_extracted_archive` arg passed to `_generate_examples` is always `None`, as the keys `validation.clean` and `validation.other` do not exists in the `local_extracted_archive`.

When defining the `audio_file` in `_generate_examples`, since `local_extracted_archive` is always `None`, we always omit the `local_extracted_archive` path from the `audio_file` path, **even** if in non-streaming mode:
https://github.com/huggingface/datasets/blob/2e7142a3c6500b560da45e8d5128e320a09fcbd4/datasets/librispeech_asr/librispeech_asr.py#L259-L263

Thus, `audio_file` will only ever be the streaming path (`audio_file`, not `os.path.join(local_extracted_archive, audio_file)`).

This PR fixes the `.get()` keys for the `local_extracted_archive` for the dev splits.


Fix CI reporting

_The documentation is not available anymore as the PR was closed or merged._

Fix CI so that it reports defaults (failed and error) besides the custom (xfailed and xpassed) in the test summary.

This PR fixes a regression introduced by:
- #4845

This introduced the reporting of xfailed and xpassed, but wrongly removed the reporting of the defaults failed and error.


Name the default config `default`

enhancement

New feature or request

question

Further information is requested

Currently, if a dataset has no configuration, a default configuration is created from the dataset name.

For example, for a dataset loaded from the hub repository, such as https://huggingface.co/datasets/user/dataset (repo id is `user/dataset`), the default configuration will be `user--dataset`.

It might be easier to handle to set it to `default`, or another reserved word.


Raise ManualDownloadError from get_dataset_config_info

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4901). All of your documentation changes will be reflected on that endpoint.

This PRs raises a specific `ManualDownloadError` when `get_dataset_config_info` is called for a dataset that requires manual download.

Related to:
- #4898

CC: @severo


Dataset Viewer issue for asaxena1990/Dummy_dataset

Seems to be linked to the use of the undocumented `_resolve_features` method in the dataset viewer backend:

```
>>> from datasets import load_dataset
>>> dataset = load_dataset("asaxena1990/Dummy_dataset", name="asaxena1990--Dummy_dataset", split="train", streaming=True)
Using custom data configuration asaxena1990--Dummy_dataset-4a704ed7e5627563
>>> dataset._resolve_features()
Failed to read file 'https://huggingface.co/datasets/asaxena1990/Dummy_dataset/resolve/06885879a8bdd767d2d27695484fc6c83244617a/dummy_dataset_train.json' with error <class 'pyarrow.lib.ArrowInvalid'>: JSON parse error: Column() changed from object to array in row 0
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/packaged_modules/json/json.py", line 109, in _generate_tables
    pa_table = paj.read_json(
  File "pyarrow/_json.pyx", line 246, in pyarrow._json.read_json
  File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: JSON parse error: Column() changed from object to array in row 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 1261, in _resolve_features
    features = _infer_features_from_batch(self._head())
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 686, in _head
    return _examples_to_batch([x for key, x in islice(self._iter(), n)])
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 686, in <listcomp>
    return _examples_to_batch([x for key, x in islice(self._iter(), n)])
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 708, in _iter
    yield from ex_iterable
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 112, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 651, in wrapper
    for key, table in generate_tables_fn(**kwargs):
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/packaged_modules/json/json.py", line 137, in _generate_tables
    f"This JSON file contain the following fields: {str(list(dataset.keys()))}. "
AttributeError: 'list' object has no attribute 'keys'
```

Pinging @huggingface/datasets

Hi ! JSON files containing a list of object are not supported yet, you can use JSON Lines files instead in the meantime
```json
{"text": "can I know this?", "intent": "Know", "type": "Test"}
{"text": "can I know this?", "intent": "Know", "type": "Test"}
...
```

### Link

_No response_

### Description

_No response_

### Owner

_No response_


Re-add code and und language tags

This PR fixes the removal of 2 language tags done by:
- #4882

The tags are:
- "code": this is not a IANA tag but needed
- "und": this is one of the special scoped tags removed by 0d53202b9abce6fd0358cb00d06fcfd904b875af
  - used in "mc4" and "udhr" datasets


Dataset Viewer issue for timit_asr

Yes, the dataset viewer is based on `datasets`, and the following does not work:

```
>>> from datasets import get_dataset_split_names
>>> get_dataset_split_names('timit_asr')
Downloading builder script: 7.48kB [00:00, 6.69MB/s]
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/inspect.py", line 354, in get_dataset_config_info
    for split_generator in builder._split_generators(
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/timit_asr/43f9448dd5db58e95ee48a277f466481b151f112ea53e27f8173784da9254fb2/timit_asr.py", line 117, in _split_generators
    data_dir = os.path.abspath(os.path.expanduser(dl_manager.manual_dir))
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/posixpath.py", line 231, in expanduser
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/inspect.py", line 404, in get_dataset_split_names
    info = get_dataset_config_info(
  File "/home/slesage/hf/datasets-server/services/worker/.venv/lib/python3.9/site-packages/datasets/inspect.py", line 359, in get_dataset_config_info
    raise SplitsNotFoundError("The split names could not be parsed from the dataset config.") from err
datasets.inspect.SplitsNotFoundError: The split names could not be parsed from the dataset config.
```

cc @huggingface/datasets

Due to license restriction, this dataset needs manual downloading of the original data.

This information is in the dataset card: https://huggingface.co/datasets/timit_asr
> The dataset needs to be downloaded manually from https://catalog.ldc.upenn.edu/LDC93S1

Maybe a better error message for datasets that need manual downloading? @severo

Maybe we can raise a specific excpetion as done from `load_dataset`...

Yes, ideally something like https://github.com/huggingface/datasets/blob/main/src/datasets/builder.py#L81


datasets generate large arrow file

bug

Something isn't working

Checking the large file in disk, and found the large cache file in the cifar10 data directory:

![image](https://user-images.githubusercontent.com/18533904/186830449-ba96cdeb-0fe8-4543-994d-2abe7145933f.png)

As we know, the size of cifar10 dataset is ~130MB, but the cache file has almost 30GB size, there may be some problems here.


Fix missing tags in dataset cards

Fix missing tags in dataset cards.

This PR partially fixes the missing tags in dataset cards. Subsequent PRs will follow to complete this task.

Related to:
- #4833
- #4891


mice may be due to chance, as it is hard to reconcile why the additional deficiency of Jnk3 in Jnk2


Furthermore, the destroying angel grows in mossy woods and lives symbiotically with spruce.

This orchid is an epiphyte which grows on the mossy trunks of several species of forest trees.

It is found in rocks, sandstone, and clay on mossy cliffs in elevations of 460 to 750 meters.

Anatomical evidence that the medial terminal nucleus of the accessory optic tract in mammals provides a visual mossy fiber input to the flocculus.

The dendrites are short with claw-like endings that form glomeruli to receive mossy fibers, similar to cerebellar granule cells.

This woodcreeper is found in mountains from 1000 m to the timberline in mossy, epiphyte-laden forest and adjacent semi-open woodland and clearings.

Breeding takes place in wet mossy areas near waterfalls and rapids.

The mossy and sticky filaments of the gall are clearly ineffective in preventing the entry of inquilines, predators, parasitoids and hyperparasitoids.

mossy


First, parents exerted some influence over the style of the performance and, as noted above, showmen seeking private commissions were required to make some modifications in the interests of respectability.

Instead, the values of the showmen changed.

Travelling showmen visit dozens of fairs each year, in many different authorities.

First, his style was that of the showman, with big parades a speciality.

The taxation on vehicles of the travelling showman is graded more highly for vehicles not travelling on pneumatic tyres.

There is one exception to this, on which arguments are sometimes based, which is that of the travelling fairground showman.

At the moment travelling showmen are harassed in practically every town.

Chadwick was a former showman with experience of the circus who used dramatic lighting techniques, and props to underline his radical messages.

showman


skin flick


hairdressing


However, these productive abilities alone might not suffice for the expression of internalized thoughts at later periods in the language acquisition process.

If the sustainable catch is fixed, improved technical efficiency is a benefit only if resources are freed from the catching sector for productive use elsewhere.

The second mover will under-invest because its samples have made less productive.

There are almost three times as many words with productive prefixes as with unproductive ones.

Arguably, productivity in word formation is a gradient notion, ranging from more to less productive cases.

The expression was not in the productive repertoire of either of the authors.

Counter-productive masters were put on the back burner (front burner did not become fashionable until the 1970s).

Redundancy statements serve to illustrate how features that were originally noncontrastive in a child's productive system may have gained distinctive status following treatment.

productive

productive land
Our discussion was not very productive.


at second hand


public int getNowYear() {
		return Integer.parseInt(sysNowTime[5]);
	}


public static void main(String[] args) {
        if (args.length >= 2 && args[0].endsWith("output:xml")) {
            File in = new File(args[1]);
            if (!in.exists()) {
                printUsage();
                System.exit(1);
            }
            File out;
            if (args.length >= 3) {
                out = new File(args[2]);
            } else {
                String filename = in.getName().replaceAll(".fsg", ".xml");
                if (filename.equals(in.getName())) {
                    filename += ".xml";
                }
                out = new File(filename);
            }
            try {
                FSGConverter fsgc = FSGConverter.getFSGConverter();
                fsgc.convertToXML(in, out);
            } catch (IOException e) {
                System.out.println("An error occured while converting the file.");
                e.printStackTrace();
                System.exit(1);
            }
        } else {
            printUsage();
            System.exit(1);
        }
    }


public void updateItem(HttpServletRequest arequest) throws Exception
   {
      for (int idx = this.getCount()-1; idx >= 0; idx--)
      {
         CNonadItem myitem = (CNonadItem) this.getItem(idx);
         String datid = "Ndate" + myitem.nonadmid;
         String serid = "NonSeries" + myitem.nonadmid;
         String reaid = "NonReason" + myitem.nonadmid;
         String nyrsid = "Nyrs" + myitem.nonadmid;
         String nmosid = "Nmos" + myitem.nonadmid;
         String nwksid = "Nwks" + myitem.nonadmid;
         String ndaysid = "Ndys" + myitem.nonadmid;

         String datstr = CParser.truncStr(arequest.getParameter(datid), CAppConsts.MaxLenDate);
         if (datstr == null || datstr.length() == 0)
         {
            this.delItem(idx);
            continue;
         }
         String serstr = arequest.getParameter(serid);
         String reastr = arequest.getParameter(reaid);

         myitem.setNonadmDate(datstr);
         myitem.seriescd = serstr;
         myitem.reasoncd = reastr;
         myitem.nageyears = getIntVal(arequest.getParameter(nyrsid));
         myitem.nagemonths = getIntVal(arequest.getParameter(nmosid));
         myitem.nageweeks = getIntVal(arequest.getParameter(nwksid));
         myitem.nagedays = getIntVal(arequest.getParameter(ndaysid));
      }

//      int nslot = Math.max(CAppConsts.NewSlotNonAdmin, CAppConsts.NumSlotNonAdmin - getCount());
      int nslot = (this.getCount()==0)?2:1;

      for (int idx = 0; idx < nslot; idx++)
      {
         CNonadItem myitem = new CNonadItem();
         String myid = "New" + Integer.toString(idx);
         String datid = "Ndate" + myid;
         String serid = "NonSeries" + myid;
         String reaid = "NonReason" + myid;
         String nyrsid = "Nyrs" + myid;
         String nmosid = "Nmos" + myid;
         String nwksid = "Nwks" + myid;
         String ndaysid = "Ndys" + myid;

         String datstr = CParser.truncStr(arequest.getParameter(datid), CAppConsts.MaxLenDate);
         if (datstr == null || datstr.length() == 0) continue; //no data here
         String serstr = arequest.getParameter(serid);
         String reastr = arequest.getParameter(reaid);

         myitem.nonadmid = this.makeNewId("nad", 6);
         myitem.setNonadmDate(datstr);
         myitem.seriescd = serstr;
         myitem.reasoncd = reastr;
         myitem.nageyears = getIntVal(arequest.getParameter(nyrsid));
         myitem.nagemonths = getIntVal(arequest.getParameter(nmosid));
         myitem.nageweeks = getIntVal(arequest.getParameter(nwksid));
         myitem.nagedays = getIntVal(arequest.getParameter(ndaysid));

         this.addItem(myitem.nonadmid, myitem);
      }
   }


public List<Player> getPlayersInRegion() {
		List<Player> rangeList = new ArrayList<Player>();
		for (Player p : FFA.getServer().getOnlinePlayers()) {
			Location loc = p.getLocation();
			Vector vec = new Vector(loc.getBlockX(), loc.getBlockY(), loc.getBlockZ());
			if (this.contains(p.getWorld(), vec)) {
				rangeList.add(p);
			}
		}
		return rangeList;
	}


@Override
	public List<Framedata> translateFrame( ByteBuffer buffer ) throws InvalidDataException {
		buffer.mark();
		List<Framedata> frames = super.translateRegularFrame( buffer );
		if( frames == null ) {
			buffer.reset();
			frames = readyframes;
			readingState = true;
			if( currentFrame == null )
				currentFrame = ByteBuffer.allocate( 2 );
			else {
				throw new InvalidFrameException();
			}
			if( buffer.remaining() > currentFrame.remaining() ) {
				throw new InvalidFrameException();
			} else {
				currentFrame.put( buffer );
			}
			if( !currentFrame.hasRemaining() ) {
				if( Arrays.equals( currentFrame.array(), closehandshake ) ) {
					frames.add( new CloseFrameBuilder( CloseFrame.NORMAL ) );
					return frames;
				}
				else{
					throw new InvalidFrameException();
				}
			} else {
				readyframes = new LinkedList<Framedata>();
				return frames;
			}
		} else {
			return frames;
		}
	}


public Object execIdCall(IdFunctionObject f, Context cx, Scriptable scope,
                             Scriptable thisObj, Object[] args)
    {
        if (!f.hasTag(FUNCTION_TAG)) {
            return super.execIdCall(f, cx, scope, thisObj, args);
        }
        int id = f.methodId();
        switch (id) {
          case Id_constructor:
            return jsConstructor(cx, scope, args);

          case Id_toString: {
            BaseFunction realf = realFunction(thisObj, f);
            int indent = ScriptRuntime.toInt32(args, 0);
            return realf.decompile(indent, 0);
          }

          case Id_toSource: {
            BaseFunction realf = realFunction(thisObj, f);
            int indent = 0;
            int flags = Decompiler.TO_SOURCE_FLAG;
            if (args.length != 0) {
                indent = ScriptRuntime.toInt32(args[0]);
                if (indent >= 0) {
                    flags = 0;
                } else {
                    indent = 0;
                }
            }
            return realf.decompile(indent, flags);
          }

          case Id_apply:
          case Id_call:
            return ScriptRuntime.applyOrCall(id == Id_apply,
                                             cx, scope, thisObj, args);
        }
        throw new IllegalArgumentException(String.valueOf(id));
    }


public void showPage(int page) {
		if (!hasPage(page))
			page = 1;
		int start = elementsPerPage * (page - 1);
		System.out.println("====[" + title + " Page " + page + "/" + pages + titleContainer + "]====");
		if (subtitle != null && !subtitle.isEmpty())
			System.out.println(subtitle);
		for (int i = start; i < start + elementsPerPage; i++) {
			if (i + 1 > elements.size())
				break;
			System.out.println(elements.get(i));
		}
	}


public int largestRectangleArea(int[] height) {
        Stack<Integer> stk = new Stack<Integer>();
        int index = 0;
        int result = 0;
        while (index<height.length) {
            if (stk.isEmpty()||height[stk.peek()]<=height[index]) {
                stk.push(index);
                index++;
            }
            else {
                int t = stk.pop();
                result = Math.max(result, height[t]*(stk.isEmpty()?index:index-stk.peek()-1));
            }
        }
        while (!stk.isEmpty()) {
            int t = stk.pop();
            result = Math.max(result, height[t]*(stk.isEmpty()?height.length:height.length-stk.peek()-1));
        }
        return result;
    }


private String getHeaderCaseInsensitive(String name) {
            Validate.notNull(name, "Header name must not be null");
            // quick evals for common case of title case, lower case, then scan for mixed
            String value = headers.get(name);
            if (value == null)
                value = headers.get(name.toLowerCase());
            if (value == null) {
                Map.Entry<String, String> entry = scanHeaders(name);
                if (entry != null)
                    value = entry.getValue();
            }
            return value;
        }


public static void startupPropositions() {
    { Object old$Module$000 = Stella.$MODULE$.get();
      Object old$Context$000 = Stella.$CONTEXT$.get();

      try {
        Native.setSpecial(Stella.$MODULE$, Stella.getStellaModule("/LOGIC", Stella.$STARTUP_TIME_PHASE$ > 1));
        Native.setSpecial(Stella.$CONTEXT$, ((Module)(Stella.$MODULE$.get())));
        if (Stella.currentStartupTimePhaseP(2)) {
          _StartupPropositions.helpStartupPropositions1();
          _StartupPropositions.helpStartupPropositions2();
          _StartupPropositions.helpStartupPropositions3();
          _StartupPropositions.helpStartupPropositions4();
        }
        if (Stella.currentStartupTimePhaseP(4)) {
          _StartupPropositions.helpStartupPropositions5();
        }
        if (Stella.currentStartupTimePhaseP(5)) {
          _StartupPropositions.helpStartupPropositions6();
        }
        if (Stella.currentStartupTimePhaseP(6)) {
          Stella.finalizeClasses();
        }
        if (Stella.currentStartupTimePhaseP(7)) {
          _StartupPropositions.helpStartupPropositions7();
          _StartupPropositions.helpStartupPropositions8();
          _StartupPropositions.helpStartupPropositions9();
          _StartupPropositions.helpStartupPropositions10();
          _StartupPropositions.helpStartupPropositions11();
          _StartupPropositions.helpStartupPropositions12();
        }
        if (Stella.currentStartupTimePhaseP(8)) {
          Stella.finalizeSlots();
          Stella.cleanupUnfinalizedClasses();
        }
        if (Stella.currentStartupTimePhaseP(9)) {
          Stella_Object.inModule(((StringWrapper)(Stella_Object.copyConsTree(StringWrapper.wrapString("LOGIC")))));
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT ZERO-VARIABLES-VECTOR VARIABLES-VECTOR (NEW VARIABLES-VECTOR :ARRAY-SIZE 0) :DOCUMENTATION \"Save space by structure-sharing zero-length variable vectors.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *PARTIAL-SUPPORT-COUNTER* INTEGER 0)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *OPERATOR-NAME-TO-SURROGATE-TABLE* (PROPERTY-LIST OF KEYWORD SURROGATE) (NEW PROPERTY-LIST :THE-PLIST (BQUOTE (:AND /PL-KERNEL-KB/@AND :OR /PL-KERNEL-KB/@OR :NOT /PL-KERNEL-KB/@NOT :FORALL /PL-KERNEL-KB/@FORALL :EXISTS /PL-KERNEL-KB/@EXISTS :EQUIVALENT /PL-KERNEL-KB/@EQUIVALENT :FAIL /PL-KERNEL-KB/@FAIL :COLLECT-INTO /PL-KERNEL-KB/@COLLECT-INTO :IMPLIES /PL-KERNEL-KB/@SUBSET-OF))) :DOCUMENTATION \"Maps names of KIF operators to relational surrogates.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *LOGIC-MODULE* MODULE (GET-STELLA-MODULE \"LOGIC\" TRUE) :PUBLIC? TRUE)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *PL-KERNEL-MODULE* MODULE (GET-STELLA-MODULE \"PL-KERNEL\" TRUE) :PUBLIC? TRUE)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *NOW-TIMESTAMP* TIMESTAMP 0 :DOCUMENTATION \"The NOW time stamp is incremented whenever a series\nof one or more updates is followed by a query.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *LAST-KB-ACTION* KEYWORD :UPDATE-PROPOSITION :DOCUMENTATION \"Records whether the last KB access was a query or\nan update.  Used to determine when to increment the NOW time stamp\ncounter.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL BOTTOM LOGIC-OBJECT NULL :PUBLIC? TRUE :DOCUMENTATION \"The undefined individual.  Denotes the non-existence of\nan individual in whatever slot it occupies.\")");
          Logic.BOTTOM = Logic.createSkolem(null, Logic.SYM_LOGIC_BOTTOM);
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *EVALUATIONMODE* KEYWORD :EXTENSIONAL-ASSERTION :DOCUMENTATION \"Indicates the context for evaluating a proposition.  One\nof :DESCRIPTION, :INTENSIONAL-ASSERTION, or :EXTENSIONAL-ASSERTION.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *CLIPPINGENABLED?* BOOLEAN TRUE :DOCUMENTATION \"When enabled, slot-value assertions can be retracted\nby later conflicting assertions.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *NATURALDEDUCTIONMODE?* BOOLEAN TRUE :DOCUMENTATION \"When enabled, blocks normalizations that significantly\nchange the behavior of inference rules.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *CONTEXT-DEPENDENT-SEARCH-MODE?* BOOLEAN FALSE :DOCUMENTATION \"Signals that we are performing search across multiple\ncontexts.  Used to disable retraction from collections, since that increases\nthe overhead of the context mechanism.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *COMPUTEDQUERY?* BOOLEAN FALSE :DOCUMENTATION \"Used to signal 'ground-value-of' that it can\ncall 'bound-to' safely.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *SKOLEM-ID-COUNTER* INTEGER 0)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *VARIABLEIDCOUNTER* INTEGER 0)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *FREESKOLEMS* CONS NULL :DOCUMENTATION \"Cons-list of top-level existentially-quantified skolems.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT TRUE-TRUTH-VALUE TRUTH-VALUE (NEW TRUTH-VALUE :POLARITY :TRUE :STRENGTH :STRICT :POSITIVE-SCORE 1.0) :DOCUMENTATION \"Value representing TRUE.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT FALSE-TRUTH-VALUE TRUTH-VALUE (NEW TRUTH-VALUE :POLARITY :FALSE :STRENGTH :STRICT :POSITIVE-SCORE -1.0) :DOCUMENTATION \"Value representing FALSE.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT DEFAULT-TRUE-TRUTH-VALUE TRUTH-VALUE (NEW TRUTH-VALUE :POLARITY :TRUE :STRENGTH :DEFAULT :POSITIVE-SCORE 0.8) :DOCUMENTATION \"Value representing DEFAULT-TRUE.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT DEFAULT-FALSE-TRUTH-VALUE TRUTH-VALUE (NEW TRUTH-VALUE :POLARITY :FALSE :STRENGTH :DEFAULT :POSITIVE-SCORE -0.8) :DOCUMENTATION \"Value representing DEFAULT-FALSE.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT UNKNOWN-TRUTH-VALUE TRUTH-VALUE (NEW TRUTH-VALUE :POLARITY :UNKNOWN) :DOCUMENTATION \"Value representing UNKNOWN.  Needed for those cases\nwhere we need a non-NULL truth value to represents UNKNOWN.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT INCONSISTENT-TRUTH-VALUE TRUTH-VALUE (NEW TRUTH-VALUE :POLARITY :INCONSISTENT :STRENGTH :STRICT) :DOCUMENTATION \"Value representing a contradiction.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *SUPPRESSUNTYPEDTYPEERROR?* BOOLEAN FALSE :DOCUMENTATION \"Used by 'safe-logical-type' to ask for a type\nwithout signalling an error if none exists.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *EQUIVALENCE-COLLECTIONS?* BOOLEAN TRUE :DOCUMENTATION \"Experiment with equality reasoning on collections.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT TRUE-PROPOSITION PROPOSITION (NEW PROPOSITION :KIND :CONSTANT :OPERATOR @TRUE :ARGUMENTS (NEW ARGUMENTS-VECTOR :ARRAY-SIZE 0) :TRUTH-VALUE TRUE-TRUTH-VALUE :HOME-CONTEXT *PL-KERNEL-MODULE*))");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT FALSE-PROPOSITION PROPOSITION (NEW PROPOSITION :KIND :CONSTANT :OPERATOR @FALSE :ARGUMENTS (NEW ARGUMENTS-VECTOR :ARRAY-SIZE 0) :TRUTH-VALUE FALSE-TRUTH-VALUE :HOME-CONTEXT *PL-KERNEL-MODULE*))");
          Logic.SGT_STELLA_TRUE.surrogateValue = Logic.TRUE_PROPOSITION;
          Logic.SGT_STELLA_FALSE.surrogateValue = Logic.FALSE_PROPOSITION;
          Proposition.findDuplicateProposition(Logic.TRUE_PROPOSITION);
          Proposition.findDuplicateProposition(Logic.FALSE_PROPOSITION);
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFCONSTANT ANONYMOUS-VARIABLE-NAME SYMBOL (QUOTE ?) :DOCUMENTATION \"Variables with name 'ANONYMOUS-VARIABLE-NAME' are considered\nanonymous, and are not assumed to be identical to any other variable also named\n'ANONYMOUS-VARIABLE-NAME'.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *MISSING-KEY-VALUE-LIST* KEY-VALUE-LIST (NEW KEY-VALUE-LIST) :DOCUMENTATION \"Represents a key-value list that should never be used.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *AUTOMATICINSTANCETABLE* (KEY-VALUE-LIST OF SYMBOL LOGIC-OBJECT) *MISSING-KEY-VALUE-LIST* :DOCUMENTATION \"Used by 'evaluate-automatic-instance' to\nrecord current bindings of automatic instance symbols.\")");
          HookList.addHook(Stella.$REDEFINE_RELATION_HOOKS$, Logic.SYM_LOGIC_TRANSFER_LOGIC_INFORMATION_FROM_RELATION_HOOK);
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *DEFAULTCREATIONTYPE* SURROGATE NULL)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *STRUCTURED-OBJECTS-INDEX* (KEY-VALUE-MAP OF INTEGER-WRAPPER (LIST OF CONTEXT-SENSITIVE-OBJECT)) (NEW KEY-VALUE-MAP) :DOCUMENTATION \"Contains a table of propositions and descriptions, indexed by a\nstructure hash code which might be shared by different objects.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *DONT-CHECK-FOR-DUPLICATE-PROPOSITIONS?* BOOLEAN FALSE :DOCUMENTATION \"If TRUE never check for the existence of duplicate\npropositions when building a new proposition.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *VISITEDUNFASTENEDDEFININGPROPOSITIONS* LIST NULL :DOCUMENTATION \"Used by 'recursively-fasten-down-propositions'.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *AUTO-COERCE-PROPOSITIONAL-ARGUMENTS?* BOOLEAN FALSE :DOCUMENTATION \"If TRUE, automatically coerce propositional arguments of a\nproposition, even if the corresponding argument type of the hosting relation\ndoesn't indicate that.\" :PUBLIC? TRUE)");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *TYPE-CHECK-POLICY* KEYWORD :AUTOMATICALLY-FIX-TYPE-VIOLATIONS :DOCUMENTATION \"Three policies are implemented:\n   :AUTOMATICALLY-FIX-TYPE-VIOLATIONS asserts missing types to fix type\n       violations (default),\n   :REPORT-TYPE-VIOLATIONS complains about missing or incorrect types,\n   :SIGNAL-TYPE-VIOLATIONS throws exception for missing or incorrect types, and\n   :IGNORE-TYPE-VIOLATIONS which disables all type checking.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFSPECIAL *TYPECHECKMODE* KEYWORD :POST-TYPE-VIOLATIONS :DOCUMENTATION \"Controls the behavior of the type-checking\nroutines in the event that a type-check fails.  Options are:\n  :POST-TYPE-VIOLATIONS              push offending proposition onto queue,\n  :REPORT-TYPE-VIOLATIONS            print occasions of failed type checks,\n  :SIGNAL-TYPE-VIOLATIONS            throw exception for failed type checks,\n  :AUTOMATICALLY-FIX-TYPE-VIOLATIONS assert missing types on propositions, and\n  :IGNORE-TYPE-VIOLATIONS            don't perform any type checking at all.\")");
          Stella.defineStellaGlobalVariableFromStringifiedSource("(DEFGLOBAL *CHECK-TYPES-AGENDA* (VECTOR-SEQUENCE OF CHECK-TYPES-RECORD) (NEW VECTOR-SEQUENCE :ARRAY-SIZE 4) :DOCUMENTATION \"List of propositions that have failed a type check,\nbut might pass once finalization is complete.\")");
          Symbol.registerNativeName(Logic.SYM_STELLA_ASSERT, Logic.KWD_CPP, Logic.KWD_FUNCTION);
          HookList.addHook(Stella.$DEFINE_MODULE_HOOKS$, Logic.SYM_LOGIC_INTRODUCE_MODULE);
          HookList.addHook(Stella.$CLEAR_MODULE_HOOKS$, Logic.SYM_LOGIC_CLEAR_LOGIC_MODULE_HOOK);
          HookList.addHook(Stella.$DESTROY_CONTEXT_HOOKS$, Logic.SYM_LOGIC_DESTROY_LOGIC_CONTEXT_HOOK);
        }

      } finally {
        Stella.$CONTEXT$.set(old$Context$000);
        Stella.$MODULE$.set(old$Module$000);
      }
    }
  }


(CNN) -- Shelling hit areas near two key cities in eastern Ukraine on Sunday morning, intensifying fears that a ceasefire that took effect less than two days ago may be falling apart. Why is the ceasefire under strain? A variety of fighting factions in the conflict zone -- on both sides -- may not fall directly under a military chain of command. The pro-Russian rebels are mostly volunteer militias; fighting against them on the Ukrainian side are at least some far-right nationalist militias. Controlling these groups is difficult and some may have different aims, including sabotaging the truce. At this point it's been nearly impossible to figure out who's doing the firing and why. The conditions of the ceasefire agreement don't help either. The conditions are vague and at this point there doesn't seem to be an effective mechanism in place inside the conflict zone to monitor and enforce the agreement. Why can't the two sides' leaders control their forces? It's unclear if Kiev has control over all of the fighting forces in eastern Ukraine. Some of the volunteer militias fighting alongside Ukrainian soldiers are far-right nationalists who've been critical of the current government in Kiev, but they're still fighting because they feel Ukraine is under attack by Russia. And who controls the pro-Russian rebels? Is it the local commanders? Is it Russian President Vladimir Putin? None of that is clear. Which side has the most to gain from the truce? If the truce leads to good-faith negotiations and a compromise, then both sides can gain. A compromise could look like something like this: The pro-Russian region of Donbas gets autonomy and self-determination under a federalized Ukrainian government, and in return the rebels drop their demand for independence and Kiev gets to protect Ukraine's sovereignty and territorial integrity. There are elements on both sides that don't want a compromise, and they could certainly have the potential of undermining the truce. What happens next? We wait to see if the overnight shellings and firings are an anomaly or if they're a sign of more violence and more fighting. If the ceasefire sticks, both sides have agreed to hold talks that will address the core issues and demands on both sides that are still unresolved -- including the disarming of the rebels, a guarantee of self-determination for the pro-Russian Donbas region, the fate of Russian-annexed Crimea, constitutional reform, and a solution to the humanitarian crisis in the conflict zone. Obviously if the fighting continues, all bets are off. What effect does this have on the rest of the world? What happens in the coming days will determine the next move by NATO and Western leaders. If the ceasefire falls apart, the West will likely turn up the pressure by following through with sanctions and bolstering Western forces in NATO's Baltic-member states. Moscow has already threatened to respond if that happens. The bottom line is, the conflict will escalate and so will the prospects of a regional conflict -- although at this point that seems unlikely.

Shelling in eastern Ukraine raises fears that a ceasefire may fall apart .
Reza Sayah: It's been nearly impossible to say who's doing firing and why .
It's unclear who has control over fighting forces on either side in east Ukraine .
There are elements on both sides that don't want a compromise .


(CNN) -- Remember when oil giant BP paid billions of dollars to clean up its own mess from the 2010 Gulf oil spill? Then the company agreed to plead guilty to criminal charges and pay an unprecedented $4 billion fine for the spill. In the latest development, Wednesday, BP was essentially told to stand in the corner when it comes to winning new federal contracts -- a potentially huge blow to the company's bottom line. And it's still likely on the hook for civil penalties the federal government hopes to recover for environmental damages. So, is BP being punished enough for the deadly and destructive Deepwater Horizon spill, or too much? It depends on who you ask. This matters. BP says it supports nearly a quarter of a million American jobs and has invested $52 billion in the United States since 2007. Experts suggest punishments ranging from massive fines to banning the company from offshore drilling. However this all shakes out, it likely will send a powerful message to the industry. Families of killed rig workers react to BP settlement . First, here's what happened Wednesday: . The U.S. Environmental Protection Agency said it will block BP from new government contracts due to its "lack of business integrity" stemming from the 2010 explosion and oil spill. It was a tragedy that killed 11 oil workers and dumped 205 million gallons of oil into the Gulf in a slow-moving, 85-day national drama that prosecutors called the worst environmental disaster in U.S. history. This month hasn't been good to BP. The decision follows the London-based company's agreement November 15 to plead guilty to criminal charges including manslaughter, obstruction of Congress and environmental violations related to the explosion and spill -- an agreement that still requires a judge's approval. What it means . Wednesday's EPA decision means BP won't be allowed to compete for new contracts, grants or other business until the company shows it meets federal business standards. As of February, BP had $9 billion in contracts to supply the U.S. government with fuel, most of it destined for military users, said Christine Tiscareno, a London-based analyst with S&P Capital IQ. Those contracts aren't affected by the EPA decision, the agency said. BP said it's already filed paperwork to get back in good graces with the EPA, said the agency is already preparing an administrative agreement to lift the suspension and highlighted what the company says it's done to rectify the situation. "As BP's submissions to the EPA have made clear, the company has made significant enhancements since the accident," the company said in a statement. "The company launched an internal investigation immediately after the accident, publicly released the results, and has been implementing all 26 of the investigation's recommendations." What's ahead . The next big step in the case comes in February, when a civil trial is expected to begin over the government's demand for civil damages under the Clean Water Act. That case could draw huge new fines for the company. The disaster, in addition to claiming 11 lives, caused disruption to countless residents and businesses, many of whom counted on the Gulf for their livelihoods, whether in the oil fields, fishing grounds or tourist beaches. For some, including environmentalists, the company couldn't be fined enough or spend enough money to get back on their good side. "Well, you know, we're discussing the worst environmental disaster in U.S. history," said Matt Dundas, acting campaign director at the environmental group Oceana. "The punishment needs to fit the crime." But, he said, the record $4 billion fine BP is expected to pay as a result of the criminal case is a tenth of the maximum it could have faced, making it small consolation to BP's harshest critics. The fine, while a record for the U.S. justice system, represents less than three months worth of profits for BP. The company reported $5.2 billion in profits in the quarter that ended September 30. Dundas is now rooting for the maximum possible verdict in the civil case -- $50 billion in fines. But he says there's only one punishment that would really get BP and other oil companies to sit up and take steps to prevent what Dundas says is the overwhelming risk of another large oil spill: being kicked out of the game entirely. "I think a terrific minimum would be to bar them from offshore drilling," he said. "That's really the only standard that's going to get oil companies to pay attention." The head of a trade association representing BP and other oil and gas operators in Louisiana and the Gulf of Mexico said Wednesday that BP has already complied with government requirements. "BP has done everything that the federal government has asked and more. ... This new EPA designation is another layer," said Chris John, president of the Louisiana Mid-Continent Oil and Gas Association. "How much is enough, when the economy in America is trying to dig out of the recession, and energy ... is a very important component to getting Americans back to work?" The EPA's decision could have a negative impact beyond BP, he said, affecting other companies in the industry who have also complied with regulators. "The companies that operate out in the Gulf of Mexico, they can go anywhere, they're global. They can drill anywhere in the world," John said. "They have chosen to come back to the Gulf in the past few years. ... This additional statement and designation by the EPA has to play into their decision of where to put their capital." BP didn't directly address whether it believes it has paid enough for the disaster, but noted Wednesday that it has spent $14 billion in response and cleanup costs, $1 million on restoration projects and $9 billion in compensation and expects to spend $7.8 billion more to resolve the majority of outstanding claims. Owning up . The company also pointed out it has acknowledged criminal responsibility, carried out recommendations made by investigators who studied the incident and has taken actions to strengthen safety and drilling standards that exceed federal regulations. Tiscareno, the oil analyst, said that while the costs to BP have been steep, they are bearable for the company. The big question is what happens in February, when the civil case goes to trial. Tiscareno said she expects BP, which has taken a long-term approach to recovering from the spill, will recover financially unless the civil trial nets an "irrational" decision. A huge verdict could pose problems for BP, Tiscareno said, but she said the government can't go too far in gutting the firm because of all the jobs and tax revenues the company provides. BP has invested $52 billion in the U.S. in the last five years, employs 23,000 U.S. residents and indirectly supports nearly 250,000 jobs, the company said Wednesday. "The U.S. needs BP as much as BP needs the U.S.," Tiscareno said. CNN's Kendra Wates and Catherine E. Shoichet contributed to this report.

NEW: Trade association: "BP has done everything that the federal government has asked"
Two years after the oil spill, experts weigh in on possible punishments for BP .
BP says it expects the temporary ban will be lifted soon .
The EPA says the oil giant showed a "lack of business integrity"


Adel Taarabt appeared for QPR for the first time since his public fall-out with Harry Redknapp in Monday afternoon's reserve game against Ipswich Town. The Moroccan's days at Loftus Road appeared to be numbered after his extraordinary spat with Redknapp last month. Taarabt claimed his manager never takes a training session after Redknapp had accused the midfielder of being 'three stone overweight' following the 3-2 defeat to Liverpool. VIDEO Scroll down to watch Harry Redknapp say  Taarabt's camp have told blatant lies . Adel Taarabt has played just three first-team games for QPR this season after returning from loan at AC Milan . Taarabt laughed off claims by QPR manager Harry Redknapp that the midfielder was 'three stone overweight' The 25-year-old was due to play for the club's Under 21s last week, but wasn't included in the draw against Swansea. However, Taarabt came through 90 minutes in the 3-0 win against Ipswich Town as he tries to prove his fitness to his boss. The former Spurs man, who has been hampered by an ankle injury, has been limited to just three appearances for the R's this season after returning from a loan spell at AC Milan. Taarabt hit back at Redknapp during an exclusive interview with Sportsmail last month . Taarabt spent last campaign on loan at Fulham and then Serie A side AC Milan .

Moroccan played 90 minutes for Under 21s in 3-0 win against Ipswich Town .
Taarabt has made just three first-team appearances for the R's this season .
Midfielder was accused of being 'overweight' by manager Harry Redknapp .
Taarabt hit back and claimed Redknapp never took training sessions .


By . Wills Robinson . PUBLISHED: . 11:14 EST, 6 March 2014 . | . UPDATED: . 12:19 EST, 6 March 2014 . Members of the feminist punk group Pussy Riot have been attacked by a gang of men inside a Russian McDonalds. Nadya Tolokonnikova and Masha Alyokhina were covered in paint while they sat and had breakfast in the city of Nizhny Novgorod. The pair, who were jailed for hooliganism in 2012, also had pepper spray, green antispetic and metal thrown in their faces during the assault. Scroll down for video . Attacked: Nadya Tolokonnikova and Masha Alyokhina were covered in paint during the assault while they sat and had breakfast in the city of Nizhny Novgorod . Distressed: The pair spoke directly into the camera after the incident . They were visiting the area to inspect a local prison with Peter Verzilov and members of NGO 'Zone of the Rights'. Tolonnikova allegedly left the restaurant with a 'cut forehead' and the group have complained to the police. She was seen getting up from her seat and clutching her face during the incident. while the gang of men continually shouted abuse. When the attack eventually stopped, one of the men held up a sign to the camera while another was seen shouting down the lens. At around 7.00 am the group barged into the fast food restaurant where the musicians and their crew were eating. Gang: They were covered in paint, antiseptic and pepper spray by the group of men . Prolonged: The gang shouted abuse while they continued to launch the liquids at the members of the feminist band . Close: One of the attackers faced up to Nadya Tolokonnikova, preparing to throw something at her . Abuse: The individuals continued to shout at the musicians while a crowd built up behind them . It is not the first time the group have been targeted by members of the public. At the start of the Sochi winter . Olympics, members of the group were whipped and tackled to the ground by Cossacks as . they sang in protest. The . band started to perform an anti-Putin song under a sign advertising the . Winter Olympics. But singer Tolokonnikova was brutally whipped . in the attack and the guitarist was left bloodied and bruised. Last month, Tolokonnikova was filmed saying the anti-government protests in Ukraine were an 'inspiation'. She was also among those arrested during protests outside a Moscow courthouse. Tolokonnikova and Alyokhina were released from seperate prisons on December 23, 2013 after the Russian State Duma approved an amnesty. Speech: An attacker spoke into a camera owned by the NGO 'Zone of the Rights' in the aftermath of the assault . Slogan: One man holds up a sign to the camera before the group leave the restaurant .

Nadya Tolokonnikova and Masha Alyokhina were also pepper sprayed .
The pair were in city of Nizhny Novgorod visiting a local prison .
Alyokhina reportedly left the fast food restaurant with a 'cut forehead'


Context: Stockit offers an online market place for the timber and paper industry. Mills and importers can make business with any device at any time all over the world. Stockit enables global reach with a few clicks. Mills can sell stock lots to Stockit's entire user base or create closed auctions for current customers. Buyers can find goods of their choice from high quality mills and buy it delivered to their preferred port without hidden costs.StockIt :: Home. the marketplace for forest products. Timber. Sell or buy timber. Stockit Timber connects Saw mills and timber buyers. Find the best stock lists online and place your bid. As a member and frequent buyer you also get exclusive access and invitations to your favorite mills stocks before everyone else. Membership is free of charge. Paper. Sell or buy paper. Stockit Paper connects paper mills and paper buyers. Find the best stock lists online and place your bid. As a member and frequent buyer you also get exclusive access and invitations to your favorite mills stocks before everyone else. Membership is free of charge. Forest. Sell or buy forest or logs. Stockit Forest connects Nordic (Sweden, Finland and Norway) forest owners with buyers of logs. Register on Stockit to get the best offers. Get in direct contact and save valuable time and money. Membership is free of charge.

Question: What is the name of this company?


Context: UPLAND CONSTRUCTION is a construction company based out of 108 N 5th Ave, Mills, Wyoming, United States.

Question: What is the name of this company?


from django.conf import settings
from django.core.paginator import Paginator, InvalidPage, EmptyPage
from django.http import HttpResponse

from tardis.tardis_portal.auth import decorators as authz
from tardis.tardis_portal.models import Dataset
from tardis.tardis_portal.shortcuts import get_experiment_referer
from tardis.tardis_portal.shortcuts import render_response_index

@authz.dataset_access_required
def view_full_dataset(request, dataset_id):
    """Displays a MX Dataset and associated information.

    Shows a full (hundreds of images) dataset its metadata and a list
    of associated files with the option to show metadata of each file
    and ways to download those files.  With write permission this page
    also allows uploading and metadata editing.

    Settings for this view:
    INSTALLED_APPS += ("tardis.apps.mx_views",)
    DATASET_VIEWS = [("http://synchrotron.org.au/views/dataset/full",
                      "tardis.apps.mx_views.views.view_full_dataset"),]

    """
    dataset = Dataset.objects.get(id=dataset_id)

    def get_datafiles_page():
        # pagination was removed by someone in the interface but not here.
        # need to fix.
        pgresults = 100

        paginator = Paginator(dataset.datafile_set.all(), pgresults)

        try:
            page = int(request.GET.get('page', '1'))
        except ValueError:
            page = 1

        # If page request (9999) is out of range, deliver last page of results.

        try:
            return paginator.page(page)
        except (EmptyPage, InvalidPage):
            return paginator.page(paginator.num_pages)

    display_images = dataset.get_images()
    image_count = len(display_images)
    if image_count > 4:
        # take 4 evenly spaced images from the set
        display_images = display_images[0::image_count / 4][:4]

    upload_method = getattr(settings, "UPLOAD_METHOD", "uploadify")

    c = {
        'dataset': dataset,
        'datafiles': get_datafiles_page(),
        'parametersets': dataset.getParameterSets()
                                .exclude(schema__hidden=True),
        'has_download_permissions':
            authz.has_dataset_download_access(request, dataset_id),
        'has_write_permissions':
            authz.has_dataset_write(request, dataset_id),
        'from_experiment': \
            get_experiment_referer(request, dataset_id),
        'other_experiments': \
            authz.get_accessible_experiments_for_dataset(request, dataset_id),
        'display_images': display_images,
        'upload_method': upload_method,
        'default_organization':
            getattr(settings, 'DEFAULT_ARCHIVE_ORGANIZATION', 'classic'),
        'default_format':
            getattr(settings, 'DEFAULT_ARCHIVE_FORMATS', ['tgz', 'tar'])[0]
    }
    return HttpResponse(render_response_index(
        request, 'mx_views/view_full_dataset.html', c))


"""Pure python implementation of the binary Tokyo Tyrant 1.1.17 protocol

Tokyo Cabinet <http://tokyocabinet.sourceforge.net/> is a "super hyper ultra
database manager" written and maintained by Mikio Hirabayashi and released
under the LGPL.

Tokyo Tyrant is the de facto database server for Tokyo Cabinet written and
maintained by the same author. It supports a REST HTTP protocol, memcached,
and its own simple binary protocol. This library implements the full binary
protocol for the Tokyo Tyrant 1.1.17 in pure Python as defined here::

    http://tokyocabinet.sourceforge.net/tyrantdoc/

Typical usage is with the PyTyrant class which provides a dict-like wrapper
for the raw Tyrant protocol::

    >>> import pytyrant
    >>> t = pytyrant.PyTyrant.open('127.0.0.1', 1978)
    >>> t['__test_key__'] = 'foo'
    >>> t.concat('__test_key__', 'bar')
    >>> print t['__test_key__']
    foobar
    >>> del t['__test_key__']

"""
import math
import socket
import struct
import UserDict

__version__ = '1.1.17'

__all__ = [
    'Tyrant', 'TyrantError', 'PyTyrant',
    'RDBMONOULOG', 'RDBXOLCKREC', 'RDBXOLCKGLB',
]

class TyrantError(Exception):
    pass

DEFAULT_PORT = 1978
MAGIC = 0xc8

RDBMONOULOG = 1 << 0
RDBXOLCKREC = 1 << 0
RDBXOLCKGLB = 1 << 1

class C(object):
    """
    Tyrant Protocol constants
    """
    put = 0x10
    putkeep = 0x11
    putcat = 0x12
    putshl = 0x13
    putnr = 0x18
    out = 0x20
    get = 0x30
    mget = 0x31
    vsiz = 0x38
    iterinit = 0x50
    iternext = 0x51
    fwmkeys = 0x58
    addint = 0x60
    adddouble = 0x61
    ext = 0x68
    sync = 0x70
    vanish = 0x71
    copy = 0x72
    restore = 0x73
    setmst = 0x78
    rnum = 0x80
    size = 0x81
    stat = 0x88
    misc = 0x90

def _t0(code):
    return [chr(MAGIC) + chr(code)]

def _t1(code, key):
    return [
        struct.pack('>BBI', MAGIC, code, len(key)),
        key,
    ]

def _t1FN(code, func, opts, args):
    outlst = [
        struct.pack('>BBIII', MAGIC, code, len(func), opts, len(args)),
        func,
    ]
    for k in args:
        outlst.extend([struct.pack('>I', len(k)), k])
    return outlst

def _t1R(code, key, msec):
    return [
        struct.pack('>BBIQ', MAGIC, code, len(key), msec),
        key,
    ]

def _t1M(code, key, count):
    return [
        struct.pack('>BBII', MAGIC, code, len(key), count),
        key,
    ]

def _tN(code, klst):
    outlst = [struct.pack('>BBI', MAGIC, code, len(klst))]
    for k in klst:
        outlst.extend([struct.pack('>I', len(k)), k])
    return outlst

def _t2(code, key, value):
    return [
        struct.pack('>BBII', MAGIC, code, len(key), len(value)),
        key,
        value,
    ]

def _t2W(code, key, value, width):
    return [
        struct.pack('>BBIII', MAGIC, code, len(key), len(value), width),
        key,
        value,
    ]

def _t3F(code, func, opts, key, value):
    return [
        struct.pack('>BBIIII', MAGIC, code, len(func), opts, len(key), len(value)),
        func,
        key,
        value,
    ]

def _tDouble(code, key, integ, fract):
    return [
        struct.pack('>BBIQQ', MAGIC, code, len(key), integ, fract),
        key,
    ]

def socksend(sock, lst):
    sock.sendall(''.join(lst))

def sockrecv(sock, bytes):
    d = ''
    while len(d) < bytes:
        c = sock.recv(min(8192, bytes - len(d)))
        if not c:
            raise TyrantError('Connection closed')
        d += c
    return d

def socksuccess(sock):
    fail_code = ord(sockrecv(sock, 1))
    if fail_code:
        raise TyrantError(fail_code)

def socklen(sock):
    return struct.unpack('>I', sockrecv(sock, 4))[0]

def socklong(sock):
    return struct.unpack('>Q', sockrecv(sock, 8))[0]

def sockstr(sock):
    return sockrecv(sock, socklen(sock))

def sockdouble(sock):
    intpart, fracpart = struct.unpack('>QQ', sockrecv(sock, 16))
    return intpart + (fracpart * 1e-12)

def sockstrpair(sock):
    klen = socklen(sock)
    vlen = socklen(sock)
    k = sockrecv(sock, klen)
    v = sockrecv(sock, vlen)
    return k, v

class PyTyrant(object, UserDict.DictMixin):
    """
    Dict-like proxy for a Tyrant instance
    """
    @classmethod
    def open(cls, *args, **kw):
        return cls(Tyrant.open(*args, **kw))

    def __init__(self, t):
        self.t = t

    def __repr__(self):
        # The __repr__ for UserDict.DictMixin isn't desirable
        # for a large KV store :)
        return object.__repr__(self)

    def has_key(self, key):
        return key in self

    def __contains__(self, key):
        try:
            self.t.vsiz(key)
        except TyrantError:
            return False
        else:
            return True

    def setdefault(self, key, value):
        try:
            self.t.putkeep(key, value)
        except TyrantError:
            return self[key]
        return value

    def __setitem__(self, key, value):
        self.t.put(key, value)

    def __getitem__(self, key):
        try:
            return self.t.get(key)
        except TyrantError:
            raise KeyError(key)

    def __delitem__(self, key):
        try:
            self.t.out(key)
        except TyrantError:
            raise KeyError(key)

    def __iter__(self):
        return self.iterkeys()

    def iterkeys(self):
        self.t.iterinit()
        try:
            while True:
                yield self.t.iternext()
        except TyrantError:
            pass

    def keys(self):
        return list(self.iterkeys())

    def __len__(self):
        return self.t.rnum()

    def clear(self):
        self.t.vanish()

    def update(self, other=None, **kwargs):
        # Make progressively weaker assumptions about "other"
        if other is None:
            pass
        elif hasattr(other, 'iteritems'):
            self.multi_set(other.iteritems())
        elif hasattr(other, 'keys'):
            self.multi_set([(k, other[k]) for k in other.keys()])
        else:
            self.multi_set(other)
        if kwargs:
            self.update(kwargs)

    def multi_del(self, keys, no_update_log=False):
        opts = (no_update_log and RDBMONOULOG or 0)
        if not isinstance(keys, (list, tuple)):
            keys = list(keys)
        self.t.misc("outlist", opts, keys)

    def multi_get(self, keys, no_update_log=False):
        opts = (no_update_log and RDBMONOULOG or 0)
        if not isinstance(keys, (list, tuple)):
            keys = list(keys)
        rval = self.t.misc("getlist", opts, keys)
        if len(rval) <= len(keys):
            # 1.1.10 protocol, may return invalid results
            if len(rval) < len(keys):
                raise KeyError("Missing a result, unusable response in 1.1.10")
            return rval
        # 1.1.11 protocol returns interleaved key, value list
        d = dict((rval[i], rval[i + 1]) for i in xrange(0, len(rval), 2))
        return map(d.get, keys)

    def multi_set(self, items, no_update_log=False):
        opts = (no_update_log and RDBMONOULOG or 0)
        lst = []
        for k, v in items:
            lst.extend((k, v))
        self.t.misc("putlist", opts, lst)

    def call_func(self, func, key, value, record_locking=False, global_locking=False):
        opts = (
            (record_locking and RDBXOLCKREC or 0) |
            (global_locking and RDBXOLCKGLB or 0))
        return self.t.ext(func, opts, key, value)

    def get_size(self, key):
        try:
            return self.t.vsiz(key)
        except TyrantError:
            raise KeyError(key)

    def get_stats(self):
        return dict(l.split('\t', 1) for l in self.t.stat().splitlines() if l)

    def prefix_keys(self, prefix, maxkeys=None):
        if maxkeys is None:
            maxkeys = len(self)
        return self.t.fwmkeys(prefix, maxkeys)

    def concat(self, key, value, width=None):
        if width is None:
            self.t.putcat(key, value)
        else:
            self.t.putshl(key, value, width)

    def sync(self):
        self.t.sync()

    def close(self):
        self.t.close()

class Tyrant(object):
    @classmethod
    def open(cls, host='127.0.0.1', port=DEFAULT_PORT, timeout=3.0):
        sock = socket.socket()
        sock.settimeout(timeout)
        sock.connect((host, port))
        sock.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
        return cls(sock)

    def __init__(self, sock):
        self.sock = sock

    def close(self):
        self.sock.close()

    def put(self, key, value):
        """Unconditionally set key to value
        """
        socksend(self.sock, _t2(C.put, key, value))
        socksuccess(self.sock)

    def putkeep(self, key, value):
        """Set key to value if key does not already exist
        """
        socksend(self.sock, _t2(C.putkeep, key, value))
        socksuccess(self.sock)

    def putcat(self, key, value):
        """Append value to the existing value for key, or set key to
        value if it does not already exist
        """
        socksend(self.sock, _t2(C.putcat, key, value))
        socksuccess(self.sock)

    def putshl(self, key, value, width):
        """Equivalent to::

            self.putcat(key, value)
            self.put(key, self.get(key)[-width:])
        """
        socksend(self.sock, _t2W(C.putshl, key, value, width))
        socksuccess(self.sock)

    def putnr(self, key, value):
        """Set key to value without waiting for a server response
        """
        socksend(self.sock, _t2(C.putnr, key, value))

    def out(self, key):
        """Remove key from server
        """
        socksend(self.sock, _t1(C.out, key))
        socksuccess(self.sock)

    def get(self, key):
        """Get the value of a key from the server
        """
        socksend(self.sock, _t1(C.get, key))
        socksuccess(self.sock)
        return sockstr(self.sock)

    def _mget(self, klst):
        socksend(self.sock, _tN(C.mget, klst))
        socksuccess(self.sock)
        numrecs = socklen(self.sock)
        for i in xrange(numrecs):
            k, v = sockstrpair(self.sock)
            yield k, v

    def mget(self, klst):
        """Get key,value pairs from the server for the given list of keys
        """
        return list(self._mget(klst))

    def vsiz(self, key):
        """Get the size of a value for key
        """
        socksend(self.sock, _t1(C.vsiz, key))
        socksuccess(self.sock)
        return socklen(self.sock)

    def iterinit(self):
        """Begin iteration over all keys of the database
        """
        socksend(self.sock, _t0(C.iterinit))
        socksuccess(self.sock)

    def iternext(self):
        """Get the next key after iterinit
        """
        socksend(self.sock, _t0(C.iternext))
        socksuccess(self.sock)
        return sockstr(self.sock)

    def _fwmkeys(self, prefix, maxkeys):
        socksend(self.sock, _t1M(C.fwmkeys, prefix, maxkeys))
        socksuccess(self.sock)
        numkeys = socklen(self.sock)
        for i in xrange(numkeys):
            yield sockstr(self.sock)

    def fwmkeys(self, prefix, maxkeys):
        """Get up to the first maxkeys starting with prefix
        """
        return list(self._fwmkeys(prefix, maxkeys))

    def addint(self, key, num):
        socksend(self.sock, _t1M(C.addint, key, num))
        socksuccess(self.sock)
        return socklen(self.sock)

    def adddouble(self, key, num):
        fracpart, intpart = math.modf(num)
        fracpart, intpart = int(fracpart * 1e12), int(intpart)
        socksend(self.sock, _tDouble(C.adddouble, key, fracpart, intpart))
        socksuccess(self.sock)
        return sockdouble(self.sock)

    def ext(self, func, opts, key, value):
        # tcrdbext opts are RDBXOLCKREC, RDBXOLCKGLB
        """Call func(key, value) with opts

        opts is a bitflag that can be RDBXOLCKREC for record locking
        and/or RDBXOLCKGLB for global locking"""
        socksend(self.sock, _t3F(C.ext, func, opts, key, value))
        socksuccess(self.sock)
        return sockstr(self.sock)

    def sync(self):
        """Synchronize the database
        """
        socksend(self.sock, _t0(C.sync))
        socksuccess(self.sock)

    def vanish(self):
        """Remove all records
        """
        socksend(self.sock, _t0(C.vanish))
        socksuccess(self.sock)

    def copy(self, path):
        """Hot-copy the database to path
        """
        socksend(self.sock, _t1(C.copy, path))
        socksuccess(self.sock)

    def restore(self, path, msec):
        """Restore the database from path at timestamp (in msec)
        """
        socksend(self.sock, _t1R(C.copy, path, msec))
        socksuccess(self.sock)

    def setmst(self, host, port):
        """Set master to host:port
        """
        socksend(self.sock, _t1M(C.setmst, host, port))
        socksuccess(self.sock)

    def rnum(self):
        """Get the number of records in the database
        """
        socksend(self.sock, _t0(C.rnum))
        socksuccess(self.sock)
        return socklong(self.sock)

    def size(self):
        """Get the size of the database
        """
        socksend(self.sock, _t0(C.size))
        socksuccess(self.sock)
        return socklong(self.sock)

    def stat(self):
        """Get some statistics about the database
        """
        socksend(self.sock, _t0(C.stat))
        socksuccess(self.sock)
        return sockstr(self.sock)

    def _misc(self, func, opts, args):
        # tcrdbmisc opts are RDBMONOULOG
        socksend(self.sock, _t1FN(C.misc, func, opts, args))
        try:
            socksuccess(self.sock)
        finally:
            numrecs = socklen(self.sock)
        for i in xrange(numrecs):
            yield sockstr(self.sock)

    def misc(self, func, opts, args):
        """All databases support "putlist", "outlist", and "getlist".
        "putlist" is to store records. It receives keys and values one after the other, and returns an empty list.
        "outlist" is to remove records. It receives keys, and returns an empty list.
        "getlist" is to retrieve records. It receives keys, and returns values.

        Table database supports "setindex", "search", "genuid".

        opts is a bitflag that can be RDBMONOULOG to prevent writing to the update log
        """
        return list(self._misc(func, opts, args))

def main():
    import doctest
    doctest.testmod()

if __name__ == '__main__':
    main()


from datetime import datetime, timedelta

import curator

from . import CuratorTestCase

class TestTimeBasedDeletion(CuratorTestCase):
    def test_curator_will_properly_delete_indices(self):
        self.create_indices(10)
        self.run_curator(delete_older=3)
        mtd = self.client.cluster.state(index=self.args['prefix'] + '*', metric='metadata')
        self.assertEquals(4, len(mtd['metadata']['indices'].keys()))

    def test_curator_will_properly_delete_hourly_indices(self):
        self.create_indices(10, 'hours')
        self.run_curator(delete_older=3, time_unit='hours')
        mtd = self.client.cluster.state(index=self.args['prefix'] + '*', metric='metadata')
        self.assertEquals(4, len(mtd['metadata']['indices'].keys()))

class TestFindExpiredIndices(CuratorTestCase):
    def test_find_indices_ignores_indices_with_different_prefix_or_time_unit(self):
        self.create_index('logstash-2012.01.01')        # wrong precision
        self.create_index('not-logstash-2012.01.01.00') # wrong prefix
        self.create_index('logstash-2012.01.01.00')

        expired = list(curator.find_expired_indices(self.client, 'hours', 1))
        self.assertEquals(1, len(expired))
        self.assertEquals('logstash-2012.01.01.00', expired[0][0])

    def test_find_reports_correct_time_interval_from_cutoff(self):
        self.create_index('l-2014.01.01')
        self.create_index('l-2014.01.02')
        # yesterday is always save since we reset to mignight and do <, not <=
        self.create_index('l-2014.01.03')

        expired = list(curator.find_expired_indices(self.client, 'days', 1,
            utc_now=datetime(2014, 1, 4, 3, 45, 50), prefix='l-'))
        self.assertEquals(
            [
                (u'l-2014.01.01', timedelta(2)),
                (u'l-2014.01.02', timedelta(1))
            ],
            expired
        )


from __future__ import print_function, absolute_import, nested_scopes, generators, division, with_statement, unicode_literals
import logging

from hytra.core.ilastik_project_options import IlastikProjectOptions
from hytra.jst.conflictingsegmentsprobabilitygenerator import ConflictingSegmentsProbabilityGenerator
from hytra.core.ilastikhypothesesgraph import IlastikHypothesesGraph
from hytra.core.fieldofview import FieldOfView

try:
    import multiHypoTracking_with_cplex as mht
except ImportError:
    try:
        import multiHypoTracking_with_gurobi as mht
    except ImportError:
        mht = None
        import dpct

def constructFov(shape, t0, t1, scale=[1, 1, 1]):
    [xshape, yshape, zshape] = shape
    [xscale, yscale, zscale] = scale

    fov = FieldOfView(t0, 0, 0, 0, t1, xscale * (xshape - 1), yscale * (yshape - 1),
                      zscale * (zshape - 1))
    return fov

# def test_twoSegmentations():
#     # set up ConflictingSegmentsProbabilityGenerator
#     ilpOptions = IlastikProjectOptions()
#     ilpOptions.divisionClassifierPath = None
#     ilpOptions.divisionClassifierFilename = None

#     ilpOptions.rawImageFilename = 'tests/multiSegmentationHypothesesTestDataset/Raw.h5'
#     ilpOptions.rawImagePath = 'exported_data'
#     ilpOptions.rawImageAxes = 'txyzc'

#     ilpOptions.labelImageFilename = 'tests/multiSegmentationHypothesesTestDataset/segmentation.h5'

#     ilpOptions.objectCountClassifierFilename = 'tests/multiSegmentationHypothesesTestDataset/tracking.ilp'

#     additionalLabelImageFilenames = ['tests/multiSegmentationHypothesesTestDataset/segmentationAlt.h5']
#     additionalLabelImagePaths = [ilpOptions.labelImagePath]

#     probabilityGenerator = ConflictingSegmentsProbabilityGenerator(
#         ilpOptions,
#         additionalLabelImageFilenames,
#         additionalLabelImagePaths,
#         useMultiprocessing=False,
#         verbose=False)
#     probabilityGenerator.fillTraxels(usePgmlink=False)

#     assert(len(probabilityGenerator.TraxelsPerFrame[0]) == 4)
#     assert(len(probabilityGenerator.TraxelsPerFrame[1]) == 3)
#     assert(len(probabilityGenerator.TraxelsPerFrame[2]) == 3)
#     assert(len(probabilityGenerator.TraxelsPerFrame[3]) == 4)
#     filenamesPerTraxel = [t.segmentationFilename for t in probabilityGenerator.TraxelsPerFrame[3].values()]
#     idsPerTraxel = [t.idInSegmentation for t in probabilityGenerator.TraxelsPerFrame[3].values()]
#     assert(idsPerTraxel.count(1) == 2)
#     assert(idsPerTraxel.count(2) == 2)
#     assert(filenamesPerTraxel.count('tests/multiSegmentationHypothesesTestDataset/segmentation.h5') == 2)
#     assert(filenamesPerTraxel.count('tests/multiSegmentationHypothesesTestDataset/segmentationAlt.h5') == 2)

#     # build hypotheses graph, check that conflicting traxels are properly detected
#     fieldOfView = constructFov(probabilityGenerator.shape,
#                                probabilityGenerator.timeRange[0],
#                                probabilityGenerator.timeRange[1],
#                                [probabilityGenerator.x_scale,
#                                 probabilityGenerator.y_scale,
#                                 probabilityGenerator.z_scale])

#     hypotheses_graph = IlastikHypothesesGraph(
#         probabilityGenerator=probabilityGenerator,
#         timeRange=probabilityGenerator.timeRange,
#         maxNumObjects=1,
#         numNearestNeighbors=2,
#         fieldOfView=fieldOfView,
#         withDivisions=False,
#         divisionThreshold=0.1
#     )

#     assert(hypotheses_graph.countNodes() == 14)
#     assert(hypotheses_graph.countArcs() == 23)
#     assert(hypotheses_graph._graph.node[(0, 1)]['traxel'].conflictingTraxelIds == [3])
#     assert(hypotheses_graph._graph.node[(0, 3)]['traxel'].conflictingTraxelIds == [1])
#     assert(hypotheses_graph._graph.node[(0, 2)]['traxel'].conflictingTraxelIds == [4])
#     assert(hypotheses_graph._graph.node[(0, 4)]['traxel'].conflictingTraxelIds == [2])
#     assert(hypotheses_graph._graph.node[(1, 1)]['traxel'].conflictingTraxelIds == [2, 3])
#     assert(hypotheses_graph._graph.node[(1, 2)]['traxel'].conflictingTraxelIds == [1])
#     assert(hypotheses_graph._graph.node[(1, 3)]['traxel'].conflictingTraxelIds == [1])

#     # track, but check that the right exclusion constraints are present
#     hypotheses_graph.insertEnergies()
#     trackingGraph = hypotheses_graph.toTrackingGraph()

#     assert(len(trackingGraph.model['exclusions']) == 8)
#     for exclusionSet in trackingGraph.model['exclusions']:
#         assert(len(exclusionSet) == 2)

#     # use multiHypoTracking, insert exclusion constraints!
#     if mht is not None:
#         result = mht.track(trackingGraph.model, {"weights": [10, 10, 500, 500]})
#     else:
#         return
#         # standard dpct cannot handle exclusion constraints yet
#         result = dpct.trackFlowBased(trackingGraph.model, {"weights": [10, 10, 500, 500]})

#     hypotheses_graph.insertSolution(result)
#     # hypotheses_graph.computeLineage()

#     numActivePerFrame = {}

#     for node in hypotheses_graph.nodeIterator():
#         timeframe = node[0]
#         if 'value' in hypotheses_graph._graph.node[node]:
#             value = hypotheses_graph._graph.node[node]['value']
#         else:
#             value = 0
#         numActivePerFrame.setdefault(timeframe, []).append(value)

#     for _, v in numActivePerFrame.items():
#         assert(sum(v) == 2)

#     edgeFlow = 0
#     for edge in hypotheses_graph.arcIterator():
#         if 'value' in hypotheses_graph._graph.edge[edge[0]][edge[1]]:
#             edgeFlow += hypotheses_graph._graph.edge[edge[0]][edge[1]]['value']
#     assert(edgeFlow == 6)

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    test_twoSegmentations()


from dis import dis
from io import StringIO
from itertools import product, chain
import random
import sys

import pytest

from codetransformer.code import Code, Flag, pycode
from codetransformer.instructions import LOAD_CONST, LOAD_FAST, uses_free

@pytest.fixture(scope='module')
def sample_flags(request):
    random.seed(8025816322119661921)  # ayy lmao
    nflags = len(Flag.__members__)
    return tuple(
        dict(zip(Flag.__members__.keys(), case)) for case in chain(
            random.sample(list(product((True, False), repeat=nflags)), 1000),
            [[True] * nflags],
            [[False] * nflags],
        )
    )

def test_lnotab_roundtrip():
    # DO NOT ADD EXTRA LINES HERE
    def f():  # pragma: no cover
        a = 1
        b = 2
        c = 3
        d = 4
        a, b, c, d

    start_line = test_lnotab_roundtrip.__code__.co_firstlineno + 3
    lines = [start_line + n for n in range(5)]
    code = Code.from_pycode(f.__code__)
    lnotab = code.lnotab
    assert lnotab.keys() == set(lines)
    assert isinstance(lnotab[lines[0]], LOAD_CONST)
    assert lnotab[lines[0]].arg == 1
    assert isinstance(lnotab[lines[1]], LOAD_CONST)
    assert lnotab[lines[1]].arg == 2
    assert isinstance(lnotab[lines[2]], LOAD_CONST)
    assert lnotab[lines[2]].arg == 3
    assert isinstance(lnotab[lines[3]], LOAD_CONST)
    assert lnotab[lines[3]].arg == 4
    assert isinstance(lnotab[lines[4]], LOAD_FAST)
    assert lnotab[lines[4]].arg == 'a'
    assert f.__code__.co_lnotab == code.py_lnotab == code.to_pycode().co_lnotab

def test_lnotab_really_dumb_whitespace():
    ns = {}
    exec('def f():\n    lol = True' + '\n' * 1024 + '    wut = True', ns)
    f = ns['f']
    code = Code.from_pycode(f.__code__)
    lines = [2, 1026]
    lnotab = code.lnotab
    assert lnotab.keys() == set(lines)
    assert isinstance(lnotab[lines[0]], LOAD_CONST)
    assert lnotab[lines[0]].arg
    assert isinstance(lnotab[lines[1]], LOAD_CONST)
    assert lnotab[lines[1]].arg
    assert f.__code__.co_lnotab == code.py_lnotab == code.to_pycode().co_lnotab

def test_flag_packing(sample_flags):
    for flags in sample_flags:
        assert Flag.unpack(Flag.pack(**flags)) == flags

def test_flag_unpack_too_big():
    assert all(Flag.unpack(Flag.max).values())
    with pytest.raises(ValueError):
        Flag.unpack(Flag.max + 1)

def test_flag_max():
    assert Flag.pack(
        CO_OPTIMIZED=True,
        CO_NEWLOCALS=True,
        CO_VARARGS=True,
        CO_VARKEYWORDS=True,
        CO_NESTED=True,
        CO_GENERATOR=True,
        CO_NOFREE=True,
        CO_COROUTINE=True,
        CO_ITERABLE_COROUTINE=True,
        CO_FUTURE_DIVISION=True,
        CO_FUTURE_ABSOLUTE_IMPORT=True,
        CO_FUTURE_WITH_STATEMENT=True,
        CO_FUTURE_PRINT_FUNCTION=True,
        CO_FUTURE_UNICODE_LITERALS=True,
        CO_FUTURE_BARRY_AS_BDFL=True,
        CO_FUTURE_GENERATOR_STOP=True,
    ) == Flag.max

def test_flag_max_immutable():
    with pytest.raises(AttributeError):
        Flag.CO_OPTIMIZED.max = None

def test_code_multiple_varargs():
    with pytest.raises(ValueError) as e:
        Code(
            (), (
                '*args',
                '*other',
            ),
        )

    assert str(e.value) == 'cannot specify *args more than once'

def test_code_multiple_kwargs():
    with pytest.raises(ValueError) as e:
        Code(
            (), (
                '**kwargs',
                '**kwargs',
            ),
        )

    assert str(e.value) == 'cannot specify **kwargs more than once'

@pytest.mark.parametrize('cls', uses_free)
def test_dangling_var(cls):
    instr = cls('dangling')
    with pytest.raises(ValueError) as e:
        Code((instr,))

    assert (
        str(e.value) ==
        "Argument to %r is not in cellvars or freevars." % instr
    )

def test_code_flags(sample_flags):
    attr_map = {
        'CO_NESTED': 'is_nested',
        'CO_GENERATOR': 'is_generator',
        'CO_COROUTINE': 'is_coroutine',
        'CO_ITERABLE_COROUTINE': 'is_iterable_coroutine',
        'CO_NEWLOCALS': 'constructs_new_locals',
    }
    for flags in sample_flags:
        if sys.version_info < (3, 6):
            codestring = b'd\x00\x00S'  # return None
        else:
            codestring = b'd\x00S'  # return None

        code = Code.from_pycode(pycode(
            argcount=0,
            kwonlyargcount=0,
            nlocals=2,
            stacksize=0,
            flags=Flag.pack(**flags),
            codestring=codestring,
            constants=(None,),
            names=(),
            varnames=('a', 'b'),
            filename='',
            name='',
            firstlineno=0,
            lnotab=b'',
        ))
        assert code.flags == flags
        for flag, attr in attr_map.items():
            if flags[flag]:
                assert getattr(code, attr)

@pytest.fixture
def abc_code():
    a = LOAD_CONST('a')
    b = LOAD_CONST('b')
    c = LOAD_CONST('c')  # not in instrs
    code = Code((a, b), argnames=())

    return (a, b, c), code

def test_instr_index(abc_code):
    (a, b, c), code = abc_code

    assert code.index(a) == 0
    assert code.index(b) == 1

    with pytest.raises(ValueError):
        code.index(c)

def test_code_contains(abc_code):
    (a, b, c), code = abc_code

    assert a in code
    assert b in code
    assert c not in code

def test_code_dis(capsys):
    @Code.from_pyfunc
    def code():  # pragma: no cover
        a = 1
        b = 2
        return a, b

    buf = StringIO()
    dis(code.to_pycode(), file=buf)
    expected = buf.getvalue()

    code.dis()
    out, err = capsys.readouterr()
    assert not err
    assert out == expected

    buf = StringIO()
    code.dis(file=buf)
    assert buf.getvalue() == expected


# coding=utf-8
# --------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# Code generated by Microsoft (R) AutoRest Code Generator.
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

from enum import Enum, EnumMeta
from six import with_metaclass

class _CaseInsensitiveEnumMeta(EnumMeta):
    def __getitem__(self, name):
        return super().__getitem__(name.upper())

    def __getattr__(cls, name):
        """Return the enum member matching `name`
        We use __getattr__ instead of descriptors or inserting into the enum
        class' __dict__ in order to support `name` and `value` being both
        properties for enum members (which live in the class' __dict__) and
        enum members themselves.
        """
        try:
            return cls._member_map_[name.upper()]
        except KeyError:
            raise AttributeError(name)

class LockLevel(with_metaclass(_CaseInsensitiveEnumMeta, str, Enum)):
    """The lock level of the management lock.
    """

    NOT_SPECIFIED = "NotSpecified"
    CAN_NOT_DELETE = "CanNotDelete"
    READ_ONLY = "ReadOnly"


from sourcer import Grammar

# This is work in progress.
# See: https://help.salesforce.com/articleView?id=customize_functions.htm&type=5

g = Grammar(r'''
    ```
    import ast
    ```

    start = Expression

    Expression = OperatorPrecedence(
        Atom | "(" >> Expression << ")",
        Postfix(ArgumentList | FieldAccess),
        Prefix("-" | "+" | "!"),
        RightAssoc("^"),
        LeftAssoc("*" | "/"),
        LeftAssoc("+" | "-" | "&"),
        NonAssoc("<=" | "<" | ">=" | ">"),
        NonAssoc("!=" | "<>" | "==" | "="),
        LeftAssoc("&&"),
        LeftAssoc("||"),
    )

    class ArgumentList {
        arguments: "(" >> (Expression /? ",") << ")"
    }

    class FieldAccess {
        field: "." >> Word
    }

    Atom = Global | Identifier | Rational | Integer | String

    class Global {
        name: "$" >> Word
    }

    class Identifier {
        name: Word
    }

    # ASK: What is the real syntax for these things?
    Word = /[_a-zA-Z][_a-zA-Z0-9]*/
    Rational = /(\d+\.\d*)|(\d*\.\d+)/ |> `float`
    Integer = /\d+/ |> `int`
    StringLiteral = /("([^"\\]|\\.)*")/ | /('([^'\\]|\\.)*')/

    # For now, just use ast module to evaluate string literals.
    class String {
        value: StringLiteral |> `ast.literal_eval`
    }

    ignore /\s+/

''', include_source=True)

aliases = {
    '=': '==',
    '<>': '!=',
}

constants = {
    'NULL': None,
    'TRUE': True,
    'FALSE': False,
}

# Incomplete collection of evaluators.
evaluators = {
    '*': lambda x, y: x * y if x is not None and y is not None else None,
    '/': lambda x, y: x / y if x is not None and y is not None else None,
    '+': lambda x, y: x + y if x is not None and y is not None else None,
    '-': lambda x, y: x - y if x is not None and y is not None else None,
    '==': lambda x, y: x == y,
    '!=': lambda x, y: x != y,
    '&&': lambda x, y: x and y,
    '||': lambda x, y: x or y,
    '>': lambda x, y: x > y if x is not None and y is not None else False,
    '<': lambda x, y: x < y if x is not None and y is not None else False,
    '>=': lambda x, y: x >= y if x is not None and y is not None else False,
    '<=': lambda x, y: x <= y if x is not None and y is not None else False,
    'AND': lambda *a: all(a),
    'CONTAINS': lambda x, y: str(y) in str(x) if x is not None else True,
    'IF': lambda x, y, z: y if x else z,
    'ISBLANK': lambda x: x is None,
    'LOG': lambda x: log10(x) if x is not None else None,
    'MAX': lambda *a: max(*a),
    'MIN': lambda *a: min(*a),
    'MOD': lambda x, y: (x % y) if x is not None and y is not None else None,
    'NOT': lambda x: not(x),
    'OR': lambda *a: any(a),
    'SQRT': lambda x: sqrt(x) if x is not None else None,
    'TEXT': lambda x: str(x),
}

def evaluate(node, bindings):
    # Look up identifiers.
    if isinstance(node, g.Identifier):
        if node.name in bindings:
            return bindings[node.name]

        name = node.name.upper()
        return bindings.get(name, name)

    # Look up fields.
    if isinstance(node, g.Postfix) and isinstance(node.operator, g.FieldAccess):
        obj, field = node.left, node.operator.field
        if hasattr(obj, field):
            return getattr(obj, field)
        elif isinstance(obj, dict):
            return obj.get(field)
        else:
            return node

    # Evaluate function calls and operators.
    if isinstance(node, g.Infix):
        x, func, y = node.left, node.operator, node.right
        args = (x, y)
    elif isinstance(node, g.Postfix) and isinstance(node.operator, g.ArgumentList):
        func, args = node.left, node.operator.arguments
    else:
        return node

    # Check if we're using an alias.
    func = aliases.get(func, func)

    if func in evaluators:
        return evaluators[func](*args)
    else:
        return node

def run(formula, bindings=None):
    updated_bindings = dict(constants)
    updated_bindings.update(bindings or {})
    tree = g.parse(formula)
    return g.transform(tree, lambda node: evaluate(node, updated_bindings))

def test_some_simple_formulas():
    result = run('1 + 2 * 3')
    assert result == 7

    result = run('foo == bar && fiz == buz', bindings={
        'foo': 1, 'bar': 1, 'fiz': 2, 'buz': 2,
    })
    assert result == True

    result = run('foo == bar && fiz == buz', bindings={
        'foo': 1, 'bar': 1, 'fiz': 2, 'buz': 3,
    })
    assert result == False

    result = run('1 <= 2 && (false || true)')
    assert result == True # Explicitly compare to True.

    result = run('1 > 2 || (true && false)')
    assert result == False # Explicitly compare to False.

    result = run('foo != bar', bindings={'foo': 10, 'bar': 10})
    assert not result

    result = run('foo != bar', bindings={'foo': 1, 'bar': 2})
    assert result

    result = run('foo.bar', bindings={'foo': {'bar': 10}})
    assert result == 10

    result = run('foo.bar.baz', bindings={'foo': {'bar': {'baz': 100}}})
    assert result == 100

    result = run('MIN(20, 10, 30)')
    assert result == 10

    result = run('MIN(20, 10, 30) + MAX(11, 12, 13)')
    assert result == 23


## Copyright 2003-2009 Luc Saffre
## This file is part of the TimTools project.
## TimTools is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## the Free Software Foundation; either version 3 of the License, or
## (at your option) any later version.
## TimTools is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public License for more details.
## You should have received a copy of the GNU General Public License
## along with TimTools; if not, see <http://www.gnu.org/licenses/>.

import re

def plain2xml(txt):
   txt = txt.replace("&","&")
   txt = txt.replace("<","<")
   return txt

memocommands = (
   ( re.compile('\[url\s+(\S+)\s*(.*?)\]',re.DOTALL),
     lambda m : ''+m.group(2)+' (' + m.group(1)+ ')'),
   )
# urlfind =
# urlrepl = re.compile('\2 (\1)')

# def urlrepl(m):

def memo2xml(txt):
   txt = plain2xml(txt)
   txt = txt.replace('[B]','')
   txt = txt.replace('[b]','')
   txt = txt.replace('[U]','')
   txt = txt.replace('[u]','')
   for find,repl in memocommands:
      txt = re.sub(find,repl,txt)
   return txt

def rst2xml(txt):
   raise "doesn't work"
   import docutils.parsers.rst
   import docutils.utils
   parser = docutils.parsers.rst.Parser()
   doc = docutils.utils.new_document("feed")
   parser.parse(txt, doc)
   raise "and now?"

_feeders={
   'xml' : lambda x : x,
   'plain' : plain2xml,
   'rst' : rst2xml,
   'memo' : memo2xml,
   }

def getFeeder(name):
   return _feeders[name]