Return a split Dataset in load_dataset

[lhoestq]

...instead of a DatasetDict.

# now supported
ds = load_dataset("squad")
ds[0]  
for example in ds:
    pass

# still works
ds["train"]
ds["validation"]

# new
ds.splits  # Dict[str, Dataset] | None

# soon to be supported (not in this PR)
ds = load_dataset("dataset_with_no_splits")
ds[0]
for example in ds:
    pass

I implemented Dataset.__getitem__ and IterableDataset.__getitem__ to be able to get a split from a dataset.
The splits are defined by the ds.info.splits dictionary.

Therefore a dataset is a table that optionally has some splits defined in the dataset info. And a split dataset is the concatenation of all its splits.

I made as little breaking changes as possible. Notable breaking changes:

• load_dataset("potato").keys() / .items() / .values() / don't work anymore, since we don't return a dict
• same for for split_name in load_dataset("potato"), since we now iterate on the examples
• ..

TODO:

• Update push_to_hub
• Update save_to_disk/load_from_disk
• check for other breaking changes
• fix existing tests
• add new tests
• docs

This is related to #5189, to extend load_dataset to return datasets without splits

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[lhoestq]

Just noticed that now we have to deal with indexed & split datasets. The remaining tests are failing because one should be able to get an indexed dataset when accessing the split of a dataset made of indexed splits (right now the index is just trashed)

[polinaeterna]

Suggested change

	new_fingerprint=new_fingerprint + f"-{split_name}",
	new_fingerprint= f"{new_fingerprint}-{split_name}" if new_fingerprint else None,

otherwise would raise unsupported operand type(s) for +: 'NoneType' and 'str'