Einsum indexing very fragile, because it tests for int (and int64 is not int)

[orichardson]

The index arrays for einsum do not accept the output of any numpy array, because these datatypes cannot be ints (see #2951; #12322). The test should be modified to accept numpy ints, because converting the datatype to 'int' explicitly is counter-intuitive and unnecessary.

Though easy to circumvent, this will incur a lot of unnecessary debugging time, as it appears to break abstraction boundaries, has strange interactions with tolist, and indices that work in test environments fail when constructed programatically.

Reproducing code example:

import numpy as np
X = np.arange(9).reshape(3,3)

# each of these results in an identical list [0] as far as equality test is concerned 
idx1 = [0]                # list[ int ] 
idx2 = np.unique(idx1)    # np.array [int64]
idx3 = idx2.tolist()      # list [int]
idx3 = list(idx2)         # list [int64]

np.einsum(X, [0], idx1 ) # succeeds
np.einsum(X, [0], idx2 ) # fails
np.einsum(X, [0], idx3 ) # succeeds
np.einsum(X, [0], idx4 ) # fails

Error message:

ValueError: each subscript must be either an integer or an ellipsis

Full error message if optimize_arg is False:

<__array_function__ internals> in einsum(*args, **kwargs)

~/.local/lib/python3.6/site-packages/numpy/core/einsumfunc.py in einsum(*operands, **kwargs)
   1354     # If no optimization, run pure einsum
   1355     if optimize_arg is False:
-> 1356         return c_einsum(*operands, **kwargs)
   1357 
   1358     valid_einsum_kwargs = ['out', 'dtype', 'order', 'casting']

ValueError: each subscript must be either an integer or an ellipsis

Full error message when optimize_arg is True:

<__array_function__ internals> in einsum(*args, **kwargs)

~/.local/lib/python3.6/site-packages/numpy/core/einsumfunc.py in einsum(*operands, **kwargs)
   1377     # Build the contraction list and operand
   1378     operands, contraction_list = einsum_path(*operands, optimize=optimize_arg,
-> 1379                                              einsum_call=True)
   1380 
   1381     handle_out = False

Numpy/Python version information:

1.17.3 3.6.9 (default, Nov 7 2019, 10:44:02)
[GCC 8.3.0]

[eric-wieser]

We ought to be calling operator.index or the equivalent C API here, rather than the PyInt_Check I assume we must be doing today.

[eric-wieser]

Relevant lines are here:

numpy/numpy/core/src/multiarray/multiarraymodule.c

Lines 2462 to 2501 in b489287

	/* Subscript */
	else if (PyInt_Check(item) || PyLong_Check(item)) {
	long s = PyInt_AsLong(item);
	npy_bool bad_input = 0;
	if (subindex + 1 >= subsize) {
	PyErr_SetString(PyExc_ValueError,
	"subscripts list is too long");
	Py_DECREF(obj);
	return -1;
	}
	if ( s < 0 ) {
	bad_input = 1;
	}
	else if (s < 26) {
	subscripts[subindex++] = 'A' + (char)s;
	}
	else if (s < 2*26) {
	subscripts[subindex++] = 'a' + (char)s - 26;
	}
	else {
	bad_input = 1;
	}
	if (bad_input) {
	PyErr_SetString(PyExc_ValueError,
	"subscript is not within the valid range [0, 52)");
	Py_DECREF(obj);
	return -1;
	}
	}
	/* Invalid */
	else {
	PyErr_SetString(PyExc_ValueError,
	"each subscript must be either an integer "
	"or an ellipsis");
	Py_DECREF(obj);
	return -1;
	}

Logic ought to be the C equivalent of:

try:
    s = operator.index(item)
except TypeError as e:
    raise TypeError("each subscript must be either an integer "
                    "or an ellipsis") from e
# do stuff with s

[guilhermeleobas]

I will take this one.

[tinaoberoi]

@guilhermeleobas you working on this ? If not I would like to take this up. Thanks

[guilhermeleobas]

Hi @tinaoberoi, it was on my plans but you can go ahead and work on it :)

[tinaoberoi]

Thanks @guilhermeleobas , I new here, any previous findings and help is appreciated. :-)

[rlintott]

I'm also a first time contributor. I might try fixing this. I don't really mind if someone else contribute first. In any case, the experience is good for me.