PERF: Using Numpy C-API when calling np.arange

[ShaharNaveh]

• closes #xxxx
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

---

Somewhat of a follow up to #32681.

---

I could not benchmark this change as this function is cdef and not def or cpdef (I would also love if someone could give me a tip on how to benchmark cdef functions from an Ipython shell for example).

What I did what I ran cython -a pandas/_libs/internals.pyx and took a screenshot of the before and after.

---

Master:

---

PR:

---

Also, is there a reason not to replace every call of np.arange with cnp.PyArray_Arange? (cython files only)

[jbrockmendel]

I would also love if someone could give me a tip on how to benchmark cdef functions from an Ipython shell for example

i usually write a def wrapper in the same file to call the cdef function in a loop. You could also find the "closest" non-cdef function that calls the affected function and profile that

[jbrockmendel]

Also, is there a reason not to replace every call of np.arange with cnp.PyArray_Arange? (cython files only)

if its in a place that gets called a zillion times (especially in a loop), then go for it.

For others is probably isn't worth the cognitive overhead for the next reader to come along.

[ShaharNaveh]

Also, is there a reason not to replace every call of np.arange with cnp.PyArray_Arange? (cython files only)

if its in a place that gets called a zillion times (especially in a loop), then go for it.

Okay, make sense.

For others is probably isn't worth the cognitive overhead for the next reader to come along.

If a comment is included (like in this PR), does it solves the cognitive overhead?

and are the there any other objections?

[jbrockmendel]

If a comment is included (like in this PR), does it solves the cognitive overhead?

I guess, but then it feels fishy to be adding identical comments in many places (then again, I do this too). Your call.

Separate topic: in core.groupby.generic L389, L1350, L1702 there are calls to ._convert that I think are superfluous. Getting a second pair of eyes to try to reason about the code and prove it would be great if you're up for it.

[jbrockmendel]

thanks @MomIsBestFriend

[jorisvandenbossche]

I guess, but then it feels fishy to be adding identical comments in many places (then again, I do this too). Your call.

BTW, I don't think we should merge such PRs in the future without a performance check showing an considerable improvement.
(as I agree it gives cognitive overhead, so we should only do it when worth it; what is worth it is of course subjective, but at least we should time it)