Skip to content

Fixes boolean indexing for strided masks #1370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Aug 28, 2023

Conversation

ndgrigorian
Copy link
Collaborator

@ndgrigorian ndgrigorian commented Aug 26, 2023

This PR fixes some edge cases to boolean indexing that were not working as expected.

The cumulative sum used to get the count of set elements in the mask was misbehaving, especially for cases with negative strides (and most noticeably, the 1D case). This seems to have been caused by stride simplification can change the order elements are traversed in, which caused incorrect results.

Additionally, the offset parameter was also unused in when the contiguous kernel was called after simplification.

A new stride simplification method called compact_iteration_space was introduced to fix the problem.

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • If this PR is a work in progress, are you opening the PR as a draft?

@github-actions
Copy link

@coveralls
Copy link
Collaborator

Coverage Status

coverage: 85.634%. remained the same when pulling 7a30569 on fix-boolean-indexing-cumsum into cf4660d on master.

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.14.6dev4=py310ha25a700_21 ran successfully.
Passed: 916
Failed: 84
Skipped: 119

@oleksandr-pavlyk
Copy link
Contributor

It was a mistake to use simplify_iteration_space_1 here. Not only will negative strides cause trouble, but so will F-contiguous mask arrays.

However I don't think removing simplification altogether is the right solution. We could use simplify_iteration_space_2 using C-contiguous strides for the shape of the input mask array for the second input.

@ndgrigorian
Copy link
Collaborator Author

It was a mistake to use simplify_iteration_space_1 here. Not only will negative strides cause trouble, but so will F-contiguous mask arrays.

However I don't think removing simplification altogether is the right solution. We could use simplify_iteration_space_2 using C-contiguous strides for the shape of the input mask array for the second input.

I've tested this out and it does also resolve the problem.

Would it be worthwhile then to write an abbreviated version of simplify_iteration_space, something like simplify_iteration_space_by_c_contig_strides, which removes logic that will never be true for a C-contig second input and doesn't have an unused strides vector and offset as byproducts?

@oleksandr-pavlyk
Copy link
Contributor

Would it be worthwhile then to write an abbreviated version of simplify_iteration_space, something like simplify_iteration_space_by_c_contig_strides, which removes logic that will never be true for a C-contig second input and doesn't have an unused strides vector and offset as byproducts?

Yes, I was thinking about this as well. It is a good idea. I'd suggest naming it compact_iteration_space (restricted simplification).

ndgrigorian and others added 4 commits August 27, 2023 20:10
- The cumulative sum was being calculated incorrectly -- the offset from stride simplification was unused and the result was incorrect for some cases with non-C-contiguous strides
- To fix this, new functions ``compact_iteration_space`` and complementary function ``compact_iteration`` have been implemented
Compacting strides can reduce dimensionality of the array, but it can not
turn an input that is not already C-contiguous into a C-contiguous one.

Hence the branch checking if the input became C-contiguous after compacting
is effectively dead.
@ndgrigorian ndgrigorian force-pushed the fix-boolean-indexing-cumsum branch from 7a30569 to 741dc7f Compare August 28, 2023 04:17
@ndgrigorian
Copy link
Collaborator Author

@oleksandr-pavlyk
I've added an implementation for compact_iteration_space to the PR.

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.14.6dev4=py310ha25a700_23 ran successfully.
Passed: 916
Failed: 84
Skipped: 119

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.14.6dev4=py310ha25a700_30 ran successfully.
Passed: 916
Failed: 84
Skipped: 119

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.14.6dev4=py310ha25a700_31 ran successfully.
Passed: 916
Failed: 84
Skipped: 119

@ndgrigorian ndgrigorian merged commit f49889c into master Aug 28, 2023
@ndgrigorian ndgrigorian deleted the fix-boolean-indexing-cumsum branch August 28, 2023 16:06
@github-actions
Copy link

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.14.6dev4=py310ha25a700_26 ran successfully.
Passed: 915
Failed: 85
Skipped: 119

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants