[Performance]: Worse prefilling with unified triton attention

[haochengxia]

Proposal to improve performance

Report of performance regression

When I used the kernel updated with #16828, I witnessed a huge prefilling (TTFT) performance drop on A100@40GB.

For an 8k token sequence, the old chunked prefilling kernel costs ~600ms while the new one uses 1500ms.

Is there any configuration I should set for this kernel?

Thanks!