Skip to content

sycl: Add more debug prints #13640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Rbiessy
Copy link
Collaborator

@Rbiessy Rbiessy commented May 19, 2025

This adds more debugging prints related to the operations being run. scope_op_debug_print is introduced to print the "call ..." and "call ... done" logs in a single line.
This adds more information for all the operations and its destination and input tensors: the tensor type, its number of elements, stride sizes and whether the tensor is strided (i.e. non-contiguous) or permuted.
For mul_mat this also add some debug prints when conversations kernels are called from and to fp16, fp32 and quantized types.
This also adds debugging prints for a few operations that were missing it.

The output for llama-2-7b.Q4_0 PP and TG is added here as an example:

@github-actions github-actions bot added ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels May 19, 2025
@@ -694,8 +694,6 @@ void ggml_sycl_cpy(ggml_backend_sycl_context & ctx, const ggml_tensor * src0, co
}

void ggml_sycl_dup(ggml_backend_sycl_context & ctx, ggml_tensor * dst) {
// TODO: why do we pass dst as src1 here?
GGML_SYCL_DEBUG("[SYCL] call %s\n", __func__);
scope_op_debug_print scope_dbg_print(__func__, dst, /*num_src=*/1);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a concern that this dynamic method will decrease the performance of running time:

The legacy method won't impact the performance when disable the debug mode.
But this method will spend time to create the object when disable the debug mode.

Have you test the performance impact with legacy code when disable the debug mode?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a check here in the function:

if (!g_ggml_sycl_debug) {
        return;
    }

So, if debugging mode is disabled, it will not do anything so no performance hit in non debug mode I think.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope_op_debug_print scope_dbg_print(__func__, dst, /*num_src=*/1);

this code will trigger constructor function when entry ggml_sycl_dup().
When call ggml_sycl_dup() n times, the constructor function will be called n times.

Legacy code is the static code. It won't take more time when entry ggml_sycl_dup().

Copy link
Collaborator

@qnixsynapse qnixsynapse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.
Is it possible to extend this to backend interface functions as well such as init, set, memset, clear, reset, etc?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants