stats: allow custom histogram buckets

[kyessenov]

Compromise to #2539, to prefer over the bootstrap EnvoyFilter.

@zirain

[kyessenov]

/retest

[zirain]

this's a feature only supported by envoy, using annotation is LGTM.

[costinm]

I think it's reasonable - we have quite a few things exposed as annotations.

One concern about pod annotations is that in Ambient they will likely cease functioning - except for Waypoints, where the annotation will go on the Gateway resource. But that's the case for most CRDs we have with workload selector, so nothing special.

On this note - would be good to add K8S Gateway as resource that will support the annotation ( they get copied to the deployment AFAIK ).

[costinm]

• document the default
• why do we need istio and envoy ? Can one be used for both ?

[howardjohn]

If we do need them separate, should we just make this json? Its not that complicated but we are essentially creating a micro-language we need to parse.

if we just make them one then its a simple CSV so no problem

[kyessenov]

Updated. Prefix is not needed but usually used.

[costinm]

Worth mentioning that customizing stats bucket for specific pods is actually best granularity for such an API - the defaults should work for most common cases, overrides are needed for pods that have unusual behavior, like long lived connections ( istiod itself ).

[howardjohn]

Looks good overall as an experimental annotation, just one comment on the formatting (doesn't need to block, just want to make sure it is at least ACKed before moving forward)

[howardjohn]

If we do need them separate, should we just make this json? Its not that complicated but we are essentially creating a micro-language we need to parse.

if we just make them one then its a simple CSV so no problem

[howardjohn]

One small comment, otherwise LGTM. feel free to remove hold

[howardjohn]

Suggested change

	description: Specifies the custom histogram buckets with a prefix matcher to separate the Istio mesh metrics from the Envoy stats, e.g. `{"istio":{1,5,10,50,100,500,1000,5000,10000},"envoy":{1,5,10,25,50,100,250,500,1000,2500,5000,10000}}`. Default buckets are `{0.5,1,5,10,25,50,100,250,500,1000,2500,5000,10000,30000,60000,300000,600000,1800000,3600000}`.
	description: Specifies the custom histogram buckets with a prefix matcher to separate the Istio mesh metrics from the Envoy stats, e.g. `{"istio":[1,5,10,50,100,500,1000,5000,10000],"envoy":[1,5,10,25,50,100,250,500,1000,2500,5000,10000]}`. Default buckets are `[0.5,1,5,10,25,50,100,250,500,1000,2500,5000,10000,30000,60000,300000,600000,1800000,3600000]`.

Current one is not valid json

[howardjohn]

can we specify what the unit is as well? milliseconds?

[kyessenov]

There's no unit. We have both byte count and milliseconds in use.