I am implementing a system using Google Cloud Speech-to-Text (STT) services via gRPC streams and need a reliable way to track usage per client for quota management. I want Google Cloud's logging system to be the authoritative source of truth for billable minutes used by each end user. The challenge is associating specific gRPC requests with my internal client IDs within the Google Cloud Log Explorer.

What I've tried: The Google documentation on Audit Logging suggests using a specific custom metadata header prefix (x-goog-custom-audit-) to include custom data within the audit logs. I attempted to inject my custom data into the gRPC metadata like this (using Python):

self.grpc_metadata = [
  ("x-goog-custom-audit-audio-topic", client_id),  # My custom client identifier
]

The gRPC streams function correctly with this metadata attached, so the API is accepting the headers.

The Issue: Despite sending the requests with the correctly formatted header, I cannot detect any custom data in the Google Cloud Log Explorer afterward. The relevant information does not appear within the protoPayload or jsonPayload of the STT service logs.

How can I effectively track usage per user using Google Cloud's built-in logging mechanisms, ensuring the custom metadata actually appears in the logs?

Alternatively, is there another recommended method for reconciling usage data when using high-throughput gRPC streams that don't involve manually aggregating local metrics?