How can I enable XNNPACK on a Raspberry Pi 5 for an INT8 TensorFlow Lite model?

I have a quantized INT8 TensorFlow Lite model that I want to run efficiently on my Raspberry Pi 5. I’ve read that XNNPACK can speed up inference on CPU, but I’m not sure how to enable or verify that it’s working on the Pi 5.

My setup:

Raspberry Pi 5 (64-bit Raspberry Pi OS)

Python 3.11

TensorFlow Lite (installed via tflite-runtime)

My questions:

Does XNNPACK work on the Raspberry Pi 5 CPU (Cortex-A76)?

How can I enable XNNPACK when using a quantized INT8 TFLite model in Python?

How can I verify that XNNPACK is actually being used during inference? your text

What I’ve tried:

Installing tflite-runtime via pip

Using the experimental_delegates argument

Searching for libtensorflowlite_delegate_xnnpack.so (but can’t find it on the Pi)

Expected result: Faster inference using XNNPACK delegate on the Raspberry Pi 5.

Actual result: Inference runs, but I don’t know if XNNPACK is active, and I don’t see any speed improvement.

import tensorflow as tf
import numpy as np

# Load model
interpreter = tf.lite.Interpreter(model_path="model_int8.tflite")

# Enable XNNPACK (if possible)`
interpreter = tf.lite.Interpreter(model_path="model_int8.tflite", 
                                  experimental_delegates=[tf.lite.experimental.load_delegate('libtensorflowlite_delegate_xnnpack.so')])

interpreter.allocate_tensors()

# Run inference
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_data = np.random.randint(0, 255, size=input_details[0]['shape'], dtype=np.uint8)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)type here