I'm somewhat new to RabbitMQ and I'm currently going through a bit of a strange situation at work that I don't really know how to handle other than by using a workaround, was curious if anyone knows exactly how to fix this.

So in a certain part of our messaging diagram, we have a fanout exchange (let's call it A) and A is connected to some queues (let's call one of them B) and it is also connected to a header exchange, which we can call C. C has around 5 different connections to the same queue depending on the message headers (old implementation), we're interested in the flow of the messages going to a particular queue which we can call D

A --> B
--> C --> D

For some reason, when I send messages from my publisher service to fanout exchange A, they reach queue B in about 8 seconds after publishing, which is alright. But it takes about 1 minute and 20s to reach queue D for some reason. I can't really find much information about it online either

Below is sort of the structure we have in our rmq script

"exchanges": [
  { "name": "FanoutA",        "vhost": "Vhost", "type": "fanout",  "durable": true },
  { "name": "HeaderC",  "vhost": "Vhost", "type": "headers", "durable": true },
],
"queues": [
  { "name": "QueueD", "vhost": "Vhost", "durable": true, "arguments": { "x-queue-type": "quorum" } },
  { "name": "QueueE",   "vhost": "Vhost", "durable": true, "arguments": { "x-queue-type": "quorum" } }
],
"bindings": [
  { "source": "FanoutA",       "vhost": "Vhost", "destination": "HeaderC",          "destination_type": "exchange", "routing_key": "" },
  { "source": "HeaderC", "vhost": "Vhost", "destination": "QueueD", "destination_type": "queue",    "routing_key": "", "arguments": { "IsoCode": 392 } },
  { "source": "HeaderC", "vhost": "Vhost", "destination": "QueueE",   "destination_type": "queue",    "routing_key": "", "arguments": { "Category": "Issue" } },
  { "source": "HeaderC", "vhost": "Vhost", "destination": "QueueE",   "destination_type": "queue",    "routing_key": "", "arguments": { "Category": "Solve" } },
  { "source": "HeaderC", "vhost": "Vhost", "destination": "QueueE",   "destination_type": "queue",    "routing_key": "", "arguments": { "Category": "Math" } },
  { "source": "HeaderC", "vhost": "Vhost", "destination": "QueueE",   "destination_type": "queue",    "routing_key": "", "arguments": { "Category": "Genre" } },
  { "source": "HeaderC", "vhost": "Vhost", "destination": "QueueE",   "destination_type": "queue",    "routing_key": "", "arguments": { "Category": "Apply" } }, },

The workaround I found is to remove the connection from fanout exchange to the header exchange and shovel the messages from Queue B directly to the header exchange. For some reason this is super fast. Any idea what might be going on? Could it be a problem with the publisher service itself? The rabbitMQ version is 4.0.1

Also important to mention, this doesnt happen under heavy load. It is occurring even with simply one message at a time