I've started to work on caching and queuing in my back-end using Redis and BullMQ.
It's my first time ever; so I don't know much about the strategies or how to ensure that I won't lose any data. Since BullMQ requires that Redis doesn't remove any data, that means I have to manage the expiring time carefully and efficiently to avoid reaching the out-of-memory error.
Here is my concern:
What would happen to the failed data?
Let's say data that is batched to the cache, after a certain period, it is saved in the db. suppose that something happened to the db connection or to the server itself. how should I handle this situation?
currently, I've decided not to set any expiration time to the cached data, and only delete BullMQ completed jobs. am I right in doing this? this means the cache would be at risk of being filled.