I am designing a Centralized Watcher (or Auditor) microservice within a distributed system. This service's primary responsibility is to cross-reference and validate data across multiple domains to ensure business compliance.
We are dealing with high-volume batch processing. For instance, importing 10,000+ records from an external provider and validating each one against existing data owned by other microservices (e.g., an OrderService or a BiddingService).
To optimize performance, I’m considering a Database-level Read-Only Integration. Since all our services use PostgreSQL and sit within the same AWS VPC, I am looking at using PostgreSQL Foreign Data Wrappers (FDW) to perform cross-database JOINs directly.
Example use cases:
1. One microservice in this domain is responsible for managing, versioning, creating, and sending orders to users. The Watcher imports operations from an external system and checks whether users have accepted their orders—if not, it creates a control case. In the worst case, the orders table that the Watcher queries can have millions of entries.
2. Another microservice in this environment is responsible for issuing biddings/tenders. Again, there is a control regarding these tenders validating certain aspects, and it will be handled by this Watcher service.
The idea is that all control cases for these different entities will be managed and centralized by this Watcher service.
Architecture Details:
Tech Stack: Java (Quarkus/Spring), PostgreSQL (AWS RDS).
Ownership: All services are owned by the same team and reside in the same domain.
Watcher Role: It manages "Control Cases" based on the state of entities in other databases (e.g., if an Order exists in
OrderDBbut has a specific flag, create a record inWatcherDB).
I have the following questions:
Is there a formal name for this "Watcher/Eye" pattern where a service has read-only access to other databases for compliance?
If FDW is acceptable, what are its best practices?
Are there better alternatives for high-performance cross-service data validation that maintain loose coupling?
I know it's not ideal, but if handled correctly, is reading directly from the database a good solution for quick retrieval of information? More so if all the services will be owned by the same team within the same domain?
How would I handle syncing information between databases? For instance, when an order is accepted, the Order Service flags it in its database but doesn't know about the Watcher service's existence. This information needs to be synced eventually.