I'm creating documents from web scraping in python and uploading them to Firestore.
To do so I'm adding them to a dictionary and uploading them from a for loop in python, one by one(ideally would be better to upload the collection at once, but that doesn't seem an option). I want to use batches, however they have the 500 limit per batch and I need to do more than 100,000 operations. The operations are merely set() operations and a couple of update()
Is there a function to know the current size of the batch so I can reinitialize it?
What is the best way to use batches for more than 500 operations in python?