I'm generating a single MBTiles file for a very large geographic area (~200 GB of source polygon data, split into 128 parquet chunks for processing).

My current pipeline:

1. Process each of the 128 source parquets through tippecanoe → produces 128 individual .mbtiles "part" files (~6.25 GB each, totaling ~800 GB)

2. Attempt to merge all parts using tile-join into a single final MBTiles file

The problem:

tile-join is unable to complete the merge at this scale. I've tried:

Approach 1: Single-shot merge (all 128 parts at once)

bash

Copy Code

tile-join -f -o final.mbtiles part_0000.mbtiles part_0001.mbtiles ... part_0127.mbtiles

This never completes (killed by cluster after hours).

Approach 2: Hierarchical batch merging (16 parts per batch)

bash

Copy Code

# Phase 1: Merge each group of 16 parts
tile-join -f -o group_000.mbtiles part_0000.mbtiles ... part_0015.mbtiles
# ... (8 groups total, each ~100 GB input)

# Phase 2: Merge the 8 groups
tile-join -f -o final.mbtiles group_000.mbtiles ... group_007.mbtiles

Even Phase 1 fails — tile-join runs for 3+ hours on a single batch of 16 parts (~100 GB input) without completing.

Environment:

• Running on Linux with 1.7 TB local disk (plenty of space)

• Writing output to local disk (SQLite writes require local FS)

• tile-join version: from mapbox/tippecanoe conda-forge package

• Sufficient RAM (>60 GB)

What I've tried:

• Increasing timeout (up to 3 hours per batch — still not enough)

• Reducing batch size (16 parts still takes too long)

• Ensuring output is on fast local disk (not network storage)

Questions:

1. Is tile-join designed to handle inputs at this scale (~800 GB total)? Or is there a practical upper limit I'm hitting?

2. Are there any flags or options to speed up merging for very large inputs (e.g., parallelism, streaming mode)?

3. Is there an alternative tool better suited for merging many large MBTiles files at continental scale?

4. Would a different partitioning strategy help — e.g., merging in smaller iterative rounds (4 parts at a time × more rounds)?

Data characteristics:

• Polygon features (square polygons around lat/lon points)

• Millions of features per part

• Zoom levels 0-14

• Feature density is high (dense flood risk polygons across the entire region)

Any guidance on scaling tile-join to this size, or alternative approaches, would be appreciated.