I'm trying to copy a a TS table from one storage account to another with an ADF pipeline, using a Copy activity. I want a copy that is 100% identical with the source table.
The table does not have a stable schema, meaning that many different kinds of entities are stored inside it with many different properties. This is why I defined an explicit mapping with all the possible properties and their data types.
Based on the documentation I realized that for tables with no stable schema I need to define an explicit mapping for all the possible columns/properties, otherwise ADF just checks the first entity in the table for schema definition and ignores all the properties that cannot be found on the first entity. Given that there are hundreds of possible property names, I'm generating the ADF mappings with a script, I don't define them in the UI.
Some example mapping look like like this:
{
"source": {
"name": "UseABCClassification",
"type": "Boolean"
},
"sink": {
"name": "UseABCClassification",
"type": "Boolean"
}
}
{
"source": {
"name": "CreationDate",
"type": "DateTimeOffset"
},
"sink": {
"name": "CreationDate",
"type": "DateTimeOffset"
}
}
Looking in the UI they are displayed correctly:


However, once I run the pipeline all the data gets converted into the String data type. Checking the result in Azure Storage Explorer:


The same thing happens, no matter what the source data type is, the results are always String. Any help on how I can obtain a 100% identical copy of my source table in this case would be highly appreciated.