With this dtype defined (note reverse-alpha field order):
Similarity = pl.Struct([
pl.Field('zavg', pl.Float64),
pl.Field('ystd', pl.Float64),
pl.Field('xbase_indices', pl.List(pl.Int64))
])
When I run this map_elements returning a fixed dict that matches Similarity in keys, type, and order:
df = pl.DataFrame({'x': [0, 1]})
df.with_columns((
pl.col('x')
.map_elements(lambda d: {'zavg':1.1, 'ystd': 0.1, 'xbase_indices': [2]},
return_dtype=Similarity)
.alias(col+'_Blah')
for col in ['item_1A', 'item_7']))
I get this SchemaError with key order mismatch (note key order):
SchemaError: expected output type
Struct([
Field { name: "zavg", dtype: Float64 },
Field { name: "ystd", dtype: Float64 },
Field { name: "xbase_indices", dtype: List(Int64) }])
got (note alphabetical key order)
Struct([
Field { name: "xbase_indices", dtype: List(Int64) },
Field { name: "ystd", dtype: Float64 },
Field { name: "zavg", dtype: Float64 }])
set return_dtype to the proper datatype
If I remove the return_dtype it works. Further experiments suggest Polars is reading the dict keys in alphabetical order.
Expected? How should I handle this?