You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The German Weather Service has stored his metadata in a poor format, which has no seperators (neither comma nor tab). Instead of reading the data like we currently do it (rowwise reading and checking of number of chunks, then processing it with respect to this chunk number),
`
from fix_metaindex
metaindex_to_fix = metaindex.iloc[:, 6:]
# Reduce the original dataframe by those columns
metaindex = metaindex.iloc[:, :6]
# Index is fixed by string operations (put together all except the last
# string which refers to state)
metaindex_to_fix = metaindex_to_fix \
.agg(lambda data: [string
for string in data
if string is not None], 1) \
.to_frame() \
.iloc[:, 0] \
.agg(lambda data: [' '.join(data[:-1]), data[-1]]) \
.apply(pd.Series)
# Finally put together again the original frame and the fixed data
metaindex = pd.concat([metaindex, metaindex_to_fix], axis=1)
`
we should find another way, where we can read it directly with pandas with some predefinitions like read_fwf (fixed width formatted). This should give as a performance boost and also fix some other issues with string handling.
The text was updated successfully, but these errors were encountered:
The German Weather Service has stored his metadata in a poor format, which has no seperators (neither comma nor tab). Instead of reading the data like we currently do it (rowwise reading and checking of number of chunks, then processing it with respect to this chunk number),
`
from fix_metaindex
`
we should find another way, where we can read it directly with pandas with some predefinitions like read_fwf (fixed width formatted). This should give as a performance boost and also fix some other issues with string handling.
The text was updated successfully, but these errors were encountered: