Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When saving to CF prepend datasets starting with a digit by CHANNEL_ #1525

Merged
merged 41 commits into from
Mar 2, 2021
Merged
Changes from 2 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
5af9f73
Prepend datasets begining with a digit by CHANNEL_
Feb 1, 2021
bda9f1c
Fix newline
Feb 1, 2021
9bc7821
Merge remote-tracking branch 'origin/master' into dataset_names_when_…
Feb 12, 2021
c537672
parsing of wavelength from netcdf variable attribute
Feb 12, 2021
1c1898a
stickler long line
Feb 12, 2021
9d55746
flake8 lint b007
Feb 12, 2021
9dc8d27
codebeat nesting to deep
Feb 12, 2021
02a3ce2
lint
Feb 12, 2021
8de0bb9
More propper name
Feb 12, 2021
a0d916c
Remove not needed code. Some test
Feb 13, 2021
7e8b651
Add test to cf writer dataset starting with a digit
Feb 14, 2021
c0859bd
Allow passing flag to turn on valid cf dataset name
Feb 15, 2021
c346926
Update satpy/writers/cf_writer.py
TAlonglong Feb 16, 2021
eb25edd
Merge remote-tracking branch 'origin/master' into dataset_names_when_…
Feb 16, 2021
3783dc7
Rewrite accurding to review
Feb 17, 2021
563b974
Merge branch 'dataset_names_when_saving_to_cf' of github.com:TAlonglo…
Feb 17, 2021
8113a47
flake8
Feb 17, 2021
d795043
Set default value
Feb 17, 2021
680cef6
codebeat
Feb 17, 2021
b9aeadf
Add some to the docs
Feb 17, 2021
3a01818
Adjustments use CHANNEL_ as default
Feb 17, 2021
436ac0c
Update tests
Feb 17, 2021
a38f08f
Merge remote-tracking branch 'origin/master' into dataset_names_when_…
Feb 24, 2021
5604246
Passing numeric_name_prefix to the reader with default as the writer
Feb 24, 2021
152c123
Added test for the satpy_cf_nc reader prefixing
Feb 25, 2021
b11317c
codebeat
Feb 25, 2021
ae54ba6
codefactor len(sequence)
Feb 25, 2021
57d5fea
Refactor test. Added test different prefix write and read
Feb 26, 2021
7dec6fb
flake8
Feb 26, 2021
e5b8c17
Possible to include original_name in nc var attrs
Mar 1, 2021
db34a63
Fix and add test
Mar 1, 2021
5a39dbc
Try fix test again
Mar 1, 2021
88d8320
fix ds_id vis ds_info
Mar 2, 2021
0351ba4
deepcode
Mar 2, 2021
19f9397
Deepcode2
Mar 2, 2021
6d66cf9
codebeat nesting to deep, complexity
Mar 2, 2021
4b4fcd1
Fix assign_ds_info
Mar 2, 2021
ebdce35
include_orig_name=True
Mar 2, 2021
0042192
use np.testing.assert_array_equal. Fix one test
Mar 2, 2021
8eaeedd
Only add original_name attr when prefix is used
Mar 2, 2021
3e9d293
replace assertequal with np.testing.assert_array_equal
Mar 2, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions satpy/writers/cf_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,11 @@ def update_encoding(dataset, to_netcdf_kwargs):
other_to_netcdf_kwargs = to_netcdf_kwargs.copy()
encoding = other_to_netcdf_kwargs.pop('encoding', {}).copy()

for var_name, variable in dataset.variables.items():
if 'satpy_dataset_name' in variable.attrs:
if variable.attrs['satpy_dataset_name'] in encoding:
encoding['CHANNEL_' + variable.attrs['satpy_dataset_name']] = encoding.pop(variable.attrs['satpy_dataset_name'])
TAlonglong marked this conversation as resolved.
Show resolved Hide resolved

_set_default_chunks(encoding, dataset)
_set_default_fill_value(encoding, dataset)
_set_default_time_encoding(encoding, dataset)
Expand Down Expand Up @@ -507,6 +512,9 @@ def da2cf(dataarray, epoch=EPOCH, flatten_attrs=False, exclude_attrs=None, compr
new_data = dataarray.copy()
if 'name' in new_data.attrs:
name = new_data.attrs.pop('name')
if name[0].isdigit():
new_data.attrs['satpy_dataset_name'] = name
name = 'CHANNEL_' + name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about renaming this keyword argument numeric_name_prefix and if it is None, don't do anything. If it is specified then it is assumed to be a string. So someone could do scn.save_datasets(writer='cf', numeric_name_prefix='ch') if they wanted to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I second that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a good idea.

I just have not figured out how to read this back if we are to skip the extra attribute I have added for now.

new_data = new_data.rename(name)

# Remove _satpy* attributes
Expand Down