Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add electricity to internal datasets #60

Merged
merged 15 commits into from Aug 29, 2023
Merged

Conversation

ostreech1997
Copy link
Collaborator

@ostreech1997 ostreech1997 commented Aug 24, 2023

Before submitting (must do checklist)

  • Did you read the contribution guide?
  • Did you update the docs? We use Numpy format for all the methods and classes.
  • Did you write any new necessary tests?
  • Did you update the CHANGELOG?

Proposed Changes

Closing issues

Closes #51.

@github-actions
Copy link

github-actions bot commented Aug 24, 2023

🚀 Deployed on https://deploy-preview-60--etna-docs.netlify.app

@github-actions github-actions bot temporarily deployed to pull request August 24, 2023 16:35 Inactive
Copy link
Collaborator

@d-a-bunin d-a-bunin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw the error with tests. I assume that the problem is in dataset size. Probably we should discuss that with other team members. Let's fix the other things first.

.binder/requirements.txt Outdated Show resolved Hide resolved
etna/datasets/internal_datasets.py Outdated Show resolved Hide resolved
etna/datasets/internal_datasets.py Outdated Show resolved Hide resolved
etna/datasets/internal_datasets.py Outdated Show resolved Hide resolved
os.makedirs(dataset_dir, exist_ok=True)
with warnings.catch_warnings():
warnings.simplefilter("ignore")
data = _download_dataset_zip(url=url, file_name="LD2011_2014.txt", sep=";")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of warnings could you meet there?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we load LD2011_2014.txt via pd.read_csv, it returns huge warning message like this:

DtypeWarning: Columns (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18, ... ,368,369,370) have mixed types. Specify dtype option on import or set low_memory=False.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we specify dtypes somehow to avoid this?

etna/datasets/internal_datasets.py Show resolved Hide resolved
etna/datasets/internal_datasets.py Outdated Show resolved Hide resolved
tests/test_datasets/test_internal_datasets.py Outdated Show resolved Hide resolved
@ostreech1997 ostreech1997 added the datasets Work with datasets label Aug 25, 2023
@ostreech1997 ostreech1997 added this to the Internal datasets milestone Aug 25, 2023
@github-actions github-actions bot temporarily deployed to pull request August 25, 2023 16:26 Inactive
@github-actions github-actions bot temporarily deployed to pull request August 28, 2023 08:02 Inactive
Copy link
Collaborator

@d-a-bunin d-a-bunin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look at comments above.

@github-actions github-actions bot temporarily deployed to pull request August 28, 2023 13:38 Inactive
@github-actions github-actions bot temporarily deployed to pull request August 28, 2023 13:48 Inactive
@github-actions github-actions bot temporarily deployed to pull request August 28, 2023 14:01 Inactive
@ostreech1997 ostreech1997 merged commit 959b3a0 into internal_datasets Aug 29, 2023
13 checks passed
@ostreech1997 ostreech1997 deleted the issue-51 branch August 29, 2023 10:38
ostreech1997 added a commit that referenced this pull request Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasets Work with datasets
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants