Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: melt method doesn't seem to preserve timezone settings. #55254

Closed
3 tasks done
sasaki000 opened this issue Sep 23, 2023 · 1 comment · Fixed by #55270
Closed
3 tasks done

BUG: melt method doesn't seem to preserve timezone settings. #55254

sasaki000 opened this issue Sep 23, 2023 · 1 comment · Fixed by #55270
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member

Comments

@sasaki000
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame(data=[
    {'type': 'A0', 'start_date': pd.Timestamp('2023/03/01', tz='Asia/Tokyo'), 'end_date': pd.Timestamp('2023/03/10', tz='Asia/Tokyo')},
    {'type': 'A1', 'start_date': pd.Timestamp('2023/03/01', tz='Asia/Tokyo'), 'end_date': pd.Timestamp('2023/03/11', tz='Asia/Tokyo')},
], index=['aaaa', 'bbbb'])
display(df)

df = df.melt(id_vars=['type'], value_vars=['start_date', 'end_date'], var_name='start/end', value_name='date')
display(df)

Issue Description

The melt method doesn't seem to preserve timezone settings.

Expected Behavior

Even after executing the melt method, it is desirable that the time zone settings remain the same as before executing the melt method.

Installed Versions

Name: pandas
Version: 2.1.1
Summary: Powerful data structures for data analysis, time series, and statistics
Home-page: https://pandas.pydata.org/
Author:
Author-email: The Pandas Development Team pandas-dev@python.org
License: BSD 3-Clause License

    Copyright (c) 2008-2011, AQR Capital Management, LLC, Lambda Foundry, Inc. and PyData Development Team
    All rights reserved.
    
    Copyright (c) 2011-2023, Open source contributors.
    
    Redistribution and use in source and binary forms, with or without
    modification, are permitted provided that the following conditions are met:
    
    * Redistributions of source code must retain the above copyright notice, this
      list of conditions and the following disclaimer.
    
    * Redistributions in binary form must reproduce the above copyright notice,
      this list of conditions and the following disclaimer in the documentation
      and/or other materials provided with the distribution.
    
    * Neither the name of the copyright holder nor the names of its
      contributors may be used to endorse or promote products derived from
      this software without specific prior written permission.
    
    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
    AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
    IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
    DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
    FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
    DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
    SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
    CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
    OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Location: /home/sasaki/workspace/orcas_proj/venv/lib/python3.10/site-packages
Requires: numpy, python-dateutil, pytz, tzdata
Required-by: bokeh, mplfinance, seaborn

@sasaki000 sasaki000 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 23, 2023
@samukweku
Copy link
Contributor

pivot_longer preserves the dtypes:

# pip install pyjanitor
import pandas as pd
import janitor

df.pivot_longer(index='type', names_to='start/end', values_to='date')

  type   start/end                      date
0   A0  start_date 2023-03-01 00:00:00+09:00
1   A1  start_date 2023-03-01 00:00:00+09:00
2   A0    end_date 2023-03-10 00:00:00+09:00
3   A1    end_date 2023-03-11 00:00:00+09:00

with respect to pd.melt, I suspect the issue lies in line 137 - the same logic applied to id_vars in line 125 should be replicated there as well. Not a bad idea @sasaki000 if you raise a PR for this

paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 25, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 25, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 25, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 27, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 27, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 28, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 28, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 29, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Sep 30, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Oct 2, 2023
paulreece added a commit to paulreece/pandas-paulreece that referenced this issue Oct 3, 2023
mroeschke pushed a commit that referenced this issue Oct 3, 2023
…e melt method. (#55270)

This fix enables you to preserve the datetime precision when using the melt method. It fixes the issues raised in #55254.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member
Projects
None yet
2 participants