Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix METdataio isn't correctly placing the database in the correct METviewer group #228

Closed
8 of 22 tasks
bikegeek opened this issue Aug 29, 2023 · 2 comments · Fixed by #229
Closed
8 of 22 tasks
Assignees
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: input data Input data issue component: METdataio METdataio: General priority: high High Priority requestor: METplus Team METplus Development Team type: bug Fix something that is not working

Comments

@bikegeek
Copy link
Collaborator

bikegeek commented Aug 29, 2023

Describe the Problem

Tina has found that when explicitly setting the group to RAL Projects in the XML specification file, the database is loaded in the METviewer mariaDB database, but is placed in the incorrect group.

Expected Behavior

on 'mohawk', using the following script to set up the METviewer database (MAKE MODIFICATIONS TO CREATE A DIFFERENT DATABASE):

#!/bin/sh -v
 
 dbname="mv_debugging"
 #host="mohawk"
 host="localhost"
 #metv="/d2/projects/METViewer/src/apps/METviewer"
 metv="/d2/personal/mwin/Debug_METdataio_for_METviewer/METdataio/METdbLoad"
 #path2proj="/d2/projects/UAE_tk/metv"
 path2proj="/d2/personal/mwin/Debug_METdataio_for_METviewer/METdataio/METdbLoad/sql"
 mvuser="mvuser"
 #mvuser="mvadmin"
 mvpass="mvuser"
 #mvpass="160GiltVa0D5M"

 # Drop the database before creating
 mysql -h${host} -u${mvuser} -p${mvpass} -e"drop database ${dbname};"
 mysql -h${host} -u${mvuser} -p${mvpass} -e"create database ${dbname};"
 mysql -h${host} -u${mvuser} -p${mvpass} ${dbname} < ${metv}/sql/mv_mysql.sql
 ${metv}/bin/mv_load.sh ${path2proj}/load_${dbname}.xml > ${path2proj}/load_${dbname}.log 2>&1


Next, use the following specification file (MAKE MODIFICATIONS TO MATCH THE DATABASE CREATED ABOVE and point to the correct paths for the schema):

<load_spec>
	<connection>
		<host>mohawk</host>
		<database>mv_uae_2022_upa</database>
		<user>mvuser</user>
		<password>mvuser</password>
	</connection>

        <date_list name="folder_dates">
        <start>2023010100</start>
        <end>2023073018</end>
        <inc>21600</inc>
        <format>yyyyMMddHH</format>
        </date_list>

	<folder_tmpl>/d2/projects/UAE_tk/data/{datadir}/{ltype}/{date}</folder_tmpl>


	<verbose>true</verbose>
	<insert_size>1</insert_size>
	<stat_header_db_check>true</stat_header_db_check>
	<mode_header_db_check>true</mode_header_db_check>
	<drop_indexes>false</drop_indexes>
	<apply_indexes>true</apply_indexes>
	<load_stat>true</load_stat>
	<load_mode>false</load_mode>
	<load_mpr>true</load_mpr>
	<group>RAL Projects</group>

	<load_val>
		<field name="datadir">
			<val>EnVar_G1V_stats</val>
                        <val>EnVar_G2V_stats</val>
		</field>
                <field name="ltype">
                        <val>point_stat_ua</val>
                </field>
                <field name="date">
			<date_list name="folder_dates"/>
                </field>
	</load_val>

	<load_xml>true</load_xml>
</load_spec>

Verify that the new database you created above is NOT in the RAL Projects group via METviewer:
http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp

by looking under the RAL Projects group for the database you created.

Environment

Describe your runtime environment:
1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop)
2. OS: (e.g. RedHat Linux, MacOS)
3. Software version number(s)

To Reproduce

Describe the steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
Post relevant sample data following these instructions:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-help-desk#ftp

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) required
  • Select no scientist required

Labels

  • Select component(s)
  • [x]Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED CYCLE ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Add any new Python packages to the METplus Components Python Requirements table.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issues
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@bikegeek bikegeek added priority: high High Priority type: bug Fix something that is not working requestor: METplus Team METplus Development Team METdataio: General component: METdataio component: input data Input data issue labels Aug 29, 2023
@bikegeek bikegeek added this to the METdataio-3.0.0 milestone Aug 29, 2023
@bikegeek bikegeek self-assigned this Aug 29, 2023
@bikegeek bikegeek added the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Aug 29, 2023
@TatianaBurek
Copy link
Collaborator

loading code expects both tags to be present and . Only in this case 'group' data is loaded into the database.
In reality, is optional. Need to change the code to load even the is missing

@TatianaBurek
Copy link
Collaborator

TatianaBurek commented Aug 30, 2023

The initial value of self.description is 'None'. it should be an empty sting.
The logic for inserting group and description also needs to be fixed also

TatianaBurek added a commit that referenced this issue Aug 30, 2023
get values for group and description separately #228
@TatianaBurek TatianaBurek linked a pull request Aug 31, 2023 that will close this issue
14 tasks
bikegeek pushed a commit that referenced this issue Aug 31, 2023
* update both - group and description - when one or both was changed #228

* change the initial value of description
get values for group and description separately #228
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: input data Input data issue component: METdataio METdataio: General priority: high High Priority requestor: METplus Team METplus Development Team type: bug Fix something that is not working
Projects
Status: ✅ Done
Development

Successfully merging a pull request may close this issue.

2 participants