New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructuring of QWIPU directories #83

Closed
srt1 opened this Issue Jan 2, 2018 · 4 comments

Comments

Projects
3 participants
@srt1
Collaborator

srt1 commented Jan 2, 2018

Heath has proposed that the QWIPU files be reorganized to eliminate the DVD directories, and change the nesting ordering to be more parallel to J2J and LODES. This would mean reordering the nesting on the national QWI, and potentially also mean moving the state qwi files into the /data directory of the website, rather than /pub. The primary reason to do this is to eliminate the second copy of the state qwipu files (though possibly this could be done through symbolic links, if permitted?).

This could also have implications for the version.txt and manifest files, which are currently separate by DVD directory. It makes little sense to have separated files, if the data files are all sitting in the same directory. Plus, there was never much reason to have separate version files for the demographic characteristics, since there is no design to ever have those different. The F/FA/FS is the only dimension on which the date ranges differ.

If we go ahead with the current Redmine ticket (number 1746), we may have some things to adjust in the schema, as far as the location and organization of the csv files.

@srt1 srt1 added this to the V4.3.0 milestone Jan 2, 2018

@srt1

This comment has been minimized.

Show comment
Hide comment
@srt1

srt1 Feb 20, 2018

Collaborator

From: Camille S Norwood (CENSUS/CES FED) Camille.S.Norwood@census.gov
Sent: Thursday, February 15, 2018 4:06:10 PM
To: Patrick Hayward (CENSUS/CES FED)
Cc: Chaoling Zheng (CENSUS/CES FED); Lars Vilhuber; Matthew Graham (CENSUS/CES FED); Stephen R Tibbets (CENSUS/CES FED)
Subject: Re: Proposal to restructure QWIPU directories

Proof-of-Concept Vintage Ready for Review: /data/production/dev5/run/qwipu/de/20180215_1003

This vintage has all DVD subdirectories removed


This vintage has stopped generating the ISO file

This vintage has collapsed all:

    version files into one version.txt file

    manifest files into one qwipu_<state>_manifest.txt file

    md5sum files into one qwi_<state>.txt file

Collaborator

srt1 commented Feb 20, 2018

From: Camille S Norwood (CENSUS/CES FED) Camille.S.Norwood@census.gov
Sent: Thursday, February 15, 2018 4:06:10 PM
To: Patrick Hayward (CENSUS/CES FED)
Cc: Chaoling Zheng (CENSUS/CES FED); Lars Vilhuber; Matthew Graham (CENSUS/CES FED); Stephen R Tibbets (CENSUS/CES FED)
Subject: Re: Proposal to restructure QWIPU directories

Proof-of-Concept Vintage Ready for Review: /data/production/dev5/run/qwipu/de/20180215_1003

This vintage has all DVD subdirectories removed


This vintage has stopped generating the ISO file

This vintage has collapsed all:

    version files into one version.txt file

    manifest files into one qwipu_<state>_manifest.txt file

    md5sum files into one qwi_<state>.txt file

@larsvilhuber larsvilhuber added this to In progress in Schemas Mar 24, 2018

@larsvilhuber

This comment has been minimized.

Show comment
Hide comment
@larsvilhuber

larsvilhuber May 25, 2018

Member

Also see discussion about CONTENT of version.txt (now version_qwi.txt) in #91

Member

larsvilhuber commented May 25, 2018

Also see discussion about CONTENT of version.txt (now version_qwi.txt) in #91

@larsvilhuber

This comment has been minimized.

Show comment
Hide comment
@larsvilhuber

larsvilhuber Jun 21, 2018

Member

Of course, now that I look at it, we haven't actually described the download locations in the past...

I will add something to the CSV_Naming file, which SHOULD actually describe the full path.

Member

larsvilhuber commented Jun 21, 2018

Of course, now that I look at it, we haven't actually described the download locations in the past...

I will add something to the CSV_Naming file, which SHOULD actually describe the full path.

larsvilhuber added a commit that referenced this issue Jul 13, 2018

larsvilhuber added a commit that referenced this issue Jul 13, 2018

larsvilhuber added a commit that referenced this issue Jul 13, 2018

larsvilhuber added a commit that referenced this issue Jul 13, 2018

larsvilhuber added a commit that referenced this issue Jul 13, 2018

Schemas automation moved this from In progress to Done Jul 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment