Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NGAS subscription can only handle files with crc32c checksums #44

Closed
smclay opened this issue Mar 6, 2021 · 2 comments
Closed

NGAS subscription can only handle files with crc32c checksums #44

smclay opened this issue Mar 6, 2021 · 2 comments
Labels
bug

Comments

@smclay
Copy link
Contributor

smclay commented Mar 6, 2021 •
edited
Loading

I created a test where I archive files specifying different checksum variants...

ngas-cmd -a localhost -p 8001 archive filename=test_sample_crc32.txt crc_variant=crc32
ngas-cmd -a localhost -p 8001 archive filename=test_sample_crc32c.txt crc_variant=crc32c
ngas-cmd -a localhost -p 8001 archive filename=test_sample_crc32z.txt crc_variant=crc32z

Only the file archived using crc32c is successfully pushed to the a subscriber. Looking at the logs from the subscriber I see the following error messages...

2021-03-06T17:54:15.181 [20092] [     R-510] [  INFO] ngamsServer.ngamsArchiveUtils#_dataHandler:1083 NGAMS_INFO_ARCHIVING_FILE:4026:INFO: Archiving file with URI: test_sample_crc32z.txt
2021-03-06T17:54:15.199 [20092] [     R-510] [  INFO] ngamsServer.ngamsArchiveUtils#_dataHandler:1101 Archiving file: test_sample_crc32z.txt with mime-type: text/plain
2021-03-06T17:54:15.200 [20092] [SUBSCRIBER] [WARNIN] ngamsServer.ngamsSrvUtils#_create_remote_subscriptions:140 Unsuccessful NGAS XML response. Status: FAILURE, message: ORA-00001: unique constraint (NGAS05_AAT.IDX_NGAS_SUBSCRIBERS_SUBSCR_ID) violated. Will try again later
2021-03-06T17:54:15.202 [20092] [     R-510] [WARNIN] ngamsServer.ngamsArchiveUtils#cleanUpStagingArea:1117 Removing Staging File: /srv/ngas1/volume1/staging/NGAMS_TMP_FILE___DAINVetest_sample_crc32z.txt
2021-03-06T17:54:15.203 [20092] [     R-510] [WARNIN] ngamsServer.ngamsArchiveUtils#cleanUpStagingArea:1117 Removing Staging File: /srv/ngas1/volume1/staging/nNpIcT-test_sample_crc32z.txt
2021-03-06T17:54:15.203 [20092] [     R-510] [WARNIN] ngamsServer.ngamsArchiveUtils#cleanUpStagingArea:1117 Removing Staging File: /srv/ngas1/volume1/staging/NGAMS_TMP_FILE___DAINVetest_sample_crc32z.txt.pickle
2021-03-06T17:54:15.203 [20092] [     R-510] [WARNIN] ngamsServer.ngamsArchiveUtils#cleanUpStagingArea:1117 Removing Staging File: /srv/ngas1/volume1/staging/nNpIcT-test_sample_crc32z.txt.pickle
2021-03-06T17:54:15.205 [20092] [     R-510] [ ERROR] ngamsServer.ngamsServer#reqCallBack:1743 Error while serving request
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ngamsServer/ngamsServer.py", line 1727, in reqCallBack
    method, path, headers)
  File "/usr/lib/python2.7/site-packages/ngamsServer/ngamsServer.py", line 1822, in handleHttpRequest
    ngamsCmdHandling.handle_cmd(self, reqPropsObj, httpRef)
  File "/usr/lib/python2.7/site-packages/ngamsServer/ngamsCmdHandling.py", line 63, in handle_cmd
    msg = _get_module(srvObj, reqPropsObj).handleCmd(srvObj, reqPropsObj, httpRef)
  File "/usr/lib/python2.7/site-packages/ngamsServer/commands/archive.py", line 69, in handleCmd
    do_replication=True)
  File "/usr/lib/python2.7/site-packages/ngamsServer/ngamsArchiveUtils.py", line 1039, in dataHandler
    do_replication=do_replication, transfer=transfer)
  File "/usr/lib/python2.7/site-packages/ngamsServer/ngamsArchiveUtils.py", line 1138, in _dataHandler
    rfile, skip_crc=skip_crc, transfer=transfer)
  File "/usr/lib/python2.7/site-packages/ngamsServer/ngamsArchiveUtils.py", line 249, in archive_contents_from_request
    raise Exception(msg)
Exception: Checksum error for file test_sample_crc32z.txt, local crc = 3225417148, but remote crc = 972511042
2021-03-06T17:54:15.205 [20092] [     R-510] [  INFO] ngamsServer.ngamsServer#send_status:350 Returning status FAILURE with message Checksum error for file test_sample_crc32z.txt, local crc = 3225417148, but remote crc = 972511042 and HTTP code 400

I also receive the following notification email message...

Notification Message:


ACCUMULATED NOTIFICATION MESSAGES:

NOTE: Distribution of retained Email Notification Messages forced at Offline

--MESSAGE#000001/2021-03-06T17:54:15.204----------
NGAMS_ER_ARCHIVE_PUSH_REQ:4011:ERROR: Problem occurred handling Archive Push Request! URI: test_sample_crc32z.txt. Error: Checksum error for file test_sample_crc32z.txt, local crc = 3225417148, but remote crc = 972511042.
--END-----------------------------------------


Note: This is an automatically generated message
@rtobar
Copy link
Contributor

rtobar commented Mar 10, 2021

I confirm this is indeed an issue. The problem actually happens with any checksum type that is not crc32 though, meaning that both crc32c and crc32z can fail, while crc32 passes.

@rtobar rtobar added the bug label Mar 10, 2021
rtobar added a commit that referenced this issue Mar 10, 2021
@rtobar
Inform crc variant on subscription push
056272a 
When pushing data downstream, the subscription mechanism informed the
remote end about the file checksum value, but not the variant/type of
checksum used to calculate such value. This left the downstream server
incapable of working with checksums other than "crc32", while files
locally checksummed with other variants would fail to be pushed because
on the remote server the checksum values wouldn't match.

We vaguely knew this was a limitation from old discussions, but it
became a problem more explicitly in #44.

Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>
rtobar added a commit that referenced this issue Mar 10, 2021
@rtobar
Test subscription with different checksum variants
38e72ac 
The new, enriched version of test_basic_subscription now checks that
basic file subscription transmission works against all supported
checksum variants. This mimics the test shown as evidence of the problem
in #44, and thus will ensure this doesn't happen again.

Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>
rtobar added a commit that referenced this issue Mar 11, 2021
@rtobar
Inform crc variant on subscription push
afe4694 
When pushing data downstream, the subscription mechanism informed the
remote end about the file checksum value, but not the variant/type of
checksum used to calculate such value. This left the downstream server
incapable of working with checksums other than "crc32", while files
locally checksummed with other variants would fail to be pushed because
on the remote server the checksum values wouldn't match.

We vaguely knew this was a limitation from old discussions, but it
became a problem more explicitly in #44.

Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>
rtobar added a commit that referenced this issue Mar 11, 2021
@rtobar
Test subscription with different checksum variants
55bbc44 
The new, enriched version of test_basic_subscription now checks that
basic file subscription transmission works against all supported
checksum variants. This mimics the test shown as evidence of the problem
in #44, and thus will ensure this doesn't happen again.

Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>
@rtobar
Copy link
Contributor

rtobar commented Mar 11, 2021

A fix for this issue, and an enriched subscription unit test that checks this works for all crc variants have now been merged to the master branch.

@rtobar rtobar closed this as completed Mar 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rtobar @smclay