Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapi.pm: short reads not handled when reading blocks from server #2897

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

Mapi.pm: short reads not handled when reading blocks from server #2897

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2011-10-06 11:43:05 +0200
From: Rémy Chibois <>
To: clients devs <>
Version: 11.5.3 (Aug2011-SP1) [obsolete]

Last updated: 2011-10-26 13:22:02 +0200

Comment 16382

Date: 2011-10-06 11:43:05 +0200
From: Rémy Chibois <>

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3
Build Identifier:

Subroutine Mapi::get block does not check return value from sysread, which, depending on the current read context, time and socket data availability, may not return the requested number of bytes (actually less). This happens rarely when connecting to a server located on the same computer, but rather frequently when using a remote server, essentially because network latency is higher

Reproducible: Always

Steps to Reproduce:

  1. Use the following snippet:

====
!/usr/bin/env perl

use strict;
use warnings;

$|++;

use DBI();

my $dbh = DBI->connect(
'dbi:monetdb:host=a_remote_host;database=a_db', 'monetdb', 'monetdb'
);

my $query = qq{
SELECT
*
FROM
some_table
};

my $sth = $dbh->prepare($query);
$sth->execute;

my $r = $sth->fetchall_arrayref();

$dbh->disconnect();

  1. You should observe random warnings and corrupt data

Actual Results:

Lots of warnings from Mapi.pm because read blocks are incomplete

Expected Results:

Correct data

Comment 16383

Date: 2011-10-06 11:44:54 +0200
From: Rémy Chibois <>

Created attachment 78
Patch handling short reads as well as read errors

This patch adds an additional method to Mapi.pm in order to handle short reads and read errors

Attached file: Mapi.pm-short_read.patch (text/plain, 1174 bytes)
Description: Patch handling short reads as well as read errors

Comment 16384

Date: 2011-10-06 11:47:49 +0200
From: Rémy Chibois <>

(Previous patch is against Mapi.pm from MonetDB-11.5.3)

Comment 16386

Date: 2011-10-07 16:17:12 +0200
From: @sjoerdmullender

Changeset 61f3aed5a5ab made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=61f3aed5a5ab

Changeset description:

Added test for bug #2897.
Unfortunately (?) the test doesn't fail because we test within a
single system.

Comment 16387

Date: 2011-10-07 16:22:31 +0200
From: Rémy Chibois <>

Changeset 76662f0c364c made by Rémy Chibois rchibois@gmail.com in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=76662f0c364c

Changeset description:

perl: make sure blocks are completely read.
Fixed bug #2897 where slow (network) reads could cause blocks to not be
fully read in one go, causing errors in the subsequent use of those
blocks.

Comment 16388

Date: 2011-10-07 16:24:51 +0200
From: @sjoerdmullender

The patch has been applied (thanks for the report and the patch).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant