Mapi.pm: short reads not handled when reading blocks from server

[monetdb-team]

Date: 2011-10-06 11:43:05 +0200
From: Rémy Chibois <>
To: clients devs <>
Version: 11.5.3 (Aug2011-SP1) [obsolete]

Last updated: 2011-10-26 13:22:02 +0200

Comment 16382

Date: 2011-10-06 11:43:05 +0200
From: Rémy Chibois <>

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari/534.48.3
Build Identifier:

Subroutine Mapi::get block does not check return value from sysread, which, depending on the current read context, time and socket data availability, may not return the requested number of bytes (actually less). This happens rarely when connecting to a server located on the same computer, but rather frequently when using a remote server, essentially because network latency is higher

Reproducible: Always

Steps to Reproduce:

1. Use the following snippet:

====
!/usr/bin/env perl

use strict;
use warnings;

$|++;

use DBI();

my $dbh = DBI->connect(
'dbi:monetdb:host=a_remote_host;database=a_db', 'monetdb', 'monetdb'
);

my $query = qq{
SELECT
*
FROM
some_table
};

my $sth = $dbh->prepare($query);
$sth->execute;

my $r = $sth->fetchall_arrayref();

$dbh->disconnect();

2. You should observe random warnings and corrupt data

Actual Results:

Lots of warnings from Mapi.pm because read blocks are incomplete

Expected Results:

Correct data

Comment 16383

Date: 2011-10-06 11:44:54 +0200
From: Rémy Chibois <>

Created attachment 78
Patch handling short reads as well as read errors

This patch adds an additional method to Mapi.pm in order to handle short reads and read errors

Attached file: Mapi.pm-short_read.patch (text/plain, 1174 bytes)
Description: Patch handling short reads as well as read errors

Comment 16384

Date: 2011-10-06 11:47:49 +0200
From: Rémy Chibois <>

(Previous patch is against Mapi.pm from MonetDB-11.5.3)

Comment 16386

Date: 2011-10-07 16:17:12 +0200
From: @sjoerdmullender

Changeset 61f3aed5a5ab made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=61f3aed5a5ab

Changeset description:

Added test for bug #2897.
Unfortunately (?) the test doesn't fail because we test within a
single system.

Comment 16387

Date: 2011-10-07 16:22:31 +0200
From: Rémy Chibois <>

Changeset 76662f0c364c made by Rémy Chibois rchibois@gmail.com in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=76662f0c364c

Changeset description:

perl: make sure blocks are completely read.
Fixed bug #2897 where slow (network) reads could cause blocks to not be
fully read in one go, causing errors in the subsequent use of those
blocks.

Comment 16388

Date: 2011-10-07 16:24:51 +0200
From: @sjoerdmullender

The patch has been applied (thanks for the report and the patch).