Memleak

[honux]

Hey there,
I was selectively migrating my db using this script:

<?php
set_time_limit(0);
include '/usr/share/nginx/teemo/vendor/autoload.php';

$old = new MongoClient('mongodb://server_1:27017');
$new = new MongoClient('mongodb://server_2:27017');

$summoners = $old->selectCollection('br', 'users')->find();

foreach ($summoners as $summoner) {
    if (isset($summoner['league']) && isset($summoner['points']) && count($summoner['points']) > 3) {
        unset($summoner['games']);
        $new->update(
                    ['_id'=>$summoner['_id']], 
                    ['$set'=>$summoner],
                    ['upsert'=>1]
                );
        $i++;
        if ($i %100 == 0) {
            echo $i.' ';
        }
    }
}

And found a huge mem leak, no idea why.

[Pomyk]

Which hhvm version are using? Can you put that code in a function and check if the leak goes away?

[honux]

HipHop VM 3.4.2 (dbg)
Compiler: tags/HHVM-3.4.2-0-g40d85d7386b3342ab848fc45c6892433c89ef5b2
Repo schema: 13cba877c4300d1d0481b3f40eb128a5115d4e08
Extension API: 20140829

<?php
set_time_limit(0);
include '/usr/share/nginx/teemo/vendor/autoload.php';

function foo ($data_, $db_, $i) {
    if (isset($data_['league']) && isset($data_['points']) && count($data_['points']) > 3) {
        unset($data_['games']);
        $db_->update(
                    ['_id'=>$data_['_id']], 
                    ['$set'=>$data_],
                    ['upsert'=>1]
                );
        if ($i %100 == 0) {
            echo $i.' ';
        }
    }
}

$old = new MongoClient('mongodb://server_1:27017');
$new = new MongoClient('mongodb://server_2:27017');

$summoners = $old->selectCollection('br', 'users')->find();
$i = 0;

foreach ($summoners as $summoner) {
    $i++;
    foo($summoner, $new, $i);
}

Still the problem.
It was hard, but got a stacktrace:

Host: Developer
ProcessID: 8318
ThreadID: 7fcfe2495c80
ThreadPID: 8318
Name: /usr/bin/hhvm
Type: Segmentation fault
Runtime: hhvm
Version: tags/HHVM-3.4.2-0-g40d85d7386b3342ab848fc45c6892433c89ef5b2
DebuggerCount: 0

Arguments: foo.php
ThreadType: CLI

# 0  ?? at hhvm:0
# 1  nallocm at /usr/lib/x86_64-linux-gnu/libjemalloc.so.1:0
# 2  void* HPHP::MemoryManager::smartMallocSizeBigHelper<false>(void*&, unsigned long&, unsigned long) at hhvm:0
# 3  HPHP::MixedArray::CopyReserve(HPHP::MixedArray const*, unsigned long) at hhvm:0
# 4  HPHP::MixedArray::Merge(HPHP::ArrayData*, HPHP::ArrayData const*) at hhvm:0
# 5  HPHP::Array::merge(HPHP::Array const&) at hhvm:0
# 6  HPHP::f_array_merge(HPHP::Variant const&, HPHP::Variant const&, HPHP::Array const&) at hhvm:0
# 7  void HPHP::Native::callFunc<false, true>(HPHP::Func const*, void*, HPHP::TypedValue*, HPHP::TypedValue&) at hhvm:0
# 8  HPHP::TypedValue* HPHP::Native::functionWrapper<false, true>(HPHP::ActRec*) at hhvm:0
# 9  HPHP::jit::x64::BackEnd::enterTCHelper(unsigned char*, HPHP::jit::TReqInfo&) at hhvm:0
# 10 HPHP::jit::MCGenerator::enterTC(unsigned char*, void*) at hhvm:0
# 11 HPHP::ExecutionContext::enterVM(HPHP::ActRec*, HPHP::ExecutionContext::StackArgsState, HPHP::Resumable*, HPHP::ObjectData*) at hhvm:0
# 12 HPHP::ExecutionContext::invokeFuncFew(HPHP::TypedValue*, HPHP::Func const*, void*, HPHP::StringData*, int, HPHP::TypedValue const*) at hhvm:0
# 13 HPHP::ObjectData::o_invoke_few_args(HPHP::String const&, int, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&) at hhvm:0
# 14 HPHP::ArrayIter::second() at hhvm:0
# 15 long HPHP::iter_next_cold<false>(HPHP::Iter*, HPHP::TypedValue*, HPHP::TypedValue*) at hhvm:0
# 16 HPHP::jit::x64::BackEnd::enterTCHelper(unsigned char*, HPHP::jit::TReqInfo&) at hhvm:0
# 17 HPHP::jit::MCGenerator::enterTC(unsigned char*, void*) at hhvm:0
# 18 HPHP::ExecutionContext::invokeFunc(HPHP::TypedValue*, HPHP::Func const*, HPHP::Variant const&, HPHP::ObjectData*, HPHP::Class*, HPHP::VarEnv*, HPHP::StringData*, HPHP::ExecutionContext::InvokeFlags) at hhvm:0
# 19 HPHP::ExecutionContext::invokeUnit(HPHP::TypedValue*, HPHP::Unit const*) at hhvm:0
# 20 HPHP::include_impl_invoke(HPHP::String const&, bool, char const*) at hhvm:0
# 21 HPHP::hphp_invoke(HPHP::ExecutionContext*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, HPHP::Array const&, HPHP::VRefParamValue const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool, bool, bool) at hhvm:0
# 22 HPHP::hphp_invoke_simple(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) at hhvm:0
# 23 ?? at hhvm:0
# 24 HPHP::execute_program(int, char**) at hhvm:0
# 25 main at hhvm:0
# 26 __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6:0
# 27 ?? at hhvm:0

PHP Stacktrace:

#0  array_merge(Array, Array)
#1  MongoCursor->setDocuments(Array)
#2  MongoCursor->fetchMoreDocuments()
#3  MongoCursor->fetchMoreDocumentsIfNeeded()
#4  MongoCursor->current() called at [/usr/share/nginx/www/foo.php:28]

[Pomyk]

I meant whole code in a function, but if that doesn't help you can report it in hhvm: https://github.com/facebook/hhvm/issues

[mcuadros]

closing due to inactivity

[Enelar]

Confirming same issue.

$client = new \MongoClient(" ");
$collection = new \MongoCollection($client, "collection");

$cursor = $collection->find();

foreach ($cursor as $row)
  echo $i++."<br>";

Unable to iterate over 1M rows - getting OOM.

Same with unset $row.

[Enelar]

@mcuadros up