Skip to content
This repository has been archived by the owner on Feb 20, 2018. It is now read-only.

Memleak #17

Closed
honux opened this issue Dec 20, 2014 · 6 comments
Closed

Memleak #17

honux opened this issue Dec 20, 2014 · 6 comments

Comments

@honux
Copy link

honux commented Dec 20, 2014

Hey there,
I was selectively migrating my db using this script:

<?php
set_time_limit(0);
include '/usr/share/nginx/teemo/vendor/autoload.php';

$old = new MongoClient('mongodb://server_1:27017');
$new = new MongoClient('mongodb://server_2:27017');

$summoners = $old->selectCollection('br', 'users')->find();

foreach ($summoners as $summoner) {
    if (isset($summoner['league']) && isset($summoner['points']) && count($summoner['points']) > 3) {
        unset($summoner['games']);
        $new->update(
                    ['_id'=>$summoner['_id']], 
                    ['$set'=>$summoner],
                    ['upsert'=>1]
                );
        $i++;
        if ($i %100 == 0) {
            echo $i.' ';
        }
    }
}

And found a huge mem leak, no idea why.

@Pomyk
Copy link
Contributor

Pomyk commented Dec 21, 2014

Which hhvm version are using? Can you put that code in a function and check if the leak goes away?

@honux
Copy link
Author

honux commented Dec 21, 2014

HipHop VM 3.4.2 (dbg)
Compiler: tags/HHVM-3.4.2-0-g40d85d7386b3342ab848fc45c6892433c89ef5b2
Repo schema: 13cba877c4300d1d0481b3f40eb128a5115d4e08
Extension API: 20140829

<?php
set_time_limit(0);
include '/usr/share/nginx/teemo/vendor/autoload.php';

function foo ($data_, $db_, $i) {
    if (isset($data_['league']) && isset($data_['points']) && count($data_['points']) > 3) {
        unset($data_['games']);
        $db_->update(
                    ['_id'=>$data_['_id']], 
                    ['$set'=>$data_],
                    ['upsert'=>1]
                );
        if ($i %100 == 0) {
            echo $i.' ';
        }
    }
}

$old = new MongoClient('mongodb://server_1:27017');
$new = new MongoClient('mongodb://server_2:27017');

$summoners = $old->selectCollection('br', 'users')->find();
$i = 0;

foreach ($summoners as $summoner) {
    $i++;
    foo($summoner, $new, $i);
}

Still the problem.
It was hard, but got a stacktrace:

Host: Developer
ProcessID: 8318
ThreadID: 7fcfe2495c80
ThreadPID: 8318
Name: /usr/bin/hhvm
Type: Segmentation fault
Runtime: hhvm
Version: tags/HHVM-3.4.2-0-g40d85d7386b3342ab848fc45c6892433c89ef5b2
DebuggerCount: 0

Arguments: foo.php
ThreadType: CLI

# 0  ?? at hhvm:0
# 1  nallocm at /usr/lib/x86_64-linux-gnu/libjemalloc.so.1:0
# 2  void* HPHP::MemoryManager::smartMallocSizeBigHelper<false>(void*&, unsigned long&, unsigned long) at hhvm:0
# 3  HPHP::MixedArray::CopyReserve(HPHP::MixedArray const*, unsigned long) at hhvm:0
# 4  HPHP::MixedArray::Merge(HPHP::ArrayData*, HPHP::ArrayData const*) at hhvm:0
# 5  HPHP::Array::merge(HPHP::Array const&) at hhvm:0
# 6  HPHP::f_array_merge(HPHP::Variant const&, HPHP::Variant const&, HPHP::Array const&) at hhvm:0
# 7  void HPHP::Native::callFunc<false, true>(HPHP::Func const*, void*, HPHP::TypedValue*, HPHP::TypedValue&) at hhvm:0
# 8  HPHP::TypedValue* HPHP::Native::functionWrapper<false, true>(HPHP::ActRec*) at hhvm:0
# 9  HPHP::jit::x64::BackEnd::enterTCHelper(unsigned char*, HPHP::jit::TReqInfo&) at hhvm:0
# 10 HPHP::jit::MCGenerator::enterTC(unsigned char*, void*) at hhvm:0
# 11 HPHP::ExecutionContext::enterVM(HPHP::ActRec*, HPHP::ExecutionContext::StackArgsState, HPHP::Resumable*, HPHP::ObjectData*) at hhvm:0
# 12 HPHP::ExecutionContext::invokeFuncFew(HPHP::TypedValue*, HPHP::Func const*, void*, HPHP::StringData*, int, HPHP::TypedValue const*) at hhvm:0
# 13 HPHP::ObjectData::o_invoke_few_args(HPHP::String const&, int, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&, HPHP::Variant const&) at hhvm:0
# 14 HPHP::ArrayIter::second() at hhvm:0
# 15 long HPHP::iter_next_cold<false>(HPHP::Iter*, HPHP::TypedValue*, HPHP::TypedValue*) at hhvm:0
# 16 HPHP::jit::x64::BackEnd::enterTCHelper(unsigned char*, HPHP::jit::TReqInfo&) at hhvm:0
# 17 HPHP::jit::MCGenerator::enterTC(unsigned char*, void*) at hhvm:0
# 18 HPHP::ExecutionContext::invokeFunc(HPHP::TypedValue*, HPHP::Func const*, HPHP::Variant const&, HPHP::ObjectData*, HPHP::Class*, HPHP::VarEnv*, HPHP::StringData*, HPHP::ExecutionContext::InvokeFlags) at hhvm:0
# 19 HPHP::ExecutionContext::invokeUnit(HPHP::TypedValue*, HPHP::Unit const*) at hhvm:0
# 20 HPHP::include_impl_invoke(HPHP::String const&, bool, char const*) at hhvm:0
# 21 HPHP::hphp_invoke(HPHP::ExecutionContext*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, HPHP::Array const&, HPHP::VRefParamValue const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool, bool, bool) at hhvm:0
# 22 HPHP::hphp_invoke_simple(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) at hhvm:0
# 23 ?? at hhvm:0
# 24 HPHP::execute_program(int, char**) at hhvm:0
# 25 main at hhvm:0
# 26 __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6:0
# 27 ?? at hhvm:0

PHP Stacktrace:

#0  array_merge(Array, Array)
#1  MongoCursor->setDocuments(Array)
#2  MongoCursor->fetchMoreDocuments()
#3  MongoCursor->fetchMoreDocumentsIfNeeded()
#4  MongoCursor->current() called at [/usr/share/nginx/www/foo.php:28]

@Pomyk
Copy link
Contributor

Pomyk commented Dec 21, 2014

I meant whole code in a function, but if that doesn't help you can report it in hhvm: https://github.com/facebook/hhvm/issues

@mcuadros
Copy link
Contributor

closing due to inactivity

@Enelar
Copy link

Enelar commented Jun 30, 2016

Confirming same issue.

$client = new \MongoClient(" ");
$collection = new \MongoCollection($client, "collection");

$cursor = $collection->find();

foreach ($cursor as $row)
  echo $i++."<br>";

Unable to iterate over 1M rows - getting OOM.

Same with unset $row.

@Enelar
Copy link

Enelar commented Jun 30, 2016

@mcuadros up

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants