Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

an AttributeError raised in mnist example 'NoneType' object has no attribute 'terminate' #311

Closed
cutechestnut opened this issue Nov 14, 2018 · 26 comments
Closed

an AttributeError raised in mnist example 'NoneType' object has no attribute 'terminate' #311

cutechestnut opened this issue Nov 14, 2018 · 26 comments
Assignees
@tl-yang
Labels
bug report wontfix

Comments

@cutechestnut
Copy link

when i run the fit function in mnist example, raised this error:
my evironment is windows 10 ,and is strictly requested due to requirements.txt.
has someone encountered such a situation? please tell me how to solve
thank you

Traceback (most recent call last):
File "", line 1, in
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\search.py", line 231, in search
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
File "C:\Anaconda3\lib\multiprocessing\process.py", line 116, in terminate
exitcode = _main(fd)
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'
prepare(preparation_data)
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Anaconda3\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Anaconda3\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\autokeras-master\examples\mnist.py", line 9, in
clf.fit(x_train,y_train)
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\image\image_supervised.py", line 159, in fit
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\cnn_module.py", line 50, in fit
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\search.py", line 231, in search
File "C:\Anaconda3\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'
@tl-yang
Copy link
Contributor

tl-yang commented Nov 16, 2018

Hi @cutechestnut. Thank you for the bug report, I'll be looking into this issue.

@EverKnows
Copy link

i got this problem too，here is my error information.（autokeras0.3.1 windows10 py36）
Traceback (most recent call last):

File "", line 1, in
runfile('F:/Ranly_Obj/AdcancedTech/auto_example.py', wdir='F:/Ranly_Obj/AdcancedTech')

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "F:/Ranly_Obj/AdcancedTech/auto_example.py", line 27, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\image\image_supervised.py", line 123, in fit
super().fit(x, y, x_test, y_test, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\supervised.py", line 140, in fit
self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\net_module.py", line 60, in fit
self.searcher.search(train_data, test_data, int(time_remain))

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\search.py", line 238, in search
p.terminate()

File "E:\anaconda\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()

AttributeError: 'NoneType' object has no attribute 'terminate'

@Timmmeyyy
Copy link

Hi, i got the same issue 5 minutes ago. May @tl-yang you have an solution ?

@tl-yang
Copy link
Contributor

tl-yang commented Nov 26, 2018

@Timmmeyyy @EverKnows @cutechestnut I'm having a hard time reproducing the bug. Can you provide me with some more detail? Like how long do you run the mnist example until the error occurred? what is the console ouput(before the error)

I think it might be a race condition. But I can't tell under what circumstances the error will occur.

@Fairasname
Copy link

Hi,

It also happens to me (Ubuntu 16.04.5 LTS), I am using a random generated dataset (np.random.rand), this is the code:

# train.py file, run with "python3.6 train.py" 
import numpy as np
import autokeras as ak
from tensorflow import set_random_seed
seed = 0
np.random.seed(seed)
set_random_seed(0)

x_train = np.random.rand(100, 30, 30, 1)
x_val  = np.random.rand(70, 30, 30, 1)
y_train = np.random.rand(100)
y_val = np.random.rand(70)
clf = ak.ImageClassifier(verbose=True)
clf.fit(x_train, y_train,  x_test=x_val, y_test=y_val)

This is the error:

Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 231, in search
    raise e
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 199, in search
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
	return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.
        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:
            if __name__ == '__main__':
                freeze_support()
                ...
        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 114, in _main
    prepare(preparation_data)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "/usr/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/s1881460/train.py", line 42, in <module>
    x_test=x_val, y_test=y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 238, in search
    p.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

Then, I have to Ctrl-C for stopping the execution, which outputs:

^CTraceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 199, in search
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 62, in _launch
    f.write(fp.getbuffer())
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "train.py", line 42, in <module>
    x_test=x_val, y_test=y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 238, in search
    p.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

Hope it helps!

@tl-yang
Copy link
Contributor

tl-yang commented Nov 27, 2018 •
edited

hi @Ivorra . I think you need to wrap your code in a if __name__ = '__main__': block. Cause in linux, we're using the spawn method to start a process.

@Fairasname
Copy link

@tl-yang Could you clarify how to do the wrapping? I am doing the following, but it just gives another error:

import numpy as np
import autokeras as ak

from tensorflow import set_random_seed

def f():
    seed = 0
     np.random.seed(seed)
     set_random_seed(seed)

    x_train = np.random.rand(100, 30, 30, 1)
    x_val  = np.random.rand(70, 30, 30, 1)
    y_train = np.random.rand(100)
    y_val = np.random.rand(70)

    clf = ak.ImageClassifier(verbose=True)

    clf.fit(x_train, y_train, x_test=x_val, y_test=y_val)
        
if __name__ == '__main__':
    f()

And the error:


Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.

+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Using TensorFlow backend.
Process SpawnProcess-1:                                                                             
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 301, in train
    verbose=verbose).train_model(**trainer_args)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/nn/model_trainer.py", line 115, in train_model
    test_loss, metric_value = self._test()
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/nn/model_trainer.py", line 179, in _test
    test_loss += float(self.loss_function(outputs, targets))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/nn/loss_function.py", line 5, in classification_loss
    labels = target.argmax(1)
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/tensor.py", line 231, in argmax
    return torch.argmax(self, dim, keepdim)
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/functional.py", line 374, in argmax
    return torch._argmax(input, dim, keepdim)
RuntimeError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff940>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ffac8>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ffbe0>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff080>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff470>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff5f8>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'

After Ctrl-C:

^CTraceback (most recent call last):
  File "run.py", line 56, in <module>
    f()
  File "run.py", line 40, in f
    x_test=x_val, y_test=y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 217, in search
    metric_value, loss, graph = q.get(timeout=remaining_time)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 104, in get
    if not self._poll(timeout):
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt

@tl-yang
Copy link
Contributor

tl-yang commented Nov 27, 2018

@Ivorra I think you just found another issue. Try to remove the x_test y_test parameter in the .fit function. It should work.

@Fairasname
Copy link

@tl-yang It seems to give the same error, I just set the line to "clf.fit(x_train, y_train, x_val, y_val)". Is that what I should change?

@tl-yang
Copy link
Contributor

tl-yang commented Nov 27, 2018

@Ivorra I mean remove the last two parameters: clf.fit(x_train, y_train) it seems that x_test and y_test parameters have some problem now.

@Fairasname
Copy link

@tl-yang It seems that it gives the same error as before.

Code:

import numpy as np
import autokeras as ak
from tensorflow import set_random_seed
seed = 0
np.random.seed(seed)
set_random_seed(0)


x_train = np.random.rand(100, 30, 30, 1)
x_val  = np.random.rand(70, 30, 30, 1)
y_train = np.ceil(np.random.rand(100))
y_val = np.ceil(np.random.rand(70))
clf = ak.ImageClassifier(verbose=True)
clf.fit(x_train, y_train) #, x_val, y_val)
Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 231, in search
    raise e
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 199, in search
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.
        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:
            if __name__ == '__main__':
                freeze_support()
                ...
        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 114, in _main
    prepare(preparation_data)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "/usr/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/s1881460/train.py", line 41, in <module>
    clf.fit(x_train, y_train) #, x_val, y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 238, in search
    p.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

Ctrl-C,

^CTraceback (most recent call last):
  File "train.py", line 41, in <module>
    clf.fit(x_train, y_train) #, x_val, y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 217, in search
    metric_value, loss, graph = q.get(timeout=remaining_time)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 104, in get
    if not self._poll(timeout):
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)

@Fairasname
Copy link

Fairasname commented Nov 27, 2018 •
edited

OK, think it works now:

import numpy as np
import autokeras as ak
def f():
        x_train = np.random.rand(100, 30, 30, 1)
        x_val  = np.random.rand(70, 30, 30, 1)
        y_train = np.ceil(np.random.rand(100))
        y_val = np.ceil(np.random.rand(70))
        clf = ak.ImageClassifier(verbose=True)
        clf.fit(x_train, y_train)
if __name__ == '__main__':
        f()

@Fairasname
Copy link

My question is, does then autokeras use the training accuracy as the main metric, instead of the validation one?

@EverKnows
Copy link

@tl-yang
here is my code
from keras.datasets import mnist
from autokeras import ImageClassifier
import tensorflow
from fashion_mnist_load import fashion_mnist_load

def f():
(x_train, y_train), (x_test, y_test) = mnist.load_data(r'F:/Ranly_Obj/AdcancedTech/mnist/mnist.npz')
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))

clf = ImageClassifier(verbose=True)
    
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
print('Done')
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y * 100)

if name=='main':
f()

and here is my error info
runfile('F:/Ranly_Obj/AdcancedTech/auto_example.py', wdir='F:/Ranly_Obj/AdcancedTech')
Using TensorFlow backend.
C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\tqdm\autonotebook_init_.py:14: TqdmExperimentalWarning: Using tqdm.autonotebook.tqdm in notebook mode. Use tqdm.tqdm instead to force console mode (e.g. in jupyter console)
" (e.g. in jupyter console)", TqdmExperimentalWarning)
Preprocessing the images.
x is (60000, 28, 28, 1)
Preprocessing finished.

Initializing search.
Initialization finished.

+----------------------------------------------+
| Training model 0 |
+----------------------------------------------+
Traceback (most recent call last):

File "", line 1, in
runfile('F:/Ranly_Obj/AdcancedTech/auto_example.py', wdir='F:/Ranly_Obj/AdcancedTech')

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "F:/Ranly_Obj/AdcancedTech/auto_example.py", line 32, in
f()

File "F:/Ranly_Obj/AdcancedTech/auto_example.py", line 26, in f
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\image\image_supervised.py", line 123, in fit
super().fit(x, y, x_test, y_test, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\supervised.py", line 140, in fit
self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\net_module.py", line 60, in fit
self.searcher.search(train_data, test_data, int(time_remain))

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\search.py", line 238, in search
p.terminate()

File "E:\anaconda\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()

AttributeError: 'NoneType' object has no attribute 'terminate'

@tl-yang
Copy link
Contributor

tl-yang commented Nov 27, 2018 •
edited

@EverKnows Are you using notebook to run the code? Can you provide some more detail about your environment?

@EverKnows
Copy link

@tl-yang i use spyder to run my code

My enviroment

windows 10
python 3.6
autokeras 0.3.1
tensorflow 1.7.0
keras 2.2.2
torch 0.4.1
spyder 3.2.6
tensorflow-gpu 1.9.0

@tl-yang tl-yang mentioned this issue Nov 28, 2018
Closed
@locys
Copy link

locys commented Dec 7, 2018 •
edited

When I put 46000 224 * 224 images for training ,I ran into this problem.
After I reduce the size to 96 * 96 , it works well
Then I tryed 128 * 128, problem occured again
I think It is something about resouces. my enviroment is win10, gtx1080ti, i7-8700k, 32g memory

@tl-yang
Copy link
Contributor

tl-yang commented Dec 7, 2018

@locys The exception actually may not have anything to do with the actual error. Could you provide me with the full console log?

@deadsoul44
Copy link

deadsoul44 commented Jan 13, 2019 •
edited

I have the same issue.

This is my code. Almost same as example.

`from keras.datasets import mnist
from autokeras import ImageClassifier

if name == 'main':
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))

clf = ImageClassifier(verbose=True, augment=False)
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y * 100)`

Error message:

Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 15s 1us/stepETA: 15s - ETA: 12s - ETA: 10s - ETA: 10s 4243456/11490434 [==========>...................] - ETA: 9s 5046272/11490434 [============>.................] - ETA: 8s - ETA: 8s - ETA: 7s - ETA: 7s - ETA: 5s 7831552/11490434 [===================>..........] - ETA: 4s - ETA: 3s - ETA: 2s 9658368/11490434 [========================>.....] - ETA: 2s - ETA: 2s - ETA: 1s - ETA: 1s - ETA: 1s - ETA: 0s
Preprocessing the images.
Preprocessing finished.

Initializing search.
Initialization finished.

+----------------------------------------------+
| Training model 0 |
+----------------------------------------------+
Traceback (most recent call last):

File "", line 1, in
runfile('E:/tm/mutlu_tm/tm_scrape/MNIST_autokeras.py', wdir='E:/tm/mutlu_tm/tm_scrape')

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 704, in runfile
execfile(filename, namespace)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/tm/mutlu_tm/tm_scrape/MNIST_autokeras.py", line 17, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\image\image_supervised.py", line 114, in fit
super().fit(x, y, time_limit)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\supervised.py", line 129, in fit
self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\net_module.py", line 65, in fit
self.searcher.search(train_data, test_data, int(time_remain))

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\search.py", line 222, in search
p.terminate()

File "C:\Anaconda\envs\tf_gpu\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()

AttributeError: 'NoneType' object has no attribute 'terminate'

Setup Details

OS type and version: Windows 8.1 Pro
Python: 3.6.8
autokeras: 0.3.5
scikit-learn: 0.20.1
numpy: 1.14.5 , 1.15.4 py36h19fb1c0_0
keras: 2.2.4 <0>, 2.2.2
scipy: 1.1.0
tensorflow: 1.10.0
pytorch: 0.4.1

I am using conda virtual env and spyder to run the code.

@last2win
Copy link

last2win commented Apr 9, 2019

i also meet this error when i use spyder, but if i use command line to run python file, no error occur!

@m-abdelrahman
Copy link

I also have the same error with Django, How can I apply this solution ?
OS : Ubuntu 18
Python: 3.6.7
django: 2.2.0
pip: 19.0.3
virtualenv: 16.4.3

@m-abdelrahman
Copy link

@tl-yang

@abensaid
Copy link

Any update regarding the "'NoneType' object has no attribute 'terminate'" bug??

@m-abdelrahman
Copy link

m-abdelrahman commented May 4, 2019 •
edited

I found some thing about that , the issue happens if some bad object in the query set doesn't contain the mentioned attribute, if all objects are fine the issue doesn't happen , at least on Django 2

@m-abdelrahman
Copy link

@abensaid

@stale
Copy link

stale bot commented Aug 12, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Aug 12, 2019
@stale stale bot closed this as completed Aug 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tl-yang tl-yang

Labels
bug report wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests