New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

an AttributeError raised in mnist example 'NoneType' object has no attribute 'terminate' #311

Open
cutechestnut opened this Issue Nov 14, 2018 · 19 comments

Comments

Projects
None yet
8 participants
@cutechestnut
Copy link

cutechestnut commented Nov 14, 2018

when i run the fit function in mnist example, raised this error:
my evironment is windows 10 ,and is strictly requested due to requirements.txt.
has someone encountered such a situation? please tell me how to solve
thank you

Traceback (most recent call last):
File "", line 1, in
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\search.py", line 231, in search
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
File "C:\Anaconda3\lib\multiprocessing\process.py", line 116, in terminate
exitcode = _main(fd)
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'
prepare(preparation_data)
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Anaconda3\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Anaconda3\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\autokeras-master\examples\mnist.py", line 9, in
clf.fit(x_train,y_train)
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\image\image_supervised.py", line 159, in fit
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\cnn_module.py", line 50, in fit
File "C:\Anaconda3\lib\site-packages\autokeras-0.2.19-py3.6.egg\autokeras\search.py", line 231, in search
File "C:\Anaconda3\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Nov 16, 2018

Hi @cutechestnut. Thank you for the bug report, I'll be looking into this issue.

@EverKnows

This comment has been minimized.

Copy link

EverKnows commented Nov 21, 2018

i got this problem too,here is my error information.(autokeras0.3.1 windows10 py36)
Traceback (most recent call last):

File "", line 1, in
runfile('F:/Ranly_Obj/AdcancedTech/auto_example.py', wdir='F:/Ranly_Obj/AdcancedTech')

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "F:/Ranly_Obj/AdcancedTech/auto_example.py", line 27, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\image\image_supervised.py", line 123, in fit
super().fit(x, y, x_test, y_test, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\supervised.py", line 140, in fit
self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\net_module.py", line 60, in fit
self.searcher.search(train_data, test_data, int(time_remain))

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\search.py", line 238, in search
p.terminate()

File "E:\anaconda\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()

AttributeError: 'NoneType' object has no attribute 'terminate'

@Timmmeyyy

This comment has been minimized.

Copy link

Timmmeyyy commented Nov 21, 2018

Hi, i got the same issue 5 minutes ago. May @tl-yang you have an solution ?

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Nov 26, 2018

@Timmmeyyy @EverKnows @cutechestnut I'm having a hard time reproducing the bug. Can you provide me with some more detail? Like how long do you run the mnist example until the error occurred? what is the console ouput(before the error)

I think it might be a race condition. But I can't tell under what circumstances the error will occur.

@Ivorra

This comment has been minimized.

Copy link

Ivorra commented Nov 26, 2018

Hi,

It also happens to me (Ubuntu 16.04.5 LTS), I am using a random generated dataset (np.random.rand), this is the code:

# train.py file, run with "python3.6 train.py" 
import numpy as np
import autokeras as ak
from tensorflow import set_random_seed
seed = 0
np.random.seed(seed)
set_random_seed(0)

x_train = np.random.rand(100, 30, 30, 1)
x_val  = np.random.rand(70, 30, 30, 1)
y_train = np.random.rand(100)
y_val = np.random.rand(70)
clf = ak.ImageClassifier(verbose=True)
clf.fit(x_train, y_train,  x_test=x_val, y_test=y_val)

This is the error:

Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 231, in search
    raise e
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 199, in search
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
	return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.
        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:
            if __name__ == '__main__':
                freeze_support()
                ...
        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 114, in _main
    prepare(preparation_data)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "/usr/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/s1881460/train.py", line 42, in <module>
    x_test=x_val, y_test=y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 238, in search
    p.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

Then, I have to Ctrl-C for stopping the execution, which outputs:

^CTraceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 199, in search
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 62, in _launch
    f.write(fp.getbuffer())
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "train.py", line 42, in <module>
    x_test=x_val, y_test=y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 238, in search
    p.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

Hope it helps!

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Nov 27, 2018

hi @Ivorra . I think you need to wrap your code in a if __name__ = '__main__': block. Cause in linux, we're using the spawn method to start a process.

@Ivorra

This comment has been minimized.

Copy link

Ivorra commented Nov 27, 2018

@tl-yang Could you clarify how to do the wrapping? I am doing the following, but it just gives another error:

import numpy as np
import autokeras as ak

from tensorflow import set_random_seed

def f():
    seed = 0
     np.random.seed(seed)
     set_random_seed(seed)

    x_train = np.random.rand(100, 30, 30, 1)
    x_val  = np.random.rand(70, 30, 30, 1)
    y_train = np.random.rand(100)
    y_val = np.random.rand(70)

    clf = ak.ImageClassifier(verbose=True)

    clf.fit(x_train, y_train, x_test=x_val, y_test=y_val)
        
if __name__ == '__main__':
    f()

And the error:


Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.

+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Using TensorFlow backend.
Process SpawnProcess-1:                                                                             
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 301, in train
    verbose=verbose).train_model(**trainer_args)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/nn/model_trainer.py", line 115, in train_model
    test_loss, metric_value = self._test()
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/nn/model_trainer.py", line 179, in _test
    test_loss += float(self.loss_function(outputs, targets))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/nn/loss_function.py", line 5, in classification_loss
    labels = target.argmax(1)
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/tensor.py", line 231, in argmax
    return torch.argmax(self, dim, keepdim)
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/functional.py", line 374, in argmax
    return torch._argmax(input, dim, keepdim)
RuntimeError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff940>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ffac8>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ffbe0>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff080>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff470>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'
Exception ignored in: <bound method StorageRef.__del__ of <torch.multiprocessing.reductions.StorageRef object at 0x7fa24c3ff5f8>>
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 26, in __del__
AttributeError: 'NoneType' object has no attribute '_free_weak_ref'

After Ctrl-C:

^CTraceback (most recent call last):
  File "run.py", line 56, in <module>
    f()
  File "run.py", line 40, in f
    x_test=x_val, y_test=y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 217, in search
    metric_value, loss, graph = q.get(timeout=remaining_time)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 104, in get
    if not self._poll(timeout):
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Nov 27, 2018

@Ivorra I think you just found another issue. Try to remove the x_test y_test parameter in the .fit function. It should work.

@Ivorra

This comment has been minimized.

Copy link

Ivorra commented Nov 27, 2018

@tl-yang It seems to give the same error, I just set the line to "clf.fit(x_train, y_train, x_val, y_val)". Is that what I should change?

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Nov 27, 2018

@Ivorra I mean remove the last two parameters: clf.fit(x_train, y_train) it seems that x_test and y_test parameters have some problem now.

@Ivorra

This comment has been minimized.

Copy link

Ivorra commented Nov 27, 2018

@tl-yang It seems that it gives the same error as before.

Code:

import numpy as np
import autokeras as ak
from tensorflow import set_random_seed
seed = 0
np.random.seed(seed)
set_random_seed(0)


x_train = np.random.rand(100, 30, 30, 1)
x_val  = np.random.rand(70, 30, 30, 1)
y_train = np.ceil(np.random.rand(100))
y_val = np.ceil(np.random.rand(70))
clf = ak.ImageClassifier(verbose=True)
clf.fit(x_train, y_train) #, x_val, y_val)
Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Using TensorFlow backend.
Preprocessing the images.
x is  (100, 30, 30, 1)
Preprocessing finished.
Initializing search.
Initialization finished.
+----------------------------------------------+
|               Training model 0               |
+----------------------------------------------+
Traceback (most recent call last):
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 231, in search
    raise e
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 199, in search
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.
        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:
            if __name__ == '__main__':
                freeze_support()
                ...
        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 114, in _main
    prepare(preparation_data)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "/usr/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/s1881460/train.py", line 41, in <module>
    clf.fit(x_train, y_train) #, x_val, y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 238, in search
    p.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

Ctrl-C,

^CTraceback (most recent call last):
  File "train.py", line 41, in <module>
    clf.fit(x_train, y_train) #, x_val, y_val)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/image/image_supervised.py", line 123, in fit
    super().fit(x, y, x_test, y_test, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/supervised.py", line 140, in fit
    self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/net_module.py", line 60, in fit
    self.searcher.search(train_data, test_data, int(time_remain))
  File "/home/s1881460/.local/lib/python3.6/site-packages/autokeras/search.py", line 217, in search
    metric_value, loss, graph = q.get(timeout=remaining_time)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 104, in get
    if not self._poll(timeout):
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
@Ivorra

This comment has been minimized.

Copy link

Ivorra commented Nov 27, 2018

OK, think it works now:

import numpy as np
import autokeras as ak
def f():
        x_train = np.random.rand(100, 30, 30, 1)
        x_val  = np.random.rand(70, 30, 30, 1)
        y_train = np.ceil(np.random.rand(100))
        y_val = np.ceil(np.random.rand(70))
        clf = ak.ImageClassifier(verbose=True)
        clf.fit(x_train, y_train)
if __name__ == '__main__':
        f()
@Ivorra

This comment has been minimized.

Copy link

Ivorra commented Nov 27, 2018

My question is, does then autokeras use the training accuracy as the main metric, instead of the validation one?

@EverKnows

This comment has been minimized.

Copy link

EverKnows commented Nov 27, 2018

@tl-yang
here is my code
from keras.datasets import mnist
from autokeras import ImageClassifier
import tensorflow
from fashion_mnist_load import fashion_mnist_load

def f():
(x_train, y_train), (x_test, y_test) = mnist.load_data(r'F:/Ranly_Obj/AdcancedTech/mnist/mnist.npz')
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))

clf = ImageClassifier(verbose=True)
    
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
print('Done')
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y * 100)

if name=='main':
f()

and here is my error info
runfile('F:/Ranly_Obj/AdcancedTech/auto_example.py', wdir='F:/Ranly_Obj/AdcancedTech')
Using TensorFlow backend.
C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\tqdm\autonotebook_init_.py:14: TqdmExperimentalWarning: Using tqdm.autonotebook.tqdm in notebook mode. Use tqdm.tqdm instead to force console mode (e.g. in jupyter console)
" (e.g. in jupyter console)", TqdmExperimentalWarning)
Preprocessing the images.
x is (60000, 28, 28, 1)
Preprocessing finished.

Initializing search.
Initialization finished.

+----------------------------------------------+
| Training model 0 |
+----------------------------------------------+
Traceback (most recent call last):

File "", line 1, in
runfile('F:/Ranly_Obj/AdcancedTech/auto_example.py', wdir='F:/Ranly_Obj/AdcancedTech')

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "E:\anaconda\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "F:/Ranly_Obj/AdcancedTech/auto_example.py", line 32, in
f()

File "F:/Ranly_Obj/AdcancedTech/auto_example.py", line 26, in f
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\image\image_supervised.py", line 123, in fit
super().fit(x, y, x_test, y_test, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\supervised.py", line 140, in fit
self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\net_module.py", line 60, in fit
self.searcher.search(train_data, test_data, int(time_remain))

File "C:\Users\lenovo\AppData\Roaming\Python\Python36\site-packages\autokeras\search.py", line 238, in search
p.terminate()

File "E:\anaconda\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()

AttributeError: 'NoneType' object has no attribute 'terminate'

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Nov 27, 2018

@EverKnows Are you using notebook to run the code? Can you provide some more detail about your environment?

@EverKnows

This comment has been minimized.

Copy link

EverKnows commented Nov 28, 2018

@tl-yang i use spyder to run my code

My enviroment

windows 10
python 3.6
autokeras 0.3.1
tensorflow 1.7.0
keras 2.2.2
torch 0.4.1
spyder 3.2.6
tensorflow-gpu 1.9.0

@locys

This comment has been minimized.

Copy link

locys commented Dec 7, 2018

When I put 46000 224 * 224 images for training ,I ran into this problem.
After I reduce the size to 96 * 96 , it works well
Then I tryed 128 * 128, problem occured again
I think It is something about resouces. my enviroment is win10, gtx1080ti, i7-8700k, 32g memory

@tl-yang

This comment has been minimized.

Copy link
Collaborator

tl-yang commented Dec 7, 2018

@locys The exception actually may not have anything to do with the actual error. Could you provide me with the full console log?

@deadsoul44

This comment has been minimized.

Copy link

deadsoul44 commented Jan 13, 2019

I have the same issue.

This is my code. Almost same as example.

`from keras.datasets import mnist
from autokeras import ImageClassifier

if name == 'main':
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))

clf = ImageClassifier(verbose=True, augment=False)
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
print(y * 100)`

Error message:

Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 15s 1us/stepETA: 15s - ETA: 12s - ETA: 10s - ETA: 10s 4243456/11490434 [==========>...................] - ETA: 9s 5046272/11490434 [============>.................] - ETA: 8s - ETA: 8s - ETA: 7s - ETA: 7s - ETA: 5s 7831552/11490434 [===================>..........] - ETA: 4s - ETA: 3s - ETA: 2s 9658368/11490434 [========================>.....] - ETA: 2s - ETA: 2s - ETA: 1s - ETA: 1s - ETA: 1s - ETA: 0s
Preprocessing the images.
Preprocessing finished.

Initializing search.
Initialization finished.

+----------------------------------------------+
| Training model 0 |
+----------------------------------------------+
Traceback (most recent call last):

File "", line 1, in
runfile('E:/tm/mutlu_tm/tm_scrape/MNIST_autokeras.py', wdir='E:/tm/mutlu_tm/tm_scrape')

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 704, in runfile
execfile(filename, namespace)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/tm/mutlu_tm/tm_scrape/MNIST_autokeras.py", line 17, in
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\image\image_supervised.py", line 114, in fit
super().fit(x, y, time_limit)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\supervised.py", line 129, in fit
self.cnn.fit(self.get_n_output_node(), x_train.shape, train_data, test_data, time_limit)

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\net_module.py", line 65, in fit
self.searcher.search(train_data, test_data, int(time_remain))

File "C:\Anaconda\envs\tf_gpu\lib\site-packages\autokeras\search.py", line 222, in search
p.terminate()

File "C:\Anaconda\envs\tf_gpu\lib\multiprocessing\process.py", line 116, in terminate
self._popen.terminate()

AttributeError: 'NoneType' object has no attribute 'terminate'

Setup Details

OS type and version: Windows 8.1 Pro
Python: 3.6.8
autokeras: 0.3.5
scikit-learn: 0.20.1
numpy: 1.14.5 , 1.15.4 py36h19fb1c0_0
keras: 2.2.4 <0>, 2.2.2
scipy: 1.1.0
tensorflow: 1.10.0
pytorch: 0.4.1

I am using conda virtual env and spyder to run the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment