Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with PyTorch model #20

Open
sharvil10 opened this issue May 22, 2021 · 0 comments
Open

Error with PyTorch model #20

sharvil10 opened this issue May 22, 2021 · 0 comments

Comments

@sharvil10
Copy link

I am trying to run the WideResNet_Trades model to run the benchmark using the benchmark.iteration_cli interface. However I am getting an error shown below.

Using TensorFlow backend.
WARNING:tensorflow:From /home/itachi/repos/ares/ares/benchmark/iteration_cli.py:100: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /home/itachi/repos/ares/ares/benchmark/iteration_cli.py:105: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From /home/itachi/repos/ares/ares/benchmark/iteration_cli.py:107: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-05-22 01:29:30.918086: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-05-22 01:29:30.946135: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2799925000 Hz
2021-05-22 01:29:30.946691: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559ebfc9e4c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-05-22 01:29:30.946708: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-05-22 01:29:30.949124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-05-22 01:29:31.011989: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-22 01:29:31.012647: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559ebfca0320 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-05-22 01:29:31.012678: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1060, Compute Capability 6.1
2021-05-22 01:29:31.012868: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-22 01:29:31.013139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: 
name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
2021-05-22 01:29:31.013309: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013430: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013538: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013642: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013730: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013799: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.016472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-05-22 01:29:31.016493: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1662] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2021-05-22 01:29:31.016511: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-22 01:29:31.016521: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 
2021-05-22 01:29:31.016529: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N 
WARNING:tensorflow:From /home/itachi/repos/ares/ares/attack/utils.py:8: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /home/itachi/repos/ares/ares/dataset/utils.py:6: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use for ... in dataset:to iterate over a dataset. If using tf.estimator, return the Dataset object directly from your input function. As a last resort, you can use tf.compat.v1.data.make_one_shot_iterator(dataset).
Traceback (most recent call last):
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_5
	 [[{{node Variable_5/read}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/itachi/repos/ares/ares/benchmark/iteration_cli.py", line 144, in <module>
    rs = benchmark.run(dataset, logger)
  File "/home/itachi/repos/ares/ares/benchmark/iteration.py", line 256, in run
    return self._run(dataset, logger)
  File "/home/itachi/repos/ares/ares/benchmark/iteration.py", line 113, in _run_basic
    labels, dists = next(g)
  File "/home/itachi/repos/ares/ares/attack/bim.py", line 115, in _batch_attack_generator
    self._session.run(self.setup_xs, feed_dict={self.xs_ph: xs})
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956, in run
    run_metadata_ptr)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
    run_metadata)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_5
	 [[node Variable_5/read (defined at /miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'Variable_5/read':
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/repos/ares/ares/benchmark/iteration_cli.py", line 138, in <module>
    session, cw_n_points=args.cw_n_points, **kwargs)
  File "/repos/ares/ares/benchmark/iteration.py", line 61, in __init__
    self.attack = load_attack(attack_name, init_kwargs)
  File "/repos/ares/ares/benchmark/utils.py", line 37, in load_attack
    return attack_class(**kwargs)
  File "/repos/ares/ares/attack/pgd.py", line 34, in __init__
    self.rand_init_eps_var = tf.Variable(tf.zeros((self.batch_size,), dtype=self.model.x_dtype))
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 258, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call
    shape=shape)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 197, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py", line 2519, in default_variable_creator
    shape=shape)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 262, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 1688, in __init__
    shape=shape)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 1872, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 203, in identity
    ret = gen_array_ops.identity(input, name=name)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py", line 4239, in identity
    "Identity", input=input, name=name)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in __init__
    self._traceback = tf_stack.extract_stack()

Loading tensorflow session...
Loading model...
Loading dataset...
Loading attack...
Configuring attack...
Running benchmark

Command:

python3 -m ares.benchmark.iteration_cli --method pgd --dataset cifar10 --count 100 --goal ut --distance-metric l_inf --offset 0 --output ./out_logs --batch-size 2 --iteration 10 example/cifar10/wideresnet_trades.py --logger

Below is my environment details:
OS: Ubuntu 20.04
Python: 3.7
Pytorch 1.5.0
Tensorflow: 1.15.4
Keras: 2.3.1

Does it have to do something with my setup or there's a bug? Please help me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant