Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

train module unittests fails when excuted with the tensorflow_gpu feature. #333

Open
dskkato opened this issue Nov 21, 2021 · 0 comments
Open

train module unittests fails when excuted with the tensorflow_gpu feature. #333

dskkato opened this issue Nov 21, 2021 · 0 comments

Comments

@dskkato
Copy link
Contributor

dskkato commented Nov 21, 2021

When I run tests in the master branch, the following two tests will always panic when executed with the tensorflow_gpu feature. I'll look into the cause when I have time.

> cargo clean && cargo test --features tensorflow_gpu
...
---- train::tests::simple_gradient_descent stdout ----
thread 'train::tests::simple_gradient_descent' panicked at 'x_output[0] = 1.9200001', src\train.rs:339:9
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- train::tests::simple_adadelta stdout ----
thread 'train::tests::simple_adadelta' panicked at 'x_output[0] = 3', src\train.rs:380:9

failures:
train::tests::simple_adadelta
train::tests::simple_gradient_descent
...

My environment is as follows:

  • branch: master
  • command: > cargo clean && cargo test --features tensorflow_gpu
  • Windows 10 Pro
    • CUDA: v11.2
    • cudnn: v8.1.0
  • rustc version is rustc 1.56.0 (09c42c458 2021-10-18)
full log 
> cargo clean && cargo test --features tensorflow_gpu
   Compiling autocfg v1.0.1
   Compiling cfg-if v1.0.0
   Compiling libc v0.2.103
   Compiling cc v1.0.70
   Compiling pkg-config v0.3.20
   Compiling winapi v0.3.9
   Compiling lazy_static v1.4.0
   Compiling proc-macro2 v1.0.29
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.78
   Compiling vcpkg v0.2.15
   Compiling crossbeam-utils v0.8.5
   Compiling crc32fast v1.2.1
   Compiling scopeguard v1.1.0
   Compiling adler v1.0.2
   Compiling crossbeam-epoch v0.9.5
   Compiling byteorder v1.4.3
   Compiling rayon-core v1.9.1
   Compiling curl v0.4.39
   Compiling semver v1.0.4
   Compiling parking_lot_core v0.8.5
   Compiling adler32 v1.2.0
   Compiling either v1.6.1
   Compiling protobuf v2.23.0
   Compiling rustversion v1.0.5
   Compiling weezl v0.1.5
   Compiling smallvec v1.7.0
   Compiling bitflags v1.3.2
   Compiling color_quant v1.1.0
   Compiling crc-catalog v1.1.1
   Compiling scoped_threadpool v0.1.9
   Compiling half v1.7.1
   Compiling bytemuck v1.7.2
   Compiling random v0.12.2
   Compiling instant v0.1.11
   Compiling miniz_oxide v0.4.4
   Compiling memoffset v0.6.4
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling rayon v1.5.1
   Compiling num-rational v0.3.2
   Compiling num-iter v0.1.42
   Compiling lock_api v0.4.5
   Compiling miniz_oxide v0.3.7
   Compiling deflate v0.8.6
   Compiling crc v2.0.0
   Compiling gif v0.11.3
   Compiling num_cpus v1.13.0
   Compiling libz-sys v1.1.3
   Compiling bzip2-sys v0.1.11+1.0.8
   Compiling curl-sys v0.4.49+curl-7.79.1
   Compiling quote v1.0.9
   Compiling crossbeam-channel v0.5.1
   Compiling png v0.16.8
   Compiling flate2 v1.0.22
   Compiling num-complex v0.4.0
   Compiling crossbeam-deque v0.8.1
   Compiling socket2 v0.4.2
   Compiling time v0.1.44
   Compiling schannel v0.1.19
   Compiling filetime v0.2.15
   Compiling tar v0.4.37
   Compiling parking_lot v0.11.2
   Compiling bzip2 v0.4.3
   Compiling jpeg-decoder v0.1.22
   Compiling tiff v0.6.1
   Compiling thiserror-impl v1.0.29
   Compiling tensorflow-internal-macros v0.0.1 (D:\tensorflow\rust\tensorflow-internal-macros)
   Compiling serial_test_derive v0.5.1
   Compiling image v0.23.14
   Compiling serial_test v0.5.1
   Compiling thiserror v1.0.29
   Compiling zip v0.5.13
   Compiling tensorflow-sys v0.20.0 (D:\tensorflow\rust\tensorflow-sys)
warning: use of deprecated associated function `zip::read::ZipFile::<'a>::sanitized_name`: by stripping `..`s from the path, the meaning of paths can change.
                `mangled_name` can be used if this behaviour is desirable
   --> tensorflow-sys\build.rs:153:60
    |
153 |         let output_path = extract_to.as_ref().join(zipfile.sanitized_name());
    |                                                            ^^^^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

warning: `tensorflow-sys` (build script) generated 1 warning
   Compiling tensorflow v0.17.0 (D:\tensorflow\rust)
warning: unused attribute `deprecated`
  --> src\ops\math_ops.rs:51:1
   |
51 | #[deprecated(note = "Use mul instead.", since = "0.15.0")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_attributes)]` on by default
note: the built-in attribute `deprecated` will be ignored, since it's applied to the macro invocation `define_op`
  --> src\ops\math_ops.rs:52:1
   |
52 | define_op!(multiply, Multiply, "Mul", args { a, b });
   | ^^^^^^^^^

warning: unused attribute `deprecated`
  --> src\ops\math_ops.rs:54:1
   |
54 | #[deprecated(note = "Use sub instead.", since = "0.15.0")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
note: the built-in attribute `deprecated` will be ignored, since it's applied to the macro invocation `define_op`
  --> src\ops\math_ops.rs:55:1
   |
55 | define_op!(subtract, Subtract, "Sub", args { a, b });
   | ^^^^^^^^^

warning: unused attribute `deprecated`
 --> src\ops\random_ops.rs:4:1
  |
4 | #[deprecated(note = "Use random_standard_normal instead.", since = "0.15.0")]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
note: the built-in attribute `deprecated` will be ignored, since it's applied to the macro invocation `define_op`
 --> src\ops\random_ops.rs:5:1
  |
5 | define_op!(random_normal, RandomNormal, "RandomStandardNormal", args{x}, attrs {
  | ^^^^^^^^^

warning: unused macro definition
    --> src\lib.rs:1998:1
     |
1998 | / macro_rules! shape_from_array {
1999 | |     ($N:expr) => {
2000 | |         impl From<[i32; $N]> for Shape {
2001 | |             fn from(data: [i32; $N]) -> Shape {
...    |
2047 | |     };
2048 | | }
     | |_^
     |
     = note: `#[warn(unused_macros)]` on by default

warning: unused import: `crate::Shape`
   --> src\train.rs:280:9
    |
280 |     use crate::Shape;
    |         ^^^^^^^^^^^^
    |
    = note: `#[warn(unused_imports)]` on by default

warning: use of deprecated field `session::SavedModelBundle::meta_graph_def`: Please use SavedModelBundle::meta_graph_def() instead
  --> src\session.rs:82:17
   |
82 |                 meta_graph_def: Vec::from(meta.as_ref()),
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated field `session::SavedModelBundle::meta_graph_def`: Please use SavedModelBundle::meta_graph_def() instead
   --> src\session.rs:653:13
    |
653 |             meta_graph_def,
    |             ^^^^^^^^^^^^^^

warning: `tensorflow` (lib test) generated 7 warnings (5 duplicates)
warning: `tensorflow` (lib) generated 5 warnings
    Finished test [unoptimized + debuginfo] target(s) in 2m 46s
     Running unittests (target\debug\deps\tensorflow-fe2d0e08ffb79370.exe)

running 75 tests
test buffer::tests::basic ... ok
test graph::tests::graph_get_op_def ... ok
test graph::tests::device ... ok
test graph::tests::control_inputs ... ok
test graph::tests::graph_versions ... ok
test graph::tests::graph_add_gradients_no_gradient ... ok
test graph::tests::graph_add_gradients_stopped_gradient ... ok
test graph::tests::import_graph_def_results_missing_unused_input_mappings ... ok
test graph::tests::import_graph_def_results_return_operations ... ok
test graph::tests::import_graph_def_set_default_device ... ok
test graph::tests::import_graph_def_results_return_outputs ... ok
test graph::tests::import_graph_def_with_return_outputs ... ok
test graph::tests::import_graph_def_uniquify_prefix ... ok
test graph::tests::import_graph_def_uniquify_names ... ok
test graph::tests::smoke ... ok
test graph::tests::output_consumers ... ok
test graph::tests::output_name ... ok
test graph::tests::operation_attributes ... ok
test graph::tests::test_get_tensor_shape ... ok
test graph::tests::test_import_graph_def ... ok
test io::tests::peek_next ... ok
test graph::tests::graph_generate_operation_name ... ok
test io::tests::seek_over_too_long ... ok
test ops::tests::device ... ok
test scope::tests::device ... ok
test scope::tests::get_unique_name_for_op ... ok
test scope::tests::control_dependencies ... ok
test scope::tests::kernel_label ... ok
test scope::tests::smoke ... ok
test scope::tests::uniquification ... ok
test scope::tests::xla_cluster ... ok
test graph::tests::graph_to_function ... ok
test io::tests::writer_identical_to_python ... ok
test graph::tests::graph_add_gradients ... ok
2021-11-21 15:45:59.996618: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-11-21 15:45:59.998778: I tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: test_resources/regression-model
2021-11-21 15:45:59.999604: I tensorflow/cc/saved_model/reader.cc:90] Reading meta graph with tags { train serve }
2021-11-21 15:45:59.999641: I tensorflow/cc/saved_model/reader.cc:132] Reading SavedModel debug info (if present) from: test_resources/regression-model
2021-11-21 15:46:00.600862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.601960: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.602296: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test session::tests::smoke ... ok
2021-11-21 15:46:00.602799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::shape_from_array0 ... ok
2021-11-21 15:46:00.603147: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.603541: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test session::tests::test_close ... ok
test tests::shape_from_array1 ... ok
2021-11-21 15:46:00.603880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::shape_from_array1_ref ... ok
test tests::shape_from_array8 ... ok
test session::tests::test_device_list ... ok
test tests::shape_from_array8_ref ... ok
test tests::shape_from_none ... ok
test tests::tensor_clone ... ok
2021-11-21 15:46:00.605395: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::tensor_display ... ok
test tests::smoke ... ok
test tests::tensor_eq ... ok
test tests::tensor_from_array ... ok
test tests::tensor_from_array_ref ... ok
test tests::test_bfloat16 ... ok
2021-11-21 15:46:00.606488: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::test_f16 ... ok
test tests::test_close ... ok
test tests::test_library_load ... ok
2021-11-21 15:46:00.607495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::test_get_registered_kernels_for_op ... ok
test tests::test_set_config ... ok
test tests::test_set_target ... ok
2021-11-21 15:46:00.622403: I tensorflow/cc/saved_model/loader.cc:211] Restoring SavedModel bundle.
2021-11-21 15:46:00.622815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::test_strings ... ok
test tests::test_tensor ... ok
test tests::test_tensor_native_type_zero ... ok
test tests::test_get_all_registered_kernels ... ok
2021-11-21 15:46:00.704288: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.705134: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.
2021-11-21 15:46:00.705153: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.
2021-11-21 15:46:00.705184: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1614] Profiler found 1 GPUs
2021-11-21 15:46:00.705814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.706916: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { train serve }; Status: success: OK. Took 708381 microseconds.
test tests::test_run ... ok
test session::tests::test_run_metadata_no_run_options ... ok
test session::tests::test_run ... ok
2021-11-21 15:46:00.711607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.712311: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test variable::tests::const_initialized_matrix ... ok
2021-11-21 15:46:00.716698: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test variable::tests::const_initialized_scalar ... ok
test variable::tests::custom_initializer_missing_dtype ... ok
test while_loop::tests::generated_name_while ... ok
2021-11-21 15:46:00.797363: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.797983: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test train::tests::simple_gradient_descent ... FAILED
test train::tests::simple_adadelta ... FAILED
2021-11-21 15:46:00.800065: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1748] CUPTI activity buffer flushed
test session::tests::test_savedmodelbundle ... ok
2021-11-21 15:46:00.832353: I tensorflow/core/profiler/internal/gpu/cupti_collector.cc:663]  GpuTracer has collected 16 callback api events and 14 activity events. 
2021-11-21 15:46:00.832618: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.
2021-11-21 15:46:00.832815: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.
2021-11-21 15:46:00.832830: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.
test variable::tests::custom_initializer ... ok
test while_loop::tests::simple_while ... ok
2021-11-21 15:46:00.933351: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1748] CUPTI activity buffer flushed
2021-11-21 15:46:00.979535: I tensorflow/core/profiler/internal/gpu/cupti_collector.cc:663]  GpuTracer has collected 5 callback api events and 4 activity events. 
2021-11-21 15:46:00.979638: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.
test session::tests::test_run_metadata ... ok
2021-11-21 15:46:01.380820: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:01.384741: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.
2021-11-21 15:46:01.384782: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.
2021-11-21 15:46:01.529294: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1748] CUPTI activity buffer flushed
2021-11-21 15:46:01.567161: I tensorflow/core/profiler/internal/gpu/cupti_collector.cc:663]  GpuTracer has collected 6 callback api events and 5 activity events. 
2021-11-21 15:46:01.567235: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.
test session::tests::test_run_options ... ok
test train::tests::xor_nn ... ok

failures:

---- train::tests::simple_gradient_descent stdout ----
thread 'train::tests::simple_gradient_descent' panicked at 'x_output[0] = 3', src\train.rs:321:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- train::tests::simple_adadelta stdout ----
thread 'train::tests::simple_adadelta' panicked at 'x_output[0] = 3', src\train.rs:380:9


failures:
    train::tests::simple_adadelta
    train::tests::simple_gradient_descent

test result: FAILED. 73 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.43s

error: test failed, to rerun pass '--lib'
   Compiling autocfg v1.0.1
   Compiling cfg-if v1.0.0
   Compiling libc v0.2.103
   Compiling cc v1.0.70
   Compiling pkg-config v0.3.20
   Compiling winapi v0.3.9
   Compiling lazy_static v1.4.0
   Compiling proc-macro2 v1.0.29
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.78
   Compiling vcpkg v0.2.15
   Compiling crossbeam-utils v0.8.5
   Compiling crc32fast v1.2.1
   Compiling scopeguard v1.1.0
   Compiling adler v1.0.2
   Compiling crossbeam-epoch v0.9.5
   Compiling byteorder v1.4.3
   Compiling rayon-core v1.9.1
   Compiling curl v0.4.39
   Compiling semver v1.0.4
   Compiling parking_lot_core v0.8.5
   Compiling adler32 v1.2.0
   Compiling either v1.6.1
   Compiling protobuf v2.23.0
   Compiling rustversion v1.0.5
   Compiling weezl v0.1.5
   Compiling smallvec v1.7.0
   Compiling bitflags v1.3.2
   Compiling color_quant v1.1.0
   Compiling crc-catalog v1.1.1
   Compiling scoped_threadpool v0.1.9
   Compiling half v1.7.1
   Compiling bytemuck v1.7.2
   Compiling random v0.12.2
   Compiling instant v0.1.11
   Compiling miniz_oxide v0.4.4
   Compiling memoffset v0.6.4
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling rayon v1.5.1
   Compiling num-rational v0.3.2
   Compiling num-iter v0.1.42
   Compiling lock_api v0.4.5
   Compiling miniz_oxide v0.3.7
   Compiling deflate v0.8.6
   Compiling crc v2.0.0
   Compiling gif v0.11.3
   Compiling num_cpus v1.13.0
   Compiling libz-sys v1.1.3
   Compiling bzip2-sys v0.1.11+1.0.8
   Compiling curl-sys v0.4.49+curl-7.79.1
   Compiling quote v1.0.9
   Compiling crossbeam-channel v0.5.1
   Compiling png v0.16.8
   Compiling flate2 v1.0.22
   Compiling num-complex v0.4.0
   Compiling crossbeam-deque v0.8.1
   Compiling socket2 v0.4.2
   Compiling time v0.1.44
   Compiling schannel v0.1.19
   Compiling filetime v0.2.15
   Compiling tar v0.4.37
   Compiling parking_lot v0.11.2
   Compiling bzip2 v0.4.3
   Compiling jpeg-decoder v0.1.22
   Compiling tiff v0.6.1
   Compiling thiserror-impl v1.0.29
   Compiling tensorflow-internal-macros v0.0.1 (D:\tensorflow\rust\tensorflow-internal-macros)
   Compiling serial_test_derive v0.5.1
   Compiling image v0.23.14
   Compiling serial_test v0.5.1
   Compiling thiserror v1.0.29
   Compiling zip v0.5.13
   Compiling tensorflow-sys v0.20.0 (D:\tensorflow\rust\tensorflow-sys)
warning: use of deprecated associated function `zip::read::ZipFile::<'a>::sanitized_name`: by stripping `..`s from the path, the meaning of paths can change.
                `mangled_name` can be used if this behaviour is desirable
   --> tensorflow-sys\build.rs:153:60
    |
153 |         let output_path = extract_to.as_ref().join(zipfile.sanitized_name());
    |                                                            ^^^^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

warning: `tensorflow-sys` (build script) generated 1 warning
   Compiling tensorflow v0.17.0 (D:\tensorflow\rust)
warning: unused attribute `deprecated`
  --> src\ops\math_ops.rs:51:1
   |
51 | #[deprecated(note = "Use mul instead.", since = "0.15.0")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_attributes)]` on by default
note: the built-in attribute `deprecated` will be ignored, since it's applied to the macro invocation `define_op`
  --> src\ops\math_ops.rs:52:1
   |
52 | define_op!(multiply, Multiply, "Mul", args { a, b });
   | ^^^^^^^^^

warning: unused attribute `deprecated`
  --> src\ops\math_ops.rs:54:1
   |
54 | #[deprecated(note = "Use sub instead.", since = "0.15.0")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
note: the built-in attribute `deprecated` will be ignored, since it's applied to the macro invocation `define_op`
  --> src\ops\math_ops.rs:55:1
   |
55 | define_op!(subtract, Subtract, "Sub", args { a, b });
   | ^^^^^^^^^

warning: unused attribute `deprecated`
 --> src\ops\random_ops.rs:4:1
  |
4 | #[deprecated(note = "Use random_standard_normal instead.", since = "0.15.0")]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
note: the built-in attribute `deprecated` will be ignored, since it's applied to the macro invocation `define_op`
 --> src\ops\random_ops.rs:5:1
  |
5 | define_op!(random_normal, RandomNormal, "RandomStandardNormal", args{x}, attrs {
  | ^^^^^^^^^

warning: unused macro definition
    --> src\lib.rs:1998:1
     |
1998 | / macro_rules! shape_from_array {
1999 | |     ($N:expr) => {
2000 | |         impl From<[i32; $N]> for Shape {
2001 | |             fn from(data: [i32; $N]) -> Shape {
...    |
2047 | |     };
2048 | | }
     | |_^
     |
     = note: `#[warn(unused_macros)]` on by default

warning: unused import: `crate::Shape`
   --> src\train.rs:280:9
    |
280 |     use crate::Shape;
    |         ^^^^^^^^^^^^
    |
    = note: `#[warn(unused_imports)]` on by default

warning: use of deprecated field `session::SavedModelBundle::meta_graph_def`: Please use SavedModelBundle::meta_graph_def() instead
  --> src\session.rs:82:17
   |
82 |                 meta_graph_def: Vec::from(meta.as_ref()),
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: use of deprecated field `session::SavedModelBundle::meta_graph_def`: Please use SavedModelBundle::meta_graph_def() instead
   --> src\session.rs:653:13
    |
653 |             meta_graph_def,
    |             ^^^^^^^^^^^^^^

warning: `tensorflow` (lib test) generated 7 warnings (5 duplicates)
warning: `tensorflow` (lib) generated 5 warnings
    Finished test [unoptimized + debuginfo] target(s) in 2m 46s
     Running unittests (target\debug\deps\tensorflow-fe2d0e08ffb79370.exe)

running 75 tests
test buffer::tests::basic ... ok
test graph::tests::graph_get_op_def ... ok
test graph::tests::device ... ok
test graph::tests::control_inputs ... ok
test graph::tests::graph_versions ... ok
test graph::tests::graph_add_gradients_no_gradient ... ok
test graph::tests::graph_add_gradients_stopped_gradient ... ok
test graph::tests::import_graph_def_results_missing_unused_input_mappings ... ok
test graph::tests::import_graph_def_results_return_operations ... ok
test graph::tests::import_graph_def_set_default_device ... ok
test graph::tests::import_graph_def_results_return_outputs ... ok
test graph::tests::import_graph_def_with_return_outputs ... ok
test graph::tests::import_graph_def_uniquify_prefix ... ok
test graph::tests::import_graph_def_uniquify_names ... ok
test graph::tests::smoke ... ok
test graph::tests::output_consumers ... ok
test graph::tests::output_name ... ok
test graph::tests::operation_attributes ... ok
test graph::tests::test_get_tensor_shape ... ok
test graph::tests::test_import_graph_def ... ok
test io::tests::peek_next ... ok
test graph::tests::graph_generate_operation_name ... ok
test io::tests::seek_over_too_long ... ok
test ops::tests::device ... ok
test scope::tests::device ... ok
test scope::tests::get_unique_name_for_op ... ok
test scope::tests::control_dependencies ... ok
test scope::tests::kernel_label ... ok
test scope::tests::smoke ... ok
test scope::tests::uniquification ... ok
test scope::tests::xla_cluster ... ok
test graph::tests::graph_to_function ... ok
test io::tests::writer_identical_to_python ... ok
test graph::tests::graph_add_gradients ... ok
2021-11-21 15:45:59.996618: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-11-21 15:45:59.998778: I tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: test_resources/regression-model
2021-11-21 15:45:59.999604: I tensorflow/cc/saved_model/reader.cc:90] Reading meta graph with tags { train serve }
2021-11-21 15:45:59.999641: I tensorflow/cc/saved_model/reader.cc:132] Reading SavedModel debug info (if present) from: test_resources/regression-model
2021-11-21 15:46:00.600862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.601960: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.602296: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test session::tests::smoke ... ok
2021-11-21 15:46:00.602799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::shape_from_array0 ... ok
2021-11-21 15:46:00.603147: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.603541: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test session::tests::test_close ... ok
test tests::shape_from_array1 ... ok
2021-11-21 15:46:00.603880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::shape_from_array1_ref ... ok
test tests::shape_from_array8 ... ok
test session::tests::test_device_list ... ok
test tests::shape_from_array8_ref ... ok
test tests::shape_from_none ... ok
test tests::tensor_clone ... ok
2021-11-21 15:46:00.605395: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::tensor_display ... ok
test tests::smoke ... ok
test tests::tensor_eq ... ok
test tests::tensor_from_array ... ok
test tests::tensor_from_array_ref ... ok
test tests::test_bfloat16 ... ok
2021-11-21 15:46:00.606488: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::test_f16 ... ok
test tests::test_close ... ok
test tests::test_library_load ... ok
2021-11-21 15:46:00.607495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::test_get_registered_kernels_for_op ... ok
test tests::test_set_config ... ok
test tests::test_set_target ... ok
2021-11-21 15:46:00.622403: I tensorflow/cc/saved_model/loader.cc:211] Restoring SavedModel bundle.
2021-11-21 15:46:00.622815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test tests::test_strings ... ok
test tests::test_tensor ... ok
test tests::test_tensor_native_type_zero ... ok
test tests::test_get_all_registered_kernels ... ok
2021-11-21 15:46:00.704288: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.705134: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.
2021-11-21 15:46:00.705153: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.
2021-11-21 15:46:00.705184: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1614] Profiler found 1 GPUs
2021-11-21 15:46:00.705814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.706916: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { train serve }; Status: success: OK. Took 708381 microseconds.
test tests::test_run ... ok
test session::tests::test_run_metadata_no_run_options ... ok
test session::tests::test_run ... ok
2021-11-21 15:46:00.711607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.712311: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test variable::tests::const_initialized_matrix ... ok
2021-11-21 15:46:00.716698: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test variable::tests::const_initialized_scalar ... ok
test variable::tests::custom_initializer_missing_dtype ... ok
test while_loop::tests::generated_name_while ... ok
2021-11-21 15:46:00.797363: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:00.797983: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
test train::tests::simple_gradient_descent ... FAILED
test train::tests::simple_adadelta ... FAILED
2021-11-21 15:46:00.800065: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1748] CUPTI activity buffer flushed
test session::tests::test_savedmodelbundle ... ok
2021-11-21 15:46:00.832353: I tensorflow/core/profiler/internal/gpu/cupti_collector.cc:663]  GpuTracer has collected 16 callback api events and 14 activity events. 
2021-11-21 15:46:00.832618: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.
2021-11-21 15:46:00.832815: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.
2021-11-21 15:46:00.832830: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.
test variable::tests::custom_initializer ... ok
test while_loop::tests::simple_while ... ok
2021-11-21 15:46:00.933351: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1748] CUPTI activity buffer flushed
2021-11-21 15:46:00.979535: I tensorflow/core/profiler/internal/gpu/cupti_collector.cc:663]  GpuTracer has collected 5 callback api events and 4 activity events. 
2021-11-21 15:46:00.979638: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.
test session::tests::test_run_metadata ... ok
2021-11-21 15:46:01.380820: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1338 MB memory:  -> device: 0, name: Quadro P400, pci bus id: 0000:01:00.0, compute capability: 6.1
2021-11-21 15:46:01.384741: I tensorflow/core/profiler/lib/profiler_session.cc:131] Profiler session initializing.
2021-11-21 15:46:01.384782: I tensorflow/core/profiler/lib/profiler_session.cc:146] Profiler session started.
2021-11-21 15:46:01.529294: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1748] CUPTI activity buffer flushed
2021-11-21 15:46:01.567161: I tensorflow/core/profiler/internal/gpu/cupti_collector.cc:663]  GpuTracer has collected 6 callback api events and 5 activity events. 
2021-11-21 15:46:01.567235: I tensorflow/core/profiler/lib/profiler_session.cc:164] Profiler session tear down.
test session::tests::test_run_options ... ok
test train::tests::xor_nn ... ok

failures:

---- train::tests::simple_gradient_descent stdout ----
thread 'train::tests::simple_gradient_descent' panicked at 'x_output[0] = 3', src\train.rs:321:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- train::tests::simple_adadelta stdout ----
thread 'train::tests::simple_adadelta' panicked at 'x_output[0] = 3', src\train.rs:380:9


failures:
    train::tests::simple_adadelta
    train::tests::simple_gradient_descent

test result: FAILED. 73 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.43s

error: test failed, to rerun pass '--lib'
@dskkato dskkato changed the title Some unittests fails when excuted with the tensorflow_gpu feature. train module unittests fails when excuted with the tensorflow_gpu feature. Nov 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dskkato