Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace torch.assert_allclose with assertEqual from pytorch #1387

Merged
merged 1 commit into from Mar 15, 2021

Conversation

discort
Copy link
Contributor

@discort discort commented Mar 12, 2021

Closes #680

  • Replace torch.assert_allclose with assertEqual from pytorch test framework.

@discort
Copy link
Contributor Author

discort commented Mar 12, 2021

@mthrok @vincentqb

I believe I didn't touch failed functionality. Any suggestions how to make the build passing?

binary_macos_conda_py3.9 - Failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/distiller/miniconda3/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/cli/main_build.py", line 481, in main
    execute(sys.argv[1:])
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/cli/main_build.py", line 470, in execute
    outputs = api.build(args.recipe, post=args.post, test_run_post=args.test_run_post,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/api.py", line 186, in build
    return build_tree(
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/build.py", line 3068, in build_tree
    packages_from_this = build(metadata, stats,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/build.py", line 2031, in build
    output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/render.py", line 789, in expand_outputs
    for (output_dict, m) in deepcopy(_m).get_output_metadata_set(permit_unsatisfiable_variants=False):
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/metadata.py", line 2115, in get_output_metadata_set
    conda_packages = finalize_outputs_pass(ref_metadata, conda_packages, pass_no=0,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/metadata.py", line 776, in finalize_outputs_pass
    fm = finalize_metadata(om, parent_metadata=parent_metadata,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/render.py", line 547, in finalize_metadata
    build_unsat, host_unsat = add_upstream_pins(m,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/render.py", line 409, in add_upstream_pins
    host_deps, host_unsat, extra_run_specs_from_host = _read_upstream_pin_files(m, 'host',
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/render.py", line 374, in _read_upstream_pin_files
    deps, actions, unsat = get_env_dependencies(m, env, m.config.variant,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/render.py", line 139, in get_env_dependencies
    actions = environ.get_install_actions(tmpdir, tuple(dependencies), env,
  File "/Users/distiller/miniconda3/lib/python3.8/site-packages/conda_build/environ.py", line 804, in get_install_actions
    raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-64: {"bzip2[version='1.0.*|>=1.0.6,<2.0a0|>=1.0.8,<2.0a0']", "libuv[version='>=1.40.0,<2.0a0']", "python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3.9,<3.10.0a0|>=3.5,<3.6.0a0']", "openssl[version='>=1.1.1h,<1.1.2a|>=1.1.1i,<1.1.2a|>=1.1.1j,<1.1.2a']", "expat[version='>=2.2.4,<3.0a0']", "libuv[version='>=1.39.0,<2.0a0|>=1.40.0,<2.0a0']", "python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.9,<3.10.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0|3.4.*']", 'pypy3.7=7.3.3', "gdbm[version='>=1.18,<1.19.0a0']", "pypy3.6[version='>=7.3.1']", "numpy[version='>=1.19']", "openssl[version='1.0.*|>=1.0.2o,<1.0.3a|>=1.0.2p,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1d,<1.1.2a|>=1.1.1e,<1.1.2a|>=1.1.1f,<1.1.2a|>=1.1.1h,<1.1.2a|>=1.1.1i,<1.1.2a|>=1.1.1j,<1.1.2a|>=1.1.1g,<1.1.2a|>=1.1.1c,<1.1.2a|>=1.0.2n,<1.0.3a|>=1.0.2m,<1.0.3a|>=1.0.2l,<1.0.3a|>=1.1.1b,<1.1.2a']", "expat[version='>=2.2.9,<2.3.0a0']", 'ninja', "python[version='3.6.*|3.7.*|3.8.*']", "python[version='3.7.*|3.9.*|3.8.*']", "python_abi[version='3.6|3.6.*|3.7.*|3.8.*',build='*_cp37m|*_cp38|*_cp36m|*_pypy36_pp73']", "python[version='>=3.8,<3.9.0a0']", "ncurses[version='>=6.1,<6.3.0a0|>=6.1,<7.0a0']", 'python_abi==3.7[build=*_pypy37_pp73]', "bzip2[version='>=1.0.6,<2.0a0|>=1.0.8,<2.0a0']", 
"pypy3.6[version='7.3.*|7.3.3.*|7.3.2.*|7.3.1.*|7.3.0.*']", "ncurses[version='>=6.2,<6.3.0a0|>=6.2,<7.0a0']", "libcxx[version='>=4.0.1|>=9.0.1']", "bzip2[version='>=1.0.8,<2.0a0']", "openssl[version='1.0.*|>=1.0.2o,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1b,<1.1.2a|>=1.1.1g,<1.1.2a|>=1.1.1h,<1.1.2a|>=1.1.1j,<1.1.2a|>=1.1.1i,<1.1.2a|>=1.1.1f,<1.1.2a|>=1.1.1d,<1.1.2a|>=1.1.1c,<1.1.2a|>=1.0.2p,<1.0.3a|>=1.0.2m,<1.0.3a|>=1.0.2l,<1.0.3a']", "openssl[version='>=1.1.1a,<1.1.2a|>=1.1.1d,<1.1.2a|>=1.1.1e,<1.1.2a|>=1.1.1g,<1.1.2a|>=1.1.1h,<1.1.2a|>=1.1.1i,<1.1.2a|>=1.1.1j,<1.1.2a']", "pypy3.7[version='>=7.3.3']", "libcxxabi[version='4.0.1|4.0.1|8.0.0|8.0.0|8.0.0|8.0.0|8.0.1',build='hebd6815_0|1|4|0|3|2|hcfea43d_1']", "pypy3.7[version='7.3.*|7.3.3.*']", "python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.9,<3.10.0a0|>=3.5,<3.6.0a0|>=3.5']", "libcxx[version='>=10.0.0|>=10.0.1|>=11.0.0|>=11.0.1']", "expat[version='2.2.*|>=2.2.10,<2.3.0a0|>=2.2.9,<2.3.0a0|>=2.2.5,<2.3.0a0|>=2.2.10,<3.0a0|>=2.2.9,<3.0a0|>=2.2.6,<3.0a0|>=2.2.5,<3.0a0']", 'ca-certificates', "openssl[version='1.0.*|>=1.0.2o,<1.0.3a|>=1.0.2p,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1d,<1.1.2a|>=1.1.1f,<1.1.2a|>=1.1.1g,<1.1.2a|>=1.1.1h,<1.1.2a|>=1.1.1i,<1.1.2a|>=1.1.1j,<1.1.2a|>=1.1.1e,<1.1.2a|>=1.1.1c,<1.1.2a|>=1.0.2n,<1.0.3a|>=1.0.2m,<1.0.3a|>=1.0.2l,<1.0.3a|>=1.1.1b,<1.1.2a']"}

Exited with code exit status 1

Copy link
Collaborator

@mthrok mthrok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @discort

Thanks for the update. It looks mostly good. Regarding the build failure, since this PR does not change anything about the library code / build process, we can ignore them. (Build failure often happens due to upstream packaging and there might be something very fragile about the way build job is setup for macOS.)


def test_mfcc_empty(self):
# Passing in an empty tensor should result in an error
self.assertRaises(AssertionError, kaldi.mfcc, torch.empty(0))

def test_resample_waveform(self):
def get_output_fn(sound, args):
output = kaldi.resample_waveform(sound, args[1], args[2])
output = kaldi.resample_waveform(sound.to(torch.float32), args[1], args[2])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain why it is necessary to add to(torch.float32) here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't why but sound equals to int, so you will get RuntimeError: expected scalar type Long but found Float

In [1]: sound.dtype
Out[1]: torch.int16

To reproduce:

sound = torch.randint(-2742, 8204, (1, 8000))
kaldi.resample_waveform(sound, 16000, 1000)
~/python/projects/torchaudio/torchaudio/compliance/kaldi.py in resample_waveform(waveform, orig_freq, new_freq, lowpass_filter_width)
    841     num_wavs, length = waveform.shape
    842     waveform = F.pad(waveform, (width, width + orig_freq))
--> 843     resampled = F.conv1d(waveform[:, None], kernel, stride=orig_freq)
    844     resampled = resampled.transpose(1, 2).reshape(num_wavs, -1)
    845     target_length = int(math.ceil(new_freq * length / orig_freq))

RuntimeError: expected scalar type Long but found Float

Is it expected? @mthrok

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @mthrok

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @discort

Sorry for the late response. I looked at the code, and it seems that this should have been caught in previous PRs but the combination of forgetting to change skipIfNoSoxBackend and the removal of torchaudio.load_wav caused this. I think this is fine. Thanks for the followup.

@discort discort requested a review from mthrok March 12, 2021 22:51

def test_mfcc_empty(self):
# Passing in an empty tensor should result in an error
self.assertRaises(AssertionError, kaldi.mfcc, torch.empty(0))

def test_resample_waveform(self):
def get_output_fn(sound, args):
output = kaldi.resample_waveform(sound, args[1], args[2])
output = kaldi.resample_waveform(sound.to(torch.float32), args[1], args[2])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @discort

Sorry for the late response. I looked at the code, and it seems that this should have been caught in previous PRs but the combination of forgetting to change skipIfNoSoxBackend and the removal of torchaudio.load_wav caused this. I think this is fine. Thanks for the followup.

@mthrok mthrok merged commit 2897f36 into pytorch:master Mar 15, 2021
@discort discort deleted the assert_equal_kaldi branch March 15, 2021 15:44
mthrok pushed a commit to mthrok/audio that referenced this pull request Dec 13, 2022
The code was not formatted properly, which caused the subsequent steps to fail
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use pytorch testing framework
3 participants