Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[user empathy day 2] dynamo raises exception when tracing super(Fraction, cls).__new__(cls) #128075

Open
masnesral opened this issue Jun 5, 2024 · 0 comments
Labels
empathy-day Label for issues from user empathy days module: dynamo oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@masnesral
Copy link
Contributor

masnesral commented Jun 5, 2024

🐛 Describe the bug

(While trying to enable torch.compile for https://github.com/nateraw/stable-diffusion-videos). We get an exception when tracing super(Fraction, cls).__new__(cls). The solution during the empathy day was to catch exceptions in _dynamo/trace_rules.py around the if not hashable(obj):

Traceback (most recent call last):
  File "/data/users/slarsen/empathy/test.py", line 28, in <module>
    video_path = get_video_path(pipeline)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 489, in _fn
    return fn(*args, **kwargs)
  File "/data/users/slarsen/empathy/test.py", line 14, in get_video_path
    video_path = pipeline.walk(
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 867, in walk
    make_video_pyav(
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 112, in make_video_pyav
    output_filepath = str(output_filepath)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 116, in resume_in_make_video_pyav
    for img in sorted(Path(frames_or_frame_dir).glob(glob_pattern)):
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 116, in resume_in_make_video_pyav
    for img in sorted(Path(frames_or_frame_dir).glob(glob_pattern)):
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 116, in resume_in_make_video_pyav
    for img in sorted(Path(frames_or_frame_dir).glob(glob_pattern)):
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 140, in resume_in_make_video_pyav
    write_video(output_filepath, frames, fps=fps, options={"crf": "10", "pix_fmt": "yuv420p"})
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torchvision/io/video.py", line 90, in write_video
    with av.open(filename, mode="w") as container:
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torchvision/io/video.py", line 91, in resume_in_write_video
    stream = container.add_stream(video_codec, rate=fps)
  File "av/container/output.pyx", line 107, in av.container.output.OutputContainer.add_stream
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/fractions.py", line 93, in __new__
    self = super(Fraction, cls).__new__(cls)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 655, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 727, in _convert_frame
    result = inner_convert(frame, cache_entry, hooks, frame_state)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 383, in _convert_frame_assert
    compiled_product = _compile(
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 665, in _compile
    raise InternalTorchDynamoError(str(e)).with_traceback(
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 646, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 244, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 562, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1033, in transform_code_object
    transformations(instructions, code_options)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 151, in _fn
    return fn(*args, **kwargs)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 527, in transform
    tracer.run()
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2128, in run
    super().run()
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 887, in STORE_FAST
    loaded_vt = loaded_vt.rename(self, name)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 90, in realize_and_forward
    return getattr(self.realize(), name)(*args, **kwargs)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 56, in realize
    self._cache.realize(self.parents_tracker)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 22, in realize
    self.vt = VariableBuilder(tx, self.source)(self.value)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 245, in __call__
    vt = self._wrap(value).clone(**self.options())
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 708, in _wrap
    elif trace_rules.lookup(value) is not None:
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/trace_rules.py", line 2798, in lookup
    if not hashable(obj):
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 420, in hashable
    hash(x)
  File "/home/slarsen/.conda/envs/empathy/lib/python3.10/fractions.py", line 646, in __hash__
    dinv = pow(self._denominator, -1, _PyHASH_MODULUS)
torch._dynamo.exc.InternalTorchDynamoError: 'Fraction' object has no attribute '_denominator'

from user code:
   File "/home/slarsen/.conda/envs/empathy/lib/python3.10/fractions.py", line 93, in resume_in___new__
    self = super(Fraction, cls).__new__(cls)

Versions

PyTorch version: 2.2.2+cu121

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng

masnesral added a commit that referenced this issue Jun 5, 2024
Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See #128075

[ghstack-poisoned]
masnesral added a commit that referenced this issue Jun 5, 2024
Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See #128075

ghstack-source-id: 04da167dfde45bf3fb310d4571095b5e9b699a37
Pull Request resolved: #128078
masnesral added a commit that referenced this issue Jun 5, 2024
Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See #128075

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang

[ghstack-poisoned]
masnesral added a commit that referenced this issue Jun 5, 2024
Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See #128075

ghstack-source-id: 16237cff4e7be9bf1bbfb37e5e1702a5104b75d1
Pull Request resolved: #128078
pytorchmergebot pushed a commit that referenced this issue Jun 6, 2024
Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See #128075

ghstack-source-id: 9b0d4896532168eba2f98ad68427931c1dbf787c
Pull Request resolved: #128078
@masnesral masnesral linked a pull request Jun 7, 2024 that will close this issue
@soulitzer soulitzer added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jun 10, 2024
pytorchmergebot pushed a commit that referenced this issue Jun 10, 2024
Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See #128075

Pull Request resolved: #128078
Approved by: https://github.com/anijain2305
TharinduRusira pushed a commit to TharinduRusira/pytorch that referenced this issue Jun 14, 2024
…8078)

Summary: Found during user empathy day when attempting to hash a fractions.Fraction object before it was fully constructed. See pytorch#128075

Pull Request resolved: pytorch#128078
Approved by: https://github.com/anijain2305
@bdhirsh bdhirsh added the empathy-day Label for issues from user empathy days label Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
empathy-day Label for issues from user empathy days module: dynamo oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants