property grad

[yueyinqiu]

close #1291

[yueyinqiu]

fixed #1300

[yueyinqiu]

Are there any suggestion about this behavior in PyTorch?

import torch

x = torch.zeros([], requires_grad=True)
y = torch.zeros([])

x.grad = y
print(y is x.grad)  # True

This is pretty strange that they are the same Python object. Perhaps PyTorch tensors are using a different technique instead of a simple wrapper of a pointer to a LibTorch tensor. (Maybe this is very common for Python. Actually I don't know how Python interacts with C++.)

The only way I can find is to also save the C# tensor when setting grad, and prefer to return it when getting.

[NiklasGustafsson]

Are there any suggestion about this behavior in PyTorch?

import torch

x = torch.zeros([], requires_grad=True)
y = torch.zeros([])

x.grad = y
print(y is x.grad)  # True

To maintain this behavior seems to be of pretty marginal value, right?

[yueyinqiu]

I suppose at least I will never rely on this.

I have made this pr ready for review/merging.

[yueyinqiu]

The test seems to fail by chance.

Perhaps we need a seeded random for unit tests :)

Would it be a good idea to include pythonnet and check the results with PyTorch?

[NiklasGustafsson]

There’s some kind of concurrency issue that is affecting things. We could use seeded random numbers, too, but it would be a lot of work and not all APIs accept generator objects. Setting the global seed won’t work because of reentrancy. Niklas From: yueyinqiu ***@***.***> Date: Wednesday, May 1, 2024 at 09:27 To: dotnet/TorchSharp ***@***.***> Cc: Niklas Gustafsson ***@***.***>, Comment ***@***.***> Subject: Re: [dotnet/TorchSharp] property grad (PR #1299) The unit test seems to fail by chance. Perhaps we need a seeded random for unit test :) Would it be a good idea to include pythonnet and check the results with PyTorch? — Reply to this email directly, view it on GitHub<#1299 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AANVMB7AKZRM537IWYROECLZAEJXNAVCNFSM6AAAAABG26L6XGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBYG4ZDKNBRGY>. You are receiving this because you commented.Message ID: ***@***.***>

[yueyinqiu]

Yes, and I've found that torch's reproducibility system is not that reliable. I'm just considering something like C# System.Random, for our unit test alone.

[NiklasGustafsson]

Hmmm. Tests are failing due to MNIST download. That has nothing to do with the PR, but I wonder if the source location has changed. If so, we may need to disable the test and then file an issue.

[yueyinqiu]

I suppose it's a temporary problem because train images, train labels and test images can all be accessed. (And test labels cannot.)

[NiklasGustafsson]

Let's hope. I'll try restarting the build a couple of times today.

[NiklasGustafsson]

It just keeps failing. Based on the stack trace, it's failing to download the first file, which is 'train-images-idx3-ubyte.gz' and when I click on the links above, it doesn't work. Maybe it's got something to do with where the requests come from? If the license allows it, we could add those files to the repo, instead. They're not big.

[yueyinqiu]

I suppose the best solution is to use mirrors.

In PyTorch there is a torchvision.datasets.MNIST.mirrors, whose default value is:

    mirrors = [
        "http://yann.lecun.com/exdb/mnist/",
        "https://ossci-datasets.s3.amazonaws.com/mnist/",
    ]

However torchvision.datasets.MNIST is not a class in TorchSharp. So

• should we add that in Modules.MNIST? But this is inconsistent with PyTorch;
• or make torchvision.datasets.MNIST a class? But then it can't be a method. Still inconsistent;
• or maybe we could just simply add an extra parameter.

---

Edit:

I have checked that only mnist-like datasets have mirrors property. (Also, they must have such property, since they are inherited from MNIST.) Since it's not a common property of datasets, it seems that it's more like a inner property, which users are not encouraged to touch.

In that case, adding mirrors into Modules.MNIST is also acceptable. And this allows us to add more properties freely, like resources, training_file, test_file, classes and etc.

[yueyinqiu]

However current Modules.MNIST is internal. Perhaps what we should do currently is just to add the another default mirror.

I have created a PR here #1308

[yueyinqiu]

Maybe it's got something to do with where the requests come from?

Perhaps it is. I have found a relevant issue in pytorch pytorch/vision#7637