Add lookup table operator

[jantonguirao]

Signed-off-by: Joaquin Anton janton@nvidia.com

Why we need this PR?

Need a lookup operator, e.g. lookup weights for label values

What happened in this PR?

• Introduced LookupTable CPU and GPU implementations
• Added python tests to cover the new operator

JIRA TASK: [DALI-1050]

[jantonguirao]

!build

[dali-automaton]

CI MESSAGE: [899100]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [899100]: BUILD PASSED

[klecki]

Btw, did you forget to add python tests?

[jantonguirao]

Btw, did you forget to add python tests?

Yes, and the GPU implementation. I pushed it now

[jantonguirao]

!build

[dali-automaton]

CI MESSAGE: [902153]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [902153]: BUILD PASSED

[jantonguirao]

!build

[dali-automaton]

CI MESSAGE: [903729]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [903729]: BUILD PASSED

[LiuHao-THU]

ops, only int supported for input.

[JanuszL]

@LiuHao-THU - what is your use case and how you want it to work?

[LiuHao-THU]

@LiuHao-THU - what is your use case and how you want it to work?
Thank for the reply, here is my situation:

I'm Reimplement bayesian personalized ranking(a recommendation algorithm), the network is very small, however, I have over 200000 users in my datasets. Now, I'm using PyTorch data_loader,
the code is here. self.cand is a dictionary(lookup-table in my datasets, user as key, rating as value), however, after I set num works in data loader to a very large value, the CPU usage is very high, the network still runs in low GPU usage. so I want to load the datasets to GPU for preprocessing, however, the lookuptable range from [0, 65535].

"""

"""
class BPRData(data.Dataset):

def __init__(self, users, items, candidates, num_items, is_training = True):
	super(BPRData, self).__init__()
	self.users = users
	self.items = items
	self.is_training = is_training
	self.cand = candidates
	self.all = set([i for i in range(num_items)])
	self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

def __len__(self):
	return len(self.users)

def __getitem__(self, idx):
	if self.is_training == True:
		neg_items = list(self.all - set(self.cand[int(self.users[idx])]))
		indices = random.randint(0, len(neg_items) - 1)

	user = self.users[idx]
	item_i = self.items[idx]
	item_j = neg_items[indices] if \
			self.is_training else self.items[idx]

	return user, item_i, item_j

"""

[JanuszL]

@LiuHao-THU - the lookup was rather designed for signal processing, than for tabular data. Have you tried to check RAPIDS?
If you want to can also extend LookupTable operator to support a wider range of indexes.