[QST] Prediction Output Length Not Matching Input Length

[dpendleton22]

❓ Questions & Help

Details

This question references following the demo which uses synthetic data: synthetic data demo

I'm trying to understand what is going on with this model and how it works. I got it installed and I'm following the demo where the synthetic data is generated. The demo uses a NextItemPrediction head which I follow and train.

I'm able to serve the model on Triton but when I generate more data to now predict on I can quite understand what's going on. I generated 1k new rows with 966 unique session_ids but the prediction have be back an array of size (34, 500). Why am I only getting back 34 rows of predictions??

I followed the demo exactly so feel free to reference that if you'd like. The demo itself shows in it's example that passing in 1k examples returned 28 predictions:

NUM_ROWS =1000
long_tailed_item_distribution = np.clip(np.random.lognormal(3., 1., int(NUM_ROWS)).astype(np.int32), 1, 50000)
# generate random item interaction features 
df = pd.DataFrame(np.random.randint(70000, 90000, int(NUM_ROWS)), columns=['session_id'])
df['item_id'] = long_tailed_item_distribution

# generate category mapping for each item-id
df['category'] = pd.cut(df['item_id'], bins=334, labels=np.arange(1, 335)).astype(np.int32)
df['age_days'] = np.random.uniform(0, 1, int(NUM_ROWS)).astype(np.float32)
df['weekday_sin']= np.random.uniform(0, 1, int(NUM_ROWS)).astype(np.float32)

# generate day mapping for each session 
map_day = dict(zip(df.session_id.unique(), np.random.randint(1, 10, size=(df.session_id.nunique()))))
df['day'] =  df.session_id.map(map_day)

print(df.head(2))

   session_id  item_id  category  age_days  weekday_sin  day
0       79856        3         2  0.327276     0.080060    2
1       74117        6         4  0.012172     0.147716    1

from merlin.systems.triton.utils import send_triton_request
response = send_triton_request(workflow.input_schema, df, output_schema.column_names, endpoint="localhost:8001")
response

{'next-item': array([[-3.9399953, -2.632081 , -4.2211075, ..., -3.6699016, -3.673493 ,
-3.1244578],
[-3.940445 , -2.6335964, -4.2203593, ..., -3.671566 , -3.6745713,
-3.1240335],
[-3.9393594, -2.6300201, -4.222065 , ..., -3.6674871, -3.672068 ,
-3.1251097],
...,
[-3.9396427, -2.6304667, -4.2218847, ..., -3.6677885, -3.6724825,
-3.1250875],
[-3.939829 , -2.6316376, -4.221267 , ..., -3.6693997, -3.6732295,
-3.1245873],
[-3.9399223, -2.631995 , -4.2210817, ..., -3.669589 , -3.6734715,
-3.1244512]], dtype=float32)}
response['next-item'].shape
(28, 495)

[dpendleton22]

Answered my own question. There is a minimum sequence length of 2 set when creating the schema so any sessions that do now have more then 1 event will be discarded