README.md

tf-neural-process

Tensorflow implementation of Neural Process family

• Full code and experiment is based on deepmind repository [GIT, LICENSE]
• This repository aims to write more generic code and split model from experiment.
• Paper:
  1. Conditional Neural Process [arxiv]
  2. Neural Process [arxiv]
  3. Attentive Neural Process [arxiv]

Sample

Jupyter notebook sample is here.

1. Conditional Neural Process

Model definition [GIT]

encoder_output_sizes = [128, 128, 128, 128]
decoder_output_sizes = [128, 128, 1]

model = neural_process.ConditionalNP(encoder_output_sizes, decoder_output_sizes)

Sample image

2. Neural Process

Model definition [GIT]

z_output_sizes = [128, 128, 128, 128]
enc_output_sizes = [128, 128, 128, 128]
dec_output_sizes = [128, 128, 1]

model = neural_process.NeuralProcess(z_output_sizes, enc_output_sizes, dec_output_sizes)

Sample image

3. Attentive Neural Process

Model definition [GIT]

z_output_sizes = [128, 128, 128, 128]
enc_output_sizes = [128, 128, 128, 128]
cross_output_sizes = [128, 128, 128, 128]
dec_output_sizes = [128, 128, 1]

self_attention = neural_process.Attention(attention_type='multihead', proj=[128, 128])
cross_attention = neural_process.Attention(attention_type='multihead', proj=[128, 128])

model = neural_process.AttentiveNP(z_output_sizes,
                                   enc_output_sizes,
                                   cross_output_sizes,
                                   dec_output_sizes,
                                   self_attention,
                                   cross_attention)

Sample image