video_path is what ?

[ucas010]

hi,dear
in the predict.py,
what's the meaning ?

    parser.add_argument(
        '--save_dir',
        type=str,
        default=os.path.join('data', 'predict_results'),
        help='directory to store results')
    parser.add_argument(
        '--video_path',
        type=str,
        default=None,
        help='directory to store results')

video_path is the video what I want to test/infer ?

thx

[ucas010]

but no,
when I set the para
default='D:/python36/new/dyrandom/mvideo_0.mp4',
also has the same problem

Traceback (most recent call last):
  File "D:\python36\new\rcdnn\image\predict.py", line 181, in <module>
    infer(args)
  File "D:\python36\new\rcdnn\image\predict.py", line 103, in infer
    infer_reader = get_reader(args.model_name.upper(), 'infer', infer_config)
  File "D:\python36\new\rcdnn\image\reader\reader_utils.py", line 67, in get_reader
    return reader_model.create_reader()
  File "D:\python36\new\rcdnn\image\reader\feature_reader.py", line 38, in create_reader
    fl = open(self.filelist).readlines()
FileNotFoundError: [Errno 2] No such file or directory: './data/dataset/youtube8m/infer.list'

So what's the para 's meaning ?
thx

[heavengate]

您好，哪个模型呢

[ucas010]

@heavengate
NeXtVLAD

[SunGaofeng]

the --video_path argument is designed to point to the video file to be processed, it can be 'D:/python36/new/dyrandom/mvideo_0.mp4'.

However, only models such as TSN, TSM, StNet and NONLOCAL in PaddleVideo support processing a video file of MP4 or other format.
The model NeXtVLAD does not support processing a video file since it should take feature data as input.
If you want to predict by NeXtVLAD, please transfer the origin video file into feature data first.

[ucas010]

@SunGaofeng
thx
So could you tell me which is the API of the feature input to the model,
or just tell me what's the data format in
./data/dataset/youtube8m/infer.list
thanks a lot

[SunGaofeng]

@SunGaofeng
thx
So could you tell me which is the API of the feature input to the model,
or just tell me what's the data format in
./data/dataset/youtube8m/infer.list
thanks a lot

This is similar to those used in the training dataset, where all the feature data are transformed from the Youtube-8M original data and stored in the pickle format. For details, you can refer to this document https://github.com/PaddlePaddle/models/tree/develop/PaddleCV/video/data/dataset#Youtube-8M数据集

If you use your own video file, you can use a pertained model to extract the rgb/audio feature data for each frame. For each video, the data format is a dictionary as:

video_data = {'video': video_id_string, 
                            'nframes':number_of_frames_int, 
                           'feature': extracted_rgb_feature_numpy, 
                           'audio': extracted_audio_feature_numpy}

./data/dataset/youtube8m/infer.list assumes that you store your data in pickle files. Each line of infer.list is the path of a pickle file. Each pickle file may store one or more video data, which is a list as [video_data1, video_data2, ...]

[ucas010]

@SunGaofeng
thanks for your kind help,
it's important to inference with the pretrained model.

[ucas010]

some other question
if I only have one frame image, several frame audio feature,
how to set the 'nframes'?
or should I separate the them, then got each of theirs inference ?just as follows

I want to get the feature before Concat Op.
but when I see the codes,I do not know how to deal with it.

    def create_reader(self):
        fl = open(self.filelist).readlines()
        fl = [line.strip() for line in fl if line.strip() != '']

        def reader():
            batch_out = []
            for filepath in fl:
                print(python_ver)
                if python_ver < (3, 0):
                    data = pickle.load(open(filepath, 'rb'))
                else:
                    data = pickle.load(open(filepath, 'rb'), encoding='bytes')
                indexes = list(range(len(data)))

                for i in indexes:
                    record = data[i]
                    nframes = record[b'nframes']
                    rgb = record[b'feature'].astype(float)
                    audio = record[b'audio'].astype(float)
                    video = record[b'video']

                    rgb = rgb[0:nframes, :]
                    audio = audio[0:nframes, :]

                    batch_out.append((rgb, audio, video))
                    if len(batch_out) == self.batch_size:
                        yield batch_out
                        batch_out = []

        return reader

So could U please help me ?
thx

[SunGaofeng]

If you have different frame numbers for audio and video, just set different 'nframes' value for them.
The feature before concat if sequence pooled, so you can concat audio and video feature even through they have different frames.

models/PaddleCV/video/models/nextvlad/nextvlad_model.py

Line 120 in 5678d3f

vlad = fluid.layers.sequence_pool(vlad, 'sum')

[ucas010]

@SunGaofeng
thanks for your kind help
Now will close the issue.