Switch from nn.DataParallel to nn.DistributedDataParallel for realtime inference

[tonyhoo]

Issue #, if available:

Description of changes:
Switch from DataParallel to DistributedDataParallel as recommended in the latest Pytorch doc

However, after switching the real time inference speed has been impacted significantly due to the overhead on spawning new process, collecting the data from these process and moving it between CPU and GPUs. See plot below:

Generating script (requires code change):

import os
import warnings
import time
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
 
from autogluon.core.utils.loaders import load_zip
from autogluon.multimodal import MultiModalPredictor
 
warnings.filterwarnings('ignore')
np.random.seed(123)
 
if __name__ == '__main__':
    download_dir = './ag_multimodal_tutorial'
 
    dataset_path = f'{download_dir}/petfinder_for_tutorial'
 
    test_data = pd.read_csv(f'{dataset_path}/test.csv', index_col=0)
 
    label_col = 'AdoptionSpeed'
 
    predictor = MultiModalPredictor.load("/home/ubuntu/workplace/autogluon/autogluon/multimodal/AutogluonModels/ag-20231016_231146")
 
    
    strategies = ["ddp", "dp"]
 
    num_runs = 30  # number of runs for each strategy
    results = []
    for strategy in strategies:
        
        for i in range(num_runs):
            start = time.time()
            predictions = predictor.predict(test_data.drop(columns=label_col), realtime=True, strategy=strategy)
            end = time.time()
            if i == 0:
                continue
            results.append({'strategy': strategy, 'execution_time': end - start})
 
    results_df = pd.DataFrame(results)
 
    # save results to a csv file
    results_df.to_csv(f"results.csv", index=False)
 
    # print the average execution time for each strategy
    average_times = results_df.groupby('strategy')['execution_time'].mean()
    print(average_times)
 
    # plot the results
    results_df.boxplot(by='strategy')
    plt.ylabel('Execution Time (s)')
    plt.show()
    plt.savefig('execution_time_plot.png')

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

Job PR-3565-9201202 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-3565/9201202/index.html

[zhiqiangdon]

Did you test realtime inference in multi-gpu environment and compare the inference time?

[tonyhoo]

Updated the codes and testing results for 30 runs between DDP and DP. It looks like DDP for real time inference has much high latency contributed by results collection operation. Based on the results, I don't recommend to use DDP for real time inference