IPC-H264-HLS-C-SDK

This is a sample code for IP Cameras (IPC for short) that use built-in MCU to generate TS format files and upload to S3.

And the S3_Simple_Put part of this SDK can also be considered as sample for Signature V4 process when calling S3 APIs.

With this SDK, IPC vendors don't need to pay for EC2 instances that receiving the video stream and only pay for the amount of storage they use to store the video. And can playback later with dynamically generated m3u8 file.

The input parameters of this SDK inclucde AK/SK/Token for AWS S3 access, region name, bucket name and s3 object prefix (ususally the iot device certificate id).

System requirement: IPC running Linux with H264 video encoding. SoC should have 3.5-4MB available memory when running @ 5Mbps data rate (Common data rate for 1080P video). Memory buffer can be adjusted according to the data rate when initialize the SDK.

3rd party library: (Already provided as submodule) OpenSSL 1.1.1 Curl 7.70

Design Overview

Data flow is as following

                          
    IPC Encoding Module
            |
            |
            v
      H264 RAW Stream
            |
            |
            v
S3_HLS_SDK Put Video/Audio Frame
    (Package to TS format)
            |
            |
            v
    S3_HLS_Buffer_Mgr (Cache)
            |
            |
            v
        Upload Queue
            |
            |
            v
    S3_HLS_S3_Put_Client
            |
            |
            v
         Amazon S3 (On Cloud)

How to Compile

1. Clone the current repository.

git clone https://github.com/aws-samples/ipc-h264-hls-c-sdk.git

2. Ensure the 3rd OpenSSL project is working.

cd ipc-h264-hls-c-sdk/
git submodule init
git submodule update

3. Configure OpenSSL project.

cd 3rd/openssl/
./Configure no-asm no-async no-egd --prefix=$PWD –-cross-compile-prefix=<YOUR_CROSS_COMPILER> <YOUR_PLATFORM>

Note:

Change the "<YOUR_CROSS_COMPILER>" to your cross compiler prefix. E.g.: mips-linux-gnu- etc.

Change the "<YOUR_PLATFORM>" to your target platform. E.g.: linux-x86_64 or linux-mips etc. And you may need to modify the generated Makefile on some platform and remove unsupported options.

4. Make OpenSSL project

make

Check the libcrypto.a & libssl.a files are generated in the openssl directory.

5. Configure Curl project

cd ../curl
autoreconf -fi
export LD_LIBRARY_PATH=../openssl
export CROSS_COMPILE=
export LDFLAGS="-L$PWD/../openssl/"
./configure ac_cv_func_RAND_egd=no --disable-shared --enable-static --with-ssl=$PWD/../openssl

6. Compile Curl project

make

Check the libcurl.a file is generated at lib/.libs/ folder.

5. Compile IPC-H264-HLS-C-SDK

cd ../..

Edit Makefile, add CROSS_COMPILE prefix if necessary.

  
CROSS_COMPILE=<YOUR_CROSS_COMPILER>
CC=$(CROSS_COMPILE)gcc
LD=$(CROSS_COMPILE)ld
AR=$(CROSS_COMPILE)ar

Compile IPC-H264-HLS-C-SDK.

make

Check the s3_hls.a file is generated in the directory. You need to link this library and corresponding libcrypto.a libssl.a library when creating your own application.

IPC-H264-HLS-C-SDK usage sudo code example:

1. Initialize the system

#define BUFFER_SIZE	4*1024*1024 // 4MB
if(S3_HLS_OK != S3_HLS_SDK_Initialize(BUFFER_SIZE, region_name, bucket_name, prefix, custom_endpoint_name) ) {
    return FAILED;
}

2. Set credential

The credential used for SDK can be ak/sk generated using IAM console. The best practise is to use AWS IoT Device Managment. Use built in certificate to exchange temporary credential using Credentials Provider. Please refer to below blog for details: https://aws.amazon.com/blogs/security/how-to-eliminate-the-need-for-hardcoded-aws-credentials-in-devices-by-using-the-aws-iot-credentials-provider/

if(S3_HLS_OK != S3_HLS_SDK_Set_Credential(argv[1], argv[2], argc >= 7 ? argv[6] : NULL)) {
    S3_HLS_SDK_Finalize();
    return FAILED;
}

During execution, user can call this function to replace credentials used to upload video clips.

3. Optionally user can specify tags that added to uploaded video clips

S3_HLS_SDK_Set_Tag("key1=value1");

4. Start upload thread for upload video clips to S3

S3_HLS_SDK_Start_Upload();

5. In the main thread of processing frames

// Main thread for processing frames:
while(!exit) {
    // Get video stream from IPC
    int nr_pack = stream->packCount;
    
	S3_HLS_FRAME_PACK s3_frame_pack;
	s3_frame_pack.item_count = nr_pack;
	
	for(int i=0; i < nr_pack; i++) {
		IMPEncoderPack *pack = &stream->pack[i];
		// use timestamp generated by encoder
		s3_frame_pack.items[i].timestamp = pack->timestamp;
		if(pack->length){
			uint32_t remSize = stream->streamSize - pack->offset;
			if(remSize < pack->length){ // IPC buffer acrossed internal ring buffer boundary
				s3_frame_pack.items[i].first_part_start = (void *)(stream->virAddr + pack->offset);
				s3_frame_pack.items[i].first_part_length = remSize;
				
				s3_frame_pack.items[i].second_part_start = (void *)stream->virAddr;
				s3_frame_pack.items[i].second_part_length = pack->length - remSize;
			}else { // IPC buffer does not across internal ring buffer boundary
				s3_frame_pack.items[i].first_part_start = (void *)(stream->virAddr + pack->offset);
				s3_frame_pack.items[i].first_part_length = pack->length;
				
				s3_frame_pack.items[i].second_part_start = NULL;
				s3_frame_pack.items[i].second_part_length = 0;
			}
	}

	S3_HLS_SDK_Put_Video_Frame(&s3_frame_pack);
}

5. When exit the program, do some clean up tasks

S3_HLS_SDK_Finalize();

For using IoT Core to get AK/SK/Token, please refer to below link: https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.