Tutorial file for Convolutional Networks

[rajeshpandit107]

Please post a tutorial for cnn.

[rajeshpandit107]

Hi,
When I run synthesis of CNN, the step where the tool starts converting partioning array.v in 1 dimenstion completely, RAM usage into the systems increase drastically and it fills up the entire RAM of 16 GB in my PC, after that vivado_hls automatically gets killed. Is there any workaround for this ? @thaarres @thesps @vloncar, Please let me know.
vivado_hls.log

[thaarres]

Hi @rajeshpandit107! Can you have a look through the notebook here part6_cnns.ipynb, specifically the section "CNNs in hls4ml", and make sure you are using the correct configuration settings?

[ynozik]

@thaarres Thanks a lot for the cnn tutorial. When I try to run the notebook I run into an error on the codeblock under "CNNs in hls4ml" (where the trained models are loaded and strip_pruning is called).

The error is:

---

ValueError Traceback (most recent call last)
in
13 'QDense': QDense,
14 'QConv2DBatchnorm': QConv2DBatchnorm,
---> 15 'QActivation': QActivation})
16 qmodel = strip_pruning(qmodel)

~/.local/lib/python3.6/site-packages/tensorflow/python/keras/saving/save.py in load_model(filepath, custom_objects, compile, options)
200 (isinstance(filepath, h5py.File) or h5py.is_hdf5(filepath))):
201 return hdf5_format.load_model_from_hdf5(filepath, custom_objects,
--> 202 compile)
203
204 filepath = path_to_string(filepath)

[... cut out intermediate calls ...]

~/.local/lib/python3.6/site-packages/tensorflow/python/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)
559 'https://www.tensorflow.org/guide/keras/save_and_serialize'
560 '#registering_the_custom_object for details.'
--> 561 .format(printable_module_name, class_name))
562
563 cls_config = config['config']

ValueError: Unknown config_item: quantized_bits. Please ensure this object is passed to the custom_objects argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.

---

I was able to run all cells before this successfully without modification, the only difference being I trained for fewer epochs to save time. The cell that causes the error is also marked uneditable, not sure why; I made it modifiable to try to pass quantized_bits as an activation but that didn't seem to work. Any idea what the issue might be?

[ynozik]

Sorry - seems that was likely fixed by adding an entry of 'quantized_bits': quantized_bits to custom_objects. I'm not completely sure if it did since I now get an error when trying to run the line hls_config_q = hls4ml.utils.config_from_keras_model(qmodel, granularity='name'), with error:

---

Exception Traceback (most recent call last)
in
4 hls4ml.model.optimizer.OutputRoundingSaturationMode.saturation_mode = 'AP_SAT'
5
----> 6 hls_config_q = hls4ml.utils.config_from_keras_model(qmodel, granularity='name')
7 hls_config_q['Model']['ReuseFactor'] = 1
8 hls_config['Model']['Precision'] = 'ap_fixed<16,6>'

~/.local/lib/python3.6/site-packages/hls4ml/utils/config.py in config_from_keras_model(model, granularity, default_precision, default_reuse_factor)
127 for keras_layer in keras_layer_config:
128 if keras_layer['class_name'] not in supported_layers:
--> 129 raise Exception('ERROR: Unsupported layer type: {}'.format(keras_layer['class_name']))
130 if keras_layer['class_name'] in skip_layers:
131 continue

Exception: ERROR: Unsupported layer type: QConv2DBatchnorm

---

The layer seems to use use_bias=True when the qKeras model is initially defined (again I didn't modify any code except for fewer training epochs), so not sure why that would be, maybe unless my adding quantized_bits earlier messed something up - I'm not too familiar with the qKeras / hls4ml workings, though.

[thaarres]

Hi Yonatan! Thanks for the e-mail and for finding this bug! Indeed, quantised bits is missing from the custom_objects list. About the second error, could it be that you are on hls4ml v0.5.0 and not on the master branch? That layer was implemented later[*]. The environment.yml[**] should be pointing to the correct version, but it differs from the version used in the master branch of the hls4ml-tutorials repository [*] https://github.com/fastmachinelearning/hls4ml/blob/master/hls4ml/model/hls_layers.py#L1823 [**]https://github.com/thaarres/hls4ml-tutorial/blob/master/environment.yml On 9 Jul 2021, at 02:49, Yonatan Nozik ***@***.******@***.***>> wrote: Sorry - seems that was likely fixed by adding an entry of 'quantized_bits': quantized_bits to custom_objects. I'm not completely sure if it did since I now get an error when trying to run the line hls_config_q = hls4ml.utils.config_from_keras_model(qmodel, granularity='name'), with error:

…

________________________________ Exception Traceback (most recent call last) in 4 hls4ml.model.optimizer.OutputRoundingSaturationMode.saturation_mode = 'AP_SAT' 5 ----> 6 hls_config_q = hls4ml.utils.config_from_keras_model(qmodel, granularity='name') 7 hls_config_q['Model']['ReuseFactor'] = 1 8 hls_config['Model']['Precision'] = 'ap_fixed<16,6>' ~/.local/lib/python3.6/site-packages/hls4ml/utils/config.py in config_from_keras_model(model, granularity, default_precision, default_reuse_factor) 127 for keras_layer in keras_layer_config: 128 if keras_layer['class_name'] not in supported_layers: --> 129 raise Exception('ERROR: Unsupported layer type: {}'.format(keras_layer['class_name'])) 130 if keras_layer['class_name'] in skip_layers: 131 continue Exception: ERROR: Unsupported layer type: QConv2DBatchnorm

________________________________ The layer seems to use use_bias=True when the qKeras model is initially defined (again I didn't modify any code except for fewer training epochs), so not sure why that would be, maybe unless my adding quantized_bits earlier messed something up - I'm not too familiar with the qKeras / hls4ml workings, though. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#14 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABKLVQBTBFC2JTCQ3R5JOKLTWZBTFANCNFSM43ZVLDVQ>.

[ynozik]

Hi @thaarres, running pip show hls4ml does show that I'm on version 0.5.0. To clarify, 0.5.0 is the latest stable release but doesn't reflect the most recent layer compatibilities?

I haven't been using a conda environment, so I suppose I should probably try that.

[ynozik]

Hi @thaarres - apologies for the repeated mentions. I revisited the notebook in the proper conda environment and was able to synthesize the default models (I did have to reduce the number of filters/neurons per layer to reduce the memory usage on my machine). However, there seems to be some error with modifying the models to use the resource strategy, since all my attempts to use it have resulted in the following:

---

***** C/RTL SYNTHESIS *****
INFO: [SCHED 204-61] Option 'relax_ii_for_timing' is enabled, will increase II to preserve clock frequency constraints.
INFO: [HLS 200-10] Analyzing design file 'firmware/myproject.cpp' ...
WARNING: [HLS 214-114] Since the only kind of statements allowed in a dataflow region are variable declarations and function calls, the compiler may not be able to correctly handle the region: firmware/nnet_utils/nnet_dense_latency.h:64:9
WARNING: [HLS 214-104] Only for-loops and functions support the dataflow: firmware/nnet_utils/nnet_dense_latency.h:76:9
WARNING: [HLS 214-114] Since the only kind of statements allowed in a dataflow region are variable declarations and function calls, the compiler may not be able to correctly handle the region: firmware/myproject.cpp:35:2
WARNING: [HLS 200-471] Dataflow form checks found 3 issue(s) in file firmware/myproject.cpp
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer2_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer5_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer6_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer7_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer10_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer11_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer12_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer15_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer16_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer17_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer20_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer21_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer24_out' is not a valid pragma.
ERROR: [HLS 200-70] '#pragma HLS STABLE variable=&layer25_out' is not a valid pragma.
Pragma processor failed:
while executing
"source [lindex $::argv 1] "
("uplevel" body line 1)
invoked from within
"uplevel #0 { source [lindex $::argv 1] } "

---

This appears to happen regardless of whether I set the overall model strategy to resource or on a per-layer basis. It also seems to happen regardless of the associated ReuseFactor. IOType is still set to io_stream, of course.

Although I have no idea if it's related, I tried to omit setting the output_softmax strategy to Stable, but that doesn't seem to have an effect either.

Keeping the latency strategy while increasing the ReuseFactor results in successful synthesis, but without too much of an effect on the results. I'm not sure what the expected synthesis behavior is of hls4ml is when you specify high reuse with a latency strategy (e.g. can it mostly ignore the ReuseFactor or does ReuseFactor take priority).

Do you happen to know if specifying the resource strategy requires any additional modifications besides simply changing Strategy?

Thanks very much.

[thaarres]

Hi @ynozik !
Sorry for the delay in following up on this! I saw you submitted an issue to the main hls4ml repository as well, did you pin this error down to the Vivado version?

[ynozik]

Hi @thaarres, yes, that was resolved by changing the Vivado version.

[lclee0577]

Hi @rajeshpandit107! Can you have a look through the notebook here part6_cnns.ipynb, specifically the section "CNNs in hls4ml", and make sure you are using the correct configuration settings?

Hi @thaarres ，in part6_cnns.inpy, the accuracy was quite low after convert to hls model(about 80% -> 20%).
Since the flatten layer was skipped, the output shape stuck with (2,2,*) .
I added another MaxPooling2D layer before Flatten layer may solve the problem. But it still confuse me why the Flatten layer was skipped.

[vloncar]

Fixed in fastmachinelearning/hls4ml#378

[lclee0577]

Fixed in fastmachinelearning/hls4ml#378

sorry ,I just reinstall the hls4ml，and thanks a lot !

[thesps]

The CNN tutorial notebook is now online with the new hls4ml release: closing this issue