-
Notifications
You must be signed in to change notification settings - Fork 844
/
lite.html
600 lines (471 loc) · 46.9 KB
/
lite.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="zh-CN" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="zh-CN" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>TensorFlow Lite（Jinpeng） — 简单粗暴TensorFlow 2.0 0.4 alpha 文档</title>
<script type="text/javascript" src="../../_static/js/modernizr.min.js"></script>
<script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../_static/jquery.js"></script>
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
<script type="text/javascript" src="../../_static/js/tw_cn.js"></script>
<script type="text/javascript" src="../../_static/js/pangu.min.js"></script>
<script type="text/javascript" src="../../_static/js/custom.js"></script>
<script type="text/javascript" src="../../_static/translations.js"></script>
<script type="text/javascript" src="../../_static/js/theme.js"></script>
<link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />
<link rel="index" title="索引" href="../../genindex.html" />
<link rel="search" title="搜索" href="../../search.html" />
<link rel="next" title="TensorFlow in JavaScript（Huan）" href="javascript.html" />
<link rel="prev" title="TensorFlow Serving" href="serving.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../../index.html" class="icon icon-home"> 简单粗暴TensorFlow 2.0
</a>
<div class="version">
0.4
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<p class="caption"><span class="caption-text">目录</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../preface.html">前言</a></li>
<li class="toctree-l1"><a class="reference internal" href="../introduction.html">TensorFlow概述</a></li>
</ul>
<p class="caption"><span class="caption-text">基础</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../basic/installation.html">TensorFlow安装与环境配置</a></li>
<li class="toctree-l1"><a class="reference internal" href="../basic/basic.html">TensorFlow基础</a></li>
<li class="toctree-l1"><a class="reference internal" href="../basic/models.html">TensorFlow 模型建立与训练</a></li>
<li class="toctree-l1"><a class="reference internal" href="../basic/tools.html">TensorFlow常用模块</a></li>
</ul>
<p class="caption"><span class="caption-text">部署</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="export.html">TensorFlow模型导出</a></li>
<li class="toctree-l1"><a class="reference internal" href="serving.html">TensorFlow Serving</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">TensorFlow Lite（Jinpeng）</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#id1">模型转换</a></li>
<li class="toctree-l2"><a class="reference internal" href="#android">Android部署</a></li>
<li class="toctree-l2"><a class="reference internal" href="#quantization">Quantization模型转换</a></li>
<li class="toctree-l2"><a class="reference internal" href="#id3">总结</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="javascript.html">TensorFlow in JavaScript（Huan）</a></li>
</ul>
<p class="caption"><span class="caption-text">大规模训练与加速</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../appendix/distributed.html">TensorFlow分布式训练</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/tpu.html">使用TPU训练TensorFlow模型（Huan）</a></li>
</ul>
<p class="caption"><span class="caption-text">扩展</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../appendix/tfhub.html">TensorFlow Hub 模型复用（Jinpeng）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/tfds.html">TensorFlow Datasets 数据集载入</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/swift.html">Swift for TensorFlow (S4TF) (Huan）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/julia.html">TensorFlow in Julia（Ziyang）</a></li>
</ul>
<p class="caption"><span class="caption-text">附录</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../appendix/static.html">图模型下的TensorFlow</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/docker.html">使用Docker部署TensorFlow环境</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/cloud.html">在云端使用TensorFlow</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/jupyterlab.html">部署自己的交互式Python开发环境JupyterLab</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/optimization.html">TensorFlow性能优化</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/recommended_books.html">参考资料与推荐阅读</a></li>
<li class="toctree-l1"><a class="reference internal" href="../appendix/terms.html">术语中英对照表</a></li>
</ul>
<p class="caption"><span class="caption-text">Preface</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../en/preface.html">Preface</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/introduction.html">TensorFlow Overview</a></li>
</ul>
<p class="caption"><span class="caption-text">Basic</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../en/basic/installation.html">Installation and Environment Configuration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/basic/basic.html">TensorFlow Basic</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/basic/models.html">Model Construction and Training</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/basic/tools.html">Common Modules in TensorFlow</a></li>
</ul>
<p class="caption"><span class="caption-text">Deployment</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../en/deployment/export.html">TensorFlow Model Saving</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/deployment/serving.html">TensorFlow Serving</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/deployment/lite.html">TensorFlow Lite</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/deployment/javascript.html">TensorFlow in JavaScript</a></li>
</ul>
<p class="caption"><span class="caption-text">Large-scale Training</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/distributed.html">Distributed Training with TensorFlow</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/tpu.html">Training TensorFlow models with TPU</a></li>
</ul>
<p class="caption"><span class="caption-text">Extensions</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/tfhub.html">TensorFlow Hub: Model Reuse</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/tfds.html">TensorFlow Datasets: Ready-to-use Datasets</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/swift.html">Swift for TensorFlow (S4TF)</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/julia.html">TensorFlow in Julia</a></li>
</ul>
<p class="caption"><span class="caption-text">Appendix</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/static.html">TensorFlow Under Graph Model</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/docker.html">Using Docker to deploy TensorFlow environment</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/cloud.html">Using TensorFlow on cloud</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/jupyterlab.html">Deploying Your Own Interactive Python Development Environment, JupyterLab</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/optimization.html">TensorFlow Performance Optimization</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/recommended_books.html">References and Recommendations for Further Reading</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../en/appendix/terms.html">Terminology comparison table between Chinese and English</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../../index.html">简单粗暴TensorFlow 2.0</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="../../index.html">Docs</a> »</li>
<li>TensorFlow Lite（Jinpeng）</li>
<li class="wy-breadcrumbs-aside">
<a href="../../_sources/zh/deployment/lite.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="tensorflow-lite-jinpeng">
<h1>TensorFlow Lite（Jinpeng）<a class="headerlink" href="#tensorflow-lite-jinpeng" title="永久链接至标题">¶</a></h1>
<p>TensorFlow Lite是TensorFlow在移动和IoT等边缘设备端的解决方案，提供了Java、Python和C++ API库，可以运行在Android、iOS和Raspberry Pi等设备上。2019年是5G元年，万物互联的时代已经来临，作为TensorFlow在边缘设备上的基础设施，TFLite将会是愈发重要的角色。</p>
<p>目前TFLite只提供了推理功能，在服务器端进行训练后，经过如下简单处理即可部署到边缘设备上。</p>
<ul class="simple">
<li><p>模型转换：由于边缘设备计算等资源有限，使用TensorFlow训练好的模型，模型太大、运行效率比较低，不能直接在移动端部署，需要通过相应工具进行转换成适合边缘设备的格式。</p></li>
<li><p>边缘设备部署：本节以android为例，简单介绍如何在android应用中部署转化后的模型，完成Mnist图片的识别。</p></li>
</ul>
<div class="section" id="id1">
<h2>模型转换<a class="headerlink" href="#id1" title="永久链接至标题">¶</a></h2>
<p>转换工具有两种：命令行工具和Python API</p>
<p>TF2.0对模型转换工具发生了非常大的变化，推荐大家使用Python API进行转换，命令行工具只提供了基本的转化功能。转换后的原模型为 <code class="docutils literal notranslate"><span class="pre">FlatBuffers</span></code> 格式。 <code class="docutils literal notranslate"><span class="pre">FlatBuffers</span></code> 原来主要应用于游戏场景，是谷歌为了高性能场景创建的序列化库，相比Protocol Buffer有更高的性能和更小的大小等优势，更适合于边缘设备部署。</p>
<p>转换方式有两种：Float格式和Quantized格式</p>
<p>为了熟悉两种方式我们都会使用，针对Float格式的，先使用命令行工具 <code class="docutils literal notranslate"><span class="pre">tflite_convert</span></code> ，其跟随TensorFlow一起安装（见 <a class="reference external" href="https://tf.wiki/zh/basic/installation.html#id1">一般安装步骤</a> ）。</p>
<p>在终端执行如下命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">tflite_convert</span> <span class="o">-</span><span class="n">h</span>
</pre></div>
</div>
<p>输出结果如下，即该命令的使用方法：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">usage</span><span class="p">:</span> <span class="n">tflite_convert</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="o">--</span><span class="n">output_file</span> <span class="n">OUTPUT_FILE</span>
<span class="p">(</span><span class="o">--</span><span class="n">saved_model_dir</span> <span class="n">SAVED_MODEL_DIR</span> <span class="o">|</span> <span class="o">--</span><span class="n">keras_model_file</span> <span class="n">KERAS_MODEL_FILE</span><span class="p">)</span>
<span class="o">--</span><span class="n">output_file</span> <span class="n">OUTPUT_FILE</span>
<span class="n">Full</span> <span class="n">filepath</span> <span class="n">of</span> <span class="n">the</span> <span class="n">output</span> <span class="n">file</span><span class="o">.</span>
<span class="o">--</span><span class="n">saved_model_dir</span> <span class="n">SAVED_MODEL_DIR</span>
<span class="n">Full</span> <span class="n">path</span> <span class="n">of</span> <span class="n">the</span> <span class="n">directory</span> <span class="n">containing</span> <span class="n">the</span> <span class="n">SavedModel</span><span class="o">.</span>
<span class="o">--</span><span class="n">keras_model_file</span> <span class="n">KERAS_MODEL_FILE</span>
<span class="n">Full</span> <span class="n">filepath</span> <span class="n">of</span> <span class="n">HDF5</span> <span class="n">file</span> <span class="n">containing</span> <span class="n">tf</span><span class="o">.</span><span class="n">Keras</span> <span class="n">model</span><span class="o">.</span>
</pre></div>
</div>
<p>在 <a class="reference external" href="https://tf.wiki/zh/deployment/export.html">TensorFlow模型导出</a> 中，我们知道TF2.0支持两种模型导出方法和格式SavedModel和Keras Sequential。</p>
<p>SavedModel导出模型转换：</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>tflite_convert --saved_model_dir<span class="o">=</span>saved/1 --output_file<span class="o">=</span>mnist_savedmodel.tflite
</pre></div>
</div>
<p>Keras Sequential导出模型转换：</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>tflite_convert --keras_model_file<span class="o">=</span>mnist_cnn.h5 --output_file<span class="o">=</span>mnist_sequential.tflite
</pre></div>
</div>
<p>到此，已经得到两个TensorFlow Lite模型。因为两者后续操作基本一致，我们只处理SavedModel格式的，Keras Sequential的转换可以按类似方法处理。</p>
</div>
<div class="section" id="android">
<h2>Android部署<a class="headerlink" href="#android" title="永久链接至标题">¶</a></h2>
<p>现在开始在Android环境部署，对于国内的读者，因为获取SDK和gradle编译环境等资源，需要先给Android Studio配置proxy或者使用国内的镜像。</p>
<p><strong>配置build.gradle</strong></p>
<p>将 <code class="docutils literal notranslate"><span class="pre">build.gradle</span></code> 中的maven源 <code class="docutils literal notranslate"><span class="pre">google()</span></code> 和 <code class="docutils literal notranslate"><span class="pre">jcenter()</span></code> 分别替换为国内镜像，如下：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">buildscript</span> <span class="p">{</span>
<span class="n">repositories</span> <span class="p">{</span>
<span class="n">maven</span> <span class="p">{</span> <span class="n">url</span> <span class="s1">'https://maven.aliyun.com/nexus/content/repositories/google'</span> <span class="p">}</span>
<span class="n">maven</span> <span class="p">{</span> <span class="n">url</span> <span class="s1">'https://maven.aliyun.com/nexus/content/repositories/jcenter'</span> <span class="p">}</span>
<span class="p">}</span>
<span class="n">dependencies</span> <span class="p">{</span>
<span class="n">classpath</span> <span class="s1">'com.android.tools.build:gradle:3.5.1'</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">allprojects</span> <span class="p">{</span>
<span class="n">repositories</span> <span class="p">{</span>
<span class="n">maven</span> <span class="p">{</span> <span class="n">url</span> <span class="s1">'https://maven.aliyun.com/nexus/content/repositories/google'</span> <span class="p">}</span>
<span class="n">maven</span> <span class="p">{</span> <span class="n">url</span> <span class="s1">'https://maven.aliyun.com/nexus/content/repositories/jcenter'</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p><strong>配置app/build.gradle</strong></p>
<p>新建一个Android Project，打开 <code class="docutils literal notranslate"><span class="pre">app/build.gradle</span></code> 添加如下信息：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>android {
aaptOptions {
noCompress "tflite" // 编译apk时，不压缩tflite文件
}
}
dependencies {
implementation 'org.tensorflow:tensorflow-lite:1.14.0'
}
</pre></div>
</div>
<p>其中，</p>
<ol class="arabic simple">
<li><p><code class="docutils literal notranslate"><span class="pre">aaptOptions</span></code> 设置tflite文件不压缩，确保后面tflite文件可以被Interpreter正确加载。</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">org.tensorflow:tensorflow-lite</span></code> 的最新版本号可以在这里查询 <a class="reference external" href="https://bintray.com/google/tensorflow/tensorflow-lite">https://bintray.com/google/tensorflow/tensorflow-lite</a></p></li>
</ol>
<p>设置好后，sync和build整个工程，如果build成功说明，配置成功。</p>
<p><strong>添加tflite文件到assets文件夹</strong></p>
<p>在app目录先新建assets目录，并将 <code class="docutils literal notranslate"><span class="pre">mnist_savedmodel.tflite</span></code> 文件保存到assets目录。重新编译apk，检查新编译出来的apk的assets文件夹是否有 <code class="docutils literal notranslate"><span class="pre">mnist_cnn.tflite</span></code> 文件。</p>
<p>点击菜单Build->Build APK(s)触发apk编译，apk编译成功点击右下角的EventLog。点击最后一条信息中的 <code class="docutils literal notranslate"><span class="pre">analyze</span></code> 链接，会触发apk analyzer查看新编译出来的apk，若在assets目录下存在 <code class="docutils literal notranslate"><span class="pre">mnist_savedmodel.tflite</span></code> ，则编译打包成功，如下：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">assets</span>
<span class="o">|</span><span class="n">__mnist_savedmodel</span><span class="o">.</span><span class="n">tflite</span>
</pre></div>
</div>
<p><strong>加载模型</strong></p>
<p>使用如下函数将 <code class="docutils literal notranslate"><span class="pre">mnist_savedmodel.tflite</span></code> 文件加载到memory-map中，作为Interpreter实例化的输入</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="cm">/** Memory-map the model file in Assets. */</span>
<span class="kd">private</span> <span class="n">MappedByteBuffer</span> <span class="nf">loadModelFile</span><span class="o">(</span><span class="n">Activity</span> <span class="n">activity</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span> <span class="o">{</span>
<span class="n">AssetFileDescriptor</span> <span class="n">fileDescriptor</span> <span class="o">=</span> <span class="n">activity</span><span class="o">.</span><span class="na">getAssets</span><span class="o">().</span><span class="na">openFd</span><span class="o">(</span><span class="n">mModelPath</span><span class="o">);</span>
<span class="n">FileInputStream</span> <span class="n">inputStream</span> <span class="o">=</span> <span class="k">new</span> <span class="n">FileInputStream</span><span class="o">(</span><span class="n">fileDescriptor</span><span class="o">.</span><span class="na">getFileDescriptor</span><span class="o">());</span>
<span class="n">FileChannel</span> <span class="n">fileChannel</span> <span class="o">=</span> <span class="n">inputStream</span><span class="o">.</span><span class="na">getChannel</span><span class="o">();</span>
<span class="kt">long</span> <span class="n">startOffset</span> <span class="o">=</span> <span class="n">fileDescriptor</span><span class="o">.</span><span class="na">getStartOffset</span><span class="o">();</span>
<span class="kt">long</span> <span class="n">declaredLength</span> <span class="o">=</span> <span class="n">fileDescriptor</span><span class="o">.</span><span class="na">getDeclaredLength</span><span class="o">();</span>
<span class="k">return</span> <span class="n">fileChannel</span><span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">FileChannel</span><span class="o">.</span><span class="na">MapMode</span><span class="o">.</span><span class="na">READ_ONLY</span><span class="o">,</span> <span class="n">startOffset</span><span class="o">,</span> <span class="n">declaredLength</span><span class="o">);</span>
<span class="o">}</span>
</pre></div>
</div>
<div class="admonition hint">
<p class="admonition-title">提示</p>
<p>memory-map可以把整个文件映射到虚拟内存中，用于提升tflite模型的读取性能。更多请参考： <a class="reference external" href="https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#map-java.nio.channels.FileChannel.MapMode-long-long-">JDK API介绍</a></p>
</div>
<p>实例化Interpreter，其中acitivity是为了从assets中获取模型，因为我们把模型编译到assets中，只能通过 <code class="docutils literal notranslate"><span class="pre">getAssets()</span></code> 打开。</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="n">mTFLite</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Interpreter</span><span class="o">(</span><span class="n">loadModelFile</span><span class="o">(</span><span class="n">activity</span><span class="o">));</span>
</pre></div>
</div>
<p>memory-map后的 <code class="docutils literal notranslate"><span class="pre">MappedByteBuffer</span></code> 直接作为 <code class="docutils literal notranslate"><span class="pre">Interpreter</span></code> 的输入， <code class="docutils literal notranslate"><span class="pre">mTFLite</span></code> （ <code class="docutils literal notranslate"><span class="pre">Interpreter</span></code> ）就是转换后模型的运行载体。</p>
<p><strong>运行输入</strong></p>
<p>我们使用MNIST test测试集中的图片作为输入，mnist图像大小28*28，单像素，因为我们输入的数据需要设置成如下格式</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="c1">//Float模型相关参数</span>
<span class="c1">// com/dpthinker/mnistclassifier/model/FloatSavedModelConfig.java</span>
<span class="kd">protected</span> <span class="kt">void</span> <span class="nf">setConfigs</span><span class="o">()</span> <span class="o">{</span>
<span class="n">setModelName</span><span class="o">(</span><span class="s">"mnist_savedmodel.tflite"</span><span class="o">);</span>
<span class="n">setNumBytesPerChannel</span><span class="o">(</span><span class="mi">4</span><span class="o">);</span>
<span class="n">setDimBatchSize</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="n">setDimPixelSize</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="n">setDimImgWeight</span><span class="o">(</span><span class="mi">28</span><span class="o">);</span>
<span class="n">setDimImgHeight</span><span class="o">(</span><span class="mi">28</span><span class="o">);</span>
<span class="n">setImageMean</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">setImageSTD</span><span class="o">(</span><span class="mf">255.0f</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// 初始化</span>
<span class="c1">// com/dpthinker/mnistclassifier/classifier/BaseClassifier.java</span>
<span class="kd">private</span> <span class="kt">void</span> <span class="nf">initConfig</span><span class="o">(</span><span class="n">BaseModelConfig</span> <span class="n">config</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">mModelConfig</span> <span class="o">=</span> <span class="n">config</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">mNumBytesPerChannel</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getNumBytesPerChannel</span><span class="o">();</span>
<span class="k">this</span><span class="o">.</span><span class="na">mDimBatchSize</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getDimBatchSize</span><span class="o">();</span>
<span class="k">this</span><span class="o">.</span><span class="na">mDimPixelSize</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getDimPixelSize</span><span class="o">();</span>
<span class="k">this</span><span class="o">.</span><span class="na">mDimImgWidth</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getDimImgWeight</span><span class="o">();</span>
<span class="k">this</span><span class="o">.</span><span class="na">mDimImgHeight</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getDimImgHeight</span><span class="o">();</span>
<span class="k">this</span><span class="o">.</span><span class="na">mModelPath</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getModelName</span><span class="o">();</span>
<span class="o">}</span>
</pre></div>
</div>
<p>将MNIST图片转化成 <code class="docutils literal notranslate"><span class="pre">ByteBuffer</span></code> ，并保持到 <code class="docutils literal notranslate"><span class="pre">imgData</span></code> （ <code class="docutils literal notranslate"><span class="pre">ByteBuffer</span></code> ）中</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="c1">// 将输入的Bitmap转化为Interpreter可以识别的ByteBuffer</span>
<span class="c1">// com/dpthinker/mnistclassifier/classifier/BaseClassifier.java</span>
<span class="kd">protected</span> <span class="n">ByteBuffer</span> <span class="nf">convertBitmapToByteBuffer</span><span class="o">(</span><span class="n">Bitmap</span> <span class="n">bitmap</span><span class="o">)</span> <span class="o">{</span>
<span class="kt">int</span><span class="o">[]</span> <span class="n">intValues</span> <span class="o">=</span> <span class="k">new</span> <span class="kt">int</span><span class="o">[</span><span class="n">mDimImgWidth</span> <span class="o">*</span> <span class="n">mDimImgHeight</span><span class="o">];</span>
<span class="n">scaleBitmap</span><span class="o">(</span><span class="n">bitmap</span><span class="o">).</span><span class="na">getPixels</span><span class="o">(</span><span class="n">intValues</span><span class="o">,</span>
<span class="mi">0</span><span class="o">,</span> <span class="n">bitmap</span><span class="o">.</span><span class="na">getWidth</span><span class="o">(),</span> <span class="mi">0</span><span class="o">,</span> <span class="mi">0</span><span class="o">,</span> <span class="n">bitmap</span><span class="o">.</span><span class="na">getWidth</span><span class="o">(),</span> <span class="n">bitmap</span><span class="o">.</span><span class="na">getHeight</span><span class="o">());</span>
<span class="n">ByteBuffer</span> <span class="n">imgData</span> <span class="o">=</span> <span class="n">ByteBuffer</span><span class="o">.</span><span class="na">allocateDirect</span><span class="o">(</span>
<span class="n">mNumBytesPerChannel</span> <span class="o">*</span> <span class="n">mDimBatchSize</span> <span class="o">*</span> <span class="n">mDimImgWidth</span> <span class="o">*</span> <span class="n">mDimImgHeight</span> <span class="o">*</span> <span class="n">mDimPixelSize</span><span class="o">);</span>
<span class="n">imgData</span><span class="o">.</span><span class="na">order</span><span class="o">(</span><span class="n">ByteOrder</span><span class="o">.</span><span class="na">nativeOrder</span><span class="o">());</span>
<span class="n">imgData</span><span class="o">.</span><span class="na">rewind</span><span class="o">();</span>
<span class="c1">// Convert the image toFloating point.</span>
<span class="kt">int</span> <span class="n">pixel</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">mDimImgWidth</span><span class="o">;</span> <span class="o">++</span><span class="n">i</span><span class="o">)</span> <span class="o">{</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">j</span> <span class="o"><</span> <span class="n">mDimImgHeight</span><span class="o">;</span> <span class="o">++</span><span class="n">j</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">//final int val = intValues[pixel++];</span>
<span class="kt">int</span> <span class="n">val</span> <span class="o">=</span> <span class="n">intValues</span><span class="o">[</span><span class="n">pixel</span><span class="o">++];</span>
<span class="n">mModelConfig</span><span class="o">.</span><span class="na">addImgValue</span><span class="o">(</span><span class="n">imgData</span><span class="o">,</span> <span class="n">val</span><span class="o">);</span> <span class="c1">//添加把Pixel数值转化并添加到ByteBuffer</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">imgData</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">// mModelConfig.addImgValue定义</span>
<span class="c1">// com/dpthinker/mnistclassifier/model/FloatSavedModelConfig.java</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">addImgValue</span><span class="o">(</span><span class="n">ByteBuffer</span> <span class="n">imgData</span><span class="o">,</span> <span class="kt">int</span> <span class="n">val</span><span class="o">)</span> <span class="o">{</span>
<span class="n">imgData</span><span class="o">.</span><span class="na">putFloat</span><span class="o">(((</span><span class="n">val</span> <span class="o">&</span> <span class="mh">0xFF</span><span class="o">)</span> <span class="o">-</span> <span class="n">getImageMean</span><span class="o">())</span> <span class="o">/</span> <span class="n">getImageSTD</span><span class="o">());</span>
<span class="o">}</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">convertBitmapToByteBuffer</span></code> 的输出即为模型运行的输入。</p>
<p><strong>运行输出</strong></p>
<p>定义一个1*10的多维数组，因为我们只有10个label，具体代码如下</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="n">privateFloat</span><span class="o">[][]</span> <span class="n">mLabelProbArray</span> <span class="o">=</span> <span class="n">newFloat</span><span class="o">[</span><span class="mi">1</span><span class="o">][</span><span class="mi">10</span><span class="o">];</span>
</pre></div>
</div>
<p>运行结束后，每个二级元素都是一个label的概率。</p>
<p><strong>运行及结果处理</strong></p>
<p>开始运行模型，具体代码如下</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="n">mTFLite</span><span class="o">.</span><span class="na">run</span><span class="o">(</span><span class="n">imgData</span><span class="o">,</span> <span class="n">mLabelProbArray</span><span class="o">);</span>
</pre></div>
</div>
<p>针对某个图片，运行后 <code class="docutils literal notranslate"><span class="pre">mLabelProbArray</span></code> 的内容就是各个label识别的概率。对他们进行排序，找出Top的label并界面呈现给用户.</p>
<p>在Android应用中，笔者使用了 <code class="docutils literal notranslate"><span class="pre">View.OnClickListener()</span></code> 触发 <code class="docutils literal notranslate"><span class="pre">"image/*"</span></code> 类型的 <code class="docutils literal notranslate"><span class="pre">Intent.ACTION_GET_CONTENT</span></code> ，进而获取设备上的图片（只支持MNIST标准图片）。然后，通过 <code class="docutils literal notranslate"><span class="pre">RadioButtion</span></code> 的选择情况，确认加载哪种转换后的模型，并触发真正分类操作。这部分比较简单，请读者自行阅读代码即可，不再展开介绍。</p>
<p>选取一张MNIST测试集中的图片进行测试，得到结果如下：</p>
<div class="figure align-center">
<a class="reference internal image-reference" href="../../_images/mnist_float.png"><img alt="../../_images/mnist_float.png" src="../../_images/mnist_float.png" style="width: 40%;" /></a>
</div>
<div class="admonition hint">
<p class="admonition-title">提示</p>
<p>注意我们这里直接用 <code class="docutils literal notranslate"><span class="pre">mLabelProbArray</span></code> 数值中的index作为label了，因为MNIST的label完全跟index从0到9匹配。如果是其他的分类问题，需要根据实际情况进行转换。</p>
</div>
</div>
<div class="section" id="quantization">
<h2>Quantization模型转换<a class="headerlink" href="#quantization" title="永久链接至标题">¶</a></h2>
<div class="admonition hint">
<p class="admonition-title">提示</p>
<p>Quantized模型是对原模型进行转换过程中，将float参数转化为uint8类型，进而产生的模型会更小、运行更快，但是精度会有所下降。</p>
</div>
<p>前面我们介绍了Float 模型的转换方法，接下来我们要展示下 Quantized 模型，在TF1.0上，可以使用命令行工具转换 Quantized模型。在笔者尝试的情况看在TF2.0上，命令行工具目前只能转换为Float 模型，Python API只能转换为 Quantized 模型。</p>
<p>Python API转换方法如下：</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>import tensorflow as tf
<span class="nv">converter</span> <span class="o">=</span> tf.lite.TFLiteConverter.from_saved_model<span class="o">(</span><span class="s1">'saved/1'</span><span class="o">)</span>
converter.optimizations <span class="o">=</span> <span class="o">[</span>tf.lite.Optimize.DEFAULT<span class="o">]</span>
<span class="nv">tflite_quant_model</span> <span class="o">=</span> converter.convert<span class="o">()</span>
open<span class="o">(</span><span class="s2">"mnist_savedmodel_quantized.tflite"</span>, <span class="s2">"wb"</span><span class="o">)</span>.write<span class="o">(</span>tflite_quant_model<span class="o">)</span>
</pre></div>
</div>
<p>最终转换后的 Quantized模型即为同级目录下的 <code class="docutils literal notranslate"><span class="pre">mnist_savedmodel_quantized.tflite</span></code> 。</p>
<p>相对TF1.0，上面的方法简化了很多，不需要考虑各种各样的参数，谷歌一直在优化开发者的使用体验。</p>
<p>在TF1.0上，我们可以使用 <code class="docutils literal notranslate"><span class="pre">tflite_convert</span></code> 获得模型具体结构，然后通过graphviz转换为pdf或png等方便查看。
在TF2.0上，提供了新的一步到位的工具 <code class="docutils literal notranslate"><span class="pre">visualize.py</span></code> ，直接转换为html文件，除了模型结构，还有更清晰的关键信息总结。</p>
<div class="admonition hint">
<p class="admonition-title">提示</p>
<p><code class="docutils literal notranslate"><span class="pre">visualize.py</span></code> 目前看应该还是开发阶段，使用前需要先从github下载最新的 <code class="docutils literal notranslate"><span class="pre">TensorFlow</span></code> 和 <code class="docutils literal notranslate"><span class="pre">FlatBuffers</span></code> 源码，并且两者要在同一目录，因为 <code class="docutils literal notranslate"><span class="pre">visualize.py</span></code> 源码中是按两者在同一目录写的调用路径。</p>
<p>下载 TensorFlow：</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>git clone git@github.com:tensorflow/tensorflow.git
</pre></div>
</div>
<p>下载 FlatBuffers：</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>git clone git@github.com:google/flatbuffers.git
</pre></div>
</div>
<p>编译 FlatBuffers：（笔者使用的Mac，其他平台请大家自行配置，应该不麻烦）</p>
<ol class="arabic simple">
<li><p>下载cmake：执行 <code class="docutils literal notranslate"><span class="pre">brew</span> <span class="pre">install</span> <span class="pre">cmake</span></code></p></li>
<li><p>设置编译环境：在 <code class="docutils literal notranslate"><span class="pre">FlatBuffers</span></code> 的根目录，执行 <code class="docutils literal notranslate"><span class="pre">cmake</span> <span class="pre">-G</span> <span class="pre">"Unix</span> <span class="pre">Makefiles"</span> <span class="pre">-DCMAKE_BUILD_TYPE=Release</span></code></p></li>
<li><p>编译：在 <code class="docutils literal notranslate"><span class="pre">FlatBuffers</span></code> 的根目录，执行 <code class="docutils literal notranslate"><span class="pre">make</span></code></p></li>
</ol>
<p>编译完成后，会在跟目录生成 <code class="docutils literal notranslate"><span class="pre">flatc</span></code>，这个可执行文件是 <code class="docutils literal notranslate"><span class="pre">visualize.py</span></code> 运行所依赖的。</p>
</div>
<p><strong>visualize.py使用方法</strong></p>
<p>在tensorflow/tensorflow/lite/tools目录下，执行如下命令</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python visualize.py mnist_savedmodel_quantized.tflite mnist_savedmodel_quantized.html
</pre></div>
</div>
<p>生成可视化报告的关键信息</p>
<div class="figure align-center">
<a class="reference internal image-reference" href="../../_images/visualize1.png"><img alt="../../_images/visualize1.png" src="../../_images/visualize1.png" style="width: 100%;" /></a>
</div>
<p>模型结构</p>
<div class="figure align-center">
<a class="reference internal image-reference" href="../../_images/visualize2.png"><img alt="../../_images/visualize2.png" src="../../_images/visualize2.png" style="width: 40%;" /></a>
</div>
<p>可见，Input/Output格式都是 <code class="docutils literal notranslate"><span class="pre">FLOAT32</span></code> 的多维数组，Input的min和max分别是0.0和255.0。</p>
<p>跟Float模型对比，Input/Output格式是一致的，所以可以复用Float模型Android部署过程中的配置。</p>
<div class="admonition hint">
<p class="admonition-title">提示</p>
<p>暂不确定这里是否是TF2.0上的优化，如果是这样的话，对开发者来说是非常友好的，如此就归一化了Float和Quantized模型处理了。</p>
</div>
<p>具体配置如下：</p>
<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="c1">// Quantized模型相关参数</span>
<span class="c1">// com/dpthinker/mnistclassifier/model/QuantSavedModelConfig.java</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">QuantSavedModelConfig</span> <span class="kd">extends</span> <span class="n">BaseModelConfig</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">protected</span> <span class="kt">void</span> <span class="nf">setConfigs</span><span class="o">()</span> <span class="o">{</span>
<span class="n">setModelName</span><span class="o">(</span><span class="s">"mnist_savedmodel_quantized.tflite"</span><span class="o">);</span>
<span class="n">setNumBytesPerChannel</span><span class="o">(</span><span class="mi">4</span><span class="o">);</span>
<span class="n">setDimBatchSize</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="n">setDimPixelSize</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="n">setDimImgWeight</span><span class="o">(</span><span class="mi">28</span><span class="o">);</span>
<span class="n">setDimImgHeight</span><span class="o">(</span><span class="mi">28</span><span class="o">);</span>
<span class="n">setImageMean</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">setImageSTD</span><span class="o">(</span><span class="mf">255.0f</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">addImgValue</span><span class="o">(</span><span class="n">ByteBuffer</span> <span class="n">imgData</span><span class="o">,</span> <span class="kt">int</span> <span class="n">val</span><span class="o">)</span> <span class="o">{</span>
<span class="n">imgData</span><span class="o">.</span><span class="na">putFloat</span><span class="o">(((</span><span class="n">val</span> <span class="o">&</span> <span class="mh">0xFF</span><span class="o">)</span> <span class="o">-</span> <span class="n">getImageMean</span><span class="o">())</span> <span class="o">/</span> <span class="n">getImageSTD</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></div>
</div>
<p>运行效果如下:</p>
<div class="figure align-center">
<a class="reference internal image-reference" href="../../_images/quantized.png"><img alt="../../_images/quantized.png" src="../../_images/quantized.png" style="width: 40%;" /></a>
</div>
<p>Float模型与 Quantized模型大小与性能对比：</p>
<p>可见， Quantized模型在模型大小和运行性能上相对Float模型都有非常大的提升。不过，在笔者试验的过程中，发现有些图片在Float模型上识别正确的，在 Quantized模型上会识别错，可见 <code class="docutils literal notranslate"><span class="pre">Quantization</span></code> 对模型的识别精度还是有影响的。在边缘设备上资源有限，需要在模型大小、运行速度与识别精度上找到一个权衡。</p>
</div>
<div class="section" id="id3">
<h2>总结<a class="headerlink" href="#id3" title="永久链接至标题">¶</a></h2>
<p>本节介绍了如何从零开始部署TFLite到Android应用中，包括：</p>
<ol class="arabic simple">
<li><p>如何将训练好的MNIST SavedModel模型，转换为Float模型和 Quantized模型</p></li>
<li><p>如何使用 <code class="docutils literal notranslate"><span class="pre">visualize.py</span></code> 和解读其结果信息</p></li>
<li><p>如何将转换后的模型部署到Android应用中</p></li>
</ol>
<p>笔者刚开始写这部分内容的时候还是TF1.0，在最近（2019年10月初）跟TF2.0的时候，发现有了很多变化，整体上是比原来更简单了。不过文档部分很多还是讲的比较模糊，很多地方还是需要看源码摸索。</p>
<div class="admonition hint">
<p class="admonition-title">提示</p>
<p>本节Android相关代码存放路径：
<code class="docutils literal notranslate"><span class="pre">https://github.com/snowkylin/tensorflow-handbook/tree/master/source/android</span></code></p>
</div>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="javascript.html" class="btn btn-neutral float-right" title="TensorFlow in JavaScript（Huan）" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="serving.html" class="btn btn-neutral float-left" title="TensorFlow Serving" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright 2018-2019, Xihan Li（雪麒）
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>