-
Notifications
You must be signed in to change notification settings - Fork 7
/
FAQ.html
695 lines (672 loc) · 62 KB
/
FAQ.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>FAQ — PyTables 3.9.2 documentation</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/graphviz.css" type="text/css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Other Material" href="other_material.html" />
<link rel="prev" title="Threading" href="cookbook/threading.html" />
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-111342564-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-111342564-1');
</script>
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="index.html">
<img src="_static/logo-pytables-small.png" class="logo" alt="Logo"/>
</a>
<div class="version">
3.9.2
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="usersguide/index.html">User’s Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="cookbook/index.html">Cookbook</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">FAQ</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#general-questions">General questions</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#what-is-pytables">What is PyTables?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#what-are-pytables-licensing-terms">What are PyTables’ licensing terms?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#i-m-having-problems-how-can-i-get-support">I’m having problems. How can I get support?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#why-hdf5">Why HDF5?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#why-python">Why Python?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#why-numpy">Why NumPy?</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#where-can-pytables-be-applied">Where can PyTables be applied?</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#is-pytables-safe">Is PyTables safe?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#can-pytables-be-used-in-concurrent-access-scenarios">Can PyTables be used in concurrent access scenarios?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#what-kind-of-containers-does-pytables-implement">What kind of containers does PyTables implement?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#cool-i-d-like-to-see-some-examples-of-use">Cool! I’d like to see some examples of use.</a></li>
<li class="toctree-l3"><a class="reference internal" href="#can-you-show-me-some-screenshots">Can you show me some screenshots?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#is-pytables-a-replacement-for-a-relational-database">Is PyTables a replacement for a relational database?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#how-can-pytables-be-fast-if-it-is-written-in-an-interpreted-language-like-python">How can PyTables be fast if it is written in an interpreted language like Python?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#if-it-is-designed-to-deal-with-very-large-datasets-then-pytables-should-consume-a-lot-of-memory-shouldn-t-it">If it is designed to deal with very large datasets, then PyTables should consume a lot of memory, shouldn’t it?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#why-was-pytables-born">Why was PyTables born?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#how-does-pytables-compare-with-the-h5py-project">How does PyTables compare with the h5py project?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#i-ve-found-a-bug-what-do-i-do">I’ve found a bug. What do I do?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#is-it-possible-to-get-involved-in-pytables-development">Is it possible to get involved in PyTables development?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#how-can-i-cite-pytables">How can I cite PyTables?</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#pytables-2-x-issues">PyTables 2.x issues</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#i-m-having-problems-migrating-my-apps-from-pytables-1-x-into-pytables-2-x-please-help">I’m having problems migrating my apps from PyTables 1.x into PyTables 2.x. Please, help!</a></li>
<li class="toctree-l3"><a class="reference internal" href="#for-combined-searches-like-table-where-x-5-x-3-why-was-a-operator-chosen-instead-of-an-and">For combined searches like <cite>table.where(‘(x<5) & (x>3)’)</cite>, why was a <cite>&</cite> operator chosen instead of an <cite>and</cite>?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#i-can-not-select-rows-using-in-kernel-queries-with-a-condition-that-involves-an-uint64col-why">I can not select rows using in-kernel queries with a condition that involves an UInt64Col. Why?</a></li>
<li class="toctree-l3"><a class="reference internal" href="#i-m-already-using-pytables-2-x-but-i-m-still-getting-numarray-objects-instead-of-numpy-ones">I’m already using PyTables 2.x but I’m still getting numarray objects instead of NumPy ones!</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#installation-issues">Installation issues</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#windows">Windows</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#error-when-importing-tables">Error when importing tables</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="other_material.html">Other Material</a></li>
<li class="toctree-l1"><a class="reference internal" href="MIGRATING_TO_3.x.html">Migrating from 2.x to 3.x</a></li>
<li class="toctree-l1"><a class="reference internal" href="downloads.html">Downloads</a></li>
<li class="toctree-l1"><a class="reference internal" href="release_notes.html">Release Notes</a></li>
<li class="toctree-l1"><a class="reference internal" href="project_pointers.html">Project pointers</a></li>
<li class="toctree-l1"><a class="reference internal" href="development.html">Development</a></li>
<li class="toctree-l1"><a class="reference internal" href="dev_team.html">Development Team</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">PyTables</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active">FAQ</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/FAQ.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="faq">
<h1>FAQ<a class="headerlink" href="#faq" title="Permalink to this heading">¶</a></h1>
<section id="general-questions">
<h2>General questions<a class="headerlink" href="#general-questions" title="Permalink to this heading">¶</a></h2>
<section id="what-is-pytables">
<h3>What is PyTables?<a class="headerlink" href="#what-is-pytables" title="Permalink to this heading">¶</a></h3>
<p>PyTables is a package for managing hierarchical datasets designed to
efficiently cope with extremely large amounts of data.</p>
<p>It is built on top of the <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id4" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> library, the <a class="reference external" href="http://www.python.org">Python language</a> <a class="footnote-reference brackets" href="#id19" id="id20" role="doc-noteref"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a> and the
<a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id22" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> package.
It features an object-oriented interface that, combined with C extensions
for the performance-critical parts of the code, makes it a fast yet
extremely easy-to-use tool for interactively storing and retrieving very
large amounts of data.</p>
</section>
<section id="what-are-pytables-licensing-terms">
<h3>What are PyTables’ licensing terms?<a class="headerlink" href="#what-are-pytables-licensing-terms" title="Permalink to this heading">¶</a></h3>
<p>PyTables is free for both commercial and non-commercial use, under the terms
of the <a class="reference external" href="http://opensource.org/licenses/BSD-3-Clause">BSD 3-Clause License</a>.</p>
</section>
<section id="i-m-having-problems-how-can-i-get-support">
<h3>I’m having problems. How can I get support?<a class="headerlink" href="#i-m-having-problems-how-can-i-get-support" title="Permalink to this heading">¶</a></h3>
<p>The most common and efficient way is to subscribe (remember you <em>need</em> to
subscribe prior to send messages) to the PyTables <a class="reference external" href="https://groups.google.com/group/pytables-users">users mailing list</a> <a class="footnote-reference brackets" href="#id32" id="id33" role="doc-noteref"><span class="fn-bracket">[</span>4<span class="fn-bracket">]</span></a>, and
send there a brief description of your issue and, if possible, a short script
that can reproduce it.
Hopefully, someone on the list will be able to help you.
It is also a good idea to check out the <a class="reference external" href="https://sourceforge.net/p/pytables/mailman/pytables-users/">archives of the user’s list</a> <a class="footnote-reference brackets" href="#id34" id="id35" role="doc-noteref"><span class="fn-bracket">[</span>5<span class="fn-bracket">]</span></a> (you may
want to check the <a class="reference external" href="http://www.mail-archive.com/pytables-users@lists.sourceforge.net/">Gmane archives</a> <a class="footnote-reference brackets" href="#id36" id="id37" role="doc-noteref"><span class="fn-bracket">[</span>6<span class="fn-bracket">]</span></a> instead) so as to see if the answer to your
question has already been dealt with.</p>
</section>
<section id="why-hdf5">
<h3>Why HDF5?<a class="headerlink" href="#why-hdf5" title="Permalink to this heading">¶</a></h3>
<p><a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id5" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> is the underlying C library and file format that enables PyTables to
efficiently deal with the data. It has been chosen for the following reasons:</p>
<ul class="simple">
<li><p>Designed to efficiently manage very large datasets.</p></li>
<li><p>Lets you organize datasets hierarchically.</p></li>
<li><p>Very flexible and well tested in scientific environments.</p></li>
<li><p>Good maintenance and improvement rate.</p></li>
<li><p>Technical excellence (<a class="reference external" href="http://www.hdfgroup.org/HDF5/RD100-2002/">R&D 100 Award</a> <a class="footnote-reference brackets" href="#id38" id="id39" role="doc-noteref"><span class="fn-bracket">[</span>7<span class="fn-bracket">]</span></a>).</p></li>
<li><p><strong>It’s Open Source software</strong></p></li>
</ul>
</section>
<section id="why-python">
<h3>Why Python?<a class="headerlink" href="#why-python" title="Permalink to this heading">¶</a></h3>
<ol class="arabic">
<li><p>Python is interactive.</p>
<p>People familiar with data processing understand how powerful command line
interfaces are for exploring mathematical relationships and scientific data
sets. Python provides an interactive environment with the added benefit of
a full featured programming language behind it.</p>
</li>
<li><p>Python is productive for beginners and experts alike.</p>
<p>PyTables is targeted at engineers, scientists, system analysts, financial
analysts, and others who consider programming a necessary evil. Any time
spent learning a language or tracking down bugs is time spent not solving
their real problem. Python has a short learning curve and most people can
do real and useful work with it in a day of learning. Its clean syntax and
interactive nature facilitate this.</p>
</li>
<li><p>Python is data-handling friendly.</p>
<p>Python comes with nice idioms that make the access to data much easier:
general slicing (i.e. <code class="docutils literal notranslate"><span class="pre">data[start:stop:step]</span></code>), list comprehensions,
iterators, generators … are constructs that make the interaction with your
data very easy.</p>
</li>
</ol>
</section>
<section id="why-numpy">
<h3>Why NumPy?<a class="headerlink" href="#why-numpy" title="Permalink to this heading">¶</a></h3>
<p><a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id23" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> is a Python package to efficiently deal with large datasets
<strong>in-memory</strong>, providing containers for homogeneous data, heterogeneous data,
and string arrays.
PyTables uses these NumPy containers as <em>in-memory buffers</em> to push the I/O
bandwith towards the platform limits.</p>
</section>
</section>
<section id="where-can-pytables-be-applied">
<h2>Where can PyTables be applied?<a class="headerlink" href="#where-can-pytables-be-applied" title="Permalink to this heading">¶</a></h2>
<p>In all the scenarios where one needs to deal with large datasets:</p>
<ul class="simple">
<li><p>Industrial applications</p>
<ul>
<li><p>Data acquisition in real time</p></li>
<li><p>Quality control</p></li>
<li><p>Fast data processing</p></li>
</ul>
</li>
<li><p>Scientific applications</p>
<ul>
<li><p>Meteorology, oceanography</p></li>
<li><p>Numerical simulations</p></li>
<li><p>Medicine (biological sensors, general data gathering & processing)</p></li>
</ul>
</li>
<li><p>Information systems</p>
<ul>
<li><p>System log monitoring & consolidation</p></li>
<li><p>Tracing of routing data</p></li>
<li><p>Alert systems in security</p></li>
</ul>
</li>
</ul>
<section id="is-pytables-safe">
<h3>Is PyTables safe?<a class="headerlink" href="#is-pytables-safe" title="Permalink to this heading">¶</a></h3>
<p>Well, first of all, let me state that PyTables does not support transactional
features yet (we don’t even know if we will ever be motivated to implement
this!), so there is always the risk that you can lose your data in case of an
unexpected event while writing (like a power outage, system shutdowns …).
Having said that, if your typical scenarios are <em>write once, read many</em>, then
the use of PyTables is perfectly safe, even for dealing extremely large amounts
of data.</p>
</section>
<section id="can-pytables-be-used-in-concurrent-access-scenarios">
<h3>Can PyTables be used in concurrent access scenarios?<a class="headerlink" href="#can-pytables-be-used-in-concurrent-access-scenarios" title="Permalink to this heading">¶</a></h3>
<p>It depends. Concurrent reads are no problem at all. However, whenever a process
(or thread) is trying to write, then problems will start to appear. First,
PyTables doesn’t support locking at any level, so several process writing
concurrently to the same PyTables file will probably end up corrupting it, so
don’t do this! Even having only one process writing and the others reading is
a hairy thing, because the reading processes might be reading incomplete data
from a concurrent data writing operation.</p>
<p>The solution would be to lock the file while writing and unlock it after a
flush over the file has been performed. Also, in order to avoid cache (<a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id6" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>,
PyTables) problems with read apps, you would need to re-open your files
whenever you are going to issue a read operation. If a re-opening operation is
unacceptable in terms of speed, you may want to do all your I/O operations in
one single process (or thread) and communicate the results via sockets,
<code class="xref py py-class docutils literal notranslate"><span class="pre">Queue.Queue</span></code> objects (in case of using threads), or whatever, with the
client process/thread.</p>
<p>The <cite>examples</cite> directory contains two scripts demonstrating methods of
accessing a PyTables file from multiple processes.</p>
<p>The first, <em>multiprocess_access_queues.py</em>, uses a
<code class="xref py py-class docutils literal notranslate"><span class="pre">multiprocessing.Queue</span></code> object to transfer read and write requests from
multiple <em>DataProcessor</em> processes to a single process responsible for all
access to the PyTables file. The results of read requests are then transferred
back to the originating processes using other <code class="xref py py-class docutils literal notranslate"><span class="pre">Queue</span></code> objects.</p>
<p>The second example script, <em>multiprocess_access_benchmarks.py</em>, demonstrates
and benchmarks four methods of transferring PyTables array data between
processes. The four methods are:</p>
<blockquote>
<div><ul class="simple">
<li><p>Using <code class="xref py py-class docutils literal notranslate"><span class="pre">multiprocessing.Pipe</span></code> from the Python standard library.</p></li>
<li><p>Using a memory mapped file that is shared between two processes. The NumPy
array associated with the file is passed as the <em>out</em> argument to the
<a class="reference internal" href="usersguide/libref/homogenous_storage.html#tables.Array.read" title="tables.Array.read"><code class="xref py py-meth docutils literal notranslate"><span class="pre">tables.Array.read()</span></code></a> method.</p></li>
<li><p>Using a Unix domain socket. Note that this example uses the ‘abstract
namespace’ and will only work under Linux.</p></li>
<li><p>Using an IPv4 socket.</p></li>
</ul>
</div></blockquote>
<p>See also the discussion in <a class="reference external" href="https://github.com/PyTables/PyTables/issues/790">gh-790</a>.</p>
</section>
<section id="what-kind-of-containers-does-pytables-implement">
<h3>What kind of containers does PyTables implement?<a class="headerlink" href="#what-kind-of-containers-does-pytables-implement" title="Permalink to this heading">¶</a></h3>
<p>PyTables does support a series of data containers that address specific needs
of the user. Below is a brief description of them:</p>
<dl class="simple">
<dt>:<a class="reference internal" href="usersguide/libref/structured_storage.html#tables.Table" title="tables.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a>:</dt><dd><p>Lets you deal with heterogeneous datasets. Allows compression. Enlargeable.
Supports nested types. Good performance for read/writing data.</p>
</dd>
<dt>:<a class="reference internal" href="usersguide/libref/homogenous_storage.html#tables.Array" title="tables.Array"><code class="xref py py-class docutils literal notranslate"><span class="pre">Array</span></code></a>:</dt><dd><p>Provides quick and dirty array handling. Not compression allowed.
Not enlargeable. Can be used only with relatively small datasets (i.e.
those that fit in memory). It provides the fastest I/O speed.</p>
</dd>
<dt>:<a class="reference internal" href="usersguide/libref/homogenous_storage.html#tables.CArray" title="tables.CArray"><code class="xref py py-class docutils literal notranslate"><span class="pre">CArray</span></code></a>:</dt><dd><p>Provides compressed array support. Not enlargeable. Good speed when
reading/writing.</p>
</dd>
<dt>:<a class="reference internal" href="usersguide/libref/homogenous_storage.html#tables.EArray" title="tables.EArray"><code class="xref py py-class docutils literal notranslate"><span class="pre">EArray</span></code></a>:</dt><dd><p>Most general array support. Compressible and enlargeable. It is pretty
fast at extending, and very good at reading.</p>
</dd>
<dt>:<a class="reference internal" href="usersguide/libref/homogenous_storage.html#tables.VLArray" title="tables.VLArray"><code class="xref py py-class docutils literal notranslate"><span class="pre">VLArray</span></code></a>:</dt><dd><p>Supports collections of homogeneous data with a variable number of entries.
Compressible and enlargeable. I/O is not very fast.</p>
</dd>
<dt>:<a class="reference internal" href="usersguide/libref/hierarchy_classes.html#tables.Group" title="tables.Group"><code class="xref py py-class docutils literal notranslate"><span class="pre">Group</span></code></a>:</dt><dd><p>The structural component.
A hierarchically-addressable container for HDF5 nodes (each of these
containers, including Group, are nodes), similar to a directory in a
UNIX filesystem.</p>
</dd>
</dl>
<p>Please refer to the <a class="reference internal" href="usersguide/libref.html"><span class="doc">Library Reference</span></a> for more specific information.</p>
</section>
<section id="cool-i-d-like-to-see-some-examples-of-use">
<h3>Cool! I’d like to see some examples of use.<a class="headerlink" href="#cool-i-d-like-to-see-some-examples-of-use" title="Permalink to this heading">¶</a></h3>
<p>Sure. Go to the HowToUse section to find simple examples that will help you
getting started.</p>
</section>
<section id="can-you-show-me-some-screenshots">
<h3>Can you show me some screenshots?<a class="headerlink" href="#can-you-show-me-some-screenshots" title="Permalink to this heading">¶</a></h3>
<p>Well, PyTables is not a graphical library by itself. However, you may want to
check out <a class="reference external" href="http://vitables.org">ViTables</a> <a class="footnote-reference brackets" href="#id40" id="id41" role="doc-noteref"><span class="fn-bracket">[</span>8<span class="fn-bracket">]</span></a>, a GUI tool to browse and edit PyTables & <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id7" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> files.</p>
</section>
<section id="is-pytables-a-replacement-for-a-relational-database">
<h3>Is PyTables a replacement for a relational database?<a class="headerlink" href="#is-pytables-a-replacement-for-a-relational-database" title="Permalink to this heading">¶</a></h3>
<p>No, by no means. PyTables lacks many features that are standard in most
relational databases. In particular, it does not have support for
relationships (beyond the hierarchical one, of course) between datasets and it
does not have transactional features. PyTables is more focused on speed and
dealing with really large datasets, than implementing the above features. In
that sense, PyTables can be best viewed as a <em>teammate</em> of a relational
database.</p>
<p>For example, if you have very large tables in your existing relational
database, they will take lots of space on disk, potentially reducing the
performance of the relational engine. In such a case, you can move those huge
tables out of your existing relational database to PyTables, and let your
relational engine do what it does best (i.e. manage relatively small or medium
datasets with potentially complex relationships), and use PyTables for what it
has been designed for (i.e. manage large amounts of data which are loosely
related).</p>
</section>
<section id="how-can-pytables-be-fast-if-it-is-written-in-an-interpreted-language-like-python">
<h3>How can PyTables be fast if it is written in an interpreted language like Python?<a class="headerlink" href="#how-can-pytables-be-fast-if-it-is-written-in-an-interpreted-language-like-python" title="Permalink to this heading">¶</a></h3>
<p>Actually, all of the critical I/O code in PyTables is a thin layer of code on
top of <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id8" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>, which is a very efficient C library. <a class="reference external" href="http://www.cython.org">Cython</a> <a class="footnote-reference brackets" href="#id42" id="id43" role="doc-noteref"><span class="fn-bracket">[</span>9<span class="fn-bracket">]</span></a> is used as the
<em>glue</em> language to generate “wrappers” around HDF5 calls so that they can be
used in Python. Also, the use of an efficient numerical package such as <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id24" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a>
makes the most costly operations effectively run at C speed. Finally,
time-critical loops are usually implemented in <a class="reference external" href="http://www.cython.org">Cython</a> <a class="footnote-reference brackets" href="#id42" id="id44" role="doc-noteref"><span class="fn-bracket">[</span>9<span class="fn-bracket">]</span></a> (which, if used
properly, allows to generate code that runs at almost pure C speeds).</p>
</section>
<section id="if-it-is-designed-to-deal-with-very-large-datasets-then-pytables-should-consume-a-lot-of-memory-shouldn-t-it">
<h3>If it is designed to deal with very large datasets, then PyTables should consume a lot of memory, shouldn’t it?<a class="headerlink" href="#if-it-is-designed-to-deal-with-very-large-datasets-then-pytables-should-consume-a-lot-of-memory-shouldn-t-it" title="Permalink to this heading">¶</a></h3>
<p>Well, you already know that PyTables sits on top of HDF5, Python and <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id25" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a>,
and if we add its own logic (~7500 lines of code in Python, ~3000 in Cython and
~4000 in C), then we should conclude that PyTables isn’t effectively a paradigm
of lightness.</p>
<p>Having said that, PyTables (as <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id9" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> itself) tries very hard to optimize the
memory consumption by implementing a series of features like dynamic
determination of buffer sizes, <em>Least Recently Used</em> cache for keeping unused
nodes out of memory, and extensive use of compact <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id26" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> data containers.
Moreover, PyTables is in a relatively mature state and most memory leaks have
been already addressed and fixed.</p>
<p>Just to give you an idea of what you can expect, a PyTables program can deal
with a table with around 30 columns and 1 million entries using as low as 13 MB
of memory (on a 32-bit platform). All in all, it is not that much, is it?.</p>
</section>
<section id="why-was-pytables-born">
<h3>Why was PyTables born?<a class="headerlink" href="#why-was-pytables-born" title="Permalink to this heading">¶</a></h3>
<p>Because, back in August 2002, one of its authors (<a class="reference external" href="https://github.com/FrancescAlted">Francesc Alted</a> <a class="footnote-reference brackets" href="#id45" id="id46" role="doc-noteref"><span class="fn-bracket">[</span>10<span class="fn-bracket">]</span></a>) had a need
to save lots of hierarchical data in an efficient way for later post-processing
it. After trying out several approaches, he found that they presented distinct
inconveniences. For example, working with file sizes larger than, say, 100 MB,
was rather painful with ZODB (it took lots of memory with the version available
by that time).</p>
<p>The <a class="reference external" href="http://www.unidata.ucar.edu/software/netcdf">netCDF3</a> <a class="footnote-reference brackets" href="#id47" id="id48" role="doc-noteref"><span class="fn-bracket">[</span>11<span class="fn-bracket">]</span></a> interface provided by <a class="reference external" href="http://dirac.cnrs-orleans.fr/ScientificPython.html">Scientific Python</a> <a class="footnote-reference brackets" href="#id51" id="id52" role="doc-noteref"><span class="fn-bracket">[</span>12<span class="fn-bracket">]</span></a> was great, but it did
not allow to structure the hierarchically; besides, <a class="reference external" href="http://www.unidata.ucar.edu/software/netcdf">netCDF3</a> <a class="footnote-reference brackets" href="#id47" id="id49" role="doc-noteref"><span class="fn-bracket">[</span>11<span class="fn-bracket">]</span></a> only supports
homogeneous datasets, not heterogeneous ones (i.e. tables). (As an aside,
<a class="reference external" href="http://www.unidata.ucar.edu/software/netcdf">netCDF4</a> <a class="footnote-reference brackets" href="#id47" id="id53" role="doc-noteref"><span class="fn-bracket">[</span>11<span class="fn-bracket">]</span></a> overcomes many of the limitations of <a class="reference external" href="http://www.unidata.ucar.edu/software/netcdf">netCDF3</a> <a class="footnote-reference brackets" href="#id47" id="id50" role="doc-noteref"><span class="fn-bracket">[</span>11<span class="fn-bracket">]</span></a>, although curiously
enough, it is based on top of <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id10" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>, the library chosen as the base for
PyTables from the very beginning.)</p>
<p>So, he decided to give <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id11" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> a try, start doing his own wrappings to it and
voilà, this is how the first public release of PyTables (0.1) saw the light in
October 2002, three months after his itch started to eat him ;-).</p>
</section>
<section id="how-does-pytables-compare-with-the-h5py-project">
<h3>How does PyTables compare with the h5py project?<a class="headerlink" href="#how-does-pytables-compare-with-the-h5py-project" title="Permalink to this heading">¶</a></h3>
<p>Well, they are similar in that both packages are Python interfaces to the <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id12" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>
library, but there are some important differences to be noted. <a class="reference external" href="http://www.h5py.org">h5py</a> <a class="footnote-reference brackets" href="#id54" id="id55" role="doc-noteref"><span class="fn-bracket">[</span>13<span class="fn-bracket">]</span></a> is an
attempt to map the <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id13" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> feature set to <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id27" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> as closely as possible. In
addition, it also provides access to nearly all of the <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id14" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> C API.</p>
<p>Instead, PyTables builds up an additional abstraction layer on top of <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id15" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> and
<a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id28" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> where it implements things like an enhanced type system, an <a class="reference internal" href="usersguide/optimization.html#searchoptim"><span class="std std-ref">engine
for enabling complex queries</span></a>, an efficient computational
kernel, advanced indexing capabilities or an undo/redo feature, to name
just a few. This additional layer also allows PyTables to be relatively
independent of its underlying libraries (and their possible limitations). For
example, PyTables can support <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id16" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> data types like <cite>enumerated</cite> or <cite>time</cite> that
are available in the <a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id17" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> library but not in the <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id29" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> package; or even
perform powerful complex queries that are not implemented directly in neither
<a class="reference external" href="http://www.hdfgroup.org/HDF5">HDF5</a> <a class="footnote-reference brackets" href="#id3" id="id18" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> nor <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id30" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a>.</p>
<p>Furthermore, PyTables also tries hard to be a high performance interface to
HDF5/NumPy, implementing niceties like internal LRU caches for nodes and other
data and metadata, <a class="reference internal" href="usersguide/optimization.html#chunksizefinetune"><span class="std std-ref">automatic computation of optimal chunk sizes</span></a> for the datasets, a variety of compressors, ranging from
slow but efficient (<a class="reference external" href="http://www.bzip.org">bzip2</a> <a class="footnote-reference brackets" href="#id56" id="id57" role="doc-noteref"><span class="fn-bracket">[</span>14<span class="fn-bracket">]</span></a>) to extremely fast ones (<a class="reference external" href="https://www.blosc.org">Blosc</a> <a class="footnote-reference brackets" href="#id58" id="id59" role="doc-noteref"><span class="fn-bracket">[</span>15<span class="fn-bracket">]</span></a>) in addition to the
standard <a class="reference external" href="http://zlib.net">zlib</a> <a class="footnote-reference brackets" href="#id60" id="id61" role="doc-noteref"><span class="fn-bracket">[</span>16<span class="fn-bracket">]</span></a>. Another difference is that PyTables makes use of <a class="reference external" href="https://github.com/pydata/numexpr">numexpr</a> <a class="footnote-reference brackets" href="#id62" id="id63" role="doc-noteref"><span class="fn-bracket">[</span>17<span class="fn-bracket">]</span></a> so
as to accelerate internal computations (for example, in evaluating complex
queries) to a maximum.</p>
<p>For contrasting with other opinions, you may want to check the PyTables/h5py
comparison in a similar entry of the <a class="reference external" href="http://docs.h5py.org/en/latest/faq.html#what-s-the-difference-between-h5py-and-pytables">FAQ of h5py</a> <a class="footnote-reference brackets" href="#id66" id="id67" role="doc-noteref"><span class="fn-bracket">[</span>18<span class="fn-bracket">]</span></a>.</p>
</section>
<section id="i-ve-found-a-bug-what-do-i-do">
<h3>I’ve found a bug. What do I do?<a class="headerlink" href="#i-ve-found-a-bug-what-do-i-do" title="Permalink to this heading">¶</a></h3>
<p>The PyTables development team works hard to make this eventuality as rare as
possible, but, as in any software made by human beings, bugs do occur. If you
find any bug, please tell us by file a bug report in the <a class="reference external" href="https://github.com/PyTables/PyTables/issues">issue tracker</a> <a class="footnote-reference brackets" href="#id68" id="id69" role="doc-noteref"><span class="fn-bracket">[</span>19<span class="fn-bracket">]</span></a> on
<a class="reference external" href="https://github.com">GitHub</a> <a class="footnote-reference brackets" href="#id70" id="id71" role="doc-noteref"><span class="fn-bracket">[</span>20<span class="fn-bracket">]</span></a>.</p>
</section>
<section id="is-it-possible-to-get-involved-in-pytables-development">
<h3>Is it possible to get involved in PyTables development?<a class="headerlink" href="#is-it-possible-to-get-involved-in-pytables-development" title="Permalink to this heading">¶</a></h3>
<p>Indeed. We are keen for more people to help out contributing code, unit tests,
documentation, and helping out maintaining this wiki. Drop us a mail on the
<cite>users mailing list</cite> and tell us in which area do you want to work.</p>
</section>
<section id="how-can-i-cite-pytables">
<h3>How can I cite PyTables?<a class="headerlink" href="#how-can-i-cite-pytables" title="Permalink to this heading">¶</a></h3>
<p>The recommended way to cite PyTables in a paper or a presentation is as
following:</p>
<ul class="simple">
<li><p>Author: Francesc Alted, Ivan Vilata and others</p></li>
<li><p>Title: PyTables: Hierarchical Datasets in Python</p></li>
<li><p>Year: 2002 -</p></li>
<li><p>URL: <a class="reference external" href="http://www.pytables.org">http://www.pytables.org</a></p></li>
</ul>
<p>Here’s an example of a BibTeX entry:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@Misc</span><span class="p">{,</span>
<span class="n">author</span> <span class="o">=</span> <span class="p">{</span><span class="n">PyTables</span> <span class="n">Developers</span> <span class="n">Team</span><span class="p">},</span>
<span class="n">title</span> <span class="o">=</span> <span class="p">{{</span><span class="n">PyTables</span><span class="p">}:</span> <span class="n">Hierarchical</span> <span class="n">Datasets</span> <span class="ow">in</span> <span class="p">{</span><span class="n">Python</span><span class="p">}},</span>
<span class="n">year</span> <span class="o">=</span> <span class="p">{</span><span class="mi">2002</span><span class="o">--</span><span class="p">},</span>
<span class="n">url</span> <span class="o">=</span> <span class="s2">"http://www.pytables.org/"</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
</section>
<section id="pytables-2-x-issues">
<h2>PyTables 2.x issues<a class="headerlink" href="#pytables-2-x-issues" title="Permalink to this heading">¶</a></h2>
<section id="i-m-having-problems-migrating-my-apps-from-pytables-1-x-into-pytables-2-x-please-help">
<h3>I’m having problems migrating my apps from PyTables 1.x into PyTables 2.x. Please, help!<a class="headerlink" href="#i-m-having-problems-migrating-my-apps-from-pytables-1-x-into-pytables-2-x-please-help" title="Permalink to this heading">¶</a></h3>
<p>Sure. However, you should first check out the <a class="reference internal" href="MIGRATING_TO_2.x.html"><span class="doc">Migrating from PyTables 1.x to 2.x</span></a>
document.
It should provide hints to the most frequently asked questions on this regard.</p>
</section>
<section id="for-combined-searches-like-table-where-x-5-x-3-why-was-a-operator-chosen-instead-of-an-and">
<h3>For combined searches like <cite>table.where(‘(x<5) & (x>3)’)</cite>, why was a <cite>&</cite> operator chosen instead of an <cite>and</cite>?<a class="headerlink" href="#for-combined-searches-like-table-where-x-5-x-3-why-was-a-operator-chosen-instead-of-an-and" title="Permalink to this heading">¶</a></h3>
<p>Search expressions are in fact Python expressions written as strings, and they
are evaluated as such. This has the advantage of not having to learn a new
syntax, but it also implies some limitations with logical <cite>and</cite> and <cite>or</cite>
operators, namely that they can not be overloaded in Python. Thus, it is
impossible right now to get an element-wise operation out of an expression like
<cite>‘array1 and array2’</cite>. That’s why one has to choose some other operator, being
<cite>&</cite> and <cite>|</cite> the most similar to their C counterparts <cite>&&</cite> and <cite>||</cite>, which
aren’t available in Python either.</p>
<p>You should be careful about expressions like <cite>‘x<5 & x>3’</cite> and others like <cite>‘3
< x < 5’</cite> which ‘’won’t work as expected’’, because of the different operator
precedence and the absence of an overloaded logical <cite>and</cite> operator. More on
this in the appendix about condition syntax in the <a class="reference external" href="https://portal.hdfgroup.org/display/HDF5/Datatypes">HDF5 manual</a> <a class="footnote-reference brackets" href="#id72" id="id73" role="doc-noteref"><span class="fn-bracket">[</span>21<span class="fn-bracket">]</span></a>.</p>
<p>There are quite a few packages affected by those limitations including <a class="reference external" href="http://www.numpy.org">NumPy</a> <a class="footnote-reference brackets" href="#id21" id="id31" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a>
themselves and <a class="reference external" href="http://sqlobject.org">SQLObject</a> <a class="footnote-reference brackets" href="#id74" id="id75" role="doc-noteref"><span class="fn-bracket">[</span>22<span class="fn-bracket">]</span></a>, and there have been quite longish discussions about
adding the possibility of overloading logical operators to Python (see <a class="reference external" href="http://www.python.org/dev/peps/pep-0335">PEP
335</a> <a class="footnote-reference brackets" href="#id76" id="id77" role="doc-noteref"><span class="fn-bracket">[</span>23<span class="fn-bracket">]</span></a> and <a class="reference external" href="https://mail.python.org/pipermail/python-dev/2004-September/048763.html">this thread</a> <a class="footnote-reference brackets" href="#id78" id="id79" role="doc-noteref"><span class="fn-bracket">[</span>24<span class="fn-bracket">]</span></a> for more details).</p>
</section>
<section id="i-can-not-select-rows-using-in-kernel-queries-with-a-condition-that-involves-an-uint64col-why">
<h3>I can not select rows using in-kernel queries with a condition that involves an UInt64Col. Why?<a class="headerlink" href="#i-can-not-select-rows-using-in-kernel-queries-with-a-condition-that-involves-an-uint64col-why" title="Permalink to this heading">¶</a></h3>
<p>This turns out to be a limitation of the <a class="reference external" href="https://github.com/pydata/numexpr">numexpr</a> <a class="footnote-reference brackets" href="#id62" id="id64" role="doc-noteref"><span class="fn-bracket">[</span>17<span class="fn-bracket">]</span></a> package. Internally,
<a class="reference external" href="https://github.com/pydata/numexpr">numexpr</a> <a class="footnote-reference brackets" href="#id62" id="id65" role="doc-noteref"><span class="fn-bracket">[</span>17<span class="fn-bracket">]</span></a> uses a limited set of types for doing calculations, and unsigned
integers are always upcasted to the immediate signed integer that can fit the
information. The problem here is that there is not a (standard) signed integer
that can be used to keep the information of a 64-bit unsigned integer.</p>
<p>So, your best bet right now is to avoid <cite>uint64</cite> types if you can. If you
absolutely need <cite>uint64</cite>, the only way for doing selections with this is
through regular Python selections. For example, if your table has a <cite>colM</cite>
column which is declared as an <cite>UInt64Col</cite>, then you can still filter its
values with:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">row</span><span class="p">[</span><span class="s1">'colN'</span><span class="p">]</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span> <span class="k">if</span> <span class="n">row</span><span class="p">[</span><span class="s1">'colM'</span><span class="p">]</span> <span class="o"><</span> <span class="n">X</span><span class="p">]</span>
</pre></div>
</div>
<p>However, this approach will generally lead to slow speed (specially on Win32
platforms, where the values will be converted to Python <cite>long</cite> values).</p>
</section>
<section id="i-m-already-using-pytables-2-x-but-i-m-still-getting-numarray-objects-instead-of-numpy-ones">
<h3>I’m already using PyTables 2.x but I’m still getting numarray objects instead of NumPy ones!<a class="headerlink" href="#i-m-already-using-pytables-2-x-but-i-m-still-getting-numarray-objects-instead-of-numpy-ones" title="Permalink to this heading">¶</a></h3>
<p>This is most probably due to the fact that you are using a file created with
PyTables 1.x series. By default, PyTables 1.x was setting an HDF5 attribute
<cite>FLAVOR</cite> with the value <cite>‘numarray’</cite> to all leaves. Now, PyTables 2.x sees
this attribute and obediently converts the internal object (truly a NumPy
object) into a <cite>numarray</cite> one. For PyTables 2.x files the <cite>FLAVOR</cite> attribute
will only be saved when explicitly set via the <cite>leaf.flavor</cite> property (or when
passing data to an <a class="reference internal" href="usersguide/libref/homogenous_storage.html#tables.Array" title="tables.Array"><code class="xref py py-class docutils literal notranslate"><span class="pre">Array</span></code></a> or <a class="reference internal" href="usersguide/libref/structured_storage.html#tables.Table" title="tables.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> at creation time), so you
will be able to distinguish default flavors from user-set ones by checking the
existence of the <cite>FLAVOR</cite> attribute.</p>
<p>Meanwhile, if you don’t want to receive <cite>numarray</cite> objects when reading old
files, you have several possibilities:</p>
<ul>
<li><p>Remove the flavor for your datasets by hand:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">for</span> <span class="n">leaf</span> <span class="ow">in</span> <span class="n">h5file</span><span class="o">.</span><span class="n">walkNodes</span><span class="p">(</span><span class="n">classname</span><span class="o">=</span><span class="s1">'Leaf'</span><span class="p">):</span>
<span class="k">del</span> <span class="n">leaf</span><span class="o">.</span><span class="n">flavor</span>
</pre></div>
</div>
</li>
<li><p>Use the :program:’ptrepack` utility with the flag <cite>–upgrade-flavors</cite>
so as to convert all flavors in old files to the default (effectively by
removing the <cite>FLAVOR</cite> attribute).</p></li>
<li><p>Remove the <cite>numarray</cite> (and/or <cite>Numeric</cite>) package from your system.
Then PyTables 2.x will return you pure NumPy objects (it can’t be
otherwise!).</p></li>
</ul>
</section>
</section>
<section id="installation-issues">
<h2>Installation issues<a class="headerlink" href="#installation-issues" title="Permalink to this heading">¶</a></h2>
<section id="windows">
<h3>Windows<a class="headerlink" href="#windows" title="Permalink to this heading">¶</a></h3>
<section id="error-when-importing-tables">
<h4>Error when importing tables<a class="headerlink" href="#error-when-importing-tables" title="Permalink to this heading">¶</a></h4>
<p>You have installed the binary installer for Windows and, when importing the
<em>tables</em> package you are getting an error like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">The</span> <span class="n">command</span> <span class="ow">in</span> <span class="s2">"0x6714a822"</span> <span class="n">refers</span> <span class="n">to</span> <span class="n">memory</span> <span class="ow">in</span> <span class="s2">"0x012011a0"</span><span class="o">.</span> <span class="n">The</span>
<span class="n">procedure</span> <span class="s2">"written"</span> <span class="n">could</span> <span class="ow">not</span> <span class="n">be</span> <span class="n">executed</span><span class="o">.</span>
<span class="n">Click</span> <span class="n">to</span> <span class="n">ok</span> <span class="n">to</span> <span class="n">terminate</span><span class="o">.</span>
<span class="n">Click</span> <span class="n">to</span> <span class="n">abort</span> <span class="n">to</span> <span class="n">debug</span> <span class="n">the</span> <span class="n">program</span><span class="o">.</span>
</pre></div>
</div>
<p>This problem can be due to a series of reasons, but the most probable one is
that you have a version of a DLL library that is needed by PyTables and it is
not at the correct version. Please, double-check the versions of the required
libraries for PyTables and install newer versions, if needed. In most cases,
this solves the issue.</p>
<p>In case you continue getting problems, there are situations where other
programs do install libraries in the PATH that are <strong>optional</strong> to PyTables
(for example BZIP2 or LZO), but that they will be used if they are found in
your system (i.e. anywhere in your <span class="target" id="index-0"></span><code class="xref std std-envvar docutils literal notranslate"><span class="pre">PATH</span></code>). So, if you find any of
these libraries in your PATH, upgrade it to the latest version available (you
don’t need to re-install PyTables).</p>
<hr class="docutils" />
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="id3" role="note">
<span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
<span class="backrefs">(<a role="doc-backlink" href="#id4">1</a>,<a role="doc-backlink" href="#id5">2</a>,<a role="doc-backlink" href="#id6">3</a>,<a role="doc-backlink" href="#id7">4</a>,<a role="doc-backlink" href="#id8">5</a>,<a role="doc-backlink" href="#id9">6</a>,<a role="doc-backlink" href="#id10">7</a>,<a role="doc-backlink" href="#id11">8</a>,<a role="doc-backlink" href="#id12">9</a>,<a role="doc-backlink" href="#id13">10</a>,<a role="doc-backlink" href="#id14">11</a>,<a role="doc-backlink" href="#id15">12</a>,<a role="doc-backlink" href="#id16">13</a>,<a role="doc-backlink" href="#id17">14</a>,<a role="doc-backlink" href="#id18">15</a>)</span>
<p><a class="reference external" href="http://www.hdfgroup.org/HDF5">http://www.hdfgroup.org/HDF5</a></p>
</aside>
<aside class="footnote brackets" id="id19" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id20">2</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://www.python.org">http://www.python.org</a></p>
</aside>
<aside class="footnote brackets" id="id21" role="note">
<span class="label"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></span>
<span class="backrefs">(<a role="doc-backlink" href="#id22">1</a>,<a role="doc-backlink" href="#id23">2</a>,<a role="doc-backlink" href="#id24">3</a>,<a role="doc-backlink" href="#id25">4</a>,<a role="doc-backlink" href="#id26">5</a>,<a role="doc-backlink" href="#id27">6</a>,<a role="doc-backlink" href="#id28">7</a>,<a role="doc-backlink" href="#id29">8</a>,<a role="doc-backlink" href="#id30">9</a>,<a role="doc-backlink" href="#id31">10</a>)</span>
<p><a class="reference external" href="http://www.numpy.org">http://www.numpy.org</a></p>
</aside>
<aside class="footnote brackets" id="id32" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id33">4</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://groups.google.com/group/pytables-users">https://groups.google.com/group/pytables-users</a></p>
</aside>
<aside class="footnote brackets" id="id34" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id35">5</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://sourceforge.net/p/pytables/mailman/pytables-users/">https://sourceforge.net/p/pytables/mailman/pytables-users/</a></p>
</aside>
<aside class="footnote brackets" id="id36" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id37">6</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://www.mail-archive.com/pytables-users@lists.sourceforge.net/">http://www.mail-archive.com/pytables-users@lists.sourceforge.net/</a></p>
</aside>
<aside class="footnote brackets" id="id38" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id39">7</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://www.hdfgroup.org/HDF5/RD100-2002/">http://www.hdfgroup.org/HDF5/RD100-2002/</a></p>
</aside>
<aside class="footnote brackets" id="id40" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id41">8</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://vitables.org">http://vitables.org</a></p>
</aside>
<aside class="footnote brackets" id="id42" role="note">
<span class="label"><span class="fn-bracket">[</span>9<span class="fn-bracket">]</span></span>
<span class="backrefs">(<a role="doc-backlink" href="#id43">1</a>,<a role="doc-backlink" href="#id44">2</a>)</span>
<p><a class="reference external" href="http://www.cython.org">http://www.cython.org</a></p>
</aside>
<aside class="footnote brackets" id="id45" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id46">10</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://github.com/FrancescAlted">https://github.com/FrancescAlted</a></p>
</aside>
<aside class="footnote brackets" id="id47" role="note">
<span class="label"><span class="fn-bracket">[</span>11<span class="fn-bracket">]</span></span>
<span class="backrefs">(<a role="doc-backlink" href="#id48">1</a>,<a role="doc-backlink" href="#id49">2</a>,<a role="doc-backlink" href="#id50">3</a>,<a role="doc-backlink" href="#id53">4</a>)</span>
<p><a class="reference external" href="http://www.unidata.ucar.edu/software/netcdf">http://www.unidata.ucar.edu/software/netcdf</a></p>
</aside>
<aside class="footnote brackets" id="id51" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id52">12</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://dirac.cnrs-orleans.fr/ScientificPython.html">http://dirac.cnrs-orleans.fr/ScientificPython.html</a></p>
</aside>
<aside class="footnote brackets" id="id54" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id55">13</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://www.h5py.org">http://www.h5py.org</a></p>
</aside>
<aside class="footnote brackets" id="id56" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id57">14</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://www.bzip.org">http://www.bzip.org</a></p>
</aside>
<aside class="footnote brackets" id="id58" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id59">15</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://www.blosc.org">https://www.blosc.org</a></p>
</aside>
<aside class="footnote brackets" id="id60" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id61">16</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://zlib.net">http://zlib.net</a></p>
</aside>
<aside class="footnote brackets" id="id62" role="note">
<span class="label"><span class="fn-bracket">[</span>17<span class="fn-bracket">]</span></span>
<span class="backrefs">(<a role="doc-backlink" href="#id63">1</a>,<a role="doc-backlink" href="#id64">2</a>,<a role="doc-backlink" href="#id65">3</a>)</span>
<p><a class="reference external" href="https://github.com/pydata/numexpr">https://github.com/pydata/numexpr</a></p>
</aside>
<aside class="footnote brackets" id="id66" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id67">18</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://docs.h5py.org/en/latest/faq.html#what-s-the-difference-between-h5py-and-pytables">http://docs.h5py.org/en/latest/faq.html#what-s-the-difference-between-h5py-and-pytables</a></p>
</aside>
<aside class="footnote brackets" id="id68" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id69">19</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://github.com/PyTables/PyTables/issues">https://github.com/PyTables/PyTables/issues</a></p>
</aside>
<aside class="footnote brackets" id="id70" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id71">20</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://github.com">https://github.com</a></p>
</aside>
<aside class="footnote brackets" id="id72" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id73">21</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://portal.hdfgroup.org/display/HDF5/Datatypes">https://portal.hdfgroup.org/display/HDF5/Datatypes</a></p>
</aside>
<aside class="footnote brackets" id="id74" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id75">22</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://sqlobject.org">http://sqlobject.org</a></p>
</aside>
<aside class="footnote brackets" id="id76" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id77">23</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="http://www.python.org/dev/peps/pep-0335">http://www.python.org/dev/peps/pep-0335</a></p>
</aside>
<aside class="footnote brackets" id="id78" role="note">
<span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id79">24</a><span class="fn-bracket">]</span></span>
<p><a class="reference external" href="https://mail.python.org/pipermail/python-dev/2004-September/048763.html">https://mail.python.org/pipermail/python-dev/2004-September/048763.html</a></p>
</aside>
</aside>
</section>
</section>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="cookbook/threading.html" class="btn btn-neutral float-left" title="Threading" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="other_material.html" class="btn btn-neutral float-right" title="Other Material" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2011–2023, PyTables maintainers.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>