org-table-jb-extras.el — Extra commands & functions for working with org-tables
- Description
- Extra commands & functions for working with org-tables
- Compatibility
- GNU Emacs 25.2.2
- Package-Requires
- ((org “9.4.6”) (cl-lib “1”) (ido-choose-function “0.1”) (dash “2.19.1”) (s “1.13.1”))
org-mode cl-lib ido-choose-function ampl-mode dash s
Bitcoin donations gratefully accepted: 16xSxS5v7rpknr4WL8unwCRab5nZEfc7eK
This library contains many extra functions & commands for dealing with org-tables which are documented below. Most of the functions useful to users can be accessed through a single command: org-table-dispatch.
When org-table-dispatch is called while point is in an org-table an ido menu of possible actions from org-table-dispatch-actions is offered to the user. These actions include: “copy table”, “copy rectangle”, “kill/clear cells”, “export to file”, “copy cols to calc”, “plot graph”, “fit curve to cols”, “transpose table”, “split/join columns”, “join rows/flatten columns”, “toggle display of row/column refs”, “Hide/show column”, “Narrow column”, “Narrow table”, “Fill empty cells”, “Insert vertical line”, etc.
Sometimes I try to export an org file to latex/pdf, and I find that the table no longer fits the page with the desired font size. In order to make the table narrower without losing content new rows need to be created to hold spillover text from cells above. You can create the rows and split the cells using org-table-wrap-region however this is tedious and error prone.
The problem of choosing which cells should be split, and where new rows should be added can be formulated as a non-linear optimization problem, and written as an AMPL program: table_widths.mod The command org-table-narrow uses this program to narrow the org-table at point to the desired width (without splitting words). To run it you need a copy of AMPL (A Mathematical Programming Language) and a solver, which can be downloaded for free from here. You also need my fork of ampl-mode.el which includes the new functions run-ampl and run-ampl-async.
Alternatively if you don’t want to depend on external binaries you can use the org-table-narrow-column command to narrow a single column of the table by introducing new rows.
If you want more fine-grained control you can use org-table-flatten-columns to select exactly which cells should be combined, and how they should be combined (see org-table-flatten-functions). When used in combination with org-table-jump-next (using flatten, see below) and keyboard macros this can be very convenient.
The org-table-jump-next & org-table-jump-prev commands provide a method for quickly & easily navigating, altering or restructuring tables cell-by-cell. When org-table-jump-next is invoked, it looks at the content of org-table-jump-condition (which can be viewed with M-x org-table-show-jump-condition) to decide where to move the cursor.
A jump condition is an sexp that uses a special syntax/DSL and functions, and is evaluated within the context of a table cell. If it evaluates to non-nil then the cursor is moved to that cell.
There are many predefined jump conditions in org-table-jump-condition-presets such as :empty (jump to the next empty cell), —/cell (jump to the next cell under a horizontal line), first<->last (toggle between first/last cell), etc. These presets can be selected, altered, or you can define you own by either pressing C-u followed by M-x org-table-jump-next/org-table-jump-prev or pressing M-x org-table-set-jump-condition.
By default the table is searched row-by-row from left to right and top to bottom to find the next position to jump to. You can select the direction of search by using a double C-u prefix when calling org-table-jump-next/org-table-jump-prev or by calling org-table-set-jump-direction. The choices are; ‘up (col-by-col upwards, right to left), ‘down (col-by-col downwards, left to right), ‘left (row-by-row leftwards, bottom to top), or ‘right (row-by-row rightwards, top to bottom).
The DSL allows the use of logical operators ! (NOT), & (AND), | (OR) for combining conditions, and a sequence operator -> for creating sequences of different jump conditions that are traversed on successive applications of org-table-jump-next (see below). It can also make use of special functions defined in org-table-filter-function-bindings, e.g. hline-p (check for horizontal lines), matchfield (match a field against a regexp), countcells (count neighbouring cells matching regexps), and also functions to alter the table, e.g. setfield (change a field), replace-in-field (replace substring of field) convertdate (change the format of a date string in field), flatten (combine cells), etc.
Example of jump condition:
("foo" & ! ("bar" 1) | (> (sumcounts 'down 0 0 "\\S-" "^\\s-*$") 5))This will jump to the next cell that contains “foo” which is NOT above a cell containing “bar”, OR the next cell with more than 4 empty cells beneath it, whichever comes first.
A detailed description of the DSL & special functions is given in the docstring for org-table-jump-condition which is displayed in a help window when entering a jump condition manually.
Normally a numeric prefix of N to org-table-jump-next or org-table-jump-prev has the effect of repeating the jump N times. However you may also assign some prefixes to different jump conditions, allowing you to quickly swap between different jump types.
To do this, define the jump condition as a list starting with a digit (usually 0), and containing digits for the other numeric prefix keys, between 1 & 9, that you want to assign to. Items coming after a digit, say N, in that list, and before any subsequent digit or the end of the list, will be used as the jump condition for any numeric prefix that ends with the digit N. The other digits of the prefix will be used as the repetition count, so e.g. a prefix of C-123 will jump 12 times using the jump condition assigned to 3. The digit 0 is used to define the jump condition when no prefix key is used.
Example of jump conditions assigned to prefix keys:
(0 :empty 1 :nonempty :hline-below 2 :empty -> :nonempty :hline-above)This will jump to the next empty cell when no prefix is used, the next non-empty cell above a horizontal line when a prefix ending in 1 is used, and when a prefix ending in 2 is used it toggles between empty cells and non-empty cells below horizontal lines (see “jump sequences” below for more info about the last one).
The history of manually entered jump conditions is stored in org-table-jump-condition-history which can be accessed by pressing the up & down arrow keys at the prompt after doing C-u M-x org-table-jump-next or M-x org-table-set-jump-condition followed by “enter manually”. If you use the session library you can make sure this history persists between emacs sessions by adding it to session-globals-include.
You can also add jump conditions to org-table-jump-condition-presets and give them descriptions which will be shown in the prompt for selecting a jump condition.
Alternatively you can store table specific jump conditions at the bottom of a table on a line that starts with #+TBLJMP:. This line should come directly after the table and any #+TBLFM: lines, and may be either the condition alone, or a cons cell whose car is a direction (see below) and whose cdr is a condition.
The following jump condition example removes a horizontal line from the table each time org-table-jump-next is executed.
(:hline-below (removeline))If you want to remove all hlines in one step, you need to alter the jump condition slightly so that it always returns nil, and hence continues searching after removing an hline:
(:hline-below (and (removeline) nil))When applied to the following table:
| a | b |
|---|---|
| 1 | 2 |
You should get this:
| a | b |
| 1 | 2 |
The following jump condition will flatten cells to the right of empty ones, and then put a horizontal line under them.
("\\S-" & ("\\S-" 0 -1) & ("^\\s-*$" 1 -1) & (flatten (1+ (sumcounts 'down 1 -1 "^\\s-*$")) 1) & (addhline))When applied to the following table:
| foo | bar |
| choo | |
| zoo | |
| aaa | bbb |
You should get a table that looks like this one:
| foo | bar choo zoo |
|---|---|
| aaa | bbb |
The following jump condition will add 1 to cells to the right of dates with year > 2022, and blank cells to the right of dates with year <= 2022. Note that (year (getdate)) will return nil for cells containing no date.
((> (or (year (getdate)) 0) 2022) (changenumber '1+ 0 1) | (getdate) (setfield "" 0 1))When applied to this table:
| Date | Amount |
| 18/06/2024 | 1.23 |
| 04/10/2023 | 4.45 |
| 12/09/2022 | 3.01 |
| 20/11/2021 | 11.2 |
You should get this:
| Date | Amount |
| 18/06/2024 | 2.23 |
| 04/10/2023 | 5.45 |
| 12/09/2022 | |
| 20/11/2021 |
You can see that once defined, jump conditions allow you to perform complex table manipulations easily. When used in combination with keyboard macros even more is possible.
Jump sequences allow different jump conditions to be used in sequence. They can be nested, and nested sequences move one step forward for each complete iteration of the parent sequence. For example:
- (A -> B -> C) results in A,B,C,A,B,C,etc.
- (A -> (B -> C) -> D) results in A,B,D,A,C,D,A,B,D,etc.
- (A -> (B -> (C -> D)) -> E) results in A,B,E,A,C,E,A,B,E,A,D,E,A,B,E,etc.
Jump sequences also keep a track of the history of visited cells so that if you move forward through a sequence using org-table-jump-next and then immediately backwards using org-table-jump-prev you will visit the exact same cells in reverse (which might not happen if we simply reversed the direction and order of jump conditions). However if you move to a different cell in between org-table-jump-next & org-table-jump-prev the history will be deleted.
Jump conditions may pass information from one jump to the next using the org-table-jump-state alist which allows you to perform more complex tasks.
This is done either using the getvar,setvar & checkvar functions, or using the following syntax:
- (KEY == VAL1 VAL2 …) : The value associated with KEY in org-table-jump-state will be compared against VAL1, VAL2, etc. and if one of them matches, then non-nil will be returned.
- (KEY := VAL) : The value associated with KEY in org-table-jump-state will be set to VAL.
In the following example we jump to cells containing “foo” until we reach one above an empty cell, at which point we jump to cells containing “bar”, and then back to “foo” cells again. This is achieved using a state variable x, which is initialized to 1 at the start, checked before each jump condition, and changed when appropriate.
((x == nil 3) (x := 1) "foo" | (x == 1) "foo" (:empty-below (x := 2) | t) | (x == 2) "bar" (x := 3))| foo | bar |
| foo | zzz |
| bar | |
| foo | bar |
| foo | zzz |
The file org-table-solve-mazelog.org contains more advanced examples.
To debug a jump conditions place a “(debug nil)” form somewhere within it.
org-dblock-write:tablefilter is a dynamic block function which can be used for filtering the rows of a table into another one according to various criteria.
To use it create add a tablefilter block like the following where you want the subtable to be inserted. Replace <NAME> with the name of the table you want to filter, and <FILTER> with an sexp that evaluates to non-nil for rows of the original table that are to be inserted in the block.
#+BEGIN: tablefilter :tblname “<NAME>” :filter <FILTER>
#+END:
The <FILTER> may use special variables, “c1”, “c2”, “row”, etc. containing the contents of the current row, and also special function defined in org-table-filter-function-bindings. For more info see the docstring for org-dblock-write:tablefilter.
For more info about dynamic blocks see here: https://orgmode.org/manual/Dynamic-Blocks.html
Below is a complete list of commands:
- org-table-insert-or-delete-vline Insert a vertical line in the current column, or delete some if NDELETE is non-nil.
- org-table-grab-columns Copy/kill columns or region of table and return as list(s).
- org-table-flatten-columns :
Apply FN to next NROWS cells in selected columns and replace cells in current row with results.
- org-table-dispatch :
Do something with column(s) of org-table at point.
- insert-file-as-org-table :
Insert a file into the current buffer at point, and convert it to an org table.
- org-table-kill-field :
Kill the org-table field under point.
- org-table-copy-field :
Copy the org-table field under point to the kill ring.
- org-table-narrow-column
Split the current column of an org-mode table to be WIDTH characters wide.
- org-table-narrow
Narrow the entire org-mode table, apart from FIXEDCOLS, to be within WIDTH characters by adding new rows.
- org-table-fill-empty-cells
Fill empty cells in current column of org-table at point by splitting non-empty cells above them.
- org-table-wrap-table Insert string BEFORE table at point, and another string AFTER.
- org-table-query-dimension
Print and return the number of columns, data lines, cells, hlines, height & width (in chars) of org-table at point.
- org-table-move-cell
Prompt for a direction and move the current cell in that direction.
- org-table-show-jump-condition
Display a message in the minibuffer showing the current jump condition.
- org-table-set-jump-condition
Set the CONDITION for org-table-jump-condition.
- org-table-set-jump-direction
Set the DIRECTION for org-table-jump-condition; ‘up, ‘down, ‘left or ‘right.
- org-table-jump-next
Jump to the STEPS next field in the org-table at point matching org-table-jump-condition.
- org-table-jump-prev
Like org-table-jump-next but jump STEPS in opposite direction.
Below is a list of customizable options:
- org-table-flatten-functions :
Alist of (NAME . FUNCTION) pairs for use with - org-table-flatten-column :.
- org-table-graph-types :
List of graph types for - org-plot/gnuplot :.
- org-table-dispatch-actions :
Actions that can be applied when - org-table-dispatch : is called.
- org-table-filter-function-bindings :
Function bindings (with descriptions) used by org-table-jump-condition & org-dblock-write:tablefilter.
- org-table-jump-condition-presets
Named presets for org-table-jump-condition.
- org-table-timestamp-patterns
List of java style date-time matching patterns as accepted by datetime-matching-regexp and related functions.
- org-table-timestamp-format
Default format for timestamps output by org-table-convert-timestamp.
- org-table-wrap-presets
Preset BEFORE and AFTER strings/functions for ‘org-table-wrap-table’.