From c0097024b0f7d3a35d8b926693737fcb0b04830d Mon Sep 17 00:00:00 2001 From: Matthew Maynes Date: Thu, 13 Jun 2019 17:03:41 +0000 Subject: [PATCH 1/2] Update training for 4.4.0 --- readme.md => README.md | 36 ++-- .../file/filterReadings.quke | 5 + kxscm/module/A.Tutorial.Data/qfn/gen.pulse | 8 + kxscm/module/A.Tutorial.Data/qfn/gen.pulses | 9 + kxscm/module/A.Tutorial.Data/qfn/gen.wave | 4 - kxscm/module/A.Tutorial.Data/qfn/sim.machine | 6 + kxscm/module/A.Tutorial.Data/qfn/sim.sensor | 5 + kxscm/module/A.Tutorial.Data/qfn/sim.system | 7 + .../module/A.Tutorial.Doc/file/walkthrough.md | 6 +- .../file/Advanced/A-Settings.md | 102 +++++------ .../file/Advanced/B-Geometries.md | 12 +- .../file/Advanced/C-StatisticalTransforms.md | 44 ++--- .../file/Advanced/D-Composing.md | 45 +++-- .../file/Advanced/E-Faceting.md | 6 +- .../file/Advanced/F-Themes.md | 19 +- .../file/Basic/A-Introduction.md | 50 +++--- .../file/Basic/B-Interaction.md | 20 +-- .../file/Basic/C-Geometries.md | 52 +++--- .../file/Basic/D-Aesthetics.md | 78 ++++---- .../file/Basic/E-Scales.md | 69 ++++--- .../file/Basic/F-Themes.md | 55 +++--- .../file/Basic/G-Composing.md | 170 ++++++++---------- .../file/DeveloperProductivity.q | 5 +- .../module/B.Section.IDE/file/FindingHelp.md | 15 +- kxscm/module/B.Section.IDE/file/Modules.md | 6 +- .../file/TransformerFunctions.md | 93 +++++++--- .../file/VIFunctions.md | 21 +-- 27 files changed, 489 insertions(+), 459 deletions(-) rename readme.md => README.md (57%) delete mode 100644 kxscm/module/A.Tutorial.Data/qfn/gen.wave diff --git a/readme.md b/README.md similarity index 57% rename from readme.md rename to README.md index 89bf512..e2e49e8 100644 --- a/readme.md +++ b/README.md @@ -18,9 +18,8 @@ Getting this repository This repository can be cloned directly into Kx Analyst or Kx Developer. Right-click the workspace area on the left of the page and select `Git > Clone...`. In the dialog, enter the URL for the training repository `https://github.com/kxsystems/analyst-training.git`. -Pressing `Ok` will open the `Pull Repository` dialog with the option to select a name -and branch for the repository. Press `Ok` to finish cloning the repository. - +Pressing `OK` will open the `Pull Repository` dialog with the option to select a name +and branch for the repository. Press `OK` to finish cloning the repository. Organization ------------ @@ -43,40 +42,43 @@ Resources For more resources, please refer to the following links. -### Analyst +### Kx Analyst -- [Home page](https://kx.com/solutions/the-enterprise/analyst/) -- [User guide](https://code.kx.com/analyst/) +- [Kx Analyst home](https://kx.com/solutions/the-enterprise/analyst/) - Full featured interactive development environment for q and kdb+ +- [Kx Analyst user guide](https://code.kx.com/analyst/) - Product documentation and example use cases #### Visual Data Transformer and Query -- (*blog*) [Data Transformer](https://kx.com/blog/kx-product-insights-modern-data-preparation-etl-in-analyst-for-kx/) -- (*video*) [Data Transformer](https://vimeo.com/183895691) -- (*video*) [Visual Query in Transformer](https://vimeo.com/184708019) +- (*blog*) [Kx Product Insights: Modern Data Preparation (ETL) in Kx Analyst](https://kx.com/blog/kx-product-insights-modern-data-preparation-etl-in-analyst-for-kx/) +- (*video*) [Data Transformer overview](https://vimeo.com/183895691) +- (*video*) [Filtering data without programming using the Transformer](https://vimeo.com/184708019) #### Visualization -- (*blog*) [Visualization for Exploratory Data Analysis](https://kx.com/blog/kx-product-insights-visualization-for-exploratory-data-analysis-eda/) -- (*video*) [Visual Inspector](https://vimeo.com/183886181) +- (*blog*) [Visualization for exploratory data analysis](https://kx.com/blog/kx-product-insights-visualization-for-exploratory-data-analysis-eda/) +- (*video*) [Visual Inspector overview](https://vimeo.com/183886181) - (*video*) [Custom Graphics with q Visualization Library](https://vimeo.com/212133060) #### Spreadsheet -- (*video*) [Spreadsheet](https://vimeo.com/183891867) +- (*video*) [Spreadsheet overview](https://vimeo.com/183891867) -### Developer +### Kx Developer -- [User guide](https://code.kx.com/developer/) +- [Kx Developer user guide](https://code.kx.com/developer/) - Product documentation and example use cases #### IDE -- (*blog*) [IDE Overview](https://kx.com/blog/kx-product-insights-analyst-for-kx-ide/) -- (*video*) [IDE Overview](https://vimeo.com/239703330) +- (*blog*) [Kx Product Insights: IDE Overview](https://kx.com/blog/kx-product-insights-analyst-for-kx-ide/) +- (*video*) [IDE overview](https://vimeo.com/239703330) #### Testing -- (*blog*) [Testing with qCumber and QuickCheq](https://kx.com/blog/kx-product-insights-testing-using-analyst-for-kx/) +- (*blog*) [Kx Product Insights: Testing with qCumber and QuickCheq](https://kx.com/blog/kx-product-insights-testing-using-analyst-for-kx/) - (*video*) [Testing with qCumber and QuickCheq](https://vimeo.com/221903630) - (*video*) [Behavioural Driven Development](https://vimeo.com/183916767) +### Demos + +- (*video*) [Kx Developer and Kx Analyst: Live demo of a data driven project using taxi data](https://www.youtube.com/watch?v=o3Sg_RHnUdw) diff --git a/kxscm/module/A.Tutorial.Analysis.test/file/filterReadings.quke b/kxscm/module/A.Tutorial.Analysis.test/file/filterReadings.quke index e183639..334e651 100644 --- a/kxscm/module/A.Tutorial.Analysis.test/file/filterReadings.quke +++ b/kxscm/module/A.Tutorial.Analysis.test/file/filterReadings.quke @@ -5,6 +5,11 @@ feature filterReadings 2 ~ count filterReadings[5] ([] x : til 10) property should filter readings of any table + // Apply a quickcheq propery of 2 inputs (using `forall2`) where + // the first input is an integer from 0-5 and the second input is + // a table of anything. This will run the test property (the function) + // 100 times with random combinations of inputs to try and prove that + // the property holds or does not. .qch.check .qch.forall2[.qch.g.int 5; .qch.g.table[]]{ if[x = 0; : .qch.discard]; (ceiling count[y] % x) ~ count filterReadings[x; y] diff --git a/kxscm/module/A.Tutorial.Data/qfn/gen.pulse b/kxscm/module/A.Tutorial.Data/qfn/gen.pulse index 5bbc206..0e22b30 100644 --- a/kxscm/module/A.Tutorial.Data/qfn/gen.pulse +++ b/kxscm/module/A.Tutorial.Data/qfn/gen.pulse @@ -1,3 +1,11 @@ +// @private +// @fileOverview +// Generate a single signal pulse for a sensor +// @param scale {timespan} time scale of pulse +// @param range {tuple (float; float)} boundaries of pulse +// @param pulse {float} size of pulse +// @param width {timespan} time width of pulse +// @return {table} generated sensor reading with a pulse {[scale; range; pulse; width] time : first[range] + ("j"$scale) * til "j"$(last[range] - first range) div "j"$scale; signal : ([] time: time; signal: 4 mavg count[time]?0.1); diff --git a/kxscm/module/A.Tutorial.Data/qfn/gen.pulses b/kxscm/module/A.Tutorial.Data/qfn/gen.pulses index 32741bb..b2e7be5 100644 --- a/kxscm/module/A.Tutorial.Data/qfn/gen.pulses +++ b/kxscm/module/A.Tutorial.Data/qfn/gen.pulses @@ -1,3 +1,12 @@ +// @private +// @fileOverview +// Generate multiple pulses for a sensor +// @param scale {timespan} time scale of pulse +// @param pulse {float} size of pulse +// @param pulseWidth {timespan} time width of pulse +// @param repeatWidth {timespan} time between pulses +// @param n {long} number of readings +// @return {table} generated sensor reading with a pulses {[scale; pulse; pulseWidth; repeatWidth; n] : raze gen.pulse[scale;;pulse;pulseWidth] each x ,' (-1 _ next x:repeatWidth * til n),n * repeatWidth } diff --git a/kxscm/module/A.Tutorial.Data/qfn/gen.wave b/kxscm/module/A.Tutorial.Data/qfn/gen.wave deleted file mode 100644 index 3ad11db..0000000 --- a/kxscm/module/A.Tutorial.Data/qfn/gen.wave +++ /dev/null @@ -1,4 +0,0 @@ -{[width; noise; n] - fuzz: 4 mavg (n?noise) - n?noise % 2; - ([] time: width * til n; signal:fuzz + sin (width % 0D00:00:01.000) * til n) - } diff --git a/kxscm/module/A.Tutorial.Data/qfn/sim.machine b/kxscm/module/A.Tutorial.Data/qfn/sim.machine index 15462a4..7233f6c 100644 --- a/kxscm/module/A.Tutorial.Data/qfn/sim.machine +++ b/kxscm/module/A.Tutorial.Data/qfn/sim.machine @@ -1,3 +1,9 @@ +// @private +// @fileOverview +// Simulates the readings for a single machine +// @param label {symbol} label to apply to machine +// @param n {long} number of readings to generate +// @return {table} {[label; n] sensors: raze {`$(string[x],"_"),/:.Q.a} each `temp`pressure`weight; : raze {[n; sensor] diff --git a/kxscm/module/A.Tutorial.Data/qfn/sim.sensor b/kxscm/module/A.Tutorial.Data/qfn/sim.sensor index f085f21..46e7528 100644 --- a/kxscm/module/A.Tutorial.Data/qfn/sim.sensor +++ b/kxscm/module/A.Tutorial.Data/qfn/sim.sensor @@ -1,3 +1,8 @@ +// @private +// @fileOverview +// Generate a number of signals for a given sensor +// @param n {long} the number of pulses +// @return {float[]} generated sensor readings {[n] : gen.pulses[0D00:00:00.02; 0.75; rand 0D00:00:40.0; 0D00:01:00.0; n]; } diff --git a/kxscm/module/A.Tutorial.Data/qfn/sim.system b/kxscm/module/A.Tutorial.Data/qfn/sim.system index d1dbb11..349fc44 100644 --- a/kxscm/module/A.Tutorial.Data/qfn/sim.system +++ b/kxscm/module/A.Tutorial.Data/qfn/sim.system @@ -1,3 +1,10 @@ +// @fileOverview +// Runs a simulation of a sensor based system with some different machines +// and sensors. The values for these sensors are randomly generated values +// between 0 and 1. The simulation takes a few seconds and will produce +// approximately 10-15 million records. +// +// @return {table} simulated sensors data {[] labels : raze {`$(string[x],"_"),/:rand[26]#.Q.a} each `feed`mach`pkg; readings : raze {sim.machine[x; 1 + rand 3]} each labels; diff --git a/kxscm/module/A.Tutorial.Doc/file/walkthrough.md b/kxscm/module/A.Tutorial.Doc/file/walkthrough.md index 511b57d..4cadb76 100644 --- a/kxscm/module/A.Tutorial.Doc/file/walkthrough.md +++ b/kxscm/module/A.Tutorial.Doc/file/walkthrough.md @@ -247,9 +247,9 @@ a closer look at this signal. We can use quick plotting library that included wi // Select only sensor that is failing. Replace the query keys with the ones from // the tooltip in the visual inspector. // -// Replace the `mach_g` value with the machine name and the `pressure_a` with the +// Replace the `mach_a` value with the machine name and the `pressure_a` with the // sensor name from the tooltip in the Visual Inspector -failing: select from sensors where machine=`mach_g, sensor=`pressure_a; +failing: select from sensors where machine=`mach_a, sensor=`pressure_a; .qp.go[800; 500] plotSignal failing ``` @@ -271,7 +271,7 @@ reduce the noise in the plot. ```q smooth: smoothReadings[10] firstId; -.qp.go[800;500] plotSignal smooth; +.qp.go[800;500] plotSignal smooth ``` Let's take a look at one last facet of the data. We can use the Grammar of Graphics to diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/A-Settings.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/A-Settings.md index 2ca278e..c096f21 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/A-Settings.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/A-Settings.md @@ -6,66 +6,66 @@ yet been discussed are introduced here. ```q - // Grouping can split categories into separate subplots - - .qp.go[700;400] .qp.area[subset; `time; `signal; ::]; - - - .qp.go[700;400] .qp.area[subset; `time; `signal] - .qp.s.aes[`group`fill; `sensor`sensor] - , .qp.s.scale[`fill; .gg.scale.colour.cat20]; - - // When grouping, many geometries provide the ability to accommodate the groups by - // stacking or dodging (bars, area, etc) +// Grouping can split categories into separate subplots + +.qp.go[700;400] .qp.area[subset; `time; `signal; ::]; + + +.qp.go[700;400] .qp.area[subset; `time; `signal] + .qp.s.aes[`group`fill; `sensor`sensor] + , .qp.s.scale[`fill; .gg.scale.colour.cat20]; - .qp.go[700;400] .qp.area[subset; `time; `signal] - .qp.s.aes[`group`fill; `sensor`sensor] - , .qp.s.scale[`fill; .gg.scale.colour.cat20] - , .qp.s.geom[enlist[`position]!enlist `stack]; +// When grouping, many geometries provide the ability to accommodate the groups by +// stacking or dodging (bars, area, etc) - // Any frame can be converted to polar coordinates to produce new plots - - .qp.go[700;400] .qp.point[subset; `signal; `time; ] - .qp.s.aes[`group`fill; `sensor`sensor] - , .qp.s.scale[`fill; .gg.scale.colour.cat20]; +.qp.go[700;400] .qp.area[subset; `time; `signal] + .qp.s.aes[`group`fill; `sensor`sensor] + , .qp.s.scale[`fill; .gg.scale.colour.cat20] + , .qp.s.geom[enlist[`position]!enlist `stack]; + +// Any frame can be converted to polar coordinates to produce new plots + +.qp.go[700;400] .qp.point[subset; `signal; `time; ] + .qp.s.aes[`group`fill; `sensor`sensor] + , .qp.s.scale[`fill; .gg.scale.colour.cat20]; + +// ... by just choosing polar coordinates + +.qp.go[700;400] .qp.point[subset; `signal; `time; ] + .qp.s.aes[`group`fill; `sensor`sensor] + , .qp.s.scale[`fill; .gg.scale.colour.cat20] + , .qp.s.coord[.gg.coords.polar]; - // ... by just choosing polar coordinates +// ... when doing this, keeping the frame aspect ratio as a square is a good idea - .qp.go[700;400] .qp.point[subset; `signal; `time; ] +.qp.go[700;400] + .qp.theme[enlist[`aspect_ratio]!enlist `square] + .qp.point[subset; `signal; `time; ] .qp.s.aes[`group`fill; `sensor`sensor] , .qp.s.scale[`fill; .gg.scale.colour.cat20] , .qp.s.coord[.gg.coords.polar]; - // ... when doing this, keeping the frame aspect ratio as a square is a good idea +// With a bit of cleverness, it is possible to create standard charts such as pie charts - .qp.go[700;400] - .qp.theme[enlist[`aspect_ratio]!enlist `square] - .qp.point[subset; `signal; `time; ] - .qp.s.aes[`group`fill; `sensor`sensor] - , .qp.s.scale[`fill; .gg.scale.colour.cat20] - , .qp.s.coord[.gg.coords.polar]; - - // With a bit of cleverness, it is possible to create standard charts such as pie charts - - .qp.go[700;400] - .qp.theme[enlist[`aspect_ratio]!enlist `square] - .qp.bar[select y:0, count i by sensor from subset; `y; `x] - .qp.s.coord[.gg.coords.polar] - , .qp.s.geom[`size`position!(1000; `stack)] - , .qp.s.aes[`group; `sensor] - , .qp.s.aes[`fill; `sensor] - , .qp.s.scale[`fill; .gg.scale.colour.cat10] - , .qp.s.scale[`y; .gg.scale.extend[0b] .gg.scale.limits[0 0N] .gg.scale.linear]; - - // But we are not *limited* to only pies - - .qp.go[700;400] - .qp.theme[enlist[`aspect_ratio]!enlist `square] - .qp.hbar[select y:0, avg signal by sensor from subset; `signal; `sensor] - .qp.s.coord[.gg.coords.polar] - , .qp.s.aes[`fill; `sensor] - , .qp.s.scale[`fill; .gg.scale.colour.cat10]; - +.qp.go[700;400] + .qp.theme[enlist[`aspect_ratio]!enlist `square] + .qp.bar[select y:0, count i by sensor from subset; `y; `x] + .qp.s.coord[.gg.coords.polar] + , .qp.s.geom[`size`position!(1000; `stack)] + , .qp.s.aes[`group; `sensor] + , .qp.s.aes[`fill; `sensor] + , .qp.s.scale[`fill; .gg.scale.colour.cat10] + , .qp.s.scale[`y; .gg.scale.extend[0b] .gg.scale.limits[0 0N] .gg.scale.linear]; + +// But we are not *limited* to only pies + +.qp.go[700;400] + .qp.theme[enlist[`aspect_ratio]!enlist `square] + .qp.hbar[select y:0, avg signal by sensor from subset; `signal; `sensor] + .qp.s.coord[.gg.coords.polar] + , .qp.s.aes[`fill; `sensor] + , .qp.s.scale[`fill; .gg.scale.colour.cat10]; + ``` Other settings have already been discussed in the Basic sections, or will diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/B-Geometries.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/B-Geometries.md index 5223194..35c1abd 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/B-Geometries.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/B-Geometries.md @@ -9,12 +9,12 @@ but are required to be stacked with another layer. ```q - .qp.go[500;300] .qp.stack ( - .qp.hline[.8; .qp.s.geom[enlist[`fill]!enlist .gg.colour.Gold]]; - .qp.hline[.5; .qp.s.geom[enlist[`fill]!enlist .gg.colour.Gold]]; - .qp.vline[0D00:01:11.999; .qp.s.geom[`size`fill!(3; .gg.colour.FireBrick)]]; - .qp.point[subset; `time; `signal] - .qp.s.geom[`size`alpha!(1; 0xf0)]) +.qp.go[500;300] .qp.stack ( + .qp.hline[.8; .qp.s.geom[enlist[`fill]!enlist .gg.colour.Gold]]; + .qp.hline[.5; .qp.s.geom[enlist[`fill]!enlist .gg.colour.Gold]]; + .qp.vline[0D00:01:11.999; .qp.s.geom[`size`fill!(3; .gg.colour.FireBrick)]]; + .qp.point[subset; `time; `signal] + .qp.s.geom[`size`alpha!(1; 0xf0)]) ``` diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/C-StatisticalTransforms.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/C-StatisticalTransforms.md index 85766fd..8f19011 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/C-StatisticalTransforms.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/C-StatisticalTransforms.md @@ -6,23 +6,19 @@ Statistical transforms apply a function to the data *before* plotting it. Some geometries come bundled with a default stat transform. ```q +// A histogram is a bar geometry paired with a 1D binning transform +.qp.go[500;200] .qp.histogram[subset; `signal; ::]; + +// This can be recreated as... +.qp.go[500;200] .qp.bar[subset; `signal; `count__] + .qp.s.stat[ .gg.stat.bin1d[`signal; ::; .st.a.count[]; ::] ]; - // A histogram is a bar geometry paired with a 1D binning transform - - .qp.go[500;200] .qp.histogram[subset; `signal; ::]; +// The `histogram` wrapper takes care of setting sizes and scales for you +.qp.go[500;200] .qp.bar[subset; `signal; `count__] + .qp.s.stat[ .gg.stat.bin1d[`signal; (`w;100;0); .st.a.count[]; ::] ] + , .qp.s.geom[enlist[`size]!enlist 100] + , .qp.s.scale[`y; .gg.scale.linear]; - // This can be recreated as... - - .qp.go[500;200] .qp.bar[subset; `signal; `count__] - .qp.s.stat[ .gg.stat.bin1d[`signal; ::; .st.a.count[]; ::] ]; - - // The `histogram` wrapper takes care of setting sizes and scales for you - - .qp.go[500;200] .qp.bar[subset; `signal; `count__] - .qp.s.stat[ .gg.stat.bin1d[`signal; (`w;100;0); .st.a.count[]; ::] ] - , .qp.s.geom[enlist[`size]!enlist 100] - , .qp.s.scale[`y; .gg.scale.linear]; - ``` A stat transform can be applied to any layer. The available built-in transforms @@ -35,14 +31,12 @@ grid, where each grid cell is split further into a series of segments depicting the duration distribution within the cell. ```q - - .qp.go[500;500] .qp.tile[subset; `signal; `sensor] - .qp.s.stat[ .gg.stat.binNd[`ma`sensor`signal; ((`c;50;0); (`w;1;0); (`c;50;0)); .st.a.count[]; ::] ] - , .qp.s.aes [`group; `signal] - , .qp.s.geom [`width`height`position!(50; 50; `dodge)] - , .qp.s.aes [`fill; `ma] - , .qp.s.scale [`fill; .gg.scale.colour.gradient . .gg.colour`SteelBlue`FireBrick] - , .qp.s.aes [`alpha; `count__] - , .qp.s.scale [`alpha; .gg.scale.alpha[50; 255]]; - +.qp.go[500;500] .qp.tile[subset; `signal; `sensor] + .qp.s.stat[ .gg.stat.binNd[`ma`sensor`signal; ((`c;50;0); (`w;1;0); (`c;50;0)); .st.a.count[]; ::] ] + , .qp.s.aes [`group; `signal] + , .qp.s.geom [`width`height`position!(50; 50; `dodge)] + , .qp.s.aes [`fill; `ma] + , .qp.s.scale [`fill; .gg.scale.colour.gradient . .gg.colour`SteelBlue`FireBrick] + , .qp.s.aes [`alpha; `count__] + , .qp.s.scale [`alpha; .gg.scale.alpha[50; 255]]; ``` diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/D-Composing.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/D-Composing.md index df89658..0260cfa 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/D-Composing.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/D-Composing.md @@ -7,14 +7,12 @@ of composing layers into a single frame. This splits two frames into a single fr both using separate y axes. ```q - - .qp.go[500;300] .qp.split ( - .qp.histogram[subset; `time] - .qp.s.geom[`alpha`fill!(0x80; `steelblue)] - , .qp.s.binx[`c;50;0]; - .qp.point[subset; `time; `signal] - .qp.s.geom[`alpha`size`fill!(0x80; 1; `firebrick)]); - +.qp.go[500;300] .qp.split ( + .qp.histogram[subset; `time] + .qp.s.geom[`alpha`fill!(0x80; `steelblue)] + , .qp.s.binx[`c;50;0]; + .qp.point[subset; `time; `signal] + .qp.s.geom[`alpha`size`fill!(0x80; 1; `firebrick)]); ``` ### Dependencies @@ -26,13 +24,11 @@ First, independent frames can rely on one another by `linking` to the same ident In the example below, zoom into either plot and notice the other updates to match. ```q - - .qp.go[600;500] .qp.vertical ( - .qp.histogram[subset; `ma] - .qp.s.link`myID; - .qp.point[subset; `time; `signal] - .qp.s.link`myID); - + .qp.go[600;500] .qp.vertical ( + .qp.histogram[subset; `ma] + .qp.s.link`myID; + .qp.point[subset; `time; `signal] + .qp.s.link`myID); ``` Second, within the same frame, a primary plot can share its data with any number @@ -44,14 +40,13 @@ on the x axis, and the point would use a 2D region if it had not been marked as secondary to the histogram. ```q - - .qp.go[500;300] .qp.stack ( - .qp.histogram[subset; `time] - .qp.s.geom[`alpha`fill!(0x80; .gg.colour.SteelBlue)] - , .qp.s.binx[`c;50;0] - , .qp.s.primary`myID; - .qp.point[subset; `time; `signal] - .qp.s.geom[`alpha`size`fill!(0x80; 1; .gg.colour.FireBrick)] - , .qp.s.secondary`myID); - +.qp.go[500;300] .qp.stack ( + .qp.histogram[subset; `time] + .qp.s.geom[`alpha`fill!(0x80; .gg.colour.SteelBlue)] + , .qp.s.binx[`c;50;0] + , .qp.s.primary`myID; + .qp.point[subset; `time; `signal] + .qp.s.geom[`alpha`size`fill!(0x80; 1; .gg.colour.FireBrick)] + , .qp.s.secondary`myID); + ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/E-Faceting.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/E-Faceting.md index d710d96..0178865 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/E-Faceting.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/E-Faceting.md @@ -6,8 +6,6 @@ In the following, we can see large spikes in the low range until the 8th, when the spikes move to the mid range, and the low range drops out. ```q - - .qp.go[1000;500] .qp.facet[update date:time.minute from subset; `sensor] - .qp.histogram[; `signal; .qp.s.binx[`c;200;0]]; - +.qp.go[1000;500] .qp.facet[update date:time.minute from subset; `sensor] + .qp.histogram[; `signal; .qp.s.binx[`c;200;0]]; ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/F-Themes.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/F-Themes.md index 07cfaf8..af3229c 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/F-Themes.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Advanced/F-Themes.md @@ -8,17 +8,14 @@ current theme. To see the available options, open `.gg.theme.default` in the Visual Inspector. ```q +// A basic chart with default theme +.qp.go[500;300] .qp.histogram[subset; `signal; ::]; - // A basic chart with default theme - - .qp.go[500;300] .qp.histogram[subset; `signal; ::]; - - // Use clean theme as a template, and override various aspects - - .qp.go[500;300] - .qp.theme[.gg.theme.clean] - .qp.theme[`axis_use_x`grid_style_x`grid_style_y`plot_background_fill!(0b; `none; `zebra; 0xffffffff)] - .qp.title["Custom Chart"] - .qp.histogram[subset; `signal; ::]; +// Use clean theme as a template, and override various aspects +.qp.go[500;300] + .qp.theme[.gg.theme.clean] + .qp.theme[`axis_use_x`grid_style_x`grid_style_y`plot_background_fill!(0b; `none; `zebra; 0xffffffff)] + .qp.title["Custom Chart"] + .qp.histogram[subset; `signal; ::]; ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/A-Introduction.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/A-Introduction.md index 4bb1f26..f328da5 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/A-Introduction.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/A-Introduction.md @@ -1,32 +1,36 @@ ### Quick Plot +The most basic way to visualize data is to use the high-level `.qp.plot[...]` +API which takes a table, a list of columns to plot, and a dictionary +of settings (or generic null). -Note, `.qp.go` specifies a width and height for the plot specification. +First, generate some data that will be used for our plotting examples. Use +the sensor data from the tutorial walkthrough. For more information on this data, +see `A.Tutorial.Doc/walkthrough.md`. ```q +// Generate some data +sensors: sim.system[]; + +count data; + +// Take a 50,000 record subset +subset: 50000#sensors; +``` + +Note, `.qp.go` specifies a width and height for the plot specification. + +```q +// No specified columns plots an n*n plot matrix +.qp.go[900;900] .qp.plot[`time`id _ subset; (); ::]; + +// One column produces a histogram +.qp.go[500;500] .qp.plot[subset; `active; ::]; - // Generate some data - data: sim.system[]; - - count data; - - subset : 50000#data; - - // No specified columns plots an n*n plot matrix - - .qp.go[900;900] .qp.plot[`time`id _ subset; (); ::]; - - // One column produces a histogram - - .qp.go[500;500] .qp.plot[data; `active; ::]; - - // Two columns produces a plot determined by the type and count of the columns - - .qp.go[500;500] .qp.plot[data; `time`signal; ::]; - - // More than two columns produces a plot matrix of the specified columns - - .qp.go[500;500] .qp.plot[data; `active`signal`ma; ::]; +// Two columns produces a plot determined by the type and count of the columns +.qp.go[500;500] .qp.plot[subset; `time`signal; ::]; +// More than two columns produces a plot matrix of the specified columns +.qp.go[500;500] .qp.plot[subset; `active`signal`ma; ::]; ``` diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/B-Interaction.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/B-Interaction.md index d1966d3..8666117 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/B-Interaction.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/B-Interaction.md @@ -4,9 +4,7 @@ Evaluate the line below: ```q - - .qp.go[500;500] .qp.point[subset; `ma; `signal; ::]; - +.qp.go[500;500] .qp.point[subset; `ma; `signal; ::]; ``` Click the a point in the resulting image. Notice a table appears under @@ -23,14 +21,12 @@ in to the data. Execute the following histogram and drag a horizontal slice across the image. ```q +.qp.go[500;500] .qp.histogram[data; `ma; ::]; + +// Settings, like number of bins, can be slotted in as the second argument +.qp.go[500;500] .qp.histogram[data; `ma] + .qp.s.binx[`c;200;0] - .qp.go[500;500] .qp.histogram[data; `ma; ::]; - - - // Settings, like number of bins, can be slotted in as the second argument - .qp.go[500;500] .qp.histogram[data; `ma] - .qp.s.binx[`c;200;0] - ``` Notice how the entire bar does not need to be selected to zoom. The zoom @@ -42,9 +38,7 @@ area, whereas a histogram zooms only into the x axis. Execute the following and click on a bar. ```q - - .qp.go[500;500] .qp.boxplot[subset; `machine; `signal; ::]; - +.qp.go[500;500] .qp.boxplot[subset; `machine; `signal; ::]; ``` Notice that many tables have been created under the image. In general, GG does diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/C-Geometries.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/C-Geometries.md index 5801665..4dfb20e 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/C-Geometries.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/C-Geometries.md @@ -5,27 +5,20 @@ Beyond the basic `.qp` API, each geometry in the grammar of graphics is split in a function. Some examples are below: ```q +// Simple 2D point takes x and y +.qp.go[500;500] .qp.point[subset; `time; `signal; ::]; + +// Bar takes x and y +.qp.go[500;500] .qp.bar[select count i by sensor from subset; `sensor; `x; ::]; + +// Most geometries have a horizontal counterpart +.qp.go[500;500] .qp.hbar[select count i by sensor from subset; `x; `sensor; ::]; +// Some geometries take more than just an (x,y) coordinate, such as (xmin, xmax, y) +.qp.go[500;500] .qp.interval[select minimum:min signal, average:avg signal by sensor from subset; `sensor; `minimum; `average; ::]; + +// All current geometries are listed in the *Analyst Function Reference* page (under *Help*) or using the cheat sheet +.gg.cheat.sheet[]; - // Simple 2D point takes x and y - - .qp.go[500;500] .qp.point[subset; `time; `signal; ::]; - - // Bar takes x and y - - .qp.go[500;500] .qp.bar[select count i by sensor from subset; `sensor; `x; ::]; - - // Most geometries have a horizontal counterpart - - .qp.go[500;500] .qp.hbar[select count i by sensor from subset; `x; `sensor; ::]; - - // Some geometries take more than just an (x,y) coordinate, such as (xmin, xmax, y) - - .qp.go[500;500] .qp.interval[select minimum:min signal, average:avg signal by sensor from subset; `sensor; `minimum; `average; ::]; - - // All current geometries are listed in the *Analyst Function Reference* page (under *Help*) or using the cheat sheet - - .gg.cheat.sheet[]; - ``` Some geometry functions pair a basic geometry with a statistical transform. The parameters @@ -34,15 +27,12 @@ can be added to any geometry to modify it's behaviour (also revisted later). ```q - - .qp.go[500;500] .qp.bar[select count i by sensor from subset; `sensor; `x; ::]; - - // A histogram pairs a 1D binning transform with a bar geometry - - .qp.go[500;500] .qp.histogram[subset; `sensor; ::]; - - // A heatmap pairs a tile geometry with a 2D binning transform - - .qp.go[500;500] .qp.heatmap[subset; `time; `signal; ::]; - +.qp.go[500;500] .qp.bar[select count i by sensor from subset; `sensor; `x; ::]; + +// A histogram pairs a 1D binning transform with a bar geometry +.qp.go[500;500] .qp.histogram[subset; `sensor; ::]; + +// A heatmap pairs a tile geometry with a 2D binning transform +.qp.go[500;500] .qp.heatmap[subset; `time; `signal; ::]; + ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/D-Aesthetics.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/D-Aesthetics.md index f5f20bf..99419d7 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/D-Aesthetics.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/D-Aesthetics.md @@ -9,58 +9,48 @@ The aesthetics that can be mapped are determined by the geometry. Information on each geometry can be found in the *Analyst Function Reference* under *Help*. ```q +// Basic plot +.qp.go[500;500] .qp.point[subset; `time; `signal; ::]; - // Basic plot - - .qp.go[500;500] .qp.point[subset; `time; `signal; ::]; +// A mapping with no scale will *try* to produce something +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.aes [`fill; `active]; + +// A mapping with a scale is much better +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.aes [`fill; `active] + , .qp.s.scale [`fill; .gg.scale.colour.gradient[`firebrick; `steelblue] ]; + +// More mappings can be joined together +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.aes [`fill`alpha; `active`ma] + , .qp.s.scale [`fill; .gg.scale.colour.gradient[`firebrick; `steelblue] ] + , .qp.s.scale [`alpha; .gg.scale.alpha[50; 255] ]; - // A mapping with no scale will *try* to produce something +// And more... +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.aes [`fill`alpha`size; `active`ma`ma] + , .qp.s.scale [`fill; .gg.scale.colour.gradient[`firebrick; `steelblue] ] + , .qp.s.scale [`alpha; .gg.scale.alpha[50; 255] ] + , .qp.s.scale [`size; .gg.scale.circle.area[1; 20] ]; - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.aes [`fill; `active]; - - // A mapping with a scale is much better - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.aes [`fill; `active] - , .qp.s.scale [`fill; .gg.scale.colour.gradient[`firebrick; `steelblue] ]; - - // More mappings can be joined together - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.aes [`fill`alpha; `active`ma] - , .qp.s.scale [`fill; .gg.scale.colour.gradient[`firebrick; `steelblue] ] - , .qp.s.scale [`alpha; .gg.scale.alpha[50; 255] ]; - - // And more... - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.aes [`fill`alpha`size; `active`ma`ma] - , .qp.s.scale [`fill; .gg.scale.colour.gradient[`firebrick; `steelblue] ] - , .qp.s.scale [`alpha; .gg.scale.alpha[50; 255] ] - , .qp.s.scale [`size; .gg.scale.circle.area[1; 20] ]; - ``` Static attributes are assigned using a `.qp.s.geom[]` setting. The static attributes that can be assigned are determined by the particular geometry. Individual geometry information is available in the *Analyst Function Reference* under *Help*. -```q - - // Set the size and fill colour of a point geometry - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.geom[`fill`size!(`firebrick; 0.5)] - - // In general, `fill refers to fill colour and `colour refers to stroke colour - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.geom[`fill`colour`size!(`gold; `black; 2)] - - // Alpha (opacity) only applies to the fill colour +```q +// Set the size and fill colour of a point geometry +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.geom[`fill`size!(`firebrick; 0.5)] + +// In general, `fill refers to fill colour and `colour refers to stroke colour +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.geom[`fill`colour`size!(`gold; `black; 2)] + +// Alpha (opacity) only applies to the fill colour +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.geom[`alpha`colour`size!(0x00; `black; 5)] - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.geom[`alpha`colour`size!(0x00; `black; 5)] - ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/E-Scales.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/E-Scales.md index d276464..b9d78dd 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/E-Scales.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/E-Scales.md @@ -6,49 +6,40 @@ well as manipulating axes. ```q +// Default scales +.qp.go[500;500] .qp.point[subset; `time; `signal; ::]; - // Default scales +// Linear scales are used by default for numeric data +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.scale[`y; .gg.scale.linear]; - .qp.go[500;500] .qp.point[subset; `time; `signal; ::]; +// But log or power scales could be used instead +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.scale[`y; .gg.scale.log]; - // Linear scales are used by default for numeric data +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.scale[`y; .gg.scale.power 2]; - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.scale[`y; .gg.scale.linear]; - - // But log or power scales could be used instead - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.scale[`y; .gg.scale.log]; - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.scale[`y; .gg.scale.power 2]; - - // Limits can be set within or beyond the data to limit the range of view - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.scale[`y; .gg.scale.limits[0 10] .gg.scale.linear]; +// Limits can be set within or beyond the data to limit the range of view +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.scale[`y; .gg.scale.limits[0 10] .gg.scale.linear]; - // Custom break points can be set - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.scale[`y; .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear]; - - // Padding/axis extension can be set - - .qp.go[500;500] .qp.point[subset; `time; `signal] - .qp.s.scale[`y; .gg.scale.extension[0.5] .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear]; - - // Both axes can be manipulated by joining the settings together +// Custom break points can be set +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.scale[`y; .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear]; + +// Padding/axis extension can be set +.qp.go[500;500] .qp.point[subset; `time; `signal] + .qp.s.scale[`y; .gg.scale.extension[0.5] .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear]; + +// Both axes can be manipulated by joining the settings together +.qp.go[500;500] .qp.point[subset; `sensor; `signal] + .qp.s.scale[`y; .gg.scale.extension[0.5] .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear] + , .qp.s.scale[`x; .gg.scale.categorical[]]; + +// Categorical scales can adjust the ordering of the axis by taking an ordering function +.qp.go[500;500] .qp.point[subset; `sensor; `signal] + .qp.s.scale[`y; .gg.scale.extension[0.5] .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear] + , .qp.s.scale[`x; .gg.scale.extension[0.5] .gg.scale.categorical desc]; - .qp.go[500;500] .qp.point[subset; `sensor; `signal] - .qp.s.scale[`y; .gg.scale.extension[0.5] .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear] - , .qp.s.scale[`x; .gg.scale.categorical[]]; - - // Categorical scales can adjust the ordering of the axis by taking an ordering function - - .qp.go[500;500] .qp.point[subset; `sensor; `signal] - .qp.s.scale[`y; .gg.scale.extension[0.5] .gg.scale.breaks[0 1 5 8 10] .gg.scale.limits[0 10] .gg.scale.linear] - , .qp.s.scale[`x; .gg.scale.extension[0.5] .gg.scale.categorical desc]; - ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/F-Themes.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/F-Themes.md index 643a56a..2b52455 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/F-Themes.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/F-Themes.md @@ -4,43 +4,36 @@ Prebuilt themes customize the look and feel of a plot. ```q +.qp.go[500;300] .qp.histogram[subset; `signal; ::]; - .qp.go[500;300] .qp.histogram[subset; `signal; ::]; +// No theme definition is the same as using the default theme +.qp.go[500;300] + .qp.theme[ .gg.theme.default ] + .qp.histogram[subset; `signal; ::]; - // No theme definition is the same as using the default theme +// The theme can be changed to one of the packaged themes, such as clean... +.qp.go[500;300] + .qp.theme[ .gg.theme.clean ] + .qp.histogram[subset; `signal; ::]; - .qp.go[500;300] - .qp.theme[ .gg.theme.default ] - .qp.histogram[subset; `signal; ::]; - - // The theme can be changed to one of the packaged themes, such as clean... - - .qp.go[500;300] - .qp.theme[ .gg.theme.clean ] - .qp.histogram[subset; `signal; ::]; - - // Or dark... - - .qp.go[500;300] - .qp.theme[ .gg.theme.dark ] - .qp.histogram[subset; `signal; ::]; +// Or dark... +.qp.go[500;300] + .qp.theme[ .gg.theme.dark ] + .qp.histogram[subset; `signal; ::]; ``` Any plot or subplot can be given a title. -```q - - .qp.go[500;300] - .qp.theme[ .gg.theme.dark ] - .qp.title["Histogram of signal"] - .qp.histogram[subset; `signal; ::]; - - // A theme declaration only applies to the subplot it is applied to - - .qp.go[500;300] - .qp.title["Histogram of signal"] - .qp.theme[ .gg.theme.dark ] - .qp.histogram[subset; `signal; ::]; - +```q +.qp.go[500;300] + .qp.theme[ .gg.theme.dark ] + .qp.title["Histogram of signal"] + .qp.histogram[subset; `signal; ::]; + +// A theme declaration only applies to the subplot it is applied to +.qp.go[500;300] + .qp.title["Histogram of signal"] + .qp.theme[ .gg.theme.dark ] + .qp.histogram[subset; `signal; ::]; ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/G-Composing.md b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/G-Composing.md index 5d3c03d..c74b0b4 100644 --- a/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/G-Composing.md +++ b/kxscm/module/B.Section.GrammarOfGraphics/file/Basic/G-Composing.md @@ -8,60 +8,54 @@ on a single pair of axes. Each layer is defined by a single geometry. Layers are into a single frame by *stacking* them together. ```q +summary : select + minDuration: min signal, + maxDuration: max signal, + avgDuration: avg signal, + maxLabel: max signal, + avgLabel: avg signal, + sensorLabel: first sensor + by sensor from subset; - summary : select - minDuration: min signal, - maxDuration: max signal, - avgDuration: avg signal, - maxLabel: max signal, - avgLabel: avg signal, - sensorLabel: first sensor - by sensor from subset; +// Any number of layers can be stacked together +.qp.go[500;500] + .qp.stack ( + .qp.interval[summary; `sensor; `minDuration; `maxDuration; ::]; + .qp.point[summary; `sensor; `avgDuration; ::]); - // Any number of layers can be stacked together - - .qp.go[500;500] - .qp.stack ( - .qp.interval[summary; `sensor; `minDuration; `maxDuration; ::]; - .qp.point[summary; `sensor; `avgDuration; ::]); - - // Layers within a stack can be arbitrarily customized +// Layers within a stack can be arbitrarily customized +.qp.go[500;500] + .qp.stack ( + .qp.interval[summary; `sensor; `minDuration; `maxDuration] + .qp.s.geom[`fill`colour!.gg.colour`Grey`Black]; + .qp.point[summary; `sensor; `avgDuration; ::]); - .qp.go[500;500] - .qp.stack ( - .qp.interval[summary; `sensor; `minDuration; `maxDuration] - .qp.s.geom[`fill`colour!.gg.colour`Grey`Black]; - .qp.point[summary; `sensor; `avgDuration; ::]); - - // Text geometries are a good way of annotating plot +// Text geometries are a good way of annotating plot +.qp.go[500;500] + .qp.stack ( + .qp.interval[summary; `sensor; `minDuration; `maxDuration] + .qp.s.geom[`fill`colour!.gg.colour`Grey`Black]; + .qp.point[summary; `sensor; `avgDuration; ::]; + .qp.text[summary; `sensor; `maxLabel; `sensor] + .qp.s.textalign[`middle] , + .qp.s.geom[``offsety!(::;-10)]); + +// By stacking multiple layers, custom plots can be expressed +.qp.go[500;500] + .qp.stack ( + .qp.interval[summary; `sensor; `minDuration; `maxDuration] + .qp.s.geom[`fill`colour!.gg.colour`Grey`Black]; + .qp.interval[summary; `sensor; `avgDuration; `avgDuration] + .qp.s.geom[enlist[`colour]!enlist .gg.colour.Gold]; + .qp.text[summary; `sensor; `maxLabel; `sensor] + .qp.s.textalign[`middle] , + .qp.s.geom[``offsety!(::;-10)]; + .qp.text[summary; `sensorLabel; `avgLabel; `avgDuration] + .qp.s.textalign[`left] , + .qp.s.geom[`angle`size`offsety`fill!(-90; 8; -10; `white)]); - .qp.go[500;500] - .qp.stack ( - .qp.interval[summary; `sensor; `minDuration; `maxDuration] - .qp.s.geom[`fill`colour!.gg.colour`Grey`Black]; - .qp.point[summary; `sensor; `avgDuration; ::]; - .qp.text[summary; `sensor; `maxLabel; `sensor] - .qp.s.textalign[`middle] , - .qp.s.geom[``offsety!(::;-10)]); - - // By stacking multiple layers, custom plots can be expressed - - .qp.go[500;500] - .qp.stack ( - .qp.interval[summary; `sensor; `minDuration; `maxDuration] - .qp.s.geom[`fill`colour!.gg.colour`Grey`Black]; - .qp.interval[summary; `sensor; `avgDuration; `avgDuration] - .qp.s.geom[enlist[`colour]!enlist .gg.colour.Gold]; - .qp.text[summary; `sensor; `maxLabel; `sensor] - .qp.s.textalign[`middle] , - .qp.s.geom[``offsety!(::;-10)]; - .qp.text[summary; `sensorLabel; `avgLabel; `avgDuration] - .qp.s.textalign[`left] , - .qp.s.geom[`angle`size`offsety`fill!(-90; 8; -10; `white)]); - - // The boxplot geometry is itself a stack of several layers - - .qp.go[500;500] .qp.boxplot[subset; `sensor; `signal; ::]; +// The boxplot geometry is itself a stack of several layers +.qp.go[500;500] .qp.boxplot[subset; `sensor; `signal; ::]; ``` @@ -70,47 +64,41 @@ into a single frame by *stacking* them together. Beyond stacking, frames can be arranged to create visual summaries. ```q - - // Horizontal arrangements - - .qp.go[500;500] .qp.horizontal ( - .qp.point[subset; `time; `signal; ::]; - .qp.point[subset; `time; `ma; ::]); - - // Vertical arrangements - - .qp.go[500;500] .qp.vertical ( - .qp.point[subset; `time; `signal; ::]; - .qp.point[subset; `time; `ma; ::]); - - // Horizontal or vertical arrangements can be weighted - - .qp.go[1000;500] .qp.layout[`hori_w; 2 1] ( - .qp.point[subset; `time; `signal; ::]; - .qp.point[subset; `time; `ma; ::]); - - .qp.go[1000;500] .qp.layout[`vert_w; 2 1] ( - .qp.point[subset; `time; `signal; ::]; - .qp.point[subset; `time; `ma; ::]); - - // ... or nested - - .qp.go[500;500] .qp.vertical ( - .qp.title["Destinations"] .qp.point[subset; `time; `signal; ::]; - .qp.horizontal ( - .qp.point[subset; `time; `ma; ::]; - .qp.histogram[subset; `signal; ::])); - - // Any number of plots can be arranged in a grid +// Horizontal arrangements +.qp.go[500;500] .qp.horizontal ( + .qp.point[subset; `time; `signal; ::]; + .qp.point[subset; `time; `ma; ::]); + +// Vertical arrangements +.qp.go[500;500] .qp.vertical ( + .qp.point[subset; `time; `signal; ::]; + .qp.point[subset; `time; `ma; ::]); + +// Horizontal or vertical arrangements can be weighted +.qp.go[1000;500] .qp.layout[`hori_w; 2 1] ( + .qp.point[subset; `time; `signal; ::]; + .qp.point[subset; `time; `ma; ::]); + +.qp.go[1000;500] .qp.layout[`vert_w; 2 1] ( + .qp.point[subset; `time; `signal; ::]; + .qp.point[subset; `time; `ma; ::]); + +// ... or nested +.qp.go[500;500] .qp.vertical ( + .qp.title["Destinations"] .qp.point[subset; `time; `signal; ::]; + .qp.horizontal ( + .qp.point[subset; `time; `ma; ::]; + .qp.histogram[subset; `signal; ::])); - .qp.go[800;500] - .qp.theme[.gg.theme.clean] - .qp.title["Distributions of major columns"] - .qp.grid[3 0N] ( - .qp.histogram[subset; `time; ::]; - .qp.histogram[subset; `ma; ::]; - .qp.histogram[subset; `signal; ::]; - .qp.histogram[subset; `sensor; ::]; - .qp.histogram[subset; `active; ::]); +// Any number of plots can be arranged in a grid +.qp.go[800;500] + .qp.theme[.gg.theme.clean] + .qp.title["Distributions of major columns"] + .qp.grid[3 0N] ( + .qp.histogram[subset; `time; ::]; + .qp.histogram[subset; `ma; ::]; + .qp.histogram[subset; `signal; ::]; + .qp.histogram[subset; `sensor; ::]; + .qp.histogram[subset; `active; ::]); ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.IDE/file/DeveloperProductivity.q b/kxscm/module/B.Section.IDE/file/DeveloperProductivity.q index 7222997..1de33fc 100644 --- a/kxscm/module/B.Section.IDE/file/DeveloperProductivity.q +++ b/kxscm/module/B.Section.IDE/file/DeveloperProductivity.q @@ -4,7 +4,7 @@ filterReadings[5] ([] x : til 10) // Then, ctrl/meta + i to open the visual inspector -3 enlist/ .analysis.CITIES +3 enlist/ CITIES // Hover over the following function to get parameter information // Open the function and notice the documentation format that provides this information @@ -19,7 +19,8 @@ filt filterReadings -// Open the function below using the hotkey and right-click the editor and choose Test to run the corresponding tests +// Open the function below using the hotkey and right-click the editor and choose Test +// to run the corresponding tests filterReadings diff --git a/kxscm/module/B.Section.IDE/file/FindingHelp.md b/kxscm/module/B.Section.IDE/file/FindingHelp.md index fcd330e..3b679ee 100644 --- a/kxscm/module/B.Section.IDE/file/FindingHelp.md +++ b/kxscm/module/B.Section.IDE/file/FindingHelp.md @@ -2,4 +2,17 @@ - Right click to bring context menu options - Menus at the top provide hotkeys -- Help > Function Reference for public libraries \ No newline at end of file +- Help > Function Reference for public libraries + +## Help with expressions + +For more information about a keyword in q or about a public API, use the hotkey +alt+? with the term selected to open the documentation. + +```q +// put your cursor on the line below and press alt+? +abs + +// put your cursor on the line below and press alt+? +.axq.parseFloat +``` diff --git a/kxscm/module/B.Section.IDE/file/Modules.md b/kxscm/module/B.Section.IDE/file/Modules.md index da148e8..507b428 100644 --- a/kxscm/module/B.Section.IDE/file/Modules.md +++ b/kxscm/module/B.Section.IDE/file/Modules.md @@ -10,6 +10,8 @@ - A repository is the top level for a group of version controlled entities - A repository contains many modules and spreadsheets -## Kickstarters +## Loading on startup -- A module can contain a "ks" function (see .analysis) +A module can contain an "onLoad" function that will be run when the workspace loads. +See [automatic initialization](https://code.kx.com/analyst/faq/#automatic-initialization-when-loading-a-module) +for more information. diff --git a/kxscm/module/B.Section.Transformer/file/TransformerFunctions.md b/kxscm/module/B.Section.Transformer/file/TransformerFunctions.md index 6799373..abb00cb 100644 --- a/kxscm/module/B.Section.Transformer/file/TransformerFunctions.md +++ b/kxscm/module/B.Section.Transformer/file/TransformerFunctions.md @@ -4,52 +4,99 @@ First, delete the table from memory, and note that it is really gone. ```q - - delete ... from `.; - tables[]; - - // `symbol$() - +delete signals from `.; +`signals in tables[]; +/=> 0b ``` Each transform is saved as a function which takes inputs and outputs. If called with generic null `(::)`, the defaults as specified in the Transformer UI are used. -The return value will be a table of nodes and generated errors. If there -are no entries in the table, everything ran without error. +The return value will be a table of nodes and generated errors. ```q - - - // name type| error - // ---------| ----- - +ImportSensors[::;::]; + +/=> name type | error message +/=> ------------------------| ------------- +/=> Graphic gg | 0 "" +/=> aggregate-signals action| 0 "" +/=> cleanup-pressure action| 0 "" +/=> cleanup-temp action| 0 "" +/=> cleanup-weight action| 0 "" +/=> filter-signals action| 0 "" +/=> pressure source| 0 "" +/=> pressure_temp join | 0 "" +/=> signals join | 0 "" +/=> signals output| 0 "" +/=> summarize op | 0 "" +/=> summary output| 0 "" +/=> temp source| 0 "" +/=> weight source| 0 "" ``` Note now that the cells table has been created in memory. ```q - - tables[] - - // ,`cells - +`signals in tables[] +/=> 1b ``` Display the function to read possible formats for inputs and outputs. ```q - - +// Replace the formatting characters and print the function to stdout +-1 "\n" sv ssr[;","; ""] each ssr[;"\""; ""] each 1 _ "\n" vs string ImportSensors; +/=> Usage +/=> +/=> Arguments: +/=> - inputs: (::) to use inputs from the UI +/=> or a dictionary from node names to IO configurations +/=> or a path to a new data source to use the schema set in the UI +/=> - outputs: (::) to use inputs from the UI +/=> or a dictionary from node names to IO configurations +/=> or a path to a new data source to use the schema set in the UI +/=> +/=> Examples: +/=> dataflow[::; ::] // To use defaults defined via the UI +/=> +/=> dataflow[enlist[`srcnode]!enlist `:/path/t.csv; ::] +/=> // To use default output and redefine the source node to be a CSV file +/=> +/=> dataflow[enlist[`srcnode]!enlist .im.io.with.target[`:/path/t.csv] .im.io.create`dsv; ::] +/=> // To use default output and redefine the source node and reset the schema +/=> +/=> dataflow[::; enlist[`outnode]!enlist .im.io.with.target[`:/path/t2.csv] .im.io.create`dsv] ``` For example, ```q - - // Pointing an input to a new csv +// Example with explicit input and output locations +ImportSensors[ + `temp`pressure!(`:A.Tutorial.Data/temp.csv; `:A.Tutorial.Data/pressure.csv); + + // Output to a csv file called `signals.csv` - we need to change the output format + // from a kdb+ table to a csv file by creating a new `.im.io` source descriptor + enlist[`signals]!enlist .im.io.with.target[`:signals_out.csv] .im.io.create `csv + ]; - // Pointing an input to a new IO configuration (including schema) +/=> name type | error message +/=> ------------------------| ------------- +/=> Graphic gg | 0 "" +/=> aggregate-signals action| 0 "" +/=> cleanup-pressure action| 0 "" +/=> cleanup-temp action| 0 "" +/=> cleanup-weight action| 0 "" +/=> filter-signals action| 0 "" +/=> pressure source| 0 "" +/=> pressure_temp join | 0 "" +/=> signals join | 0 "" +/=> signals output| 0 "" +/=> summarize op | 0 "" +/=> summary output| 0 "" +/=> temp source| 0 "" +/=> weight source| 0 "" ``` \ No newline at end of file diff --git a/kxscm/module/B.Section.VisualInspector/file/VIFunctions.md b/kxscm/module/B.Section.VisualInspector/file/VIFunctions.md index 133e3a2..12e4dbf 100644 --- a/kxscm/module/B.Section.VisualInspector/file/VIFunctions.md +++ b/kxscm/module/B.Section.VisualInspector/file/VIFunctions.md @@ -6,18 +6,13 @@ contains one of the Grammar Of Graphics powered visuals, then running the function will produce the specification for the visual. ```q +// Running with generic null uses the query defined in the UI +ReadingsOverTime[::]; + +// The result can be displayed in Analyst by passing to .qp.go +.qp.go[800;300] ReadingsOverTime[::]; + +// Custom data can be given to the function to overwrite the query in the UI +.qp.go[800;300] ReadingsByMachine select from signals where sensor = `temp_a; - // Running with generic null uses the query defined in the UI - ReadingsOverTime[::] - - - // The result can be displayed in Analyst by passing to .qp.go - .qp.go[800;300] ReadingsOverTime[::] - - - - // Custom data can be given to the function to overwrite the query in the UI - - .qp.go[800;300] ReadingsByMachine select from signals where sensor = `temp_a - ``` \ No newline at end of file From 50d43d50c936cd344177791d42dbe9339d56b85a Mon Sep 17 00:00:00 2001 From: Matthew Maynes Date: Thu, 13 Jun 2019 18:12:49 +0000 Subject: [PATCH 2/2] Remove incomplete section --- .../module/B.Section.IDE/file/VersionControl.md | 16 ---------------- 1 file changed, 16 deletions(-) delete mode 100644 kxscm/module/B.Section.IDE/file/VersionControl.md diff --git a/kxscm/module/B.Section.IDE/file/VersionControl.md b/kxscm/module/B.Section.IDE/file/VersionControl.md deleted file mode 100644 index e096ff8..0000000 --- a/kxscm/module/B.Section.IDE/file/VersionControl.md +++ /dev/null @@ -1,16 +0,0 @@ -## Modules, Collaboration, Conflicts - -- Modules + Contexts -- Branch -- Modify and Save -- Push -- Pull master -- Modify locally same spot -- Merge -- Resolve conflict - -## Updating and Reverting Changes - -- Make a change -- Update Base -- Update -> Workspace \ No newline at end of file