[TRAFODION-2734] Create Trafodion SQL Large Objects Guide 1 #1229

+  <artifactId>sql-reference-manual</artifactId>
+  <version>${env.TRAFODION_VER}</version>
+  <packaging>pom</packaging>
+  <name>Trafodion SQL Reference Manual</name>


Shouldn't this be different? Trafodion SQL Large Objects Guide perhaps?

Thanks for your reminding. :)
Yes I've changed it to Trafodion SQL Large Objects Guide.

DaveBirdsall · 2017-09-12T23:31:42Z

docs/lob_guide/pom.xml

+  <version>${env.TRAFODION_VER}</version>
+  <packaging>pom</packaging>
+  <name>Trafodion SQL Reference Manual</name>
+  <description>This manual describes reference information about the syntax of SQL statements, 


The description probably should be changed as well.

Thanks Dave.
Yes I've changed. :)

DaveBirdsall · 2017-09-12T23:33:12Z

docs/lob_guide/src/asciidoc/_chapters/about.adoc

+
+[[About_This_Document]]
+= About This Document
+This guide dwscribes how to use Large Object (LOB) datatypes.


Typo, "dwscribes". Also, might want to say "...how to use Large Object (LOB) datatypes in Trafodion SQL."

Thanks Dave.
I've corrected. :)

DaveBirdsall · 2017-09-12T23:34:09Z

docs/lob_guide/src/asciidoc/_chapters/about.adoc

+[cols="30%,70%",options="header"]
+|===
+|Chapter                                       | Description
+| <<Introduction,Introduction>>                | Introduces LOB and covers following topics: +


Grammar: "Introduces LOBs ..." is better.

Thanks Dave.
I've corrected. :)

DaveBirdsall · 2017-09-12T23:34:33Z

docs/lob_guide/src/asciidoc/_chapters/about.adoc

+|===
+|Chapter                                       | Description
+| <<Introduction,Introduction>>                | Introduces LOB and covers following topics: +
+&#8226; 1.1 <<what is lob,What is LOB>> +


Grammar: "what is a lob,What is a LOB" is better.

Thanks Dave.
I've corrected. :)

DaveBirdsall · 2017-09-12T23:40:00Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+
+LOB (Large Object), a set of large object data types used to store large volumes of data, provides random and piece-wise access to the data. 
+
+Database support for LOB is not universal.


In this section, are you trying to define the concept of "LOB" in general (that is, for any database), or just for Trafodion? Are you trying to define the notion of a LOB data type? Or a value that might be stored there?

Just for Trafodion.

Both. Define the notion first, and then the value (1.4 LOB Handle).

DaveBirdsall · 2017-09-12T23:41:10Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+
+Stores unusually large amounts of string data in character set format. +
+
+CLOB is ideal for storing and processing semi-structured data, such as large document or string.


"... such as a large document or string" is better. Alternatively, "...such as large documents or strings"

Thanks Dave.
I've corrected. :)

DaveBirdsall · 2017-09-12T23:41:54Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+[#lob storage]
+== LOB Storage
+
+The basic design idea is to allow a database row to have multiple LOB columns and a table to have multiple such rows. The design takes a combination approach of rows store and column store. 


"row store" is better ("row" is being used as an adjective)

Thanks Dave.
I've corrected. :)

DaveBirdsall · 2017-09-12T23:42:24Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+
+The basic design idea is to allow a database row to have multiple LOB columns and a table to have multiple such rows. The design takes a combination approach of rows store and column store. 
+
+When a SQL table with LOB column is created, there are several dependent objects that are created to hold metadata information on the LOB as well as the data.


"with a LOB column" is better

Thanks Dave.
I've corrected. :)

DaveBirdsall · 2017-09-12T23:43:10Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+
+When a SQL table with LOB column is created, there are several dependent objects that are created to hold metadata information on the LOB as well as the data.
+
+The LOB data itself is stored in HDFS while the Trafodion table contains a unique LOB handle for each row/column. The LOB handle provides the identifier that is used to query the LOB metadata tables to find the location of the LOB data files.


"an HDFS file" is better. A question: Is each LOB value stored in a separate HDFS file? Or are several LOB values stored in one HDFS file? Also, "...a unique LOB handle for each row/column" might be better stated, "...a unique LOB handle for each LOB value".

Thanks Dave.
I've corrected. :)

Answer from Sandhya:
Each lob column in any table will have it’s own unique lob data file. So any lob data belonging to that column will be stored in that HDFS file.

So if a table has 3 columns, you will see 3 lob data files with the first part of the name being the same and the column number will be appended to it.

For example, for a table with 2 lob columns:
Location: /user/trafodion/lobs
DataFile: LOBP_04474425229029907479_0001

Location: /user/trafodion/lobs
DataFile: LOBP_04474425229029907479_0002

Traf-Jenkins · 2017-09-13T07:45:12Z

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/2045/

Traf-Jenkins · 2017-09-13T07:50:25Z

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/2045/

DaveBirdsall

The manual is looking very good. There is so much detail to be documented. I had some questions, and I've made a few suggestions.

DaveBirdsall · 2017-09-13T23:48:33Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+[#external structure]
+=== External Structure
+
+The following is an external structure that is stored in the row of a SQL table:


Do you mean, "The following is an example of an external structure..."?

Thanks Dave.
Yes, I've corrected. :)

DaveBirdsall · 2017-09-13T23:50:00Z

docs/lob_guide/src/asciidoc/_chapters/introduction.adoc

+* LOB columns cannot be used as primary keys.
+* LOB columns cannot be used in CREATE INDEX statements.
+* LOB columns cannot be used in statistics update statements.
+* The file that contains data to insert from or to extract to need to be on platform linux or HDFS files.


Grammar: "need to be" should be "needs to be" (because "needs" refers back to "The file" which is singular).

Also, "on platform linux or HDFS files." I think you mean, "on platform as a Linux or HDFS file."

Thanks Dave.
Yes, I've corrected. :)

DaveBirdsall · 2017-09-13T23:52:35Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+////
+
+[#work with lob]
+= Work with LOB


Consider "Working with LOBs"

Thanks Dave.

Using gerund or verb (base) in title/headline is a style issue that should be followed consistently.

Gerundive (continuous) form is common in title and verb form seems more imperative, but given that

the topic title plays a huge role in the search results

Trafodion has many users of non-native English speakers

I think the verb form might be better as it more likely to match what people will search for, and uses the more familiar form of the verb.

Any thoughts?

It seems ungrammatical to my ear. An imperative tense would clearly not be correct. We aren't telling the reader to "Work with LOBs!" (note the plural would be required, or alternatively the indefinite article, "Work with a LOB!"). Rather we are trying to describe how one works with LOBs. To me using the gerund seems most natural (in all cases, not just here); I would recommend changing the style to do that. Another possibility is to use the infinitive, e.g. "How to work with LOBs" (note again the plural). That might be more natural for English-as-a-second-language users whose native language lacks gerund-like inflections.

Thanks for the detailed explanation! :)
I've changed to "Working with LOBs".

DaveBirdsall · 2017-09-14T00:00:53Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+== Syntax
+
+```
+CREATE TABLE table-name (lob-column-spec[, lob-column-spec]…)


This syntax suggests that when you have lob columns in a table, you can't have other columns as well. Perhaps something like "CREATE TABLE table-name ( column_spec [, column_spec] ... [, primary_key_spec] )", then "column_spec is: lob-column-spec | other column spec"

Thanks Dave! :)

I'm planning to change it to:

CREATE TABLE table-name (column-spec[, column-spec]…)

column-spec is:
lob-column-spec |
other-column-spec

lob-column-spec is:
column-name {lob-data-type} [column-constraint]

other-column-spec is:
column-name {data-type} [column-constraint]

lob-data-type is:
BLOB | CLOB [({numeric literal} [unit])] [STORAGE 'storage literal']

...

Any comments?

I think this is fine.

Thanks Dave! :)
I've corrected.

DaveBirdsall · 2017-09-14T00:01:32Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+* `_storage literal_`
+
+
+Currently Trafodion only support `'EXTERNAL'` here. 


Grammar. Should be "... supports ..."

Thanks Dave, I've corrected. :)

DaveBirdsall · 2017-09-14T00:31:35Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+
+Trafodion engine will retrieve exactly the amount of requested data. 
+
+The user/application will be responsible for redriving the extract until EOD. 


Instead of "EOD", I'd suggest "end of data" (we haven't defined this acronym anywhere, and it's not clear that we need to).

Thanks Dave, I've corrected. :)

DaveBirdsall · 2017-09-14T00:32:57Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+
+If the target file exists, Trafodion will truncate and write to it. 
+
+If the target file does not exist, create a table, then write to it. 


Suggest "If the target file does not exist, Trafodion will create it and write to it." (I assume Trafodion is creating the file here and not creating a table somewhere.)

Thanks Dave, I've corrected. :)

DaveBirdsall · 2017-09-14T00:33:23Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+
+If the target file exists, Trafodion will append to it. 
+
+If the target file does not exist, create a table, then write to it. 


Same comment as above

Thanks Dave, I've corrected. :)

DaveBirdsall · 2017-09-14T00:34:16Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+[#extract lob data into a user specified buffer]
+=== Extract Lob Data into a User Specified Buffer
+
+Extract from a LOB column into a target user buffer of a specified size. Trafodion engine will retrieve exactly the amount of requested data. The user/application will be responsible for redriving the extract until EOD. Then the extracted buffer data can be written to files or another location as the application chooses. 


Suggest "The Trafodion engine..."

Suggest also "end of data" instead of EOD.

Thanks Dave, I've corrected. :)

DaveBirdsall · 2017-09-14T00:36:00Z

docs/lob_guide/src/asciidoc/_chapters/work_with_lob.adoc

+
+Extract LOB data into a user specified buffer like a cursor until EOD is returned. For this method, the user specifies an input buffer and specifies the input length in an in/out variable. 
+
+For each execution of the extract, Trafodion will return SUCCESS, ERROR or EOD. 


Are SUCCESS, ERROR and EOD literals in some programming language (such as Java or C++)? Or is it a string that is returned by the Trafodion engine? Or something else? (The sqlci examples below don't mention SUCCESS, ERROR or EOD.) Seems like some additional explanation or examples are needed.

Suggestion from Sandhya:

Change
"For each execution of the extract, Trafodion will return SUCCESS, ERROR or EOD.
For the cases of SUCCESS or EOD, a length will also be returned to the user, so the user knows exactly how much data was actually extracted and returned. "

to

"Each successful execution of the extract will also return the length in the parameter specified by the caller, so the user knows how much data was actually extracted and returned. "

Traf-Jenkins · 2017-09-17T07:51:12Z

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/2058/

Traf-Jenkins · 2017-09-17T07:56:45Z

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/2058/

Traf-Jenkins · 2017-09-23T04:27:13Z

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/2076/

Traf-Jenkins · 2017-09-23T04:33:08Z

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/2076/

DaveBirdsall

+1 This looks great! Thanks for making this contribution. I think it is ready to merge.

add lob folder and 1st chapter

4c54b74

Add 2nd Chapter and 2 Sections

c79e5ae

liuyu000 changed the title [Trafodion-2734] Create *Trafodion LOB Guide* and Add 1st Chapter *Introduction* [Trafodion-2734] Create *Trafodion LOB Guide* Sep 11, 2017

Add Chapter 2, Section 2.3 & 2.4

5f9b3e9

Add Copyright

4282d41

Add Chapter 2, Section 2.5 & 2.6

9421e11

DaveBirdsall suggested changes Sep 12, 2017

View reviewed changes

Incorporate Comments 1 for SQL LOB Guide

3b704d7

DaveBirdsall suggested changes Sep 14, 2017

View reviewed changes

Incorporate Comments 2 for SQL LOB Guide

48fa8aa

liuyu000 changed the title [Trafodion-2734] Create *Trafodion LOB Guide* [TRAFODION-2734] Create *Trafodion LOB Guide* Sep 17, 2017

Incorporate Comments 3 for SQL LOB Guide

0c32cb7

DaveBirdsall approved these changes Sep 25, 2017

View reviewed changes

asfgit merged commit 0c32cb7 into apache:master Sep 27, 2017

liuyu000 changed the title [TRAFODION-2734] Create *Trafodion LOB Guide* [TRAFODION-2734] Create *Trafodion SQL Large Objects Guide* 1 Sep 28, 2017


		LOB (Large Object), a set of large object data types used to store large volumes of data, provides random and piece-wise access to the data.

		Database support for LOB is not universal.


		Stores unusually large amounts of string data in character set format. +

		CLOB is ideal for storing and processing semi-structured data, such as large document or string.


		The basic design idea is to allow a database row to have multiple LOB columns and a table to have multiple such rows. The design takes a combination approach of rows store and column store.

		When a SQL table with LOB column is created, there are several dependent objects that are created to hold metadata information on the LOB as well as the data.


		When a SQL table with LOB column is created, there are several dependent objects that are created to hold metadata information on the LOB as well as the data.

		The LOB data itself is stored in HDFS while the Trafodion table contains a unique LOB handle for each row/column. The LOB handle provides the identifier that is used to query the LOB metadata tables to find the location of the LOB data files.


		Extract LOB data into a user specified buffer like a cursor until EOD is returned. For this method, the user specifies an input buffer and specifies the input length in an in/out variable.

		For each execution of the extract, Trafodion will return SUCCESS, ERROR or EOD.

[TRAFODION-2734] Create *Trafodion SQL Large Objects Guide* 1 #1229

[TRAFODION-2734] Create *Trafodion SQL Large Objects Guide* 1 #1229

Conversation

liuyu000 commented Sep 10, 2017 • edited

Traf-Jenkins commented Sep 10, 2017

Traf-Jenkins commented Sep 10, 2017

Traf-Jenkins commented Sep 11, 2017

Traf-Jenkins commented Sep 11, 2017

Traf-Jenkins commented Sep 11, 2017

Traf-Jenkins commented Sep 11, 2017

liuyu000 commented Sep 12, 2017

Traf-Jenkins commented Sep 12, 2017

Traf-Jenkins commented Sep 12, 2017

Traf-Jenkins commented Sep 12, 2017

Traf-Jenkins commented Sep 12, 2017

Traf-Jenkins commented Sep 12, 2017

Traf-Jenkins commented Sep 12, 2017

DaveBirdsall left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuyu000 Sep 13, 2017 • edited

Choose a reason for hiding this comment

Traf-Jenkins commented Sep 13, 2017

Traf-Jenkins commented Sep 13, 2017

DaveBirdsall left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuyu000 Sep 21, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuyu000 Sep 14, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Traf-Jenkins commented Sep 17, 2017

Traf-Jenkins commented Sep 17, 2017

Traf-Jenkins commented Sep 23, 2017

Traf-Jenkins commented Sep 23, 2017

DaveBirdsall left a comment

Choose a reason for hiding this comment

[TRAFODION-2734] Create Trafodion SQL Large Objects Guide 1 #1229

[TRAFODION-2734] Create Trafodion SQL Large Objects Guide 1 #1229

liuyu000 commented Sep 10, 2017 •

edited

liuyu000 Sep 13, 2017 •

edited

liuyu000 Sep 21, 2017 •

edited

liuyu000 Sep 14, 2017 •

edited