Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/candidate-7.4.x'
Browse files Browse the repository at this point in the history 
Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
  • Loading branch information
@ghalliday
ghalliday committed Aug 29, 2019
2 parents eb49aca + d1d9553 commit 3199443
Show file tree
Hide file tree
Showing 108 changed files with 2,504 additions and 767 deletions.
5 changes: 4 additions & 1 deletion common/fileview2/fvsource.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -460,7 +460,10 @@ unsigned DataSourceMetaData::numColumns() const

void DataSourceMetaData::patchIndexFileposition()
{
fields.tos().type.setown(makeSwapIntType(8, false));
DataSourceMetaItem & lastfield = fields.tos();
//MORE: An approximation to check for fileposition(false) - should switch to using new meta info.
if (lastfield.type->isInteger())
lastfield.type.setown(makeSwapIntType(8, false));
}

ITypeInfo * DataSourceMetaData::queryType(unsigned column) const
Expand Down
8 changes: 8 additions & 0 deletions common/thorhelper/thorcommon.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2034,6 +2034,14 @@ static IOutputMetaData *_getDaliLayoutInfo(MemoryBuffer &layoutBin, IPropertyTre
props.getPropBin("_record_layout", mb);
expr.setown(patchEclRecordDefinitionFromRecordLayout(expr, mb));
}
else if (!expr->hasAttribute(_payload_Atom))
{
//Very old records before _record_layout was added to the meta information (November 2006!)
IHqlExpression * lastField = queryLastField(expr);
if (lastField && lastField->queryType()->isInteger())
expr.setown(prependOwnedOperand(expr, createAttribute(_payload_Atom, createConstant(1))));
}

if (exportBinaryType(layoutBin, expr, isIndex))
return createTypeInfoOutputMetaData(layoutBin, isGrouped);
}
Expand Down
2 changes: 1 addition & 1 deletion dali/dfu/dfurun.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1230,7 +1230,7 @@ class CDFUengine: public CInterface, implements IDFUengine
srcName.set(tmp);
srcFdesc.setown(srcFile->getFileDescriptor());
iskey = isFileKey(srcFile);
if ((cmd==DFUcmd_copy) && (srcFile->querySuperFile() != nullptr) && iskey)
if ((cmd==DFUcmd_copy) && (srcFile->querySuperFile() != nullptr) && (srcFile->querySuperFile()->numSubFiles() > 1) && iskey)
throwError1(DFTERR_InvalidSuperindexCopy, srcName.str());

oldRoxiePrefix.set(srcFile->queryAttributes().queryProp("@roxiePrefix"));
Expand Down
68 changes: 34 additions & 34 deletions docs/EN_US/ConfiguringHPCC/ConfiguringHPCC.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<book xml:base="../">
<bookinfo>
<title>Configuring the HPCC System</title>
<title>Configuring the HPCC Systems® Platform</title>

<mediaobject>
<imageobject>
Expand Down Expand Up @@ -65,7 +65,7 @@
<title>Using Configuration Manager</title>

<para>Configuration Manager is the utility with which we configure the
HPCC platform. The HPCC platform's configuration is stored in an XML file
HPCC Systems platform. The HPCC Systems platform's configuration is stored in an XML file
named <emphasis role="bold">environment.xml</emphasis>. When you install a
package, a default single-node environment.xml is generated. After that,
you can use the Configuration Manager to modify it and add nodes and
Expand All @@ -81,16 +81,16 @@
View, it is a good idea to start with a wizard generated configuration and
use Advanced View to finish it.</para>

<para>This document will guide you through configuring an HPCC environment
<para>This document will guide you through configuring an HPCC Systems environment
using the Configuration Manager.</para>

<sect1 id="configuring-a-multi-node-system">
<title>Running the Configuration Manager</title>

<para>This document will guide you through configuring an HPCC
<para>This document will guide you through configuring an HPCC Systems
environment using the Configuration Manager.</para>

<para>The HPCC package should already be installed on ALL nodes.</para>
<para>The HPCC Systems package should already be installed on ALL nodes.</para>

<para>You can use any tool or shell script you choose.</para>

Expand Down Expand Up @@ -123,7 +123,7 @@
</listitem>
</orderedlist>

<para>There are different ways to configure your HPCC system. You can
<para>There are different ways to configure your HPCC Systems platform. You can
use the <emphasis role="bold">Generate environment wizard</emphasis> and
use that environment or experienced users can then use the <emphasis
role="bold">Advanced View</emphasis> for more specific customization.
Expand Down Expand Up @@ -159,8 +159,8 @@
<listitem>
<para>Press the Next button.</para>

<para>Next you will need to define the IP addresses that your HPCC
system will be using.</para>
<para>Next you will need to define the IP addresses that your HPCC Systems
platform will be using.</para>
</listitem>

<listitem>
Expand Down Expand Up @@ -284,7 +284,7 @@
<row>
<entry><inlinegraphic fileref="images/OSSgr3.png" /></entry>

<entry>Keep in mind, that your HPCC configuration may be
<entry>Keep in mind, that your HPCC Systems configuration may be
different depending on your needs. For example, you may not
need a Roxie or you may need several smaller Roxie clusters.
In addition, in a production [Thor] system, you would ensure
Expand All @@ -304,9 +304,9 @@
<listitem>
<?dbfo keep-together="always"?>

<para>Stop the HPCC system.</para>
<para>Stop the HPCC Systems platform.</para>

<para>If it is running stop the HPCC system (on every node),
<para>If it is running stop the HPCC Systems platform (on every node),
using a command such as this:</para>

<xi:include href="Installing_and_RunningTheHPCCPlatform/Inst-Mods/SysDStop.xml"
Expand All @@ -320,7 +320,7 @@
<listitem>
<para>You may have a multi-node system and a custom script
such as the one illustrated in Appendix of the <emphasis
role="bluebold">Installing and Running the HPCC
role="bluebold">Installing and Running the HPCC Systems
Platform</emphasis> document to start and stop your
system. If that is the case please use the appropriate
command for stopping your system on every node.</para>
Expand All @@ -339,7 +339,7 @@
<entry><inlinegraphic
fileref="images/caution.png" /></entry>

<entry>Be sure HPCC is stopped before attempting to
<entry>Be sure the HPCC Systems platform is stopped before attempting to
copy the environment.xml file.</entry>
</row>
</tbody>
Expand Down Expand Up @@ -395,13 +395,13 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
<para>You may want to use a script to push out the XML file to
all nodes. See the <emphasis>Example Scripts</emphasis> section
in the Appendix of the <emphasis role="bluebold">Installing and
Running the HPCC Platform</emphasis> document. You can use the
Running the HPCC Systems Platform</emphasis> document. You can use the
scripts as a model to create your own script to copy the
environment.xml file out to all your nodes.</para>
</listitem>

<listitem>
<para>Restart the HPCC platform on all nodes.</para>
<para>Restart the HPCC Systems platform on all nodes.</para>
</listitem>
</orderedlist>
</sect3>
Expand Down Expand Up @@ -432,7 +432,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
</listitem>

<listitem>
<para>If it is running, stop the HPCC system using this command on
<para>If it is running, stop the HPCC Systems platform using this command on
every node:</para>

<xi:include href="Installing_and_RunningTheHPCCPlatform/Inst-Mods/SysDStop.xml"
Expand All @@ -448,7 +448,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
to use a script to perform this step. See the
<emphasis>Example Scripts</emphasis> section in the Appendix
of the <emphasis role="bluebold">Installing and Running the
HPCC Platform</emphasis> document.</para>
HPCC Systems Platform</emphasis> document.</para>
</listitem>
</varlistentry>
</variablelist>
Expand All @@ -466,7 +466,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
<entry><inlinegraphic
fileref="images/OSSgr3.png" /></entry>

<entry>You can use this command to confirm HPCC processes
<entry>You can use this command to confirm HPCC Systems processes
are stopped: <para><programlisting>sudo systemctl status hpccsystems-platform.target</programlisting></para></entry>
</row>
</tbody>
Expand Down Expand Up @@ -698,7 +698,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
<title>Software Section</title>

<para>Use the software components section to configure software
components of the HPCC platform. Most software components are actual
components of the HPCC Systems platform. Most software components are actual
running processes; however, some are just definitions used by the
system. These definitions are used by the configuration
generator.</para>
Expand Down Expand Up @@ -967,7 +967,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
<entry>keyspace</entry>

<entry>The name of the Cassandra keyspace to use for the
HPCC data store. The default is
HPCC Systems data store. The default is
<emphasis>hpcc</emphasis>.</entry>
</row>

Expand Down Expand Up @@ -1232,7 +1232,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
<title>Drop Zone</title>

<para>A Drop Zone (or landing zone) is a location where files can be
transferred to or from your HPCC system. The drop zone is a logical
transferred to or from your HPCC Systems installation. The drop zone is a logical
combination of a path and one or more servers.</para>

<para>Multiple drop zones allow you to configure different top level
Expand Down Expand Up @@ -1842,9 +1842,9 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
integration</emphasis>
</para>

<para>HPCC provides a mechanism to integrate ELK (ElasticSearch,
<para>HPCC Systems provides a mechanism to integrate ELK (ElasticSearch,
Logstash, and Kibana) based visualizations into ECL Watch. This
feature can be configured and enabled through the HPCC Configuration
feature can be configured and enabled through the HPCC Systems Configuration
manager.</para>

<para>To configure the ELK Log Visualization component, click and
Expand Down Expand Up @@ -2010,7 +2010,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
LDAPServer Installation in ConfigManager. For a complete description
of how to add LDAP Authentication see <emphasis>Using LDAP
Authentication</emphasis> section in the <emphasis
role="bluebold">Installing and Running The HPCC Platform</emphasis>
role="bluebold">Installing and Running The HPCC Systems Platform</emphasis>
document.</para>

<para>
Expand All @@ -2031,7 +2031,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
added the LDAP computers in the Hardware section. For a complete
description of how to add LDAP Authentication see <emphasis>Using
LDAP Authentication</emphasis> section in the <emphasis
role="bluebold">Installing and Running The HPCC Platform</emphasis>
role="bluebold">Installing and Running The HPCC Systems Platform</emphasis>
document.</para>

<orderedlist>
Expand All @@ -2053,12 +2053,12 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
</sect3>

<sect3>
<title>The HPCC Administrator's account</title>
<title>The HPCC Systems Administrator's account</title>

<para>You can set up an HPCC Administrator's account with limited
administrative rights. The HPCC Administrator does not have to be an
LDAP Administrator's account. This allows the HPCC Administrator to
be able to set up users, groups, and set permissions in HPCC without
<para>You can set up an HPCC Systems Administrator's account with limited
administrative rights. The HPCC Systems Administrator does not have to be an
LDAP Administrator's account. This allows the HPCC Systems Administrator to
be able to set up users, groups, and set permissions in HPCC Systems without
having rights to perform other LDAP administrative functions. To use
this feature: <orderedlist>
<listitem>
Expand All @@ -2067,14 +2067,14 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
</listitem>

<listitem>
<para>In the HPCC configuration manager navigate to this (LDAP
Server Process) page and enter the HPCC Administrator group
<para>In the HPCC Systems configuration manager navigate to this (LDAP
Server Process) page and enter the HPCC Systems Administrator group
name as the value in the <emphasis
role="bold">adminGroupName</emphasis> field.</para>
</listitem>

<listitem>
<para>Add (HPCC Administrator) users to this new group.</para>
<para>Add (HPCC Systems Administrator) users to this new group.</para>
</listitem>

<listitem>
Expand Down Expand Up @@ -2897,7 +2897,7 @@ sudo -u hpcc cp /etc/HPCCSystems/source/NewEnvironment.xml /etc/HPCCSystems/envi
</listitem>

<listitem>
<para>Restart the HPCC system</para>
<para>Restart the HPCC Systems platform</para>

<para>You might prefer to script this process, especially if you
have many nodes. See the Example Scripts section in the Appendix
Expand Down
4 changes: 2 additions & 2 deletions docs/EN_US/DynamicESDL/DynamicESDL_Includer.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@

<itemizedlist>
<listitem>
<para>Access to an HPCC Cluster (version 7.0 or later). This can be
<para>Access to an HPCC Systems Cluster (version 7.0 or later). This can be
one running in a Virtual Machine.</para>
</listitem>
</itemizedlist>
Expand Down Expand Up @@ -134,7 +134,7 @@

<para><emphasis role="bold">dESDL and LDAP Security</emphasis></para>

<para>If your HPCC platform is configured to use LDAP security, you must
<para>If your HPCC Systems platform is configured to use LDAP security, you must
ensure any user who will be publishing ESDL Definitions has Access to
<emphasis role="bold">ESDL configuration service</emphasis> set to
<emphasis role="bold">Allow Full</emphasis>, as shown below.</para>
Expand Down
2 changes: 1 addition & 1 deletion docs/EN_US/ECLLanguageReference/ECLR_mods/BltInFunc-PIPE.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@

<entry>The name of a program to execute, which must take any input
data through stdin and produce its output through stdout. This
program must have already been deployed on the HPCC cluster in the
program must have already been deployed on the HPCC Systems cluster in the
Thor instance directory (such as: /var/lib/HPCCSystems/mythor/) but that can be
overridden by the externalProgDir environment setting for the Thor
cluster). </entry>
Expand Down
2 changes: 1 addition & 1 deletion docs/EN_US/ECLLanguageReference/ECLR_mods/Templ-OVERVIEW.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<primary>Template Language</primary>
</indexterm> Overview</title>

<para>ECL was created to be the programming language for all of our HPCC
<para>ECL was created to be the programming language for all of our HPCC Systems
technology. Therefore, it must be able to meet all the demands of a complete
business solution: from data ingest, through querying and processing, and
all the way to fulfillment and customer output.</para>
Expand Down
4 changes: 2 additions & 2 deletions docs/EN_US/ECLPlayground/ECLPlay-Mods/ECL_Playground.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@

<para>ECL Playground is a tool hosted on an ESP server. A page runs in
your browser, allowing you to access and execute self-contained ECL code
on your HPCC system without the use of any other tools. The ECL
on your HPCC Systems platform without the use of any other tools. The ECL
Playground then shows you the results and the graph in your browser. The
view is very similar to what the ECL IDE displays.</para>

<sect2 id="ECLPlayground_AccessingThePlayground">
<title>Accessing ECL Playground</title>

<para>ECL Playground is installed with the HPCC platform. You can
<para>ECL Playground is installed with the HPCC Systems platform. You can
access it through the ECL Watch page.</para>

<para><orderedlist>
Expand Down
2 changes: 1 addition & 1 deletion docs/EN_US/ECLProgrammersGuide/PRG_Mods/CodeSign.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@
unchanged.</para>

<para>We have signed the SERVICE definitions provided by the ECL standard
plugins and included the public key in the HPCC platform installation.
plugins and included the public key in the HPCC Systems platform installation.
Code that tries to use service definitions that are signed will continue
to work as before but, code that tries to call arbitrary library functions
using user-supplied SERVICE definitions will give compile errors, if the
Expand Down
2 changes: 1 addition & 1 deletion docs/EN_US/ECLProgrammersGuide/PRG_Mods/PrG_Automated_ECL.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ B 3

<para>How you create this text file is up to you. To fully automate the
process you may want to write a daemon application that watches a
directory (such as your HPCC environment's landing zone) to detect new
directory (such as your HPCC Systems environment's landing zone) to detect new
files dropped in (by whatever means) and generate the appropriate ECL code
file to process that new file in some standard fashion (typically using
MACRO calls), then execute it from ECLplus command line as described
Expand Down
8 changes: 4 additions & 4 deletions docs/EN_US/ECLProgrammersGuide/PRG_Mods/PrG_Roxie_Overview.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,17 +16,17 @@
<row>
<entry><emphasis role="bold">Thor</emphasis></entry>

<entry>An HPCC cluster specifically designed to perform massive
<entry>An HPCC Systems cluster specifically designed to perform massive
data manipulation (ETL) processes. This is a back-office data
preparation tool and not meant for end-user production-level
queries. See the HPCC operational manuals for complete
queries. See the HPCC Systems operational manuals for complete
documentation.</entry>
</row>

<row>
<entry><emphasis role="bold">Roxie</emphasis></entry>

<entry>An HPCC cluster specifically designed to service standard
<entry>An HPCC Systems cluster specifically designed to service standard
queries, providing a throughput rate of a thousand-plus responses
per second (actual response rate for any given query is, of
course, dependent on its complexity). This is a production-level
Expand All @@ -40,7 +40,7 @@
<entry>An R&amp;D platform designed for iterative, interactive
development and testing of Roxie queries. This is not a separate
cluster, but a "piggyback" implementation of ECL Agent and Thor.
See the HPCC operational manuals for complete
See the HPCC Systems operational manuals for complete
documentation.</entry>
</row>
</tbody>
Expand Down
Loading

0 comments on commit 3199443

Please sign in to comment.