Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concise call method for RequireRecentData() subroutine #3

Open
JohnathonStoneMS opened this issue Jan 28, 2021 · 0 comments
Open

Concise call method for RequireRecentData() subroutine #3

JohnathonStoneMS opened this issue Jan 28, 2021 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@JohnathonStoneMS
Copy link
Contributor

If you want to check the recency of data before using that data to create a mapping table, you have to first load it into a non-mapping table. Then, you have to call the data recency subroutine and drop the temp table.

Example code:

[Data Recency Check]:
Load
    Min([Data Source Access Time]) AS [Data Source Access Time]
From [lib://Warehouse - Extracts/Foo.qvd] (qvd)
;

Let vIncludePath = 'lib://Qlik Utility Library/RequireRecentData.qvs';
$(Must_Include=$(vIncludePath));
Call RequireRecentData('Data Recency Check', 'Data Source Access Time', vDataRecencyThreshold, vAllowStaleData);

Drop Table [Data Recency Check];

Since these tables are usually named Data Recency Check and the metadata field containing the access timestamp is usually named Data Source Access Time, you should be able to invoke the policy by providing only the data source file path.

Desired call structure:

Let vSourceQVDPath = 'lib://Warehouse - Extracts/Foo.qvd';
Call RequireRecentDataFromQVD(vSourceQVDPath);

In this shorter-form call, the invoked subroutine will make use of environmental variables: vDataRecencyThreshold and vAllowStaleData. By default, the environment-wide configuration system controls the values of these variables. They can also be overwritten at the app-level. Finally, it would automatically use the table name, Data Recency Check, and assume the timestamp field name is Data Source Access Time.

The new subroutine, RequireRecentDataFromQVD(), should invoke RequireRecentData().

One last request: an optional 2nd parameter, an override value for the timestamp field name, to allow use of the concise call method when the timestamp field has been aliased.

@JohnathonStoneMS JohnathonStoneMS self-assigned this Jan 28, 2021
@JohnathonStoneMS JohnathonStoneMS added the enhancement New feature or request label Jan 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant