feat(data): use modern data format and truncate result if too large

[tripodsan]

fixes #119

BREAKING CHANGE: the response body is no longer a JSON array but an object that contains a limit, offset, total and data property.

see examples:

helix-data-embed/test/index.test.js

Lines 78 to 158 in e84a023

	describe('Index result Tests', () => {
	it('handles limit correctly', async () => {
	const { main: customMain } = proxyquire('../src/index.js', {
	'./embed.js': () => ({
	body: TEST_DATA,
	}),
	});
	const result = await customMain({
	src: 'https://foo.com',
	'hlx_p.limit': 10,
	});
	assert.deepEqual(result, {
	body: {
	data: TEST_DATA.slice(0, 10),
	limit: 10,
	offset: 0,
	total: 10000,
	},
	});
	});
	it('handles offset correctly', async () => {
	const { main: customMain } = proxyquire('../src/index.js', {
	'./embed.js': () => ({
	body: TEST_DATA,
	}),
	});
	const result = await customMain({
	src: 'https://foo.com',
	'hlx_p.offset': 9000,
	});
	assert.deepEqual(result, {
	body: {
	data: TEST_DATA.slice(9000),
	limit: 1000,
	offset: 9000,
	total: 10000,
	},
	});
	});
	it('handles limit and offset correctly', async () => {
	const { main: customMain } = proxyquire('../src/index.js', {
	'./embed.js': () => ({
	body: TEST_DATA,
	}),
	});
	const result = await customMain({
	src: 'https://foo.com',
	'hlx_p.limit': 50,
	'hlx_p.offset': 100,
	});
	assert.deepEqual(result, {
	body: {
	data: TEST_DATA.slice(100, 150),
	limit: 50,
	offset: 100,
	total: 10000,
	},
	});
	});
	it('truncates result if too large for action response', async () => {
	const { main: customMain } = proxyquire('../src/index.js', {
	'./embed.js': () => ({
	body: TEST_DATA,
	}),
	});
	const result = await customMain({
	src: 'https://foo.com',
	});
	assert.deepEqual(result, {
	body: {
	data: TEST_DATA.slice(0, 4970),
	limit: 4970,
	offset: 0,
	total: 10000,
	},
	});
	});
	});

[github-actions]

This PR will trigger a major release when merged.

[kptdobe]

Since this is a breaking change and I suspect it is consumed by theblog, I think we should create the follow tickets in the various consumer repos (if there are any...)

[tripodsan]

Since this is a breaking change and I suspect it is consumed by theblog, I think we should create the follow tickets in the various consumer repos (if there are any...)

yes, we should. we can also prepare the js clients:

const data = Array.isArray(response) ? response : response.data;

It will also effect the data-embed in the pipeline.

[trieloff]

Not a fan of breaking compatibility and complicating the format, but if it works, ok. The helix-pipeline fix should make sure to be able to handle both the old and the new format.

[codecov]

Codecov Report

Merging #136 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master      #136   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           11        11           
  Lines          320       331   +11     
=========================================
+ Hits           320       331   +11     

Impacted Files	Coverage Δ
src/index.js	100.00% <100.00%> (ø)
src/querybuilder/filter.js	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c43d03...02f4351. Read the comment docs.

[trieloff]

🎉 This PR is included in version 2.0.0 🎉

The release is available on:

• v2.0.0
• GitHub release

Your semantic-release bot 📦🚀