Passing parameters to magiclinks

[arminus]

First of all, thanks for the very helpful https://docs.xyops.io/#Docs/compare page!

As an n8n user, I have one question though: I have a set of n8n workflows which get triggered by webhooks which can also receive parameters in n8n as well as return data. I kind of miss this in xycop's magiclink concept. Is this something you plan or does that not fit your strategy?

[nickdollimount]

Hey @arminus,

You can send data to Magic Links now. You can either pass the parameters through the Query Parameters in the URL on a GET request or pass them in the body as JSON in a POST request.

Examples (PowerShell):

GET Request

Invoke-RestMethod -Uri 'https://xyops.domain.com/api/app/magic/v1/94XlHVdhN5j5wbW5J3Lu4eGBwZhqwerdyJCFjXZp268FWNoPJL2ryYs6goae0wNAF?firstname=John&lastname=Doe'

POST Request

Invoke-RestMethod -Uri 'https://xyops.domain.com/api/app/magic/v1/94XlHVdhN5j5wbW5J3Lu4eGBwZhqwerdyJCFjXZp268FWNoPJL2ryYs6goae0wNAF' -Method Post -Body (@{firstname = 'John'; lastname = 'Doe'} | ConvertTo-Json -Depth 100) -ContentType 'application/json'

The results provide an id and stream fields. These can be used with the stream_job API to stream the job updates from the server.

References:
https://docs.xyops.io/#Docs/api/magic
https://docs.xyops.io/#Docs/api/stream_job

Though I haven't been able to find anything regarding the job just returning a custom result. There is a bit about setting a custom success result but it's not programmatic based on your script itself, but instead it's instant and the job just runs in the background.

https://docs.xyops.io/#Docs/triggers/magic-link

I hope this helps. It would be a good feature request to allow the job to return custom HTTP results at the end of the script run to allow returning of data.

[jhuckaby]

Ah, Nick covered this quite well.

One additional detail I just want to add about returning data: the stream_job API actually includes the final job output in its SSE stream, including the job's custom output data.

When the job completes, the final update event contains the job’s code, description, elapsed, and, when applicable, its output data and files. This is followed by an empty end event.

This means the calling application can keep the HTTP connection open, process updates as they arrive, and then use the data object from the completed update as the returned result. Functionally, this should provide the same capability as returning job data in an n8n webhook response, except that xyOps delivers it through an SSE response.

Here is a small Node.js example which calls magic to launch the job, and then streams the job response with stream_job:

const magicLinkUrl = 'https://xyops.example.com/api/app/magic/v1/YOUR_MAGIC_LINK_TOKEN';

// Add optional event parameters to the Magic Link URL.
const magicUrl = new URL(magicLinkUrl);
magicUrl.searchParams.set('firstname', 'John');
magicUrl.searchParams.set('lastname', 'Doe');

// Start the job.
const magicResponse = await fetch(magicUrl);

if (!magicResponse.ok) {
	throw new Error(`Magic Link request failed: HTTP ${magicResponse.status}`);
}

const job = await magicResponse.json();

if (job.code !== 0) {
	throw new Error(job.description || 'Failed to start job');
}

// Connect to the SSE stream using the returned job ID and stream token.
const streamUrl = new URL('/api/app/stream_job/v1', magicUrl.origin);
streamUrl.searchParams.set('id', job.id);
streamUrl.searchParams.set('token', job.stream);

const streamResponse = await fetch(streamUrl);

if (!streamResponse.ok) {
	throw new Error(`Job stream failed: HTTP ${streamResponse.status}`);
}

const decoder = new TextDecoder();
let buffer = '';
let jobOutput;

for await (const chunk of streamResponse.body) {
	buffer += decoder.decode(chunk, { stream: true });

	const messages = buffer.split(/\r?\n\r?\n/);
	buffer = messages.pop();

	for (const message of messages) {
		const event = message.match(/^event:\s*(.+)$/m)?.[1];
		const json = message.match(/^data:\s*(.+)$/m)?.[1];

		if (!json) continue;

		const data = JSON.parse(json);

		// Display each SSE update as it arrives.
		console.log(event, data);

		// The completed update contains the job's final output.
		if (event === 'update' && Object.hasOwn(data, 'completed')) {
			jobOutput = data.data;
		}
	}
}

console.log('Job output:', jobOutput);

Hope this helps!

[arminus]

Thanks a lot for the explanations and also the example!

"My problem" is that this relies on the caller to implement its own async waiting loop, which is ok if the caller is implemented in code, but is a showstopper if called from something like https://grafana.com/grafana/plugins/yesoreyeram-infinity-datasource/ or similar.

Example: I have an n8n flow which provides a webhook to return the number of failed events in the last x hours as a simple number response. That flow calls the xyops API /api/app/search_jobs/v1, parses the response and returns a simple number for consumption through the Respond to webhook node. This is just an example, I'm actually looking into ways to potentially port (and centralize in xyops) "stuff" I run elsewhere.

[jhuckaby]

Ah, got it. Thank you for the grafana example. I understand the distinction now.

Yeah, so the current xyOps flow requires two requests:

1. Call the Magic Link API, which launches the job and immediately returns its ID and stream token.
2. Connect to stream_job and consume the SSE stream until the final job output arrives.

While stream_job does contain the final output data, it still requires the caller to understand SSE and manage that second, long-lived request. That does not help clients such as the Grafana Infinity data source, which expect a single conventional HTTP request followed by a regular response.

What you are describing is effectively a synchronous job HTTP API endpoint: launch a job, wait for it to complete, and then return its output data directly in the HTTP response. That is comparable to n8n's "Respond to Webhook" behavior.

xyOps does not currently have an API that combines those steps into one request. I agree that this would be useful, particularly for short-running jobs used as lightweight API endpoints. We would need to account for normal HTTP timeout limits, but the use case makes sense and fits the overall strategy.

Adding to the TODO list! I'll try to get this into the next release.

[jhuckaby]

Added in xyOps v1.0.88 🚀

Check out the updated API docs to see how to use the feature:

• run_event
• magic

Basically, just add /wait to the URLs, and both of the APIs will "wait" for the job to complete before returning, AND the HTTP response will contain the FULL Job object, including all output data and files 👍🏻