Skip to content

[Documentation] Update xtokenize-language.adoc #25977

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
davsclaus merged 3 commits into from
Sep 1, 2026
+246 −14
Merged

[Documentation] Update xtokenize-language.adoc #25977

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
59513bb
Update xtokenize-language.adoc
skin27 Aug 31, 2026
d7ffe99
Apply suggestion from @davsclaus
skin27 Aug 31, 2026
3204492
Update xtokenize-language.adoc
skin27 Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 123 additions & 7 deletions ...g/src/generated/resources/org/apache/camel/catalog/docs/xtokenize-language.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,32 @@

*Since Camel {since}*

The XML Tokenize language is a built-in language in `camel-stax`, which
is a truly XML-aware tokenizer that can be used with the xref:eips:split-eip.adoc[Split] EIP
as the conventional xref:tokenize-language.adoc[Tokenize] to efficiently and
effectively tokenize XML documents.
The XML Tokenize language (xtokenize) is a tokenizer specifically designed for
XML documents. Unlike the conventional xref:tokenize-language.adoc[Tokenize language], which is primarily a
text-based tokenizer, XML Tokenize uses a StAX parser to interpret the XML
structure while producing tokens.

XML Tokenize is capable of not only recognizing XML namespaces and hierarchical structures of the document
but also more efficiently tokenizing XML documents than the conventional xref:tokenize-language.adoc[Tokenize] language.
The conventional Tokenize language also provides an xml option for XML-aware
tokenization (xml=true). This should not be confused with XML Tokenize: xtokenize
uses different tokenization semantics and is intended for different use cases.

Use xtokenize when XML structure or namespaces are important, or when you want
a tokenizer specifically designed for XML documents. Use the conventional
Tokenize language when you primarily need text-based
tokenization and XML awareness is sufficient for your use case.

Maven users will need to add the following dependency to their `pom.xml`
for this language:

[source,xml]
----
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-stax</artifactId>
<version>x.x.x</version>
<!-- use the same version as your Camel core version -->
</dependency>
----

== XML Tokenizer Options

Expand All @@ -25,4 +44,101 @@ include::partial$language-options.adoc[]

== Example

See xref:eips:split-eip.adoc[Split EIP], which has examples using the XML Tokenize language.
Suppose the input XML contains multiple orders:

[source,xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<orders xmlns="urn:shop">
<order>
<id>1001</id>
<customer>John</customer>
</order>
<order>
<id>1002</id>
<customer>Jane</customer>
</order>
</orders>
----

The XML Tokenize language can be used with the Split EIP to split the document
into one message for each `order` element.

[tabs]
====
Java::
+
[source,java]
----
var ns = new org.apache.camel.support.builder.Namespaces("shop", "urn:shop");

from("direct:start")
.split()
.xtokenize("//shop:order", 'i', ns)
.streaming()
.to("mock:order");
----

XML::
+
[source,xml]
----
<route>
<from uri="direct:start"/>
<split streaming="true">
<xtokenize>//shop:order
<namespace key="shop" value="urn:shop"/>
</xtokenize>
<to uri="mock:order"/>
</split>
</route>
----

YAML::
+
[source,yaml]
----
- route:
from:
uri: 'direct:start'
steps:
- split:
streaming: 'true'
expression:
xtokenize:
expression: '//shop:order'
namespace:
- key: shop
value: 'urn:shop'
steps:
- to:
uri: 'mock:order'
----
====

The Split EIP produces two messages. The body of the first message is:

.Output message 1
[source,xml]
----
<order xmlns="urn:shop">
<id>1001</id>
<customer>John</customer>
</order>
----

The body of the second message is:

.Output message 2
[source,xml]
----
<order xmlns="urn:shop">
<id>1002</id>
<customer>Jane</customer>
</order>
----

The `shop` namespace is mapped to `urn:shop`, allowing the XPath expression
`//shop:order` to identify the namespaced `order` elements.

See the xref:eips:split-eip.adoc[Split EIP] for more examples.
130 changes: 123 additions & 7 deletions components/camel-stax/src/main/docs/xtokenize-language.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,32 @@

*Since Camel {since}*

The XML Tokenize language is a built-in language in `camel-stax`, which
is a truly XML-aware tokenizer that can be used with the xref:eips:split-eip.adoc[Split] EIP
as the conventional xref:tokenize-language.adoc[Tokenize] to efficiently and
effectively tokenize XML documents.
The XML Tokenize language (xtokenize) is a tokenizer specifically designed for
XML documents. Unlike the conventional xref:tokenize-language.adoc[Tokenize language], which is primarily a
text-based tokenizer, XML Tokenize uses a StAX parser to interpret the XML
structure while producing tokens.

XML Tokenize is capable of not only recognizing XML namespaces and hierarchical structures of the document
but also more efficiently tokenizing XML documents than the conventional xref:tokenize-language.adoc[Tokenize] language.
The conventional Tokenize language also provides an xml option for XML-aware
tokenization (xml=true). This should not be confused with XML Tokenize: xtokenize
uses different tokenization semantics and is intended for different use cases.

Use xtokenize when XML structure or namespaces are important, or when you want
a tokenizer specifically designed for XML documents. Use the conventional
Tokenize language when you primarily need text-based
tokenization and XML awareness is sufficient for your use case.

Maven users will need to add the following dependency to their `pom.xml`
for this language:

[source,xml]
----
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-stax</artifactId>
<version>x.x.x</version>
<!-- use the same version as your Camel core version -->
</dependency>
----

== XML Tokenizer Options

Expand All @@ -25,4 +44,101 @@ include::partial$language-options.adoc[]

== Example

See xref:eips:split-eip.adoc[Split EIP], which has examples using the XML Tokenize language.
Suppose the input XML contains multiple orders:

[source,xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<orders xmlns="urn:shop">
<order>
<id>1001</id>
<customer>John</customer>
</order>
<order>
<id>1002</id>
<customer>Jane</customer>
</order>
</orders>
----

The XML Tokenize language can be used with the Split EIP to split the document
into one message for each `order` element.

[tabs]
====
Java::
+
[source,java]
----
var ns = new org.apache.camel.support.builder.Namespaces("shop", "urn:shop");

from("direct:start")
.split()
.xtokenize("//shop:order", 'i', ns)
.streaming()
.to("mock:order");
----

XML::
+
[source,xml]
----
<route>
<from uri="direct:start"/>
<split streaming="true">
<xtokenize>//shop:order
<namespace key="shop" value="urn:shop"/>
</xtokenize>
<to uri="mock:order"/>
</split>
</route>
----

YAML::
+
[source,yaml]
----
- route:
from:
uri: 'direct:start'
steps:
- split:
streaming: 'true'
expression:
xtokenize:
expression: '//shop:order'
namespace:
- key: shop
value: 'urn:shop'
steps:
- to:
uri: 'mock:order'
----
====

The Split EIP produces two messages. The body of the first message is:

.Output message 1
[source,xml]
----
<order xmlns="urn:shop">
<id>1001</id>
<customer>John</customer>
</order>
----

The body of the second message is:

.Output message 2
[source,xml]
----
<order xmlns="urn:shop">
<id>1002</id>
<customer>Jane</customer>
</order>
----

The `shop` namespace is mapped to `urn:shop`, allowing the XPath expression
`//shop:order` to identify the namespaced `order` elements.

See the xref:eips:split-eip.adoc[Split EIP] for more examples.