Duaer

Integrations

Use Token Splitter in Duaer

In Duaer, Token Splitter is an AI sub-node. Connect it to the Text Splitter input of Default Data Loader. Split text into chunks by tokens.

Connect Token Splitter in Duaer

  1. On the Duaer canvas, select the Text Splitter input under Default Data Loader and pick Token Splitter.
  2. Check Chunk Size and Chunk Overlap on the node.
  3. Run Default Data Loader. Token Splitter runs as part of it, and its calls show under that node in the execution.

Token Splitter parameters in Duaer

  • Chunk Size: Maximum number of tokens per chunk.
  • Chunk Overlap: Number of tokens shared between consecutive chunks to preserve context.

Questions

Where does Token Splitter connect in Duaer?

In Duaer, Token Splitter connects to the Text Splitter input of Default Data Loader. It does not run on its own.

In this section