Build
Add a process source in Duaer
A process source tells Duaer where to read the steps of your cases: a CSV or XLSX file, files dropped in an S3 or SharePoint folder, PostgreSQL, MySQL, Microsoft SQL, Oracle, TiDB, OceanBase, KingbaseES, SAP HANA, IBM Db2, Dameng DM8, Teradata, Vertica, Snowflake, BigQuery, Databricks, Microsoft Fabric, Amazon Redshift, Azure Synapse Analytics, ClickHouse, Trino, a Feishu Base, or Feishu, DingTalk or WeCom approvals. The wizard takes you through columns, mapping, a preview of issues and a schedule; after you confirm, Duaer reads new rows on that schedule.
Open the wizard
- As an owner or admin, open Data sources in the sidebar: it lists the organization’s knowledge sources and every project’s process sources together, those that need attention first; select a row to see its details, Read now, Pause or Resume and its runs in a side panel. Select Add source, choose the system (for one that is also a knowledge source, such as Salesforce, choose Process source under Add as) and the project, then select Continue; the wizard opens at that system’s form. You can also go to your project › Data tables › Duaer cases › Process sources.
- From the project, select Add source and pick where the data is from the catalog. By process comes first: Purchase to pay (SAP S/4HANA, NetSuite, Kingdee Cloud Galaxy, Yonyou BIP, Yonyou U8, Business Central, Workday, Coupa), Order to cash (SAP S/4HANA, NetSuite, Kingdee Cloud Galaxy, Yonyou BIP, Yonyou U8, Business Central), Accounts payable (Xero bills, QuickBooks bills) and Accounts receivable (Xero sales invoices, QuickBooks sales invoices); then Files, Databases, Online spreadsheets, ERP and accounting, CRM, Approvals, API and Feishu. Type in Search systems to find one by name. Your choice shows at the top of the form; select Change to go back to the catalog.
- The wizard has six steps: Connect, Columns, Mapping, Preview, Schedule and audience, Confirm. Select a finished step to go back to it. Only owners and admins can add a source. SAP S/4HANA, Xero, QuickBooks, HubSpot and Salesforce have a fixed mapping: Next on Connect saves the source, previews it and, when nothing blocks, goes straight to Confirm with a read every 60 minutes and owners and admins as the audience. The skipped steps stay open from the step bar.
Connect: where the data is
The source name is filled in from the file or table you choose; you can change it, using letters in any language, digits, spaces, dot, dash or underscore. Then pick the time zone of the times in the data from the list; it starts as your browser’s.
- Not sure what to fill in: once you choose a system, Where to find it under the form says where to find each value; when the connection fails, the cause and the fix show under the error in plain words. Owners and admins can also have Duaer Designer guide them: on Data sources, select Set up in Duaer Designer beside Add source, or the same button beside Continue once a system is chosen; Where to find it has it too. The Designer says what to prepare and where to find each value, and gives the link to the wizard. Fill in passwords and keys only in the wizard, never in the conversation. Once you tell it the connection is saved, it checks how many cases it would read and which kind of process they follow and, with no blocking issue, shows a card; Allow once starts reading every 60 minutes, seen by owners and admins. When the table holds no process, the Designer says so and shows no card.
- CSV, XLSX or OCEL 2.0 file: upload a file of up to 20 MB; for XLSX, choose a sheet. OCEL 2.0 is the standard format for object-centric event logs, in JSON (.jsonocel, .json) or XML (.xmlocel, .xml); its mapping is fixed: each object is a case of the type the log gives it, and the other objects of each event and the relationships between objects are read as linked objects.
- S3 bucket, SharePoint or OneDrive folder (under Files; for an S3 bucket choose Process source): when a system exports a file a day to one folder, Duaer reads the new files on the schedule. For S3, choose an S3 credential (with the endpoint: Alibaba Cloud OSS, Tencent COS, MinIO and other S3-compatible stores work) or an AWS (IAM) credential, then the bucket and prefix; for SharePoint or OneDrive, choose a Microsoft credential and paste the folder’s sharing link. Choose the format (CSV, XLSX or OCEL 2.0) and select Look in folder: it shows how many files there are and the newest ones, and the next step maps the newest file’s columns; each row gets a source_file column with the name of its file. Each file is read once, oldest first, a large one over several runs. A file over 20 MB, or one that cannot be read, is skipped with its reason under Files on the source card, and owners and admins are told; replace it in the folder and select Read again. A store inside your network or with an http:// address only is read through an on-premises data gateway that lists the bucket’s address in ALLOWED_TARGETS.
- PostgreSQL, MySQL, Microsoft SQL, Oracle: choose a credential of that kind in the project, or select New credential to create one; it is picked once saved. Then choose the table. In the next step every column is ticked; Duaer does not read the ones you untick. Joined tables lists tables of the same database that relate to this one: by the primary and foreign keys the database declares first, by names otherwise (for example both have the sample number); a table with several rows per row of this one (such as several retests per sample) is not suggested, so cases are not repeated; Add reads their columns too, such as the test result or the customer’s region, and Add another table joins one on two columns you pick, three at most. Under Changes when a row changes, tick every column that may change later, such as each step’s time: a row is read again when any of them is newer than the last read, so steps filled in later still arrive. If the database is only inside your network, run an on-premises data gateway (0.2.0 or later) there with tcp://database-host:port in ALLOWED_TARGETS; through the gateway, Oracle takes the host:port/service form without TLS.
- TiDB, OceanBase, KingbaseES (under Databases) and Amazon Redshift, Azure Synapse Analytics (under Data warehouses): choose a credential and a table as for the databases above. TiDB and OceanBase take a MySQL credential (OceanBase in a MySQL mode tenant), KingbaseES and Redshift a PostgreSQL one (Redshift usually listens on port 5439), Synapse a Microsoft SQL one. Synapse dedicated SQL pools declare no foreign keys, so no joined tables are suggested; add them by hand. If the database is only inside your network, start the Duaer on-premises data gateway (0.2.0 or later) there with tcp://database-host:port in ALLOWED_TARGETS; the credential’s host and port must match it.
- ClickHouse and Trino (under Data warehouses): choose a credential and a table as for the databases above. A ClickHouse credential takes the server address (the HTTP interface, for example https://….clickhouse.cloud:8443), the user, the password and the database; Duaer only reads, never changes data. A Trino credential (Starburst too) takes the server address, the user, the password (empty when the server asks for none), the catalog and an optional schema; tables are listed from that catalog. Direct calls use https only; for a server inside your network or with an http:// address only, start an on-premises data gateway that lists the address, and the source reads once the gateway is online. Neither declares foreign keys, so no joined tables are suggested; add them by hand.
- SAP HANA, IBM Db2, Dameng DM8 (under Databases) and Teradata, Vertica (under Data warehouses): read only through an on-premises data gateway (0.2.0 or later). Put the database’s JDBC driver jar in the gateway’s /drivers folder (run the gateway with -v /opt/jdbc:/drivers:ro), list tcp://database-host:port in ALLOWED_TARGETS, then create a Database Through a Data Gateway credential in Duaer: the same host and port, the database (the HANA tenant, the Db2 database, the Teradata default database, the Vertica database or the Dameng schema), user, password and TLS. Without a gateway that lists the database, Load tables asks you to set one up first. Keys of these databases are not read yet, so add joined tables by hand; Teradata views have no column types, so their columns cannot be the ones that change.
- Snowflake and BigQuery (under Data warehouses): choose a credential and a table as for the databases above. Snowflake takes a Snowflake credential (password or key pair) and reads the credential’s database; BigQuery takes a Google service account or BigQuery OAuth2 credential, and needs the Project ID and Dataset before Load tables. Both bill by query, so they are read once an hour by default, which you can change in the Schedule step. A BigQuery source also sets Most a query may scan (GB), 10 GB by default: BigQuery refuses a query that would scan more, and the source pauses with the reason. A table partitioned on its update time, or a view, keeps each read small.
- Databricks (under Data warehouses): takes a Databricks credential (personal access token or OAuth2) and the ID of a SQL warehouse, found on the warehouse’s Connection details tab; Load tables lists the tables in Unity Catalog as catalog.schema. Duaer runs its queries on that warehouse; a stopped warehouse starts on the first query, which can take a few minutes, and bills while it runs. Like Snowflake and BigQuery, it is read once an hour by default.
- Microsoft Fabric (under Data warehouses): a warehouse, or a lakehouse’s SQL analytics endpoint, read as SQL Server. Fabric accepts only Microsoft Entra sign-in: create a Microsoft SQL credential, set Authentication to Microsoft Entra Service Principal and fill in Tenant ID, Client ID and Client Secret of an app registration. Server is the SQL connection string from the warehouse’s settings (it ends in datawarehouse.fabric.microsoft.com), Database the warehouse or lakehouse name. A Fabric admin must allow service principals to use Fabric APIs, and the app needs at least Viewer access to the workspace. The same credential option signs in to Azure SQL Database and Azure Synapse Analytics with Microsoft Entra.
- Google Sheets, Excel (OneDrive / SharePoint): choose a Google Sheets or Microsoft Excel OAuth2 credential (or Google’s or Microsoft’s generic OAuth2 credential), paste the spreadsheet link, select Open and choose the sheet. For Google Sheets, use the link in the address bar; the sheet it points to is chosen for you. For Excel, select Share, then Copy link in OneDrive or SharePoint. The first row holds the headers; every read reads the whole sheet again, and steps already written are not written twice. The mapping comes next.
- Xero: choose a Xero OAuth2 credential, select Load organisations, then choose the Xero organisation and the documents: Sales invoices (accounts receivable) or Bills (accounts payable). Each document’s issue, the steps in its history (created, approved, sent, voided and so on; edits, notes and attachments are left out), each payment and the full payment become steps; the mapping is fixed, so there are no columns to choose. Each case keeps contact, due_date, currency, total, amount_due (what is still owed), invoice_status (which tells a draft from a sent invoice) and reference, as they are at the last read. With the Invoice due card rule on, each owner gets a card seven days before the due date and once it is overdue, with the customer and the amount owed; a payment closes it. If the owner has already decided a card and a part payment comes in, the card comes back with the remainder. An open card also has a draft to the customer, in the rule’s language, to copy and send yourself; Duaer sends nothing to customers. Save a rule made earlier to get the draft on new cards. Ask the Assistant for overdue receivables by customer to get them from the cases you may see, by what is still owed and with currencies kept apart. A page made from the answer shows the figures of that moment. With the daily digest on, its first chosen day of the week also lists these figures for the rule’s owners, each with its change since the first recorded day. On the employee portal, a case lists what Duaer did on it: each card, the rule behind it, what the owner chose and when the source closed it. Each day Duaer also records the overdue amount, days sales outstanding, invoices paid after a card and promises kept as metric cards for the rule’s owners, each showing its change since the first day. A promise not kept by its date, or a chase with no payment seven days later, brings a new card that says why. Each read takes the documents changed since the last one, 100 at most, and keeps within Xero’s limits (60 calls a minute, 5,000 a day); a first read of many documents finishes over several runs.
- QuickBooks Online: choose a QuickBooks Online OAuth2 credential, select Load company to see the company it is connected to, then choose the documents: Invoices or Bills. QuickBooks does not give a document’s change history, so the steps come from the document and the documents linked to it: an invoice gives the estimate and its acceptance (from the linked estimate), created, issued, sent (when it was emailed), each payment and paid in full; a bill gives the purchase order, created, issued, each bill payment and paid in full. The mapping is fixed, so there are no columns to choose. Each case keeps contact, due_date, currency, total and balance (what is still owed), as they are at the last read. With the Invoice due card rule on, each owner gets a card seven days before the due date and once it is overdue, with the customer and the amount owed; a payment closes it. If the owner has already decided a card and a part payment comes in, the card comes back with the remainder. An open card also has a draft to the customer, in the rule’s language, to copy and send yourself; Duaer sends nothing to customers. Save a rule made earlier to get the draft on new cards. Ask the Assistant for overdue receivables by customer to get them from the cases you may see, by what is still owed and with currencies kept apart. A page made from the answer shows the figures of that moment. With the daily digest on, its first chosen day of the week also lists these figures for the rule’s owners, each with its change since the first recorded day. On the employee portal, a case lists what Duaer did on it: each card, the rule behind it, what the owner chose and when the source closed it. Each day Duaer also records the overdue amount, days sales outstanding, invoices paid after a card and promises kept as metric cards for the rule’s owners, each showing its change since the first day. A promise not kept by its date, or a chase with no payment seven days later, brings a new card that says why. Each read takes the documents changed since the last one, 100 at most.
- HubSpot: choose a HubSpot OAuth2 or private app token credential, select Load pipelines, then choose the object: Deals or Tickets (and one pipeline), or Contact lifecycle stage. Each stage a record enters becomes a step, named as in the pipeline; contacts’ names and emails are not read. The mapping is fixed, so there are no columns to choose. Each read takes the records changed since the last one, 100 at most.
- Salesforce: choose a Salesforce OAuth2 credential, select Load organisation to see the organisation it reaches, then choose the object: Opportunities, Cases or Leads. Each stage an opportunity enters is a step (changes to amount or dates alone are not); a case gives created and each status; a lead gives created, each status and converted. Case and lead status need field history tracking turned on in Salesforce. The mapping is fixed, so there are no columns to choose. Each read takes the records changed since the last one, 100 at most; when the day’s API requests run out, the next run goes on. With Read as records change, Duaer reads within seconds of a change; this needs Change Data Capture turned on for the object in Salesforce. Without it, the source card says so and the source still reads on schedule.
- Jira (under Chat, mail and work): choose a Jira Software Cloud credential (the kind knowledge search uses), select Load projects and choose a project. Each issue is a case: created, then each status it moves to, named as in the project. The mapping is fixed, so there are no columns to choose. Each read takes the issues changed since the last one. See Jira issue process.
- Zendesk (under Service ticket and CRM): choose a Zendesk credential (API token or OAuth2, the kind knowledge search uses) and select Check connection. The credential’s user must be a Zendesk admin to use the incremental export. Each ticket is a case: opened, assigned, responded, resolved and closed, the service ticket template; deleted tickets are left out. See Zendesk ticket process.
- Outlook mail (under Chat, mail and work: the Outlook mail and calendar tile, then Process source): choose a Microsoft Outlook OAuth2 credential; leave the mailbox empty for the account’s own, or enter a shared mailbox’s address (such as invoices@your domain; the account needs Mail.Read.Shared and access to it), and select Check connection. Each conversation is a case: each message in the inbox is Received and each sent one Sent, by its sender; the subject and the other party’s domain are kept on the case. Only these are read, never message bodies or attachments; the first read goes back 90 days, later ones read new mail. The mapping is fixed. Exchange Server on premises is not supported yet.
- UiPath and Yingdao RPA (under RPA): for UiPath, choose a UiPath Orchestrator credential with the Orchestrator URL (https://cloud.uipath.com/organization/tenant/orchestrator_ for Automation Cloud, your own address on premises, read through an on-premises data gateway when it is only inside your network) and an external application’s App ID and App Secret, with the application scopes OR.Folders.Read, OR.Jobs.Read and OR.Queues.Read; select Check connection, choose the folder, then Jobs or Queue items. Each job is a case: created, started, then finished, failed or stopped; each queue item, by queue and reference: added, started, then finished, failed or abandoned. For Yingdao, choose a Yingdao RPA credential with the enterprise’s AccessKey ID and Secret (with the run record API permission) and select Check connection. Each run is a case: started, then finished, failed or stopped, by who ran it. The process, robot, state and error are kept on the case; only finished records are read, and a run still going is read once it ends. The mapping is fixed.
- REST API (under API): any GET endpoint that answers with JSON. Enter the endpoint, the records path, filters and the pagination, and when the API allows it an incremental parameter and field, then select Read fields to check it reads; read again after any change. For an API that needs sign-in, choose a Header, Query, Basic or Bearer credential. An address inside your network is read only through an on-premises data gateway. The mapping is then chosen as for a spreadsheet. See A REST API as a process source.
- SAP S/4HANA, Cloud or on-premise: choose an SAP S/4HANA API credential (the system URL, a communication user and password, and the client on-premise) and select Check system. It shows Connected only when the standard OData APIs of the chosen process all answer, and lists any that do not with its status code. Choose the Purchase to pay or Order to cash process. In purchase to pay each purchase order is a case, with requisition created and released, order created, goods received and reversed, invoice posted and paid as steps; the mapping is fixed, so there are no columns to choose. Each read takes one kind of document, 100 at most, the four kinds in turn from where each stopped. In order to cash each sales order is a case, with order created, delivery created, goods issued, delivered, invoice created and paid as steps. Refer to SAP S/4HANA purchase to pay.
- Feishu Base, Feishu approval: choose the project’s Feishu credential, then the base or the approval definition. Each approval node’s finish time becomes a step.
- DingTalk approval: choose a DingTalk Robot credential whose app may read OA approvals, and enter the Process code (the processCode starting with PROC- in the form editor’s address in the DingTalk admin). WeCom approval: choose a WeCom credential with the approval app’s Secret, and enter the Template ID. Both take optional Form fields to read. Each submission, each node’s or level’s approval or refusal, and each withdrawal becomes a step; the first read takes the past year, and approvals still running are read again later.
- Kingdee Cloud Galaxy: in K/3 Cloud, add an app under Third-party system login and note its app ID and app secret; then create a Kingdee Cloud Galaxy API credential with the server URL (https, ending in /k3cloud), data center ID, user name, app ID and app secret. Select Check account: Duaer signs in and queries each document type of the chosen process, and lists any the user may not view. Only approved documents are read: purchase orders, purchase receipts, payables and payments for purchase to pay; sales orders, sales outbound, receivables and receipts for order to cash. A payment reaches its order through the payable it settles.
- Yonyou BIP: on the Yonyou open platform, create a self-built app with the purchase, sales and finance list APIs granted, and note its App Key and App Secret and the tenant’s API gateway and token addresses; then create a Yonyou BIP API credential with them. Select Check account: Duaer gets a token and calls each list API of the chosen process, and lists any not granted. Only approved documents are read: purchase orders, purchase receipts, purchase invoices and payments for purchase to pay; sales orders, sales outbound, sales invoices and receipts for order to cash. Receipts, outbounds and invoices reach their order through the first document of their chain; payments through the invoice they settle.
- Yonyou U8 / U8+: Duaer reads the SQL Server database of the U8 account set. Ask your database administrator for a read-only login to the account set’s database (UFDATA_<account>_<year>); then create a Microsoft SQL credential with that database. Select Check account: Duaer shows the database it reached and lists any table of the chosen process the login cannot read. Only audited documents are read: purchase orders, purchase receipts, purchase invoices and payments for purchase to pay; sales orders, deliveries, sales invoices and receipts for order to cash. A payment reaches its order through the invoice it settles. Each year’s account set is its own database, so a new year needs a new source. Duaer must be able to reach the SQL Server.
- Dynamics 365 Business Central: in the Microsoft Entra admin center, register an app, add a client secret and grant it the Dynamics 365 Business Central API.ReadWrite.All application permission; then add the app on the Microsoft Entra Applications page in Business Central with a permission set that reads. Create a Business Central API credential with the tenant ID (or domain), environment name, client ID and client secret. Select Check account: Duaer lists the environment’s companies and checks that the chosen company’s documents of the chosen process can be read; choosing another company or process checks again. Purchase to pay reads purchase orders, purchase receipts and posted purchase invoices, with Paid when an invoice is marked paid; order to cash reads sales orders, sales shipments, sales invoices and paid sales invoices. An order Business Central deleted after invoicing takes its vendor or customer, currency and amount from its invoice.
Once a database credential is chosen, Duaer lists its tables and draws them under Tables with how they relate: a solid line is a key the database declares, a dashed line a link matched by column names; each line joins a table to the one it points to (orders to customers). Duaer picks the main table, preferring one with several time columns or a status column, such as orders over the customers they point to, and joins the tables it points to. Click another table to read from it instead; + beside a table the main one points to joins or removes it. Next then picks the columns, the columns that change and the row key, suggests the mapping and goes straight to Preview; Columns and Mapping in the steps above change any of it. Up to 60 tables are drawn, the most connected first; the rest are in the Table list below the drawing. BigQuery and Databricks are drawn after Load tables.
Mapping: what each row is
Setting one up for the first time? Start with Your first process mapping: telling whether a table holds a process, which table to pick, the three questions Mapping asks and how to read the sample.
Rows of the table come in two shapes:
- One row per case, a time column per step: for example a samples table with received, test started, reviewed and report issued times. Choose the case id column, then name a step for each time column.
- One row per status change: for example an order status history. Choose the status column and the changed-at column; the value map can rename raw statuses into clearer step names. A status that comes back counts as rework.
Duaer also recognises the kind of business process: purchase to pay, order to cash, expense approval, lab testing, production order or service ticket, shown under Process kind with the reason; you can choose another or None. With a kind, steps take its usual names and order; findings judge waits against its usual limits (for example a lab sample is usually tested within five days), and cards in generated Watch and Execute workflows say it is the usual limit; Data description lists its key measures (cycle time, on time, completed, first pass) and the usual steps the data lacks.
A process often involves several kinds of object, such as orders and order lines. Read each kind with its own source and fill in Object type under Object levels (optional), for example Order line. For a lower level, choose the upper level’s source in Parent source and the column holding its id in Parent case id column. When the id is made of two columns, use Case id also uses: the order number with the line number gives SO-1001-10. When the table holds another source’s id column, Duaer suggests this; select Use. One id belongs to one kind of object: an id another kind already uses is not written, and the read says how many were left out.
One event may also involve several objects at once, such as one shipment for three orders, or an order with several invoices. Under Linked objects (optional), select Add linked objects, pick the column holding those objects' ids and fill in Their object type (for example Invoice); when a cell holds several ids, fill in Separator (for example ;); when each value says its own type, such as order:SO-1, tick Each value says its type (type:id). Up to 10 columns, and up to 20 objects per row. A mapping with linked objects needs an Object type; when each row of the log is its own kind of object, choose Object type column. SAP, NetSuite, Kingdee Cloud Galaxy, Yonyou BIP, Yonyou U8, Business Central and Coupa sources link each step to its document by themselves: purchase requisition, goods receipt, delivery, invoice and payment, with the order itself a Purchase order or Sales order. An existing source of these kinds gains linked objects once its mapping is confirmed again.
Once levels are set, the upper level’s source card gains Cases by level (owners and admins only): View lists the 50 most recently active cases, each with its current step, how many lower-level cases it has and the longest any of them has been idle; open one to see where each lower-level case stands and for how many days, or find a case by its id. On a large source the first view shows “Reading cases: n of total” until the cases appear. In the employee portal, a case’s timeline lists its lower-level cases; open one to see its own timeline, and Parent leads back. Only cases the person could already see are listed.
At the top of the Mapping step, Data description tells the table in business words once you select View: what one row is, what each column holds, how full it is and its common values, and which columns have problems. It also lists what the data can answer (such as how long cases wait between steps, where they get stuck, who is slow) and what it lacks (such as no person or owner email column), and follows the mapping as you change its columns. A source in use offers it on its card too.
Duaer first suggests a mapping from the column names and contents. It reads Chinese headers, pinyin initials (such as DDH, SHRQ) and common SAP, Kingdee and Yonyou fields (such as FBILLNO, FAPPROVEDATE), and gives time columns readable step names; a column for when the row was last written (such as updated_at) is never taken as a step. Under each choice it says why, and marks an uncertain one "Unsure, check this". Mapping shows Process kind, Rows of the table, the case id column and the steps first; everything else is under Advanced, which opens by itself when the suggestion or the existing mapping uses it. As you change a field, the first 200 rows are worked out again to show how many cases there are, the steps each on average and two cases' steps; a likely wrong case id or steps are pointed out, and Next stays on Mapping. When the table holds no process (one row per thing with one time, or no time column), Mapping says only that and leaves the form empty; when the database has tables that look like process records, Read this table switches to one. When the AI's suggestion was not used, the top says why (no AI model enabled, no answer, an answer that could not be used, or one the data contradicts), and Retry AI suggestion asks again; while the AI reads the data, "The AI is reading the data. This takes about half a minute." shows under the buttons. The AI also sees the values of status and category columns, to read statuses and fill the value map; ids, people, emails and amounts are never shown to it, and it does not guess what a number-only status code means: fill those in yourself. Three optional columns: Who did the step is the person behind each step (the owner email column when nothing else is suggested), Owner email is who owns the case now (cards go to them), and the two may be the same column; Attributes for root cause, up to 20 columns such as lab, line or priority, let the findings say which kind of case a problem happens to. If dates look like 03/04/2026, choose Day first or Month first.
Preview: see the issues first
The preview shows how many rows, cases and events were read, and the steps of the first cases. Issues are listed with examples: rows without a case id, steps without a time, cases with steps out of order, times in the future, most times at night (usually a wrong time zone), statuses not in the value map. Each issue has a one-line hint on what to check. Issues marked Blocks must be fixed before you continue.
Schedule, audience and confirm
- Under Read new rows every, choose minutes, hours or once a day. Duaer reads only rows newer than the last read; with many rows it shows More rows waiting and continues next time. A file source has no interval to choose.
- Under Who sees every case on the portal, tick Everyone in the organization so every member sees them, including people who join later, or search the organization’s members by name or email and pick them. Owners and admins always see them.
- Select Confirm and start. The source is Enabled and the first read starts at once.
Later you can Read now, Pause or Resume it; Edit, Read again from the start and Delete are in the More (⋯) menu. After a mapping change, Duaer rewrites this source’s steps with the new mapping; decisions written back by a Coordinate workflow are kept.
Runs: select View on the source card to list each read of the last year, newest first: rows read, events written and already stored, issues and the error. 50 show at a time; Show more lists older ones.
Read again from the start: after you confirm, Duaer forgets how far it read and an enabled source reads everything again at once; steps already stored are not written twice. A paused source reads from the start once resumed.
A network failure, or a cloud service that is busy or down, does not stop a source: it reads again on schedule, and the third failed read in a row stops it. When a read fails or pauses a source, the person who set it up and each owner of the organization get one Duaer card with the reason and how to resume. No second card comes for that source while one is open, and the next successful read marks them done.
How many events a project holds
A project holds up to 50,000,000 events by default. An admin can lower it with the DUAER_PROCESS_MAX_EVENTS_PER_PROJECT environment variable. At that size Duaer reads the copy of the process tables, and its main process needs about 5 GB of memory, 2 GB of it for the Node.js heap (--max-old-space-size=2048). Process tables are counted on their own and do not use the shared data table space.
- Each source card shows the events it holds and the project’s against the limit, for example “120,000 events · 300,000 of 50,000,000 in this project”.
- From 80% of the limit the line turns to a warning and adds “reading stops at the limit”.
- At the limit reading stops and the source pauses. The card says why and what to do: delete events the project no longer needs from Duaer activities, or ask an admin to raise the limit, then resume. Events already read are kept.
A process map of more than 20,000 cases, and the findings, value and metric values of a source of more than 200,000 cases, are computed from a copy of the process tables that Duaer keeps in its data folder, in duaer-process-analytics. The copy is brought up to date in the background about 30 seconds after a source run reads new events and after Duaer starts, checked again before such a map is computed, and made again if it cannot be read. At 50,000,000 events the first copy takes about 12 minutes, and the first map after it about 7 minutes; it takes about 55 bytes of disk per event, about 2.7 GB at 50,000,000 events. An admin can move it with DUAER_PROCESS_DUCKDB_DIR, change the memory it may use with DUAER_PROCESS_DUCKDB_MEMORY (1GB by default), or turn it off with DUAER_PROCESS_DUCKDB=false; without it, maps read the database and large ones take longer.
Send events with the push API
For a system Duaer has no connector for, or when events should arrive as they happen, use the push API: a system or workflow sends events to Duaer instead of Duaer reading on a schedule.
- In Add source, choose Push API under API, enter a name and time zone, and select Create endpoint.
- Copy the Endpoint and the Key. The key is shown this once; if it is lost, choose More > Replace key on the source card, and the old key stops at once.
- Send events as in the Example request: the header Authorization: Bearer <key>, and the body {"events": [...]} with 1 to 1,000 events. Each event has case_id, activity and timestamp (ISO 8601; without an offset, the source’s time zone), and optionally resource (who did it), object_type and up to 50 attributes.
- Select Refresh to see the events received, then Next to confirm the mapping and audience. Once confirmed, the events received are read at once, and later pushes within seconds.
If one event is wrong, the whole request is refused with its position and field. A wrong key or an unknown source both answer 401. A source keeps up to 2,000,000 events; past that, 413. Deleting the source deletes its events.
Questions
Does Duaer change the data in my database?
No. Duaer uses the credential you choose to read the columns you tick, only rows newer than the last read, and never writes back to your database.
My table has several time columns and a status column. Which shape do I choose?
If every step has its own time column, choose One row per case, a time column per step. If there is one status column and one changed-at column with a row per change, choose One row per status change.
Is the owner email required?
Not for findings. To generate workflows that send cards it is: a case without an owner gets no card. Owners must be members of the organization.
Can people find these cases in the knowledge base?
Yes. A search for a customer, an amount, a reference or a case id finds the cases a source wrote, including those from Xero, QuickBooks, HubSpot, Salesforce and SAP. Each person finds only the cases they see on the portal: owners and admins, the members picked under Who sees every case on the portal, and people with column rules for the cases their rules match.
In this section
See the process in your data and generate workflows in Duaer
Duaer reads the steps every case went through from the tables, databases or Feishu apps you already have: where cases wait too long, where they branch and who did each step. From that it generates three kinds of workflow, Watch, Coordinate and Execute, and replays each over past days before you publish it.
ProcessYour first process mapping: pick the right table, answer three questions
For anyone adding a database or file source for the first time: how to tell whether a table holds a process, which table to pick in a database, the three questions Mapping asks, and how to read the sample under the fields to know the mapping is right.
ProcessProcess findings and SOP conformance in Duaer
Process findings list the steps where cases wait too long or are done again, and which kind of case they happen to most. Propose any of them as a card rule and compare the 30 days before and after it was adopted. SOP conformance compares the SOP in your knowledge base with the steps cases really took.
ProcessWatch, Coordinate and Execute: the workflows Duaer generates from a process
From one process source Duaer can generate three workflows. Watch reminds an owner when a case waits too long. Coordinate asks the owner which way a case goes where cases branch, and writes the choice back. Execute asks the owner to approve starting the next step and then reaches a suggested action that a person reviews and enables.
DataKeep rows in a Duaer data table
In Duaer, a data table lives in the current project. A new table stays closed to other workflows until you grant one of them view or edit. A table that already exists stays open until that first grant.