Skip to main content

Build a File-Driven Integration

TimeUnder 10 minutes
What you'll buildA file integration that watches a folder on an FTP or SFTP server and logs a message each time a new file arrives.

File integrations are ideal for batch uploads, scheduled file processing, and ETL workflows triggered by files appearing in a folder or FTP server. This quick start shows the full cycle: add an FTP integration artifact, add a handler that runs when a new file is created, and test it by uploading a file.

Prerequisites​

A working WSO2 Integrator environment.

You'll also need access to an FTP, SFTP, or FTPS server that is reachable over the internet. Have the following details ready:

  • The server's host and port.
  • A username and password for the server.
  • A folder on the server for the integration to watch, for example, /quickstarts-file-integration. Create this folder before you start.

To test the integration, you'll also need an FTP or SFTP client to upload a file to the specified folder.

Get a free test server

If you don't have a server, create a free test server on SFTPCloud by clicking Create test FTP server. Use the host, port, username, and password it shows.

Note

The cloud editor runs in WSO2 Cloud and cannot connect to a server running on localhost on your computer. Make sure the server is accessible over the internet.

Step 1: Create the integration​

  1. On the project home page, click Create on Cloud.

    WSO2 Cloud project home showing the Create on Cloud optionWSO2 Cloud project home showing the Create on Cloud option
  2. The WSO2 Integrator cloud editor opens in your browser.

    WSO2 Integrator cloud editor landing page showing an empty integration and the Deploy to WSO2 Cloud panelWSO2 Integrator cloud editor landing page showing an empty integration and the Deploy to WSO2 Cloud panel
Note

The cloud editor creates a default integration for you, so you don't need to name it now. You can change its name when you deploy it. For more information, see Save and deploy.

Step 2: Add a file integration artifact​

  1. In the design view, click Add Artifact manually.

    Integration design view with the Add Artifact manually button highlightedIntegration design view with the Add Artifact manually button highlighted
  2. Select FTP / SFTP under File Integration.

    Artifacts page with FTP / SFTP highlighted under File IntegrationArtifacts page with FTP / SFTP highlighted under File Integration
  3. Set Host to your server's host.

  4. Select the protocol your server uses: FTP, SFTP, or FTPS.

  5. Set Port to your server's port. For example, SFTP usually uses port 22.

    Create FTP Integration form with Host, the SFTP protocol, and Port setCreate FTP Integration form with Host, the SFTP protocol, and Port set
  6. Select Basic Authentication.

  7. Set Username and Password to your server's username and password.

  8. Set Monitored Path to the folder you want to watch, for example /quickstarts-file-integration. The folder must already exist on the server.

  9. Click Create.

    Create FTP Integration form with Username, Password, and Monitored Path setCreate FTP Integration form with Username, Password, and Monitored Path set

Step 3: Add onCreate event handler​

  1. In the FTP Integration design view, click + Add Handler.

  2. In the Select Handler to Add panel, select On Create.

    FTP Integration design view with Add Handler and the On Create option highlightedFTP Integration design view with Add Handler and the On Create option highlighted
  3. Set Format to Text.

  4. Keep the default File Handling Options. After processing a file, the integration moves it to the /home/processed folder on your server to prevent it from being processed again.

  5. Click Save.

    Configure On Create Handler panel with Format set to Text and the default File Handling OptionsConfigure On Create Handler panel with Format set to Text and the default File Handling Options

Step 4: Add file processing logic​

  1. Click + inside the onFileText flow.

    onFileText flow with the add node button highlighted below the Start nodeonFileText flow with the add node button highlighted below the Start node
  2. Select Call Function under Statement.

    Node panel with Call Function highlighted under StatementNode panel with Call Function highlighted under Statement
  3. Search for printInfo in the functions panel.

  4. In the search results, go to the Standard Library section and select the printInfo function under log.

    Functions panel search results with printInfo highlighted under log in the Standard Library sectionFunctions panel search results with printInfo highlighted under log in the Standard Library section
  5. Set Msg to New file received.

  6. Click Save.

    log:printInfo panel with Msg set to New file receivedlog:printInfo panel with Msg set to New file received

Step 5: Run and test​

  1. Click Run. The Debug Console opens at the bottom of the editor. It shows some messages while the integration connects to your server, such as Authentication succeeded. These messages are expected.

  2. Using your FTP or SFTP client, upload a text file, such as test.txt, to the monitored folder.

  3. Wait up to a minute. The integration checks the folder every 60 seconds by default.

  4. Confirm the Debug Console shows a log line that ends with message="New file received".

    Debug Console showing the New file received log line after a file is uploadedDebug Console showing the New file received log line after a file is uploaded

Step 6: Deploy to WSO2 Cloud​

To deploy your integration to WSO2 Cloud from the cloud editor, see Save and deploy.

Source code​

View the generated Ballerina code

The steps above generate the following complete, runnable Ballerina program:

import ballerina/ftp;
import ballerina/log;

listener ftp:Listener ftpListener = new (host = "<your-server-host>", protocol = ftp:SFTP, port = 22, auth = {credentials: {username: "<your-username>", password: "<your-password>"}});

@ftp:ServiceConfig {path: "/quickstarts-file-integration"}
service ftp:Service on ftpListener {
@ftp:FunctionConfig {afterProcess: {moveTo: "/home/processed"}, afterError: {moveTo: "/home/failed"}}
remote function onFileText(string content) returns error? {
do {
log:printInfo("New file received");
} on fail error err {
// handle error
return error("unhandled error", err);
}
}
}

Learn more​

  • FTP/SFTP — Watch and process files on remote FTP or SFTP servers
  • Local files — Full Local Files listener reference (events, recursive watching, file handlers)
  • Streaming large files — Process large files without loading them fully into memory
  • CSV fault tolerance — Handle errors and partial failures when processing CSV files