CSV Import Scheduled Provider

🎯 Functional documentation

This provider is designed to periodically import a CSV file from a remote SFTP/FTP server and use it to create or update Parties and Dossiers in Harmoney.

Functionally, the provider performs the following steps:

1. It connects to the configured file transfer provider (ABSTRACT_FILE_TRANSFER_PROVIDER) and downloads the CSV file from the configured directory and filename (CSV_DIRECTORY / CSV_FILENAME). If the file is not present, execution stops silently. When CSV_FILENAME_IS_REGEX is true, every file matching CSV_FILENAME is imported instead (see Multiple files with a regular expression).
2. The CSV file is parsed using the configured import template (IMPORT_TEMPLATE) and delimiter (CSV_DELIMITER). Entities (Parties, Dossiers, etc.) are created or updated for each configured Financial Service Provider (SUPPORTED_FSP). If multiple FSPs are configured, all imports share the same downloaded file. When UPLOAD_RESULT_FILES is enabled, the result file of each FSP import is uploaded into MOVE_TO_FOLDER right after that import (see Result file).
3. After all FSP imports have completed (whether successful or not), optional post-processing is applied to the source file on the remote server:
   • Rename (RENAME_PATTERN): the file is renamed using a configurable pattern supporting {originalname}, {date}, and {datetime} placeholders. A file rejected as a whole is renamed with ERROR_RENAME_PATTERN instead, when set.
   • Move (MOVE_TO_FOLDER): the file (after renaming, if applicable) is moved to a target folder. Before moving, the provider checks whether a file with the same name already exists in the target folder. If it does, a yyyyMMddHHmmss timestamp is automatically appended to the filename (e.g. import_20260401143022.csv) to prevent silent overwrites. This ensures the same file is not re-processed on the next scheduled run.

Result file

Every import produces a result file with one line per row of the CSV: the row number and either OK or the reason the row was rejected. It is the same file a user receives after a manual CSV import from the UI, and it is always stored in the documents of the FSP.

With UPLOAD_RESULT_FILES set to true, this file is also uploaded to the file server, into MOVE_TO_FOLDER, so the sending system can check the outcome of each row.

• Name: {originalname}_RESULT_{datetime}.csv, for example parties-20260710_RESULT_20260710063000.csv. When several FSPs are configured, each FSP gets its own result file with the FSP identifier in the name, for example parties-20260710_RESULT_MY_FSP_20260710063000.csv.
• Delimiter: the same as the imported file (CSV_DELIMITER).
• Upload failure: a failed upload is logged and does not stop the import or the rename and move of the imported file.
• Rejected file: when the whole file is rejected (for example because required columns are missing), an error file {originalname}_ERROR_{datetime}.csv is created instead, in the same format, with a single line giving the reason. Like the result file, it is stored in the documents of the FSP and uploaded when UPLOAD_RESULT_FILES is enabled. Unexpected technical errors are reported with a generic reason; the details are in the automation logs in Harmoney. The rejected file is still renamed and moved like any other file, so it is not imported again; after correcting it, deliver it again under a new name. To make the rejection visible in the name of the file itself, with or without UPLOAD_RESULT_FILES, set ERROR_RENAME_PATTERN, for example {originalname}.error.

Multiple files with a regular expression

Some systems deliver a new file every day with a date or time stamp in its name, for example parties-20260710.csv. To import these files, set CSV_FILENAME_IS_REGEX to true and CSV_FILENAME to a regular expression, for example parties-\d{8}\.csv. The expression must match the full file name.

• When it is a regular expression: only when CSV_FILENAME_IS_REGEX is true. Without it, CSV_FILENAME is always an exact file name, even when it contains characters such as ( or [, so existing configurations keep importing that exact file. An invalid expression is rejected when the configuration is saved.

• Order: the matching files are imported one by one, sorted by file name in ascending order. A stamp in yyyyMMdd or yyyyMMddHHmmss format therefore imports the oldest file first. A stamp such as ddMMyyyy does not sort chronologically and must not be used.

• Per file: each file is imported for all configured FSPs and then renamed and moved before the next file starts.

• Download failure: if a file cannot be downloaded, the run stops and the remaining files stay in the folder for the next run, so a newer file is never imported before an older one. Errors on individual rows do not stop the run.

• Avoid importing twice: set RENAME_PATTERN or MOVE_TO_FOLDER, otherwise the same files are imported on every run; the configuration validation warns about this. Prefer MOVE_TO_FOLDER; when only renaming, make sure the new name no longer matches CSV_FILENAME.

⚙️ Configuration

• Implementation class: CsvImportScheduledProvider

Settings

Required parameters:

• ABSTRACT_FILE_TRANSFER_PROVIDER: The identifier of an existing FTP Provider configuration. This provider will be used to connect to the remote file server and download the CSV file.
• CSV_FILENAME: The name of the CSV file (including extension) to download from the configured remote folder. Example: import.csv. When CSV_FILENAME_IS_REGEX is true, a regular expression matched against the full file name (see Multiple files with a regular expression). Example: parties-\d{8}\.csv.
• IMPORT_TEMPLATE: The import template used to process the CSV file and create entities. Example: HARMONEY_PARTY_IMPORT.
• SUPPORTED_FSP: A comma-separated list of Financial Service Provider (FSP) IDs for which the CSV import needs to be executed.
• SYSTEM_USER: The ID of the system user that will be linked to the performed actions, such as creating parties and logging events.

Optional parameters

• CRON_SCHEDULE: Determines the periodic interval to run the provider. When not provided, the provider can only be triggered manually via the admin UI: organizations > org x > triggers.
• CSV_DELIMITER: Configures the CSV delimiter used for CSV file parsing. Defaults to COMMA when not set.
• CSV_FILENAME_IS_REGEX: When true, CSV_FILENAME is a regular expression and every matching file is imported, oldest first (see Multiple files with a regular expression). Defaults to false.
• UPLOAD_RESULT_FILES: When true, the result file of each import is uploaded into MOVE_TO_FOLDER. Requires MOVE_TO_FOLDER. Defaults to false.
• CSV_DIRECTORY: The folder on the remote file server from which the CSV file will be downloaded. If omitted, the DIRECTORY configured on the selected FTP provider is used.
• RENAME_PATTERN: An optional pattern to rename the file on the remote server after import (success or failure). Supported placeholders: {originalname} (filename without extension), {date} (today's date in yyyyMMdd format), and {datetime} (current date and time in yyyyMMddHHmmss format). Example: {originalname}_DONE_{datetime}.csv → import_DONE_20260401143022.csv. When omitted, the file is not renamed.
• ERROR_RENAME_PATTERN: An optional pattern to rename a file that is rejected as a whole (for example because required columns are missing), used instead of RENAME_PATTERN. Rows rejected one by one do not count. Same placeholders as RENAME_PATTERN. Example: {originalname}.error → import.error. Works with or without UPLOAD_RESULT_FILES. When omitted, a rejected file is renamed like any other file.
• MOVE_TO_FOLDER: An optional remote folder to move the file into after import (success or failure). If RENAME_PATTERN is also set, the rename is applied first and the resulting filename is moved. Before moving, the provider checks whether a file with the same name already exists in the target folder — if it does, a yyyyMMddHHmmss timestamp is appended to avoid overwriting (e.g. import_20260401143022.csv). When omitted, the file stays in the source directory.

Settings Configuration