The basic view of Data Manager is the Dataflow Overview. Here, all established dataflows can be viewed, filtered, searched out, edited, cloned, paused, deleted, and created. You can use the Search feature to scan both names and descriptions for the desired term, Filter by Status, Source or Schema, and Sort by Created Date or Update Date. A pair of buttons on the far left lets you switch between the compact Card View and the older List View. In either case, the available options will be the same.
Options
To start, let us take a quick look at the options this view presents you with:
- View Details – Shows you a log of recent activity on this dataflow, including successful runs and errors. The activity log can be viewed in detail - or limited to only showing errors to avoid clutter. You can further explore the errors to learn the cause and in which lines they occurred.
- Clone – Copies the dataflow as a new draft with all details intact, for simpler and more convenient creation of closely-related dataflows. It will be located in the 'Drafts' tab.
- Clone to sister account - In addition to creating a copy of the dataflow within the same account, it is possible to clone the dataflow into one or more sister accounts by selecting ‘to sister account’ when clicking on the Clone-command and selecting the required accounts in the pop-up. Press 'Clone dataflow'. The cloned dataflow will be in draft mode, and you should go through the dataflow to check if everything is as expected before setting the dataflow into production.
Note: for cloning to a sister account to work, following must be observed:
- The recipient account must have the same source (with same name) that the dataflow uses. If not, cloning will fail.
- If the recipient account does not contain the same schemas (with the same name(s)), the cloned dataflow will not contain these schemas (you can clone schemas between sister accounts before cloning the dataflow, if you need schemas to be the same - see the Schemas & Sources guide for more on this.)
- If a schema on the recipient account is missing mappings or contains different mappings, these mappings will not be present in the cloned dataflow
- After cloning a dataflow, it is necessary to download the file of the dataflow on the recipient account. An information box will guide you to the download page. The cloned dataflow will be in draft mode, and you should go through the dataflow to check if everything is as expected before setting the dataflow into production.
- Run Now – bypasses scheduling to manually trigger a full run of the process. Be aware that this may take some time and cannot be canceled once started.
- Pause/Unpause – Disables or reenables the scheduling of a finished dataflow.
- Delete – Permanently removes the dataflow. This cannot be reversed.
Status
The Status of a dataflow has a large impact on what options you have for interacting with it. The possible states and their limitations are as follows:
- In Progress – This Dataflow is currently running, which effectively sets it as read-only until it is finished. Only purely cosmetic information (Name, Description) can be edited, and the only available options are Clone and Delete, the later of which will stop the ongoing progress. Once finished, the Dataflow will switch to Ready or Waiting.
- Waiting – This Dataflow is complete and active, and will run again as soon as the next file finishes downloading. This state only occurs when a Dataflow is set to run continuously, and the datasource likewise continous – namely Website Tracking or StreamFile. Identical to the Ready-state in terms of available options.
- Completed – This Dataflow is complete and active, awaiting its next scheduled or manual activation. Most options are available, and clicking on the Dataflow allows for much of the information within it to be edited. This and Waiting are the only states from which a Dataflow can be Paused
- Paused – This Dataflow has been paused, interrupting its scheduled or continuous runs. Options are generally identical to the Ready-state, and the Dataflow can still be run manually with the ‘Run Now’ option. This is the only state in which the ‘Unpause’ option can be used, returning the Paused Dataflow to the Ready- or Waiting-state.
- Failed – This Dataflow has crashed due to some kind of serious issue. The ‘Details’ option may provide useful information about why and how. This can be freely edited, giving you extensive options for correcting the issue. The Dataflow will not attempt to run according to its set schedule, but the ‘Run Now’ is available, letting you give it another go.
- Draft – The dataflow is unfinished – a draft that has yet to be completed. The only options will thus be to delete it, or else open and finish the dataflow. This is signified by most of the option-bar beneath being replaced with a checklist that shows how far along the the draft is. The dataflow will appear as a Draft when clicking the Save as draft button during creation of a dataflow, or after being copied with the Clone command. Drafts are located in their own discreet Tab, and will not be visible unless this Tab is selected.
- Test Completed – This is essentially a finished Draft, which has already completed all relevant steps including the final test-run. All that remains is to activate it. These are found in the 'Drafts' Tab as well.
- Deleted – This Dataflow has been deleted, and can neither be accessed nor reactivated. Its name and general details, including the log accessed via View Details, can be found in the 'Deleted' Tab, for the purpose of record-keeping.