{section}
{column}

h1. Rationale and Context

A chain of shops has completed its information system by setting up a new collecting mechanism.
Every evening, when a shop closes its doors, the list of commands and sells is written down as a set of CSV files, which are then sent to central.
On reception, these CSV files are parsed and stored into a database.

Since the information system already includes Petals and that Petals can be distributed geographically, it sounded relevant and cost free to reuse it for this new process.
{column}
{hide-if:display=pdf}
{column:width=350px}
{panel:title=Table of contents}{toc}{panel}
{panel:title=Contributors}{contributors:order=name|mode=list}{panel}
{column}
{hide-if}
{section}

h1. Solution

h2. Overall solution

Each shop owns a Petals node, as well as the collecting central.
Each shop node has installed the Petals-BC-FileTransfer component to monitor a drop-in directory.
The collecting central node has installed the Petals-SE-Talend component. This one is in charge of loading the CSV files it receives and put their data into a database.

\\
Thus, every time a CSV file is copied into the drop-in directory of a shop, this CSV files will be sent as an attachment to a Talend job to be processed.


h2. Petals Settings and Message Flows

!ArchiUC2.jpg!
*Deployed configurations:*

# On each shop node, there is an instance of the Petals-BC-FileTransfer component, with one configuration to load CSV files and send them to the job.
# On the collecting central, there is one configuration for the Petals-SE-Talend, which embeds the job that will process the CSV files.

*Message flows:*

# From Petals-BC-FileTransfer to Petals-SE-Talend

Notice that one node can emit *several successive messages*, if several files were dropped at once.
Besides, several nodes can send messages at the same time. Therefore, *several instances of the job may have to be executed concurrently*.


h1. Configuration for the Petals-SE-Talend component

h2. Creating the job

The job is made up of two Talend components:
* The *tFileInputDelimited* component is in charge of loading a CSV file.
* The *tMySqlOutput* component is in charge of writing the sent data into a MySQL database.

\\
Here is what the job looks like.

!CommandsJob.jpg!

\\
The schema of the CSV files is the same for every shop, and for the database.
It is shown on the following screen shot.

!CommandsSchema.jpg!


\\
The properties of the tMySqlOutput are not described here, since there are very common.
And the only specific property of the tFileInputDelimited is that the location of the file to load is defined by a String context variable, called _inLocation_.
Its value will be set by Petals when the job is instantiated.



h2. Exporting the job for Petals

The job is exported as a non-singleton job.
It is very important to uncheck the singleton option during the export. Otherwise, concurrent execution will be impossible and messages will be rejected (meaning that some CSV files will not be processed).
And the _inLocation_ context variable must be exported as as *In-Attachment*.

!CommandsExport.jpg!


h1. Configuration for the Petals-BC-FileTransfer component

This component is in charge of scrutinize a drop-in directory and detect every CSV file that would be moved into it.
When a CSV file is found, it is loaded and sent as an attachment to the processing Talend job.

\\
The directory to monitor is the drop-in directory (that is here defined using an environment variable called _DropInDirectory_).
This directory will be polled every five seconds.
And the files to check are CSV files, i.e. files having a \*.csv extension.

\\
The service to consume is the Talend service that was exported above.
The Message Exchange Pattern must be *InOnly* (no other MEP is allowed for this component in consume mode).
The only operation that supports this MEP in the WSDL of the Talend job, is *executeJobOnly*.
Eventually, the skeleton of the message to send can be generated with a tool like SoapUI.
In this case, the sent message will only contain a reference to the attachment.

\\
Once exported, this configuration can be used for all the distributed nodes.
Here is the skeleton of the message to send to the Talend service.

{code:lang=xml}
<tal:executeJob xmlns:tal="http://petals.ow2.org/talend/">
<tal:contexts />
<tal:in-attachments>
<tal:fileLocation>
<tal:fileContent>$attachment</tal:fileContent>
</tal:fileLocation>
</tal:in-attachments>
</tal:executeJob>
{code}

\\
And here is the jbi.xml of the FileTransfer configuration.

{code:lang=xml}
<?xml version="1.0" encoding="UTF-8"?>
<!--
JBI descriptor for the Petals' "petals-bc-filetransfer" component (FileTransfer).
Originally created for the version 2.5-snapshot of the component.
-->
<jbi:jbi version="1.0"
xmlns:filetransfer="http://petals.ow2.org/components/filetransfer/version-2"
xmlns:gen0="http://petals.ow2.org/talend/"
xmlns:generatedNs="http://petals.ow2.org/talend/"
xmlns:jbi="http://java.sun.com/xml/ns/jbi"
xmlns:petalsCDK="http://petals.ow2.org/components/extensions/version-5"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<!-- Import a Service into Petals or Expose a Petals Service => use a BC. -->
<jbi:services binding-component="true">

<!-- Expose a Petals Service => consumes a Service. -->
<jbi:consumes
interface-name="generatedNs:CommandsCollectorServicePortType"
service-name="generatedNs:CommandsCollectorService"
endpoint-name="CommandsCollectorEndpoint">

<!-- CDK specific elements -->
<petalsCDK:timeout>30000</petalsCDK:timeout>
<petalsCDK:operation>gen0:executeJobOnly</petalsCDK:operation>
<petalsCDK:mep>InOnly</petalsCDK:mep>

<!-- Component specific elements -->
<filetransfer:read-directory>${DropInDirectory}</filetransfer:read-directory>
<filetransfer:transfer-mode>content</filetransfer:transfer-mode>
<filetransfer:file-pattern>*.csv</filetransfer:file-pattern>
<filetransfer:polling-period>5000</filetransfer:polling-period>
<filetransfer:base-message><![CDATA[<tal:executeJob xmlns:tal="http://petals.ow2.org/talend/">
<tal:contexts />
<tal:in-attachments>
<tal:fileLocation>
<tal:fileContent>$attachment</tal:fileContent>
</tal:fileLocation>
</tal:in-attachments>
</tal:executeJob>]]></filetransfer:base-message>
</jbi:consumes>
</jbi:services>
</jbi:jbi>
{code}


h1. Running the use case

To test this use case, install the Petals-SE-Talend and the Petals-BC-FileTransfer components on the target nodes.
Define the environment variable _DropInDirectory_.
Then, deploy the associated configurations in Petals.

\\
To run the test, simply drop a CSV file in the drop-in directory and check the database then.