Data Flow - From a Job to Petals using a tPetalsOutput

compared with
Key
This line was removed.
This word was removed. This word was added.
This line was added.

Changes (27)

View Page History
{section}
{column}

h1. Preliminary notes

This use case can only be reproduced with Talend Integration Suite.
People using Talend Open Studio can find an alternative in the use cases "Data Flow - From a job to Petals using attachments" and "Data Flow - From a job to Petals using a tBufferOutput".
{column}
{column:width=360px}
{panel}{toc}{panel}
{column}{section}

h1. Rationale

Execute a Talend job into Petals and passing it a context as a parameter described in the service's WSDL interface.
The resulting file will be returned as an attachement
Generate a data flow in a job and send them to Petals.

The job is exposed as a service into Petals. When this service is called, the parameter is passed to the job, which is then executed.
The file created by this job is then returned as an attchment.

The input message provides the value of of the context which is exposed as a parameter in the service's interface.
Only the job's result is expected in the response.
The input message provides nothing.
The job's result and the generated data flow are expected in the response.

h1. Creating and exporting the job

The job to be executed performs the following actions:
# The job is passed the value of the context variable.
# The job connects to a database.
# It retrieves the content of a table.
# It serializes part of the extracted data as a CSV file on the disk.
# Once the execution terminated, the created file is attached to the returned message.
# The job generates a data flow.
# This data flow is sent into Petals.

This job has one context variable, which indicates the location of the CSV file.
This location will be read by the Petals-SE-Talend component once the execution is terminated. The file will then be attached to the returned message.
This job has no context variable.

In the scope of this use case, it is assumed there is a database "formationtalend" on the localhost, having a table named customers.
The schema of the customers table includes two columns named "CustomerName" and CustomerAddress, both being of type varchar(255).

h2. Creating the job

The job creation is detailled in the use case "A Simple Talend Job".
There is no difference.
The job is made up of two components:
# The tRowGenerator generates the data flow.
# The tPetalsOutput makes this data flow available in Petals.

Here is the overall aspect of the job.

!JobToPetals-tpo_job.jpg!

\\
Here is the schema and the generation settings for the tRowGenerator component.

!JobToPetals_tpo_trgschema.jpg!

\\
Here is the schema of the tPetalsOutput component.

!JobToPetals_tpo_tposchema.jpg!



h2. Exporting the job

Select the job and right-click it. Select *Export to Petals ESB*.
Update the target destination.
Let the job be exposed as a singleton.


You should have the following dialog:

!JobToPetals_tpo_export.jpg!

Click *Finish*.



h1. Deploying and testing in Petals


h2. Looking at the created archive generated WSDL

The created archive is a Petals service assembly.
More details are available in the documentation of the petals-SE-Talend.
In the created Petals service assembly, the most interesting thing to look at is the WSDL.
Indeed, the WSDL will determine the way the exported service will be called.

What must be taken care of is the jbi.xml and the WSDL files available in the service-unit.
If you open the created archive, it contains another archive. This second archive contains a jbi.xml file.
\\
The input message's description expects no parameter.

The WSDL file exposes no parameter.
The possible input parameters are:
{code:lang=xml}
<xs:element name="executeJob" type="tns:executeJob" />
<xs:complexType name="executeJob">
<xs:sequence>
<xs:element minOccurs="0" name="contexts" type="tns:talendContexts" />
<xs:element minOccurs="0" name="in-attachments" type="tns:inAttachments" />
<xs:element maxOccurs="unbounded" minOccurs="0" name="in-data-bean" type="tns:inRow" />
<xs:element maxOccurs="unbounded" minOccurs="0" name="talend-option" type="xs:string" />
</xs:sequence>
</xs:complexType>

<xs:complexType name="talendContexts">
<xs:sequence>
</xs:sequence>
</xs:complexType>

<xs:complexType name="inAttachments">
<xs:sequence>
</xs:sequence>
</xs:complexType>

<xs:complexType name="inRow">
<xs:sequence>
</xs:sequence>
</xs:complexType>
{code}

\\
And the output message includes the job's result and the generated data flow.

{code:lang=xml}
<xs:element name="executeJobResponse" type="tns:executeJobResponse" />
<xs:complexType name="executeJobResponse">
<xs:sequence>
<xs:element minOccurs="0" name="talend-job-output" type="tns:talendJobOutput" />
</xs:sequence>
</xs:complexType>

<xs:complexType name="talendJobOutput">
<xs:sequence>
<xs:element maxOccurs="unbounded" minOccurs="0" name="executionResult" nillable="true" type="ns1:stringArray" />
<xs:element minOccurs="0" name="outAttachment" type="tns:outAttachments" />
<xs:element maxOccurs="unbounded" minOccurs="0" name="outDataBean" nillable="true" type="tns:outRow" />
</xs:sequence>
</xs:complexType>

<xs:complexType name="outAttachments">
<xs:sequence>
</xs:sequence>
</xs:complexType>

<xs:complexType name="outRow">
<xs:sequence>
<xs:element name="name" type="xs:string" nillable="true" />
<xs:element name="age" type="xs:integer" nillable="true" />
</xs:sequence>
</xs:complexType>
{code}

\\
The WSDL generation has taken into account the schema of the tPetalsOutput.

{tip}
Notice that a Java *Date* type in the component's schema would have been translated to a *timestamp* (long) during the export.
{tip}


h2. Deploying and testing this new service

SoapUI is used for this test...
To test this service, you can use a tool like SoapUI.
This way, you can see what the XML messages look like.

The first thing to do is to create a service-unit for the Petals-BC-SOAP component, that exposes (consumes) our _Talend job as a service_ outside the bus.
This step is not described here. You can take a look at the Petals-BC-SOAP documentation and the Petals Studio documentation.
Just make sure the SOAP configuration uses the InOut MEP.

\\
Now, your input message (in SoapUI) should look like this:

{code:lang=xml}
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tal="http://petals.ow2.org/talend/">
<soapenv:Header/>
<soapenv:Body>
<tal:executeJob>
<!--Optional:-->
<tal:contexts/>
<!--Optional:-->
<tal:in-attachments/>
<!--Zero or more repetitions:-->
<tal:in-data-bean/>
<!--Zero or more repetitions:-->
<!--tal:talend-option>?</tal:talend-option-->
</tal:executeJob>
</soapenv:Body>
</soapenv:Envelope>
{code}

\\
The returned message, when everything works, is:

{code:lang=xml}
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<tns:executeJobResponse xmlns:tns="http://petals.ow2.org/talend/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<tns:talend-job-output>
<tns:executionResult>
<jaxb:item xmlns:jaxb="http://jaxb.dev.java.net/array">0</jaxb:item>
</tns:executionResult>
<tns:outAttachment/>
<tns:outDataBean>
<tns:name>onq1slKprbEZN2k</tns:name>
<tns:age>70</tns:age>
</tns:outDataBean>
<tns:outDataBean>
<tns:name>oghHxi7r85k04Nu</tns:name>
<tns:age>61</tns:age>
</tns:outDataBean>

<!-- ... -->

<tns:outDataBean>
<tns:name>JppSgHonz7d7z0h</tns:name>
<tns:age>7</tns:age>
</tns:outDataBean>
<tns:outDataBean>
<tns:name>IVDkn38bDRAAcXi</tns:name>
<tns:age>52</tns:age>
</tns:outDataBean>
<tns:outDataBean>
<tns:name>McT8WFoqX1QGCqG</tns:name>
<tns:age>50</tns:age>
</tns:outDataBean>
<tns:outDataBean>
<tns:name>bw6NIKoICJdwvXk</tns:name>
<tns:age>20</tns:age>
</tns:outDataBean>
<tns:outDataBean>
<tns:name>zMKQMzp6cpTSUN7</tns:name>
<tns:age>13</tns:age>
</tns:outDataBean>
</tns:talend-job-output>
</tns:executeJobResponse>
</soapenv:Body>
</soapenv:Envelope>
{code}

\\
Notice the XML shape.
The first child is the job's result.
If the job execution fails, the 0 is replaced by another integer, e.g. 1.
One failure reason can be, as an example, that the database is not started.

\\
The list of _outDataBean_ is made available to Petals through the tPetalsOutput.
Every _outDataBean_ has the same list of children, each child being a column in the schema of the tPetalsOutput.

Thus, the expected data schema is defined by the job, and not by the service's contract.
In fact, the service's contract is partially generated from this schema.
As said in the other use cases, it is the job's content which define what the service contract will be.