Petals-SE-Talend 1.2.0+

Version 1 by Christophe DENEUX
on Mar 12, 2018 12:16.

compared with

Current by Christophe DENEUX
on Mar 30, 2018 12:46.

Key

This line was removed.

This word was removed. This word was added.

This line was added.

Changes (1)

View Page History

...

{show-if:display=pdf}
{toc}
{show-if}
{section}
{column}

{warning}This version must be installed on [Petals ESB 5.2.0|petalsesb520:Petals ESB 5.2.0]+{warning}

h1. Features

This component allows to expose Talend jobs as services into Petals and to execute them inside the bus.
It only acts as service provider, not as a service consumer.

\\
This component supports jobs created with both Talend Open Studio and Talend Integration Suite (see [http://www.talend.com] for more details).

It provides several mechanisms to pass information and data to a job.
Il also supports several mechanisms to retrieve information and data from a job.

\\
Before going further, it must be clear that no configuration for this component is intended to be created by hand. Neither by the Petals Studio.
In fact, only Talend Open Studio and Talend Integration Suite have the ability to generate a correct configuration for this component.
{column}
{hide-if:display=printable}
{column:width=350px}
{panel:title=Table of contents}{toc}{panel}
{panel:title=Contributors}{contributors:order=name|mode=list}{panel}
{column}
{hide-if}
{section}

h1. Recommended usage

The Petals-SE-Talend component is intended to execute jobs created with Talend Open Studio and Talend Integration Suite.
Whenever you want to work with both Talend and Petals ESB, it is the component to use.

\\
There are three ways of seeing the jobs that will be executed in Petals ESB.

First, they can be seen as *an extension of the platform capabilities*.
Indeed, a job can easily interact with almost any database, any kind of file, or other systems (SAP, Alfresco, etc...). Rather than developing a specific component, it can be interesting to create a job to cover this task, and expose it as a service through Petals. *The job here acts as a mediation way with data stores or systems.*

\\
Second, a job can be seen as *a set of transformation means*.
Indeed, Talend products provide _Talend components_ that are very efficient and convenient for transformations (e.g. the tMap component). However, it must be clear that these transformations only cover flat structures, like schemas of relational databases. Object or XML schemas are not covered. From this point of view, *the Petals-SE-Talend component cannot replace the Petals-SE-XSLT component*. But it can be an alternative in few cases. Hence, transformations will either rely on attachment files (the content of the attached files is transformed by the job) or on Talend components for Petals (known as _tPetalsInput_ and _tPetalsOutput_). The later solution provides a means to place the content to transform inside the XML message, rather than as an attachment. But still with constraints on the XML shape.

\\
Eventually, the last way to apprehend a Talend job inside Petals, is to see it as a *an application to execute inside Petals*.
This application may work with data stores (files, databases...), may involve data transformations, but may also use other (if not redundant with Petals) features. As an example, it is possible to send mails inside a job, connect to FTPs, etc... Obviously, these features are also available inside Petals. But in some cases, it can be more interesting and more simple to integrate them directly in the job than use the Petals'ones (where you will have to use EIPs or BPELs to link the calls). In some other cases though, the exact opposite may be the best option, i.e. externalize some parts to Petals components. *It all depends on the expected granularity and reusability*.

\\
The wide variety of possibilities (allowed by the non-less important variety of _Talend components_, and by the features of the Petals-SE-Talend component) makes this solution a very flexible one. However, as a Swiss-knife component, *the Petals-SE-Talend component should mainly be seen as a functional service-engine*. Performances, without being bad, cannot be the best ones offered. People looking for a very specific and high-performance usage will prefer develop their own Petals component, or use Petals-SE-Pojo or Petals-SE-Jsr181 components.

h1. Talend Open Studio vs. Talend Integration Suite

Two products can be used to create Talend jobs and export them into Petals: Talend Open Studio, which is an open source product, and Talend Integration Suite, which is the upgraded version of Talend Open Studio (but not free).
There are slight differences between what can be made with these two products.

h2. Talend Open Studio

Talend Open Studio supports the following features for the Petals-SE-Component:
* Expose a context as a parameter into the service's WSDL.
* Pass attachment files to a job.
* Pass native parameters and options to a job.
* Get the job's execution result.
* Get output attachments from a job.

Other features are only supported by Talend Integration Suite.
In particular, it does not support any specific component for Petals.
However, it is possible to use alternative ways, depending on the desired behavior.

h2. Talend Integration Suite

In Talend Integration Suite, all the features of Talend Open Studio are supported.
It also embeds additional capabilities to work with Petals ESB.

* tPetalsInput and tPetalsOutput components can be used in a job.
** These components respectively allow you to pass data from Petals to the job and from the job to Petals.
** These two components do not have any specific parameter. They just need their schema to be defined.

* The Petals behavior can be simulated for these two components. It means a job created in Talend Integration Suite which would contain a tPetalsInput or a tPetalsOutput component, can be executed inside Talend Integration Suite.
** Thus, for a tPetalsInput, the Petals input message can be mocked by (loaded from) an XML file during the simulation.
** For the tPetalsOutput, the returned document will be printed into the console.

\\
Here is a sample job using tPetalsInput and tPetalsOutput components.
This job simply takes an input message from Petals and transforms it before returning it in the bus.

!tPetalsInputOutput.jpg!

h2. Limitations

There are some restrictions in the Petals export and in the use of the Talend components for Petals.

\\
About the export:
* Only one job can be exported at once for Petals.
** One job = One deployment archive for Petals.

* Some data types are not supported in the export for Petals:
** Lists, Objects, passwords, List of values are not supported.
** It also applies to exported contexts.

\\
About the tPetalsInput and tPetalsOutput components:
* A job can contain several instances of any of these components.

* All the tPetalsInput must have the same schema.
** This is checked during the export.

* All the tPetalsOutput must have the same schema.
** This is checked during the export.

* All the tPetalsInput instances read the same input message.

* All the tPetalsOutput instances write in the same output message.
** If two instances of tPetalsOutput are on the same processing flow, the second will erase what is written by the first instance.
** tPetalsOutput instances should be placed on different _branches_ (onSubjobOK, onSubjobError and so on...).

h2. The Talend export for Petals

The export is the same in both Talend Open Studio and Talend Integration Suite.
The generated artifacts are the same in both of them. And the content of the jbi.xml is (almost) the same in both of them.
The only differences are the job's content (which Talend components have been added into the job) and the generated WSDL (which depends on the job content and the export options).

The service-unit structure and the jbi.xml are described farther.
This section only introduces the export options and their impact on the result.
The generated WSDL is not discussed in this documentation.
However, you will find samples in the simple use cases for this component.

Here is a snapshot of the Talend export for Petals.

{center}
!FinalTalendExportForPetals_1.jpg!
{center}

The target file is the location of the service-assembly to generate.
The job version meaning is explicit.

The export options are the following:

|| Export Option || Description ||
| Singleton job | True to make the job singleton. A singleton job can have only one instance running at once on a given Petals-SE-Talend component. |
| Generate the end-point | True to let Petals generate the end-point at deployment time. If false, the end-point name is the job name with the suffix "Endpoint". |
| Validate Petals messages | True to validate all the messages / requests against the WSDL. \\
Be careful, enabling this option reduces the performances (disk access). \\ |
| User routines | Embed the user routines in the service-unit. |
| Source files | True to embed the source files in the generated service-unit. |
| Jobs contexts | Select the context that will be used by default by the job. |

\\
Eventually, there is the edition link to specify how contexts should be exposed in the generated WSDL.

{center}
!FinalTalendExportForPetals_2.jpg!
{center}

When this link is clicked, a dialog shows up.
It lists all the context variables from the job, with their name, the type they will be associated with if exported, and the way they are exported.
By default, no context is exported. Said differently, the export mode for all the contexts is *Not exported*.

\\
Here is a small description of the export modes.

|| Export Mode || Description ||
| Not exported | The context is not exported (not visible as a parameter). But the context can still be overridden using the native parameters (options) of the job. |
| Parameter | The context is exported as a parameter in the WSDL's operation. |
| In-Attachment | The context will be passed the location of a temporary file whose content was attached in the input message. |
| Out-Attachment | The context will be read after the job was executed.
* This context must point to a file.
* The file content will be read by the Petals-SE-Talend component and put as an attachment into the response. 
* The context name will be used as the attachment name.
* The file will be deleted by the component right after its content was loaded. |
| Parameter and Out-Attachment | A mix between the Parameter and the Out-Attachment modes.
* The context is exposed as a parameter.
* It will also be read after the job execution.
* The file will be deleted anyway.
* The advantage of this export mode is to define the output file destination dynamically. |

h1. Exposing a Talend job as a Petals service (Provides mode)

The most important thing to understand is that this component is not intended to be used without Talend products.
It is useless to create JBI descriptor and a service-unit by hand for this component.
In fact, you should even care about the operations this component supports, and only rely on the WSDL generated at the export.

The operations contained by the generated WSDL only contain the operations that this component supports.

\\
This component supports two operations:
* *executeJob:* this operation creates a new job instance, passes it the received parameters, executes it and gets the result back before sending it in the response.
* *executeJobOnly:* this operation creates a new job instance, passes it the received parameters and executes it. This operation does not send back a response.

h2. The "executeJob" operation

The fully qualified name of this operation is:
* Name space URI: any URI, provided it is not null (e.g. *{html}http://petals.ow2.org/talend/{html}*)
* Local part: *executeJob*

\\
{warning}
In the version 1.0 (or 1.0.0), the only name space URI that is accepted for this operation is *{html}http://petals.ow2.org/talend/{html}*.
From the version 1.0.1, you can use any name space URI, as long as it matches a WSDL operation.
Once again, rely on the WSDL generated by Talend products.
{warning}

\\
This operation only supports the *InOut* message exchange pattern (MEP).
When invoking this operation, you must call it using its fully qualified name.

\\
The input and output depend on the job itself and is fully described in the generated WSDL definitions.
Once again, rely on the generated WSDL.

h2. The "executeJobOnly" operation

The fully qualified name of this operation is:
* Name space URI: any URI, provided it is not null (e.g. *{html}http://petals.ow2.org/talend/{html}*)
* Local part: *executeJobOnly*

\\
{warning}
In the version 1.0 (or 1.0.0), the only name space URI that is accepted for this operation is *{html}http://petals.ow2.org/talend/{html}*.
From the version 1.0.1, you can use any name space URI, as long as it matches a WSDL operation.
Once again, rely on the WSDL generated by Talend products.
{warning}

\\
This operation only supports the *InOnly* message exchange pattern (MEP).
When invoking this operation, you must call it using its fully qualified name.

\\
The input depends on the job itself and is fully described in the generated WSDL definitions.
This operation does not send any response. Once again, rely on the generated WSDL.

h2. JBI Descriptor

The service-unit descriptor file ( jbi.xml ) looks like this:
{code:lang=xml}
<?xml version="1.0" encoding="UTF-8"?>
<jbi:jbi version="1.0"
xmlns:jbi="http://java.sun.com/xml/ns/jbi"
xmlns:petalsCDK="http://petals.ow2.org/components/extensions/version-5"
xmlns:talend="http://petals.ow2.org/components/talend/version-1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<jbi:services binding-component="false">
<jbi:provides
interface-name="generatedNs:AttachmentInOutServicePortType"
service-name="generatedNs:AttachmentInOutService"
endpoint-name="AttachmentInOutEndpoint"
xmlns:generatedNs="http://petals.ow2.org/talend/">


<petalsCDK:wsdl>AttachmentInOut.wsdl</petalsCDK:wsdl>
<petalsCDK:validate-wsdl>true</petalsCDK:validate-wsdl>


<talend:name>AttachmentInOut</talend:name>
<talend:class-name>
talenddemosjava.attachmentinout_0_1.AttachmentInOut
</talend:class-name>
<talend:context>Default</talend:context>
<talend:singleton>true</talend:singleton>
<talend:validate-exchange-by-wsdl>false</talend:validate-exchange-by-wsdl>


<talend:output-attachment>outputFile</talend:output-attachment>
</jbi:provides>
</jbi:services>
</jbi:jbi>

{code}

\\
*Configuration of a Service-Unit to expose a Talend job as a service into Petals ESB :*

|| Parameter || Description || Default || Required ||
| name | The job's name \\ | \- | Yes |
| class-name | The job's class name \\ | \- | Yes |
| context | The context to use (if invalid, the job will use the default context) \\ | \- | Yes |
| singleton | The singleton property \\ | \- | Yes |
| validate-exchange-by-wsdl | Validate the messages against the WSDLs' schemas | False | No \\ |
| output-attachment \\ | The name of a context variable that points to a file that must be attached to the returned message. \\
The cardinality for this element is 0-*. \\ | \- | No \\ |

{include:0 CDK SU Provide Configuration 5.7.0}

...

Petals Components Wiki

All versions of Petals ESB

Helpful links

Petals-SE-Talend 1.2.0+

Changes (1)