Backup Logs Periodically

Rationale and Context

The application XXX is a legacy application that produces important and different stacks of logs. Each log is a positional file, where information is ordered by column, each column having the same size and separed by a tabulation character.
To track events on a long term, and improve the safety of these logs, it was decided the main information from these logs have to be archived every day in a data center.

Besides, additional processes and procedures, relying on Petals ESB, are set up around this application.
In particular, BPEL processes are defined and foreseen to backup the logs if an emergency event occurs.

Without this last point, running the job outside would have been, a priori, the best solution.
But interacting with services definitely elects the job's execution into Petals as a better solution.

This use case only describes the way the logs are saved periodically.

Solution

The logs are loaded, filtered and written to a database through a Talend job.
This Talend job is exposed as a service into Petals. This way, it can be called by any Petals service, including a BPEL process.
And it can also be scheduled to be executed on a regular basis, every day in this case.

The execution schedule is handled by the Petals-SE-Quartz component.
Every day, at the same time, it will send a message to the Talend job.

Configuration for the Petals-SE-Quartz component

Configuration for the Petals-SE-Talend component

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.