{section}
{column}

{note}This is the documentation section for the *development version* of Petals ESB Deployer.
Please, note that such a documentation and product may be incomplete and/or unstable. For production environment, please use the latest public release of product and documentation.
{note}

h1. Introduction

Petals ESB Deployer is a tool to manage the deployment of a whole Petals ESB bus.

Deploying a whole Petals ESB bus includes to deploy Petals ESB containers, Petals ESB Hazelcast Registry, and service consumers or providers running on their binding components or service-engines correctly configured. The architecture of the Petals ESB bus and service providers/consumers to deploy are defined into a model.

Petals ESB Deployer is embedded into Petals ESB CLI through different commands:
* the command {{deploy}} is extended to deploy a whole Petals ESB bus, and to migrate an existing deployment to a new version,
* the command {{undeploy}} is extended to undeploy a whole Petals ESB bus,
* the dedicated command {{check-deployment}} will check an existing Petals ESB bus against a given model describing the expected Petals ESB bus.

{warning}
Petals ESB Deployer *1.0.0* is provided with [Petals ESB CLI *3.2.0*|petalsclisnapshot:Petals ESB CLI 3.2.0-SNAPSHOT].
{warning}

{column}
{column:width=40%}
{panel:title=Table of contents}{toc}{panel}
{panel:title=Contributors}{contributors:order=name|mode=list}{panel}
{column}
{section}

h1. Use cases

The goal of Petals ESB Deployer is to resolve several deployment problematics:
* how to simplify the deployment of a Petals ESB bus avoiding to use scripts for automating deployment ?
* how to upgrade an existing deployment of a Petals ESB bus to a new version of this Petals ESB bus,
* how to deploy a set of service providers and consumers on different environments, for example DEV, TEST or PROD where the Petals ESB bus topology and external resources are different ?
* how to deploy a Petals ESB bus on different environments, for example STAGING, PRE-PROD, PROD where the Petals ESB bus topology is the same but where machines running Petals ESB bus and external resources are differents ?

h2. Deployment samples

h3. Classic: Different topologies following project progress

In this use case sample, service providers/consumers are deployed on a topology that is different according to the project step: DEV, TEST, PRE-PROD and PROD. Service providers/consumers are the same on each environment, but their instance number can be different.

h4. 1st step: development

In the first step of a project, developers will use a standalone topology running on their computer as following. The external web-service is provided by ressource {{host-dev-ws-0}} and must be configured at service unit 'SU P#1' level:
!deployment-samples-classic-DEV.png|width=40%,align=center!

h4. 2nd step: testing

Once the development is finished, some artefacts are delivered to the team "TEST":
* the list of service-unit that must be deployed,
* the service-assemblies or deployable service units as archives to deploy.

With the Petals ESB architect, the team "TEST" defines a simplified version of the Petals ESB bus topology that will be used for testing. Finally we can have such a topology using machines:
* {{host-test-0}} and {{host-test-1}} for both Petals ESB containers,
* {{host-test-2}} for the unique Petals ESB Hazelcast Registry member,
* {{host-test-ws-0}} hosts the external web-service resource that must be configured at service unit 'SU P#1' level.
!deployment-samples-classic-TEST.png|width=40%,align=center!

h4. 3rd step: pre-production validation

Once tests are passed, we validate deliverables on a pre-production environment for which the Petals ESB bus topology is the same than the production environment. As the pre-production environment is a mirror of the production environnment, this topology was defined by the Petals ESB architect in agreement with production operators. Service units deployed are the same than the one of the testing environment, only machines and few configuration items are different between environments PRE-PROD and PROD. Machines of the pre-production environment are:
* {{host-prep-0}}, {{host-prep-1}}, {{host-prep-2}} and , {{host-prep-3}} where are running Petals ESB containers,
* {{host-prep-4}} and {{host-prep-5}} where are running the Petals ESB Hazelcast Registry members,
* {{host-prep-ws-0}} hosts the external web-service resource that must be configured at service unit 'SU P#1' level.
!deployment-samples-classic-PRE-PROD.png|width=40%,align=center!

h4. 4th step: production deployment
If no problem has been detected in pre-production stage, the project is deployed on the production environment with the same topology and the same service-units than the one of the pre-production environment, except machines and few parameters:
* {{host-prod-0}}, {{host-prod-1}}, {{host-prod-2}} and , {{host-prod-3}} where are running Petals ESB containers,
* {{host-prod-4}} and {{host-prod-5}} where are running the Petals ESB Hazelcast Registry members,
* {{host-prod-ws-0}} hosts the external web-service resource that must be configured at service unit 'SU P#1' level.
!deployment-samples-classic-PROD.png|width=40%,align=center!

h3. Upgrade of a existing Petals ESB bus

In this use case sample, a Petals ESB bus is already deployed, and we want to upgrade it to a new version where:
* a set of service-units must be upgraded, including:
** some service-units to upgrade to an upper version,
** some new service-units to deploy,
** some existing service-units to undeploy,
* on a topology that can be also upgraded.
!deployment-samples-upgrading.png|width=60%,align=center!

h3. Isolating partners using several Petals ESB bus

In this use case sample, an organisation want to give access to some of its internal resources through its ESB. Moreover, to minimize problems between partners, the organisation wants to isolate each partner, but few partners can use same services.

So, the organisation decides to design an ESB based on several distinct Petals ESB buses. Each partner has its own set of service consumers/providers. Because the service level agreement can be different between partners, the Petals ESB bus topology associated to each partner can be the same or different. For mutualisation reasons, a unique Petals ESB Hazelcast Registry instance is used to host each registry of Petals ESB bus.

All Petals ESB buses are deployed in the same time, only one deployment for all Petals ESB buses. Adding or removing a partner is as an upgrade of the ESB of the organisation.

!deployment-samples-partners-isolation.png|width=60%,align=center!

h1. Modeling your deployments

h2. Model overview

Petals ESB Deployer is based on a model defining your Petals ESB bus to deploy. This model contains all components and their configurations forming your Petals ESB bus.

The model of your Petals ESB bus is composed of 3 separated parts whose each goal is different:
* a 1st model defines the *applicative content* of your Petals ESB bus. The *Service-units Object Model* lists all service-units that must be deployed into your Petals ESB bus,
* a 2nd model defines the *topologic architecture* of your Petals ESBG bus. The *Topology Object Model* designs your Petals ESB bus topology,
* and a *physical view* of your Petals ESB bus defining the placement of each components. The *Bus Object Model* locates:
** where are running the service-units, on which Petals ESB container of your topology,
** where are running your Petals ESB containers, on which machine,
** where are running your Petals ESB Hazelcast Registry nodes, on which machine,
** which kind of machines are running all needed components,
** where are running "virtual machine" (VM, DOcker container, ...), on which host,
** which external resources are to be used (as placeholder values).

Information relative to the target environment can be expressed as hard-coded value and/or properties in the Bus Object Model. Each property value will be set when deploying the model as deployment extra parameters.

!model.png!

{petalslink}
Ce modèle est dessiner avec le logiciel DIA. Fichier source du modèle disponible en attachement.
{petalslink}

{anchor:serviceUnitsObjectModel}
h2. Service-units Object Model

This model defines the set of service units to deploy on your Petals ESB bus. This model will be mainly written by development teams because they have the knowledge of service units to deploy.

!serviceUnitsObjectModel.png!

This model is composed of following elements:
* {{service-units}}, the list of service units that must be deployed,
* {{service-unit}}, one service unit to deploy:
** {{id}}, the identifier of this service-unit in the model. Can be different from the service unit name,
** {{url}}, the URL of the associated archive that can be a service assembly or a deployable service unit.
** {{placeholders}}, the set of placeholders that the service unit requires for its execution,
* {{placeholder}}, a placeholder required by a service-unit:
** {{key}}, the place holder name as used in the service-unit. A value is associated to this placeholder in the [bus object model|#busObjectModel],
** {{description}}, a description of the placeholder. Optional,
** {{default-value}}, default value of the placeholder. Optional.

{note}
Placeholders without a default value will have to be set in the [Bus Object Model model|#busObjectModel] or the associated property '{{<service-unit-id>.<placeholder-key>}}' will have to be set when deploying the model.
{note}

{anchor:topologyObjectModel}
h2. Topology Object Model

This model defines the topology of your Petals ESB bus. This model will be mainly written by the Petals ESB bus architect.

!topologyObjectModel.png!

This model is composed of following elements:
* {{topology}}, the topology definition of your Petals ESB bus, with:
** {{containers}}, its different Petals ESB containers as a list of {{container}},
** {{registry}}, the registry used in your topology. Optional.
* {{container}}, a Petals ESB container member of your topology:
** {{id}}, identifier of the Petals ESB container as used as name in the file {{topology.xml}},
** {{default-jmx-port}}, default JMX port value for Petals ESB container instance of this container definition. A [deployment property|#deployment_properties] can be used. Optional, if not set, the internal default value '{{7700}}' will be used,
** {{default-jmx-user}}, default JMX username value of the Petals ESB container instance of this container definition. A [deployment property|#deployment_properties] can be used. Optional, if not set, the internal default value '{{petals}}' will be used,
** {{default-jmx-password}}, default JMX password value of the Petals ESB container instance of this container definition. A [deployment property|#deployment_properties] can be used. Optional, if not set, the internal default value '{{petals}}' will be used,
* {{standalone-registry}}, the registry is implemented by Petals ESB Standalone Registry, default implementation for a standalone topology. Can not be used for a distributed topology,
* {{hazelcast-registry}}, the registry is implemented by Petals ESB Hazelcast Registry, default implementation for a distributed topology:
** {{default-group-name}}, default group name value for the instance of this Petals ESB Hazelcast Registry definition. A [deployment property|#deployment_properties] can be used. Optional, if not set, the internal default value '{{default-sample}}' will be used,
** {{default-group-password}}, default password value for the instance of this Petals ESB Hazelcast Registry definition. A [deployment property|#deployment_properties] can be used. Optional, if not set, the internal default value '{{s3cr3t}}' will be used,
** {{members}}, the members of the Petals ESB Hazelcast Registry,
* {{hazelcast-registry-member}}, a member of the Petals ESB Hazelcast Registry:
** {{id}}, identifier of the Petals ESB Hazelcast Registry member as used as name in the file {{cluster.xml}},
** {{default-port}}, default communication port value for the instance of this Petals ESB Hazelcast Registry member definition. A [deployment property|#deployment_properties] can be used. Optional, if not set, the internal default value '{{7900}}' will be used,

{anchor:busObjectModel}
h2. Bus Object Model

This model defines your Petals ESB bus placing Petals ESB components on their containers:
* service-unit on its Petals ESB container,
* Petals ESB container on its machine,
* Petals ESB Hazelcast Registry member on its machine.

This model will be mainly written by the Petals ESB bus architect in agreement with operators.

!busObjectModel.png!

This model is composed of following elements:

*** {{machines}}, a list of {{machine}} on which the current Petals ESB bus is running,
** {{machine}}, a machine on which a part of the Petals ESB bus is running:
*** {{id}}, identifier of the machine used as reference key for {{container-instance}} and {{hazelcast-registry-member-instance},
*** several types of machines are supported:
**** {{provisioned-machine}}, a provisioned machine as a physical machine or a virtual machine already existing:
***** {{hostname}}, hostname, IP address or [deployment property|#deployment_properties] name for the current provisioned machine,
**** {{docker-container}}, a Docker container running a Petals ESB component:
***** {{docker-host-hostname}}, hostname, IP address or [deployment property|#deployment_properties] name of the Docker host on which the Docker container will be started,
***** {{docker-host-port}}, port or [deployment property|#deployment_properties] of the Docker daemon of the Docker host on which the Docker container will be started,
** {{container-instance}}, the instance of a Petals ESB container defined in the topology model:
*** {{reference}}, reference to a {{container}} of the [topology object model|#topologyObjectModel],
*** {{machine-reference}}, the reference of the machine on which the current Petals ESB Container is running,
*** {{jmxPort}}, port JMX of the current Petals ESB container. A [deployment property|#deployment_properties] can be used. Optional, if not set, the value defined by the [referenced container|#topologyObjectModel] object will be used,
*** {{jmxUser}}, JMX username of the current Petals ESB container. A [deployment property|#deployment_properties] can be used. Optional, if not set, the value defined by the [referenced container|#topologyObjectModel] object will be used,
*** {{jmxPassword}}, JMX password of the current Petals ESB container. A [deployment property|#deployment_properties] can be used. Optional, if not set, the value defined by the [referenced container|#topologyObjectModel] object will be used,
** {{service-unit-instance}}, an instance of a service unit running on a Petals ESB container:
*** {{reference}}, reference to a {{service-unit}} of the [service-units object model|#serviceUnitsObjectModel],
*** {{container-reference}}, reference to a {{container}} of the [topology object model|#topologyObjectModel],
*** {{placeholders}}, the set of {{placeholder-instance}} that must be applied to the current service-unit instance to set the right placeholder values. Optional,
** {{placeholder-instance}}, an instance of placeholder to define its value:
*** {{reference}}, reference to a {{placeholder}} of the [service-units object model|#serviceUnitsObjectModel],
*** {{value}}, value to set to the current placeholder,
** {{hazelcast-registry-instance}} defines the registry instance of the current Petals ESB bus as a Petals ESB Hazelcast Registry instance:
*** {{reference}}, the reference of the Petals ESB Hazelcast Registry in the [topology object model|#topologyObjectModel],
*** {{group-name}}, name identifying the Petals ESB Hazelcast Registry instance inside the Hazelcast cluster. A [deployment property|#deployment_properties] can be used. Optional, if not set, the value defined by the [referenced Petals ESB Hazelcast Registry|#topologyObjectModel] object will be used,
*** {{group-password}}, password of the Petals ESB Hazelcast Registry instance. A [deployment property|#deployment_properties] can be used. Optional, if not set, the value defined by the [referenced Petals ESB Hazelcast Registry|#topologyObjectModel] object will be used,
*** {{member-instances}}, list of member instances ({{hazelcast-registry-member-instance}}) composing the current Petals ESB Hazelcast Registry instance,
** {{hazelcast-registry-member-instance}}, a member of the current Petals ESB Hazelcast Registry instance:
*** {{reference}}, the reference of the member of the Petals ESB Hazelcast Registry in the [topology object model|#topologyObjectModel],
*** {{host-machine}}, the reference of the machine on which the current Petals ESB Hazelcast Registry is running,
*** {{port}}, the communication port of the current member instance. A [deployment property|#deployment_properties] can be used. Optional, if not set, the value defined by the [referenced Petals ESB Hazelcast Registry member|#topologyObjectModel] object will be used.

{anchor:deployment_properties}
h2. Deployment properties

Deployment properties can be used in models to specify the following elements:
* in the service units object model:
** a default value of a placeholder
* in the topology object model:
** default values of Petals ESB container parameters,
** default values of Petals ESB Hazelcast Registry parameters,
* in the bus object model:
** configuration values for machines: IP address, Docker host parameters,
** configuration values for Petals ESB container instances overriding the default ones,
** configuration values for Petals ESB Hazelcast Registry instance overriding the default ones,
* or to set the value of a service unit placeholder that has neither value nor default value set.

Each deployment property values will be required at deployment time.

h2. Axioms

* A service-unit model must contain at least one service-unit,
* A topology object model must contain at least one container,
* In a topology object model:
** Petals ESB Hazelcast Registry and Petals ESB Standalone registries cannot coexist,
** A Petals ESB Standalone registry can be used only when there is a unique container,
** The Petals ESB Hazelcast Registry must be used if at least two Petals ESB containers are declared,
** The Petals ESB Hazelcast Registry must contain at least one Petals ESB Hazelcast Registry member,
* A bus object model always refers to a service-unit object model,
* A bus object model always refers to a topology object model,
* A bus object model must contain at least one machine,
* In a bus object model:
** a machine must host at least one Petals ESB container or one Petals ESB Hazelcast Registry member,
** two machines can refer to a same physical machine if they will have the same configuration values or deployment properties,
** Petals ESB container instances and Petals ESB Hazelcast Registry member instances can reference the same machine to force a co-localization.

h2. Writing your deployment model

The best practice is to write your deployment model with different files, but you are able to write your deployment model in only one file.

h3. Writing your deployment model in separated files

The best practice is to write your deployment model with different files:
* a 1st file for your service units object model, written by your development team,
* a 2nd file for your topology object model, written by the Petals ESB architect,
* and a 3rd file for your bus object model, written by the Petals ESB bus architect in agreement with operators.

h4. Writing a service units object model

Example of a service unit object model:
{code:lang=xml}
<service-units xmlns="http://petals.ow2.org/deployer/models/service-units.om/1.0" targetNamespace="http://petals.ow2.org/deployer/sample.suom">
<service-unit id="su-filetransfer-consume">
<url>mvn://org.ow2.petals.samples.filetransfer/su-filetransfer-consume/4.1.0-1.0.0</url>
</service-unit>
<service-unit id="su-filetransfer-provide">
<url>mvn://org.ow2.petals.samples.filetransfer/su-filetransfer-provide/4.1.0-1.0.0</url>
</service-unit>
</service-units>
{code}

h4. Writing a topology object model

Example of a standalone topology object model:
{code:lang=xml}
<topology xmlns="http://petals.ow2.org/deployer/models/topology.om/1.0" targetNamespace="http://petals.ow2.org/deployer/sample-standalone.tom">
<containers>
<container id="cont-0" />
</containers>
</topology>
{code}

Example of a distributed topology object model:
{code:lang=xml}
<topology xmlns="http://petals.ow2.org/deployer/models/topology.om/1.0" targetNamespace="http://petals.ow2.org/deployer/sample-distributed.tom">
<containers>
<container id="cont-0" />
<container id="cont-1" />
</containers>
<hazelcast-registry>
<members>
<member id="hzlreg-0" />
</members>
</hazelcast-registry>
</topology>
{code}

h4. Writing a bus object model

Example of a bus based on a standalone topology:
{code:lang=xml}
<bus
xmlns="http://petals.ow2.org/deployer/models/bus.om/1.0"
xmlns:suom="http://petals.ow2.org/deployer/sample.suom"
xmlns:tom="http://petals.ow2.org/deployer/sample-standalone.tom"
targetNamespace="http://petals.ow2.org/deployer/sample-standalone.bom">

<!-- Import of the service unit object model -->
<import namespace="http://petals.ow2.org/deployer/sample.suom" location="sample.suom"
importType="http://petals.ow2.org/deployer/models/service-units.om/1.0">
<!-- Import of the topology object model -->
<import namespace="http://petals.ow2.org/deployer/sample-standalone.tom" location="sample-standalone.tom"
importType="http://petals.ow2.org/deployer/models/topology.om/1.0">

<machines>
<provisioned-machine id="machine-00">
<hostname>machine-00.petals.org</hostname>
</provisioned-machine>
</machines>

<topology-instance>
<container-instances>
<container-instance ref="tom:cont-0" machine-ref="machine-00" jmx-port="7700" jmx-user="petals" jmx-password="petals"/>
</container-instances>
</topology-instance>

<service-unit-instances>
<service-unit-instance ref="suom:su-filetransfer-consume" container-ref="tom:cont-0" />
<service-unit-instance ref="suom:su-filetransfer-provide" container-ref="tom:cont-0" />
</service-unit>
</bus>
{code}

Example of a bus based on a distributed topology:
{code:lang=xml}
<bus
xmlns="http://petals.ow2.org/deployer/models/bus.om/1.0"
xmlns:suom="http://petals.ow2.org/deployer/sample.suom"
xmlns:tom="http://petals.ow2.org/deployer/sample-distributed.tom"
targetNamespace="http://petals.ow2.org/deployer/sample-standalone.bom">

<!-- Import of the service unit object model -->
<import namespace="http://petals.ow2.org/deployer/sample.suom" location="sample.suom"
importType="http://petals.ow2.org/deployer/models/service-units.om/1.0">
<!-- Import of the topology object model -->
<import namespace="http://petals.ow2.org/deployer/sample-distributed.tom" location="sample-distributed.tom"
importType="http://petals.ow2.org/deployer/models/topology.om/1.0">

<machines>
<provisioned-machine id="machine-00">
<hostname>machine-00.petals.org</hostname>
</provisioned-machine>
<provisioned-machine id="machine-01">
<hostname>machine-01.petals.org</hostname>
</provisioned-machine>
<provisioned-machine id="machine-02">
<hostname>machine-02.petals.org</hostname>
</provisioned-machine>
</machines>

<topology-instance domain-name="PEtALS">
<hazelcast-registry-instance group-name="default-sample" group-password="s3cr3t">
<hazelcast-registry-member-instances>
<hazelcast-registry-member-instance machine-ref="machine-02" port="7900" />
</hazelcast-registry-member-instances>
</hazelcast-registry-instance>

<container-instances>
<container-instance ref="tom:cont-0" machine-ref="machine-00" jmx-port="7700" jmx-user="petals" jmx-password="petals"/>
<container-instance ref="tom:cont-1" machine-ref="machine-01" jmx-port="7700" jmx-user="petals" jmx-password="petals"/>
</container-instances>
</topology-instance>

<service-unit-instances>
<!-- Service unit deployed on container 'cont-0' -->
<service-unit-instance ref="suom:su-filetransfer-consume" container-ref="tom:cont-0" />
<service-unit-instance ref="suom:su-filetransfer-provide" container-ref="tom:cont-0" />

<!-- Service unit deployed on container 'cont-1' -->
<service-unit-instance ref="suom:su-filetransfer-consume" container-ref="tom:cont-1" />
<service-unit-instance ref="suom:su-filetransfer-provide" container-ref="tom:cont-1" />
</service-unit>
</bus>
{code}

h3. Writing your deployment model in only one file

Example of a bus based on a standalone topology:
{code:lang=xml}
<bus
xmlns="http://petals.ow2.org/deployer/models/bus.om/1.0"
xmlns:tns="http://petals.ow2.org/deployer/sample-standalone.bom"

targetNamespace="http://petals.ow2.org/deployer/sample-standalone.bom">

<suom:service-units xmlns:suom="http://petals.ow2.org/deployer/models/service-units.om/1.0">
<service-unit id="su-filetransfer-consume">
<url>mvn://org.ow2.petals.samples.filetransfer/su-filetransfer-consume/4.1.0-1.0.0</url>
</service-unit>
<service-unit id="su-filetransfer-provide">
<url>mvn://org.ow2.petals.samples.filetransfer/su-filetransfer-provide/4.1.0-1.0.0</url>
</service-unit>
</suom:service-units>

<tom:topology xmlns:tom="http://petals.ow2.org/deployer/models/topology.om/1.0">
<containers>
<container id="cont-0" />
</containers>
</tom:topology>

<machines>
<provisioned-machine id="machine-00">
<hostname>machine-00.petals.org</hostname>
</provisioned-machine>
</machines>

<topology-instance domain-name="PEtALS">
<container-instances>
<container-instance ref="tns:cont-0" machine-ref="machine-00" jmx-port="7700" jmx-user="petals" jmx-password="petals"/>
</container-instances>
</topology-instance>

<service-unit-instances>
<service-unit-instance ref="tns:su-filetransfer-consume" container-ref="tns:cont-0" />
<service-unit-instance ref="tns:su-filetransfer-provide" container-ref="tns:cont-0" />
</service-unit>
</bus>
{code}

h1. Launching a deployment

A Petals ESB bus deployment is launched through the command {{deploy}} of [Petals ESB CLI|petalsclisnapshot:Petals ESB CLI 3.2.0-SNAPSHOT#deploy]. See the documentation of the command to get more information.

h1. Executing deployment

h2. Concurrent deployments

To be able to redeploy a Petals ESB bus from scratch in a minimum time, the deployment of the Petals ESB bus must be the fastest as possible. So, Petals ESB containers are deployed concurrently.

h2. Error management

Different errors can occurs during the full deployment process of a Petals ESB bus:
* errors linked to invalid model definitions,
* errors linked to unavailable artefacts,
* errors linked to unreachable machines.

Petals ESB Deployer minimizes the appearance of these errors executing pre-checks:
* to validate model definitions,
* to validate that artefacts are available (this check is not sufficient to assume that artefacts will be always reachable, but this avoid error on their URL),
* to check that all machines are reachable (this check is not sufficient to assume that machine will be always reachable, but this avoid error on hostname, IP address and port).

h1. Petals Deployment Model

h2. Source code

Maven project work in progress (POC ?)

[Petals ESB Deployer 1.0.0^petals-deployment-model.zip]

h2. Example project

Here is a project to try making deployment model for an existing project (with script CLI)

[Petals ESB Deployer 1.0.0^hdbank-deployment-model.zip]