This cookbook describes how to configure an assembly at build time. The target audience are developers or build engineers creating new assemblies or maintaining existing ones.
Note
The cookbook is valid for Gradle tools version 2.11.6 and higher.
It is assumed that you are familiar with the Concept - Gradle Assembly Tools.
Topics not covered in this document are:
How to work with an existing (and therefore configured) assembly as part of the developer workflow. This includes execution of the assembly process.
Phrase | Meaning |
---|---|
Version Control System (VCS) | Also known as source control, source code management systems (SCM), or revision control systems (RCS). VCS is a mechanism for keeping multiple versions of your files, so that when you modify a file you can still access the previous revisions. |
Artifact Repository | Place, where build and package software components are located. Provide a common interface to a dependency management system. |
Code Analysis | Process to analyze source code to calculate metrics, find bugs, etc. |
Continuous Delivery Pipeline | Sometimes called Deployment Pipeline, describes the stages, which code artifacts runs through source to production system. |
System Component | A software package of different code artifacts and files, that have to be deployed together. |
System Component Set | Is a container for system components, that needs to be build and branched together. |
Assembly | An assembly references one or more system components residing in the same or a configured artifact repository in order to deploy or deliver them together. |
Build Process | Compiles and packages files and code artifacts from a source project to deployable artifacts. |
Publish Process | The process which transfers the deployable artifacts to a configured artifact repository. |
Assembly Process | This process combines several system components to an assembly. |
Deployment Process | This process extracts files and code artifacts from an artifact repository and applies the configuration. |
Project Gradle Distribution | This is a customized Gradle distribution with the preconfigured artifact repositories and Gradle plugins. |
Gradle Plugin | A Gradle plugin packages up reusable pieces of build logic, which can be used across many different projects and builds. |
Project Gradle Plugin | This is a Gradle plugin which contains special corporate respectively project settings. |
Corporate Plugin | The term is used as a synonym for Project Gradle Plugin. |
Gradle Extension Object | Java Bean compliant class holding configurations for Gradle plugins. |
Gradle Wrapper | The Gradle Wrapper is the preferred way of starting a Gradle build. The wrapper is a batch script on Windows, and a shell script for other operating systems. When you start a Gradle build via the wrapper, Gradle will be automatically downloaded and used to run the build. See for more information The Gradle Wrapper in the Gradle documentation (2.11, 2.7, 2.3, 2.0, 1.8) |
Intershop Cluster | A number of hosts of different types serving an Intershop 7. |
Cluster Node | One separately deployable part of an Intershop cluster. A host can run multiple nodes of one Intershop cluster. |
Developers need to create custom assemblies to deploy custom cartridges or a custom combination of existing cartridges.
An assembly can also be created with the scripts in the development-setup of an Intershop delivery package (See Cookbook - Setup of CI Infrastructure).
This recipe provides a more in-depth explanation on how to create an assembly manually.
This covers the more common case of creating an assembly that inherits from an existing one.
Create a single root folder and the following files. Add them to your source repository, once you are done. See Concept - Gradle Assembly Tools for the purposes of these files.
file | initial content |
---|---|
build.gradle | Apply the 'ish-assembly' plugin and inherit from one of our base assemblies - see discussion. |
gradle.properties | Specify a version - see discussion. |
cartridge-order.txt | Copy from inherited assembly. Deprecated for Gradle tools version >= 1.1, see recipe Cookbook - Gradle Assembly Tools#Add Cartridges to an Assembly. |
gradlew , gradlew.bat files, gradle folder | Copy from other Gradle build process |
To verify that your initial assembly works, publish and/or deploy it.
Copy the following minimum build.gradle file and follow through the TODO
s:
Gradle tools version 1.0:
// Make the assembly plugin available to the build.gradle script buildscript { dependencies { classpath 'com.intershop.build.gradle:ish-assembly' classpath 'com.intershop.build.gradle:plugin-tests' } } // Apply the 'ish-assembly' plugin to the project apply plugin: 'ish-assembly' // Specify the namespace in which the assembly is published into the repository. (The name inside this namespace is automatically derived from the folder name.) // TODO: Change to your desired namespace group = 'com.example.assembly' assembly { // Inherit from another assembly // TODO: Replace by <org>:<name> of the inherited assembly inheritFrom('<org>:<name>') { // Inherit assembly deploy.gradle and deploy-settings.gradle scripts includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle'] // By default will also inherit all host-type, environments // and the contained components/cartridges } // Supply additional meta-data that will be displayed in the SMC/at logon screens extraAttributes = [ //TODO: Change to appropriate values 'productName': 'Intershop 7', 'copyrightOwner': 'Intershop Communications', 'copyrightFrom': '2005' ] }
Gradle tools version >= 1.1:
// Make the assembly plugin available to the build.gradle script buildscript { dependencies { classpath 'com.intershop.build.gradle:ish-assembly' classpath 'com.intershop.build.gradle:plugin-tests' } } // Apply the 'ish-assembly' plugin to the project apply plugin: 'ish-assembly' // Specify the namespace in which the assembly is published into the repository. (The name inside this namespace is automatically derived from the folder name.) // TODO: Change to your desired namespace group = 'com.example.assembly' assembly { // Inherit from another assembly // TODO: Replace by <org>:<name> of the inherited assembly inheritFrom('<org>:<name>') { // Inherit assembly deploy.gradle and deploy-settings.gradle scripts includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle'] // By default will also inherit all host-type, environments // and the contained components/cartridges } cartridges { // Inherit cartridge order from another assembly // TODO: Replace by <org>:<name> of the inherited assembly order = listFromAssembly('<org>:<name>') } // Supply additional meta-data that will be displayed in the SMC/at logon screens extraAttributes = [ //TODO: Change to appropriate values 'productName': 'Intershop 7', 'copyrightOwner': 'Intershop Communications', 'copyrightFrom': '2005' ] }
The ish-assembly
plugin extends the Gradle project by:
assembly
for configuring the assembly's contents. (See other recipes.)assemblyBuild
for configuring how a database dump is created during build. (See other recipes.)You have created an assembly that is basically the same like the inherited one, except for a different name. To customize the assembly, see the other recipes.
Copy the following minimum gradle.properties file and follow through the TODO
s:
# TODO: Provide version number of your assembly (excluding build number/local/snapshot suffix) version = 7.5.0.0 # build plugin versions filter.com.intershop.build.set.cartridge-plugins = 1.0.0.0.+ filter.com.intershop.build.set.extension-plugins = 1.0.0.0.+
I need to add custom deployment logic to my inherited assembly.
When adding my deploy.gradle, I get an exception like:
Execution failed for task ':publishBuildAssemblyPublicationToBuildRepository'. > Failed to publish publication 'buildAssembly' to repository 'build' > Invalid publication 'buildAssembly': multiple artifacts with the identical name, extension, type and classifier ('deploy', 'gradle', 'deploy-gradle', 'null').
Example until Gradle Deployment Tools 2.7
assembly { inheritFrom('com.intershop.assembly:commerce_management_b2x') { includeArtifacts type:['deploy-settings-gradle'] // Exclude the artifact when inheriting from the parent assembly.... // includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle'] } }
To find the parent assembly's deployment logic, browse your software repository and look for the deploy-gradle artifact of the according assembly.
To exclude the artifact from inheritance, see Recipe: Customize Assembly Inheritance.
The developer needs to add cartridges to an assembly.
See the discussion for details on the steps.
For Gradle tools version 1.0:
assembly.cartridges.include
in the build.gradle file.
For Gradle tools version >= 1.1:
assembly.cartridges.include
in the build.gradle file. assembly.cartridges.order
. This ensures that the cartridgelist.properties file is appropriately generated.To add cartridges to one or more environments, add the following declaration:
assembly { cartridges { //TODO: Specify organization and name of the cartridges, also choose which environments to include them in include 'com.example:cartridge1', 'com.example:cartridge2', in: [production, development, test] } }
Note
Component subsets (host-types, environment, configurations) are available as object literals by their name inside the assembly
closure (instead of being passed as strings).
To add a cartridge as init-only-cartridge (see Concept - Gradle Assembly Tools for more details) use the same method, but specify in:init
.
assembly { cartridges { //TODO: Specify organization and name of the cartridges include 'com.example:cartridge1', 'com.example:cartridge2', in: init } }
By default adding a cartridge also adds all its transitive dependencies, i.e., all dependencies declared in the configuration default
of its ivy.xml file, those cartridges' dependencies etc.
For most applications this makes sense, since you do not want to deploy a cartridge without its dependencies. Also you can minimize declaration effort by including the application suite cartridge. Such a cartridge already has dependencies to all application type cartridges, which in turn should already have dependencies to all cartridges contained in that application.
For more control you may include a cartridge without its dependencies using the following declaration:
assembly { cartridges { //TODO: Specify organization and name of the cartridges, also choose which environments to include them in include( 'com.example:cartridge1', 'com.example:cartridge2', in: [production, development, test] ) { transitive = false } } }
Note
Dependency Management for Third-Party Libraries
Beware that the dependency management for third-party libraries as implemented by Intershop 7 version 7.6 and newer mandates the use of transitive declaration for cartridge dependencies during assembly.
(For Gradle tools version 1.0)
To generate the <SHARE>/system/cartridges/cartridgelist.properties file, which serves as basis for look-up precedence of ISML templates, pipelines, queries, etc. and also for the order in which DBInit preparers are executed, the assembly process needs to order cartridges. This is performed by looking up the order of cartridges in the cartridge-order.txt file.
The format of the cartridge-order.txt file is:
The cartridge-order.txt must contain all cartridges that are contained in the assembly, but may contain any number of additional cartridges. It is only used to determine the order of cartridges, not which cartridges to include in the assembly.
If you use multiple assemblies it is therefore recommended to share the cartridge-order.txt file across them.
When adding cartridges with their transitive dependencies, it is necessary that all these dependencies are also included in cartridge-order.txt.
assembly.cartridges.order
(For Gradle tools version >= 1.1)
To generate the <SHARE>/system/cartridges/cartridgelist.properties file, which serves as basis for look-up precedence of ISML templates, pipelines, queries, etc. and also for the order in which DBInit preparers are executed, the assembly process needs to order cartridges. This is performed by looking up the order of cartridges in the property assembly.cartridges.order
.
Order vs. Inclusion
The set of cartridges to include and their order are declared separately. This may seem odd because they are coupled tightly. There are a few reasons for this decision:
See subsection Order from Groovy List Literal for how to specify both included cartridges and their order without redundancies.
The property assembly.cartridges.order
is of type List<String>
. Each entry of the list should be the unqualified name of a cartridge, e.g., bc_foundation.
Constraints
All cartridges included in the assembly must be contained in the order list exactly once to know how to order them. If a cartridge is included in the assembly, but missing from the order list, an exception is thrown. Another exception is thrown if a cartridge is included in the assembly but occurs multiple times in the order list.
When adding cartridges with their transitive dependencies, it is necessary that all these dependencies are also included in the order list. Also the order list must contain all cartridges inherited from other assemblies.
The order list must contain all cartridges that are contained in the assembly, but may contain any number of additional cartridges.
The property assembly.cartridges.order
can be filled in several ways.
The following way is considered best practice to fill the assembly.cartridges.order
property:
assembly { cartridges { def productionCartridges = ['customer_cartridge_1', 'customer_cartridge_2'] // TODO: Replace by organization of your cartridges include(*(productionCartridges.collect {"<org>:$it"}), in:[development, test, production]) { transitive = false } // TODO: Replace by <org>:<name> of the inherited assembly order = listFromAssembly('<org>:<name>') + productionCartridges } }
Read the order of cartridges from an existing assembly using the method listFromAssembly
:
assembly { cartridges { //TODO: Adapt name and organization of assembly order = listFromAssembly('com.intershop.assembly:intershop7') } }
Read the order from an external file using the method listFromFile
:
assembly { cartridges { //TODO: Adapt name/location of the file. In this example the file is expected in the same directory as the build.gradle file order = listFromFile(file('cartridge-order.txt')) } }
The expected file format is:
Create the list from a Groovy list literal directly written in the build.gradle file:
assembly { cartridges { //TODO: Adapt names of cartridges order = ['core', 'bc_foundation', 'bc_marketing'] } }
You may also create a list using a list literal once and pass it to both property assembly.cartridges.order
and method assembly.cartridges.include
. This assumes that you include your cartridges intransitively.
Note
The property expects assembly.cartridges.order
unqualified cartridge names, while the assembly.cartridges.include
method requires dependency expressions (e.g., qualified names).
assembly { cartridges { def productionCartridges = ['core', 'bc_foundation'] include(*(productionCartridges.collect {"com.intershop:$it"}), in:[development, test, production]) { transitive = false } order = productionCartridges } }
Create the list from any combination of the above, by concatenating multiple other lists, inserting or removing items. You may use any method/operation offered by the JDK or GDK (Groovy extensions to JDK) on java.util.List
.
assembly { cartridges { // Order cartridges: // - Cartridge 'customer_java_patches' is first // - Then all cartridges inherited from intershop7 // - Then all cartridges mentioned in 'custom-cartridge-order.txt' // - Order cartridge 'storefront_app_patch' directly after cartridge 'storefront_app' def intermediaryOrder = ['customer_java_patches'] + listFromAssembly('com.intershop.assembly:intershop7') + listFromFile(file('custom-cartridge-order.txt')) order = intermediaryOrder.plus(intermediaryOrder.find('storefront_app')+1, 'storefront_app_patch') } }
To be compatible with Gradle tools version 1.0 the property assembly.cartridges.order
is filled with the contents of a file cartridge-order.txt if it exists. If it does not it is empty by default.
To adjust versions more easily, they should be declared in a properties-file instead of the build.gradle file directly. Alter the gradle.properties file of your assembly by either:
version.<organization of cartridge>.<name of cartridge>=<version>
for each added cartridge, orfilter.<organization of assembly>.<name of assembly>=<version>
of an assembly or component set containing the added cartridges.Also add the declaration to the gradle.properties of your component set if your cartridges have dependencies to the added cartridges.
See section Configuring Dependency Versions in Concept - Gradle Build Tools for details.
The developer needs to add infrastructure components to an assembly.
Add infrastructure components to a host-type by using the method assembly.hostTypes.<host-type name>.include.
assembly { hostTypes { //TODO: replace 'appserver' by the host-type's name appserver { // TODO: Specify organization and names of the cartridges include( 'com.example:component1', 'com.example:component2', ) { transitive = false } } } }
Note
Order of Declarations
There is a bug in Gradle tools version 1.0-1.1 (ISTOOLS-461): If the build.gradle file of the assembly contains
the flags includeShare
and includeLocal
will not be inherited correctly. This may result in an (almost) empty deployment.
To avoid this error inherit the host-type first and then add additional components.
In Gradle tools version >= 2.0, the order between these two declarations is no longer significant.
The same consideration like in the recipe Cookbook - Gradle Assembly Tools#Add Cartridges to an Assembly regarding transitive vs. intransitive dependencies applies here.
To add infrastructure components including their transitive dependencies use:
assembly { hostTypes { //TODO: replace 'appserver' by the host-type's name appserver { // TODO: Specify organization and names of the cartridges include 'com.example:component1', 'com.example:component2' } } }
Recommendation
In the case of infrastructure components, Intershop recommends using non-transitive dependency inclusion for precise control over the generated assembly.
To offer a different set of cartridges in a new deployment scenario, the developer needs to define a new environment within an assembly.
Use the method assembly.environments.create:
assembly { environments { //TODO: Specify the name of the new environment create 'preproduction' } }
The new environment initially contains no cartridges. Refer to the recipe Cookbook - Gradle Assembly Tools#Add Cartridges to an Assembly for adding cartridges to this environment.
Example: Add a new environment to your assembly that inherits all the cartridges from the production environment
Solution: Add the environment “prelive” to your assembly build.gradle
assembly { … environments { create ('prelive') { production.cartridgesConfiguration.dependencies.all { cartridgesConfiguration.dependencies.add(it) } } } … }
For Platform Customers
In order to use the new environment types for the deployment, it is necessary to include this change in the settings.gradle file for each environment. To do this, please create a new service request via Service Portal and ask for changing the environment type for the affected environments.
To offer a different set of infrastructure components in a new deployment scenario, the developer needs to create a new host type in an assembly.
Add a declaration in the assembly.hostTypes:
assembly { hostTypes { //TODO: Replace 'imageServer' by the name of your host-type imageServer { //TODO: Set the flags (see Concept for meaning of the flags) includeLocal = false includeShare = false includeCartridges = false includeJavadoc = false } } }
The new host type initially contains no components. Refer to the recipe Cookbook - Gradle Assembly Tools#Add Infrastructure Components to an Assembly for adding components to this environment.
I am using dependency management for third-party libraries with Intershop 7 version 7.6 or later, but the default behavior does not work the way I need it.
When including cartridges transitively (see Recipe: Add Cartridges to an Assembly), the depending libraries become system components themselves to the assembly. Therefore dealing with libraries equals the dealing with any other system component.
When including cartridges transitively, Gradle will not only include their direct dependencies, but also the dependencies of those dependencies and so on.
You can use standard Gradle mechanisms if you need to exclude libraries that come in via this transitive closure:
configurations.all { exclude group:'com.example', module:'excluded-lib' }
The assembly does not allow conflicting versions for any system component. In case some library gets included in multiple versions, you will see an exception like:
> Could not resolve all dependencies for configuration ':development-cartridges'. > A conflict was found between the following modules: - com.example:conflicting-lib:1.2.3 - com.example:conflicting-lib:1.0.0
You may either
Resolve the version conflict by using version properties, e.g.:
version.com.example.conflicting-lib=1.2.3
Overriding Library Versions
or
A new verification has been added, that inspects the Jar files of all libraries in the assembly. In case some Java classes are contained in more than one dependency an error is thrown, e.g.:
> There are class collisions in your dependencies > Collision between com.google.code.findbugs:annotations:2.0.0 and com.google.code.findbugs:jsr305:2.0.1 > javax.annotation.CheckForNull > [...]
You can inspect the origin of different dependencies as described below and can use Transitive Dependency Management to exclude unwanted dependencies to resolve collisions.
The Gradle task checkClassCollisions
is part of the life cycle task check
and usually run during CI build. However, it is not executed automatically in the developer workflow, but can be invoked explicitly:
gradlew checkClassCollisions
In case you are wondering about the origin of some unwanted dependency, you can use Gradle to get some insight, e.g.:
gradlew dependencyInsight --dependency slf4j-api
You do not need to specify the configuration to introspect, as the assembly process defaults to the environment development.
Debugging Version Conflicts
dependencyInsight
task requires you to first resolve the conflict before it can generate its report. The resulting report will still contain information about the origin of conflicting dependencies.In general libraries are inherited from the parent assembly just like other system components. See Recipe: Customize Assembly Inheritance for more details.
Libraries excluded in the parent assembly will not be inherited to the child assembly. However, if you add cartridges to your child assembly, that yield the same unwanted transitive dependencies, you have to exclude those dependencies again.
Example
Suppose you have an assembly parent, that includes cartridge A, but excludes the transitive dependency evil.
When the assembly child inherits from parent, evil will not get inherited. However, when child adds cartridge B transitively, this will include the evil dependency again. This state is depicted in the following illustration:
To get rid of lib evil in the child assembly, it must also be excluded there.
The same holds true for version conflict resolutions.They are inherited for the transitive dependencies of inherited system components, but may reappear for newly added system components.
The developer needs to create an assembly inheriting contents from an existing assembly.
Use the method assembly.inheritFrom
to declare inheritance from an existing assembly:
assembly { inheritFrom('com.example.assembly:base-assembly') { // Control which dependencies/artifacts to inherit, see discussion // By default all dependencies in all component subsets and all artifacts are inherited } }
To select which artifacts and dependencies should be inherited, pass a closure to assembly.inheritFrom
and inside it call includeArtifacts
/ excludeArtifacts
as well as includeDependencies
/ excludeDependencies
any number of times.
This API is largely inspired from Gradle's way to declare which files to copy in a copy operation, called a "copy spec". See the Gradle user guide for drawing parallels. To stay in the naming, the closure passed to the inheritFrom
method is called an "assembly inheritance spec".
Each call to includeDependencies
/ excludeDependencies
defines a predicate, i.e., a function that returns true or false given a dependency from the inherited assembly. A certain dependency is inherited if and only if it matches:
As an exception to this rule - if there are no includes declared, all dependencies are considered included. (Excludes are still effective in this case.)
This process is performed for each assembly subset (e.g., host-type, environment, configuration - see Concept - Gradle Assembly Tools). For example, if a cartridge is included in the environments production and development in the inherited assembly, you may select to inherit it only for environment development.
The following listing shows the different ways to define a predicate using different overloadings of includeDependencies
. The same overloadings are available for excludeDependencies
.
Note
Component subsets (host-types, environment, configurations) are available as object literals by their name inside the inheritFrom
closure. Examples in the code below are production
and development
.
assembly { inheritFrom('com.example.assembly:base-assembly') { // Include all dependencies in the given component subsets. includeDependencies from:[production, development] // Include dependencies by their (unqualified) name in all component subsets they were originally in includeDependencies 'cartridge1', 'cartridge2' // Include dependencies by their (unqualified) name, but only in the given component subsets includeDependencies 'cartridge1', 'cartridge2', from:[production, development] // Include dependencies based on the return value of a closure for arbitrary complex predicates includeDependencies { ModuleVersionIdentifier dependency, componentSubset -> //TODO: Return true or false, based on properties of the dependency. } } }
There are two pitfalls here that may not be obvious:
Pitfall 1: No warning for missing includes
Includes are only a statement for filtering, they do not ensure the existence of a certain dependency. If you specify includeDependencies 'cartridge1'
and the base assembly contains 'cartridge1', it is inherited. If it does not however, there is nothing to inherit, but no warning will be given about that. Think of it as how a file filter in a directory copy operation behaves, instead of how a command to copy a specific file would do.
To explicitly check that a certain cartridge is included, add the following line after the inheritFrom
closure:
//TODO: Replace 'production' by name of the environment and 'cartridge1' by the name of the of the cartridge assert project.configurations.'production-cartridges'.dependencies.find { Dependency dependency -> dependency.name == 'cartridge1' }
Pitfall 2: A single include changes it all
If you do not specify any includes, all dependencies are inherited. Note that if you start to specify any include, they will start to kick in and only explicitly included dependencies are inherited. See the next section for how to overcome this.
includeDependencies
and excludeDependencies
If you want to declare includes explicitly for some known subset of dependencies, but just inherit everything in others, you can limit the effect of includes and excludes by scopes. This is very similar to Gradle's nesting of copy specs, like providing different include patterns for different sub-directories.
Available scopes are:
environments
, that contains all environmentshostTypes
, that contains all host typesScopes are opened by calling the method within
in the assembly inheritance spec. This method must be passed a closure, which is an assembly inheritance spec itself, i.e., it can also call includeDependencies
and excludeDependencies
.
Consider the following examples:
assembly { inheritFrom('com.example.assembly:base-assembly') { includeDependencies 'cartridge1', from:production } }
assembly { inheritFrom('com.example.assembly:base-assembly') { within(production) { includeDependencies 'cartridge1' } } }
In both examples the environment only contains the cartridge1 inherited from the environment production. In example A however, cartridge1 is the only dependency inherited from the assembly. No dependencies are inherited from other environments or host-types.
In example B, the includeDependencies
call does not affect any other component subset than production. So - because this is default when there are no explicit includes - other environments, host-types and configurations are inherited with all their dependencies.
Scoping also works for excludeDependencies
calls, but calls within a scope and calls using the from
notation are equivalent.
The following example shows, how to:
assembly { inheritFrom('com.example.assembly:base-assembly') { within(environments) { includeDependencies from:[production, development] } within(appserver) { includeDependencies 'runtime' } } }
The selection of artifacts works very similar to those of dependencies. Currently all artifacts are associated with the Ivy configuration default
, so it is not necessary to look for artifacts by their assembly subset.
The following listing shows the different ways to define a statement using different overloadings of includeArtifacts
. The same overloadings are available for excludeArtifacts
.
assembly { inheritFrom('com.example.assembly:base-assembly') { // Include artifact by their type includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle'] // Include artifacts by their name (excluding their extension) includeArtifacts name:'branding' // Include artifacts based on the return value of a closure for arbitrary complex predicates includeArtifacts { ResolvedArtifact artifact, componentSubset -> //TODO: Return true or false, based on properties of the artifact. } } }
Host-types, environments and configurations that exist in the assembly are automatically copied if any dependency or artifact is inherited from them. If no dependency / artifact is inherited, they are omitted.
Note
If you define a component subset in your assembly which already exists in the inherited assembly, then your subset takes precedence (this specifically applies to host-type attributes).
(For Gradle tools version 1.0-1.1, for Gradle tools version >= 2.0 see Recipe: Run DBInit/DBMigrate )
The Continuous Integration server should create a database dump during the assembly process and include it as artifact of the assembly, so it can be used for faster setup of demos / developer environments. The maintainer of the assembly needs to declare this.
Perform the following steps, see discussion for details:
To include the database dump as an artifact, add the following line to gradle.properties file in your Gradle user home:
releaseWithDump = true
If this property is set, the database dump will be generated before publishing the assembly and the published assembly will include the database dump as artifact of type dump. Typically this is used only in the Gradle user home of the CI server, as publishing assemblies for developers must be fast and developers do not need to generate dumps themselves. You may still set it for a developer temporarily for testing the behavior.
This also includes running the deployment, before the dbinit upon every call of the 'publish' task. (See Concept - Gradle Assembly Tools for an overview of the assembly process.)
You may skip the deployment if you are sure that the last deployment succeeded and nothing relevant has changed. To do so add the project property skipDeployment
to your execution. For example:
gradlew publish -PskipDeployment
You may choose between:
A typical use case for incremental dumps is adding demo data in a special assembly with a demo cartridge, that otherwise inherits from a non-demo assembly.
To create a dump incrementally insert the following block in to the build.gradle file of the assembly (note that this uses an assemblyBuild
block as root element):
assemblyBuild { database { //TODO: Replace by <org>:<name> of the assembly you want to inherit database dump from inherit('com.example.assembly:base-assembly') //TODO: Insert list of cartridges for which the partial DBInit should be executed initCartridges = ['cartridge1', 'cartridge2'] } }
For this to work, you have to find an assembly that contains a suitable dump (e.g., containing a subset of the cartridges in your assembly). This is typically the same assembly that you inherit dependencies from.
No Changes of Locales
One known restriction of incremental database dumps is that you cannot add or remove locales. If you want to reuse the dump, you have to make sure to configure the same locales in <IS_SHARE>/system/config/cluster/localization.properties file as where used for the base dump.
If an incremental generation of the database dump is not possible because of some reason, you can always revert to the slower generation from scratch.
If you inherit from another assembly, that you built locally, make sure that it was built with releaseWithDump
enabled (otherwise there is no dump to inherit and the incremental dump generation will fail).
The BrandingPreparer
in bc_foundation is used to install branding packages during DBInit. It extracts branding files into the IS_SHARE/sites folder. When initializing a database with a pre-generated dump, instead of running DBInit, these files are missing.
To work around this, the assembly process can also package the extracted branding installations from the sites folder and include them for deployment. There are two cases to consider:
Case: The DBInit in your assembly runs the BrandingPreparer
, but the DBInit of the base assembly does not. (Or you do not have a base assembly / perform a full DBInit anyway.) Add the following line to build.gradle:
apply plugin: 'ish-assembly-branding'
This activates packaging of branding installations after generation of the dump and publishs them as an artifact branding of type share. The package will be automatically deployed by standard Intershop 7 build plugins.
Case: The DBInit in your assembly does not run the BrandingPreparer
, but the DBInit of the base assembly did. (If unsure, check out the ivy.xml file of the base assembly in the repository for an artifact of name branding and type share.) Add the following line to the build.gradle file (or just add an include to the existing inheritFrom
):
assembly { // TODO: Replace by <org>:<name> of your inherited assembly inheritFrom('com.example.assembly:base-assembly') { // Control which dependencies/artifacts to inherit, see discussion // By default all dependencies in all component subsets and all artifacts are inherited includeArtifacts name:'branding' } }
This will inherit the packaged branding installations from the base assembly.
It is currently not possible to run the BrandingPreparer
both, in the base assembly and in a partial DBInit in your assembly. In this case you have to revert to a full DBInit.
Some of the database content is encrypted. To decrypt the content again:
.
algorithm=PLAIN
, e.g., no encryption at all.Before running the DBInit for creating a dump, the assembly process will automatically modify the encryption.properties to use PLAIN
as encryption algorithm:
intershop.encryption.0.id = demo intershop.encryption.0.algorithm = PLAIN intershop.encryption.0.passphrase = topsecret intershop.encryption.0.default=true
(For Gradle tools version >= 2.0, for Gradle tools version 1.0-1.1 see Recipe: Generate a Datatabase Dump)
The Continuous Integration server should run DBInit and/or DBMigrate during the assembly process for two reasons:
The maintainer of the assembly needs to declare this.
releaseWithDump
.dbinit
or dbmigrate
task.You may configure DBInit and DBMigrate in the same assembly, but never run both in the same Gradle execution.
You may choose between:
A typical use case for incremental dumps is adding demo data in a special assembly with a demo cartridge, that otherwise inherits from a non-demo assembly.
To create a dump incrementally insert the following block in to the build.gradle file of the assembly (note that this uses an assemblyBuild
block as root element):
assemblyBuild { dbinit { //TODO: Replace by <org>:<name> of the assembly you want to inherit database dump from importFrom('com.example.assembly:base-assembly') //TODO: Insert list of cartridges for which the partial DBInit should be executed initCartridges = ['cartridge1', 'cartridge2'] } }
For this to work you have to find an assembly, that contains a suitable dump (e.g., containing a subset of the cartridges in your assembly). This is typically the same assembly that you inherit dependencies from.
No Changes of Locales
One known restriction of incremental database dumps is that you cannot add or remove locales. If you want to reuse the dump, you have to make sure to configure the same locales in <IS_SHARE>/system/config/cluster/localization.properties file as where used for the base dump.
If an incremental generation of the database dump is not possible because of some reason, you can always revert to the slower generation from scratch.
If you inherit from another assembly, that does not include a dump and the project property releaseWithDump
is false, a warning is logged and a full dbinit is executed (this is typically the case for developers).
The BrandingPreparer
in bc_foundation is used to install branding packages during DBInit. It extracts branding files into the IS_SHARE/sites folder. When initializing a database with a pregenerated dump, instead of running DBInit, these files are missing.
To work around this, the assembly process can also package the extracted branding installations from the sites folder and include them for deployment. There are two cases to consider:
Case: The DBInit in your assembly runs the BrandingPreparer
, but the DBInit of the base assembly does not (or you do not have a base assembly / perform a full DBInit anyway). Add the following line to build.gradle:
apply plugin: 'ish-assembly-branding'
This activates packaging of branding installations after generation of the dump and publishs them as an artifact branding of type share. The package will be automatically deployed by standard Intershop 7 build plugins.
Case: The DBInit in your assembly does not run the BrandingPreparer
, but the DBInit of the base assembly did (if unsure, check out the ivy.xml file of the base assembly in the repository for an artifact of name branding and type share). Add the following line to the build.gradle file (or just add an include to the existing inheritFrom
):
assembly { // TODO: Replace by <org>:<name> of your inherited assembly inheritFrom('com.example.assembly:base-assembly') { // Control which dependencies/artifacts to inherit, see discussion // By default all dependencies in all component subsets and all artifacts are inherited includeArtifacts name:'branding' } }
This will inherit the packaged branding installations from the base assembly.
It is currently not possible to run the BrandingPreparer
both, in the base assembly and in a partial DBInit in your assembly. In this case you have to revert to a full DBInit.
Before running DBMigrate a database dump must be imported as start point of the migration.
To import a dump created by the CI server in an older version of the assembly insert the following block in to the build.gradle file of the assembly (note that this uses an assemblyBuild
block as root element):
assemblyBuild { dbmigrate { //TODO: Replace by <org>:<name>:<version> of assembly to import old dump from importFrom('com.example.assembly:assembly:1.0.0.0.+') } }
Alternatively provide a dump file explicitly:
assemblyBuild { dbmigrate { //TODO: Replace by dump file to import. This example assumes that the dump file is located in the source directory of the assembly importFile = file('dump-1.0.0.0.dmp') } }
To include the database dump as an artifact, decide whether it should originate from running DBInit or DBMigrate.
Add one the following lines to gradle.properties file in your Gradle user home:
# include DBInit dump releaseWithDump=dbinit # include DBMigrate dump releaseWithDump=dbmigrate
If this property is set and you execute the task publish
:
dbinit
/dbmigrate
task and their dependencies (see next section) are executed.Typically this is used only in the Gradle user home of the CI server. The default value of releaseWithDump
is false
.
If releaseWithDump
is false
and you execute the task publish:
dbinit
nor dbmigrate
are executed andThis way publishing assemblies for developers is fast. You may still set it for a developer temporarily for testing the behavior.
You may also run dbmigrate
/ dbinit
tasks explicitly. This will also trigger their dependencies:
If any of the prerequisites or dbmigrate
/ dbinit
itself fail, the build will fail as well. This can be used to test dbinit
/dbmigrate
.
It will not automatically export the dump, nor include it as an artifact in the assembly.
You may skip the deployment if you are sure that the last deployment succeeded and nothing relevant has changed. To do so add the project property skipDeployment
to your execution. For example:
gradlew dbinit -PskipDeployment
Note
This recipe is only valid for the additional Solr6 assembly for 7.9. If this assembly is not used, the original recipe from 7.8 remains valid.
To use Solr6-based search in the Commerce Management application and storefront, it is necessary to configure the application server so it can access the Solr6 cluster.
Note
Precondition
A Solr cloud cluster has to be set up. For information about the setup see Getting Started with SolrCloud in the Apache Solr Reference Guide.
The Solr6 extension needs to be configured for the host type share.
... assembly { ... config{ solrcloud { zooKeeperHostList = '123.456.789.012:1234';'123.456.789.013:1234' clusterIndexPrefix = 'myHost-1' } } }
The SolrCloudDeploymentPlugin
configures <IS_SHARE>/system/config/cartridges/ac_search_solr.properties to enable Intershop 7 to connect with the Solr cluster.
Properties relevant for the Solr-based search service are:
Property | Description | Example | Default Value |
---|---|---|---|
solr.zooKeeperHostList | The list of ZooKeeper connection URLs (separated by ';'), which manage the Solr cluster | 'localhost:9983' | no |
solr.clusterIndexPrefix | The prefix to distinguish Solr indexes within the Solr cluster |
'
| <HOSTNAME>-<INSTANCE-ID> |
For further information about setting up a Solr cluster and more detailed description, please see Guide - Deployment Solr6.
Two or more host types should be used to form a new one.
includeJavadoc, includeCartridges, includeShare, includeLocal
for the new new host type, the corresponding values of the previous host types are OR
linkedThe following example refers to the merge of the host types:
to a new host type
assembly { hostTypes { appserver{ include ( 'com.intershop:3rd_tomcat', 'com.intershop:3rd_ant', // further components from appserver ) { transitive = false } includeLocal = true includeCartridges = false } } }
assembly { hostTypes { discoveryserver { include ( 'com.intershop.microservice:DiscoveryServer', ) { transitive = false } includeLocal = true includeShare = false includeCartridges = false } } }
assembly { hostTypes { discoveryappserver { include ( 'com.intershop:3rd_tomcat', 'com.intershop:3rd_ant', // further components from appserver 'com.intershop.microservice:DiscoveryServer', ) { transitive = false } includeLocal = true includeShare = false includeCartridges = false } } }
The use case is that you want to include a newer version of an external library which is already included (and delivered) by gradle. This example explains this use case with the library hierynomus that depends on bouncycastle.
Perform the following stept, see discussion for details:
In your project there is a dependency on library hierynomus which has internal depency of bouncycastle. The build of the cartridge and "refresh gradle dependencies" with the new version of bouncycastle work fine. However, during deployment, the old version of bouncycastle is still used and the created jar file is put into the lib directory share\system\cartridges\libs\release\lib. You need to tell gradle to use your specific configuration, which will be created in the following.
This is how your gradle file looks currently:
To configure gradle you need to tell gradle that there is a separate configuration file for a specific library (bouncycastle in this case) which has to be used. The new configuration must be at the first place (first one wins).
The configured build.gradle file is shown below:
In the following file (overrides.version) you specify the version of the library.
The build and refresh gradle dependencies still works. The last step is the deployment of the system/cartridge. To make sure that the specific configuration file was picked, you should find the correct version of the file in the lib folder: share\system\cartridges\libs\release\lib.
In the gradle dependy graph you will see, e. g., the following output:
Error rendering macro 'code': Invalid value specified for parameter 'firstline'+--- org.bouncycastle:bcprov-jdk15on:1.49 -> 1.57