Introduction

Glossary

References

• Gradle Userguide

• Gradle Build Language Reference (DSL)

• Gradle Java Documentation

• Apache Ivy

Build Infrastructure and Environment

This section gives an overview about the servers and their configuration, that are needed to run the Intershop build tools in a continuous integration environment. The following figure describes on a very high-level, how these servers interact with each other as well as where the particular build artifacts are located. Please read the Concept - Continuous Delivery Tools -> Infrastructurebefore, if you are not familiar with the overall concept:

Build Environment

The Intershop build tools require some tools and environment variables before they can be executed.

• A Client of the Version Control System is needed to checkout the sources of the system components to be built. There is no dependency to a particular version control system.

• The Oracle Java Development Kit (JDK) is required by Gradle to execute the Gradle Wrapper and which is in the same version, which is required by the Intershop Application Server. The executables (java, javac, etc.) of this installation should be in the execution path of the according build shell. It is also possible to set the JAVA_HOME environment variable.
• Gradle stores caches and configuration in a directory called Gradle User Home . The default location is the folder <OS_USER_HOME>/.gradle . Because the cache in the Gradle user home may consume a lot of disk space, it is recommended to override this default value by setting the environment variable GRADLE_USER_HOME. See also   "Appendix D. Gradle Command Line" of the "Gradle User Guide" .

Project/Corporate Specific Artifacts and Configuration

Gradle comes with a fresh approach for storing environment specific build configuration. Project/corporate specific build configuration is stored / linked from within the VCS with the sources of the project. This way:

• Starting a Gradle build with the correct configuration for a developer mostly consists of checking out the source project and running Gradle (no installation or configuration required).
• Developer and CI server use mostly the same build configuration. This way both builds are kept consistent more easily.
  

This section describes artifacts required to implement this approach. Intershop provides templates from which you can easily create them for your project, see Cookbook - Setup CI Infrastructure (valid to GradleTools 1.1), resp. Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1).

The approach is based on the Gradle webinar "Standardizing and Administering your Enterprise Build Environment".

Corporate Gradle Plugin

The corporate Gradle plugin handles all necessary repository configurations for the company or project. It contains the URL of the repository server and the directory layout of the repositories.

Since Gradle Tools 2.0 the corporate Gradle plugin is replaced by:

• The corporate plugin configuration: a single file corporate.properties wrapped into a jar-file containing all company/project specific details of the repository server.
• An IntershopPlugin defining repositories based on the conventions established in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1). These conventions are common enough that it is not necessary for every development project/company to duplicate this code. This IntershopPlugin reads the corporate.properties to find company/project specific details.

Corporate Gradle Distribution

A corporate Gradle distribution consists of:

• a standard Gradle distribution (as provided by gradle.org) plus and
  
• a company specific Gradle init script applying project specific plugins and configuration. One of these plugins is the corporate Gradle plugin as described above.

Gradle Wrapper

The Gradle wrapper is a small Java program that downloads a Gradle distribution and starts it. It is stored in the VCS with every source project and contains a URL to the corporate Gradle distribution. By checking out this wrapper and invoking Gradle through it, the project specific configuration is automatically loaded on every Gradle invocation .

This makes the Gradle wrapper the preferred way of starting a Gradle build.

Intershop Build Tools

There are different kinds of Gradle plugins in the component oriented build of Intershop. Most plugins come with a default configuration, so for the most system components it is not necessary to create a special configuration. The Intershop Gradle plugins extend the Gradle DSL with their own extensions.

Gradle Plugin Overview

Plugins for Managing Build Configuration

Plugins for managing the repository chain, Ivy patterns and versions. These plugins are typically applied by the Project Gradle Distribution.

Plugins for Different Types of Cartridges

Depending on the contents of a cartridge different plugins must be applied for building it. These plugins cover building most common artifacts in Intershop cartridges.

Plugins for Special Types of Artifacts

The following plugins cover building less common artifacts.

Source Projects

System Component Sets

Intershop components are organized in system component sets. A system component set is build as a multi-project build (see Build Lifecycle in Concept - Continous Delivery Tools  ).

The system component set itself is the root project. Its project directory is the system component set directory. Components are direct child projects, their project directories are direct subdirectories of the system component set directory. The Gradle settings file in the component set directory determines which subdirectories should be treated as components.

Each component directory contains a build.gradle file for component specific build configuration. Common configuration for all sub projects is consolidated in the  build.gradle of the system component set directory.

The system component set directory also contains a Gradle wrapper (see GradleWrapper).

Multi-project Layout

The system component set directory contains the following files and directories:

Multi-project Build Configuration

This configuration is used for all components in a component set. This should reduce the overhead when changes on this level in the configuration are necessary.

apply plugin: 'component-set'
 
description 'Components Business Platform'
group = 'com.intershop.set'
subprojects {
    apply plugin: 'intershop-base'
    group = 'com.intershop'
    intershop {
        productName = 'Intershop'
        productCopyrightOwner = 'Intershop Communications'
    }
    
    project.plugins.withType(JavaBasePlugin) {
        sourceCompatibility = 1.7
        targetCompatibility = 1.7
    }
	
	project.tasks.withType(Test) {
		test.ignoreFailures = true
	}
}
buildscript {
    dependencies {
        classpath group: 'com.intershop.build.gradle', name: 'ish-component-plugin'
		classpath group: 'com.intershop.build.gradle', name: 'assembly'
    }
}

The version of the components in the component set is stored in the gradle.properties.

# project settings
version = 7.4.6.0
 
# build script version
filter.com.intershop.build.set.cartridge-plugins = 1.0.0.0.+

Properties of the form "filter.*" and "version.*" are interpreted by the Versioning plugin, see Versioning.

System Components

The system components reside in a system component set and are part of a multi-project build process as described above. The Intershop Build Tools are based on the standard plugins delivered with Gradle, e.g., the Java Plugin.

The Intershop Build Tools (Gradle build plugins) require a defined layout of particular source project residing in the version control system. The conventions for folder names and contents have two origins:

1. Most folder names are like in the Ant based Intershop build easing the migrations.
2. Additionally, some of the default source folders of the Gradle standard plugins are supported.

Project Layout

The following table gives an overview about the project layout of a system component / cartridge:

Build Configuration

The plugins for different system component types can be combined for an Intershop Cartridge.

Intershop provides a special version plugin, this plugin sets the correct versions from the filter and versions settings in the gradle.properties.

apply plugin: 'java-cartridge'
apply plugin: 'static-cartridge'

intershop {
    displayName = 'Component - SubProject'
	description = 'This is an example'
}

dependencies {
    compile project(':bc_foundation')
    compile project(':bc_mail')
    compile group: 'com.intershop.platform', name: 'core'
    compile group: 'com.intershop.platform', name: 'orm'
    compile group: 'com.intershop.platform', name: 'tools'
    compile group: 'com.intershop.platform', name: 'cache'
    compile group: 'com.google.guava', name: 'guava'
    compile group: 'org.slf4j', name: 'slf4j-api'
}

Published Projects

The publishing process is always a part of the build process. Publishing makes built artifacts available for the usage in other builds and for deployment. In the most cases artifacts are published to shared repositories as part of the CI build. Developers publish artifacts always to a local repository.

An Intershop component consists of more than one artifact. Therefore Intershop uses Ivy Publishing for sharing of artifacts.

The Intershop base plugin applies the Gradle "ivy-publish" Plugin.

Configuring Dependency Versions

Dependency Versions in  build.gradle Files

There are two types of dependencies:

• Dependencies between system components inside the same system component set, i.e., within the same Gradle multiproject build. These are called project dependencies.
• Dependencies between system components in different system component sets. These are called external dependencies.

During build dependencies must be resolved, i.e., to retrieve jar files and other artifacts to build/deploy. Project dependencies are resolved within the source and target folder of the system component set. External dependencies are resolved from one or more repositories, which may contain different versions of the same component. For external dependencies it must therefore be possible to control which of these versions will be used.

It is possible to write dependency version numbers directly into the  build.gradle file, as described by the Gradle User Guide. This however has several disadvantages:

1. The  build.gradle file is under source control. The developer cannot easily switch dependencies to a locally build version.
2. The  build.gradle file cannot be edited automatically, like for automatic integration of newer versions when they become available.
3. Version numbers must be declared separately for each dependency. There is no way to define a common pool of third party versions and apply them with a single declaration.

Using version expressions, like 1.0.0.+, that resolve dynamically to the latest matching version does help with the first two items. It however introduces a strong instability. Executing a command once and then a second time can lead to different results, if in between a new version was published. Also previous build processes cannot be repeated to diagnose problems.

Versioning Plugin

The Gradle build tools come with a versioning plugin that is applied by the project Gradle distribution. The versioning plugin allows to specify dependency versions via Gradle project properties.

The versioning plugin recognizes two different kinds of properties:

Version Properties

Version properties have the syntax: version.<component organization>.<component name> = <version number>, e.g., version.com.intershop.core = 7.4.6.1.20131212121212.

They determine the version of a single component. In the above case all dependencies to core are resolved to version 7.4.6.1.20131212121212.

Filter Properties

Filter properties have the syntax: filter.<assembly organization>.<assembly name> = <version number>, e.g., filter.com.intershop.assembly.intershop7= 7.4.6.1.201312121212.

They determine the version of multiple components at once by specifying an assembly or system component set. In the example above all dependencies to components contained in version 7.4.6.1.201312121212 of the assembly intershop7 will be resolved using the versions given in the assembly's ivy.xml file. For example if intershop7 in version 7.4.6.1.201312121212 contains version 2.2.25.0.20131125224905 of com.intershop:3rd_apache, dependencies to 3rd_apache will resolve to this version. Another example using a component set: Specifying filter.com.customer.set.platform-cartridges = 1.1.0.0-local causes a build against local versions for all system components in this set.

Precedence and Conflicts

In certain situations multiple versions may be selected based on the given version and filter properties. For example, you may set filter.com.intershop.assembly.intershop7= 7.4.6.1.201312121212, containing 3rd_tomcat version 2.2.25.0.20131125224905 and version.com.intershop.3rd_tomcat=2.4.13.0.20141125224905.

The following rules are used to determine which version to choose:

1. If the version of a dependency is given by a filter and a version property, the version property is used. This allows fine grained overriding of filters.
2. If two filter assemblies contain the same component in different versions:
   1. If one of the contained component versions ends on '-local' it will be used. This allows easy replacement of components with locally built versions for developers.
   2. Otherwise an exception will be thrown indicating which filters are conflicting in which component versions.

Filter conflicts can be solved by disambiguation. You can either:

1. Remove one of the filters (which is also possible by setting an empty value for the filter property) or
2. Specify a version property for the conflicted component. This - by above rule - will be used and settle the conflict.

Physical Locations

Gradle project properties can be set in different physical locations:

1. Adding them to the gradle.properties file of a system component set or assembly. This file is checked into source control.
2. Adding them to the  gradle.properties file in the Gradle user home directory. This file can be freely changed by the developer.
3. Specifying as command line parameter using the syntax -P<property>=<value>.


If different locations contain different values for the same property, they override each other in this order. That is:

• Command line parameters win over any gradle.properties entry.
• The gradle.properties in the Gradle User home win over those in the system component set/assembly.

Best Practices

1. Omit versions in dependency declarations in build.gradle file, e.g., write:
compile group: 'com.intershop.platform', name: 'core'
   instead of
   compile group: 'com.intershop.platform', name: 'core', version: '7.5.0.+'


1. Use fixed dependency versions, not dependency expressions for version and filter properties. E.g., it is technically possible to specify a filter filter.com.customer.set.shop-cartridges=1.0.0.+ which will always use the latest available build of the branch 1.0.0. This is discouraged. As mentioned in the beginning of this chapter this can lead to surprises and timing issues. Instead always give full version and build numbers.
2. Check-in all version properties that are unlikely to differ between multiple developers or between developers and the CI server into the gradle.properties file of the system component set and assembly. Especially dependencies to components by other vendors and build and deployment plugins. Developers can always still override them if they have to in their Gradle User Home.
3. Creating or changing version and filter properties can be automated easily , including updating to the latest available version. Differing from version expressions, this is a controlled action and visible in the file - especially if the file is under source control. Specification of filter properties is mostly automated for the developer, see Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7). This way developers do not have to manage them, but they may.

The templates provided for the system component set and assembly already follow these practices. See Cookbook - Setup CI Infrastructure (valid to GradleTools 1.1).

Summary

Intershop extends the Gradle DSL for building cartridges and components following Intershop conventions. Most Intershop Gradle plugins come with a default configuration.

Since the Intershop component oriented build only extends Gradle, a project build configuration can be extended with third-party or own Gradle plugins. For special requirements, the build configuration can be adapted and expanded to include additional build steps.