Introduction

Introduction

This document describes the integration of Cloudinay CDN in ICM 7.10.31.0

This includes collecting images and ZIP archives containing images from external sources (e.g.: SFTP) supported by the Guide - Transport Framework (valid to 7.10) and copying them to a temporary location of the ICM shared file system (SFS).
From there the images get uploaded to the Cloudinary CDN space. In the last step the images get assigned to products.

Furthermore, this document describes how to set up the SMC job and the Transport Framework in the Intershop Organization Management for different scenarios and different image locations.

Preconditions

• Either the managed service or the properties configuration for Cloudinary must be set up and activated. Without it, none of the Cloudinary-related features described here will work.
• The automatic assignment only works if the image file names handled match the following pattern: <product-sku-the-image-must-be-assigned-to>_<imageView-existing-in-ICM>.<image-file-extension>.
• Images for different domains should use different locations in the SFS and the Cloudinary-CDN sub-directory. (e.g.: inSPIRED or inSPIRED-inTRONICS)

Glossary

References

• Cookbook - Cloudinary Image Server
• Guide - Transport Framework (valid to 7.10)
• Concept - Image Framework
• Concept - Managed Service Framework
• Cloudinary - Documentation

Cloudinary CDN Account

To set up an ICM image server, a Cloudinary CDN account is required.

This can be either an existing account or an account provided by Intershop.

Account Data

The following data is required for the setup:

• Cloud name
• API Key
• Cloudinary Account Secret

This information is used to access the account and store images or other data.

Configuration

The configuration can be done either with properties or database preferences. These are retrieved via the Configuration Framework or stored in a Managed Service configuration.

It is possible to have a configuration for an organization that is shared to all channels or to have an individual configuration for each channel.

Image Management - Image Type

The following section deals with Image Types and related settings, specifically in the Cloudinary Context.

Image Type Source - Cloudinary

A new image type source called Cloudinary was introduced. This image type source is used for everything related to Cloudinary-CDN:

• Automatic image product assignment

Image Types Setup

To show images in storefront pages, the image types 'S', 'M' and 'L' are used in various isml templates.

To enable all Cloudinary-related functionality, the image types must use 'Cloudinary' as Source. 

Furthermore, these image types and associated image views must be configured as described in Concept - Image Framework.

Image Type Prefix

The image type prefix contains the commonly used first part of each image URL. The last part of the image URL is the image base name which is stored when the image is assigned to a product or category.

Example for a complete image URL with Cloudinary: https://res.cloudinary.com/intershop-cdn/h_270,w_270/inSPIRED/08154714_front.jpg.

Prefix for Source Cloudinary

The following table explains each part of the prefix used for the example:

https://res.cloudinary.com/intershop-cdn/h_270,w_270/q_auto/

Configuration with Properties

This section describes how to configure Cloudinary via properties files for two different scenarios.

The configuration framework allows to place properties for a specific domain like: 'inSPIRED' or 'inSPIRED-inTRONICS'.

Location of the Properties Files

For this scenario, two properties files are required. One for the organization and one for the channel to enable sharing the organization's configuration.

1. The properties file for the organization is located in: server/share/system/config/domains/<domain-name-of-the-organization> and its name could be imageServerConfiguration.properties.
   Example: server/share/system/config/domains/inSPIRED/imageServerConfiguration.properties.
2. The properties file for the channel is located in: server/share/system/config/domains/<domain-name-of-the-channel> and its name could be imageServerConfiguration.properties.
   Example: server/share/system/config/domains/inSPIRED-inTRONICS/imageServerConfiguration.properties

Setup for Organization

This describes the setup for the Organization and the sharing to individual channels.

Content of the Properties File for the Organization

The basic access data to the Cloudinary CDN account must be provided here. This is required to access the Cloudinary API and stored image files.

intershop.image.server.configuration.available=true
intershop.image.server.configuration.cloud_name=<cloud-name-from-cloudinary-account>
intershop.image.server.configuration.api_key=<api-key-from-cloudinary-account>
intershop.image.server.configuration.api_secret=<api-secret-from-cloudinary-account>

Example:

intershop.image.server.configuration.available=true
intershop.image.server.configuration.cloud_name=intershop-cdn
intershop.image.server.configuration.api_key=7337XXXXXXXXXXX
intershop.image.server.configuration.api_secret=6OKQXXXXXXXXXXXXXXXXXXXXXXX

Content of the Properties File for the Channel

Here is the configuration to use the configuration of the organization in the channel as well.

intershop.image.server.configuration.available=true
intershop.image.server.configuration.share_from_active=true
intershop.image.server.configuration.share_from_domain=<domain-name-of-the-organization>

Example:

intershop.image.server.configuration.available=true
intershop.image.server.configuration.share_from_active=true
intershop.image.server.configuration.share_from_domain=inSPIRED

Setup for Channel

This describes an individual setup for the channel that works independently of the organization.

With multiple Channels in ICM, some channels can use the organization's setup and some channels can use their own.

Content of the Properties File for the Channel

The basic access data to the Cloudinary CDN account must be provided here. This is required to access the Cloudinary API and store image files for the Channel.

intershop.image.server.configuration.available=true
intershop.image.server.configuration.cloud_name=<cloud-name-from-cloudinary-account>
intershop.image.server.configuration.api_key=<api-key-from-cloudinary-account>
intershop.image.server.configuration.api_secret=<api-secret-from-cloudinary-account>

Example:

intershop.image.server.configuration.available=true
intershop.image.server.configuration.cloud_name=intershop-cdn-channel-1
intershop.image.server.configuration.api_key=7337XXXXXXXXXXX
intershop.image.server.configuration.api_secret=6OKQXXXXXXXXXXXXXXXXXXXXXXX

Configuration with Managed Service

Another way to configure Cloudinary CDN image server is to use a managed service.

To use an available managed service (e.g. an image server), it must be activated for the use in an organization in Intershop Organization Management (Operations back office). 

Then it can be created and configured for the organization and its channels - as a shared service and as an individual service, individually for each channel.

Activation - Cloudinary CDN Image Server Managed Service

To activate the Cloudinary CDN  Service, perform the following steps:

1. Log in to Intershop Organization Management and go to Sales Organizations | inSPIRED | Services.
2. Select Image Services and enable Cloudinary CDN Service.

Configuration - Cloudinary CDN Image Server Managed Service

1. Log in to Intershop Commerce Management and open Services (on organization level - inSPIRED).
2. Click New at the bottom of the page and select Image Services - Cloudinary CDN Service as service type.
3. Click Next and provide the following data: 
   • ID
   • Name
   • Description (optional)
     
4. Check both, Active and Available
5. Click Next and provide configuration settings. The minimum data required for the setup are:
   • Cloud name
   • API Key
   • Cloudinary Account Secret
     
     
6. Click Next to select sharing rules.
   Like in properties configuration, the sharing options are required if the managed service of the organization is to be used for channels as well.
   Available sharing options are described in the following screenshot:
   
7. Click Finish.
   This creates the managed service.

Cloudinary Managed Service Setup

The Cloudinary managed service can be set up for the organization (e.g. inSPIRED) and can optionally be shared to channels (e.g. inTRONICS) and applications below.

The domain (e.g. inSPIRED or inSPIRED-inTRONICS) in which the managed service is created, relates to the folders used when uploading images to Cloudinary and for the image product assignments.

In case the Cloudinary managed service is not visible, it has to be activated for your sales Organization, see Activation - Cloudinary CDN Image Server Managed Service.

If the Cloudinary managed service is visible create the new service with configurations like this.

Configure Cloudinary Account

The first (top) part of the configuration contains login data from the Cloudinary account used.

• Cloud name
• API Key
• Cloudinary Account Secret

Full Cloudinary Service Configuration Example

Transport Framework Setup

The transport framework provides the means to copy (pull) images from an external SFTP Server and store them in the SFS for further processing, e.g. for image upload and assignments. There are several possibilities to get images into the SFS via the transport framework, such as E-Mail, Azure, etc. These can be configured, see Guide - Transport Framework (valid to 7.10).

The 'images' folder in the SFS is used as temporary container before the images get uploaded onto Cloudinary-CDN and removed from SFS when all steps are successfully done.

The Transport Configuration TransportProcessID and the domain (Unit) for which it was created (e.g.: inSPIRED-MasterRepository or inSPIRED-inTRONICS) are later required in the Job configuration.

Images can be located in and used for organization (e.g.: inSPIRED) and channel (e.g.: inTRONICS). If both (organization and channel) are required, an individual configuration for both is necessary.

Transport Configuration for Organization

To view or create a transport configuration for the inSPIRED organization, choose inSPIRED-MasterRepository as Domain and click Apply.

For information on creating a new transport configuration, refer to Organization Management Online Help | Transport Framework | Creating a New Configuration

The following sections show an example configuration for the organization:

Configuration - Common Settings

Configuration - Remote Location Settings, Authentication Settings, Transfer Settings

Transport Configuration for Channel

To view or create a transport configuration for the inTRONICS-channel, choose inSPIRED-inTRONICS as Domain and click Apply.

For information on creating a new transport configuration, refer to Organization Management Online Help | Transport Framework | Creating a New Configuration.

The following sections show an example configuration for the channel:

Configuration - Common Settings

Configuration - Remote Locations, Authentication, Transfer Settings

Job Creation for Import Cloudinary-CDN

A job that triggers and performs the pull from SFTP, the upload to Cloudinary-CDN and the product image assignment process must be created in Intershop System Management.

Configuration Details

For general information on creating a new scheduled job, refer to Intershop System Management Online Help | Schedules | Creating New Scheduled Jobs.

Use the following data:

General Tab

Provide the following data in the General tab:

• Pipeline name: ImageImportJob
• Startnode: UploadWithSFTP

The Application is set to the organization or the channel (e.g.: inSPIRED-Site / enterprise (default) or  inSPIRED-Site / inTRONICS )

Attributes Tab

In the Attributes-Tab - the TransportProcessID, TransportUnitName and UnitName must be configured for all the Jobs pipelines steps to work properly.

• The TransportProcessID, TransportUnitName values combined are used to determine the Transport FW configured specifically for this job, see Transport Framework Setup.
• If the Transport FW configuration cannot be determined, the first step of copying files from an SFTP server into the SFS does not work.
• All following steps (uploading images to Cloudinary and creating product image assignments) will not do anything either.
• It is possible to use another process (other than the OOTB transport FW) to copy files into the SFS folder. This has to be done before this Job runs.

Furthermore, it is possible to provide an optional parameter FileExtensions. This  can be configured to overwrite the default values of handled file extensions.

• The job processes all files, in the images SFS folder, filtered by a configured list of file extensions.
• The default file extensions are *.jpg *.png *.zip.
• If the file extensions need to be customized, they must be stored in a space-separated list (e.g.: '*.bmp *.gif *.jpg *.png *.zip')

• ZIP archives containing images will be automatically extracted into the same images SFS folder and then processed like all other images.

  Note

  The order in which the files are processed cannot be guaranteed, that means images in ZIP archives might replace image files with the same name.

  Not having *.zip in the list of file extensions, will prevent zip archives from being processed.

The following table shows some configuration details and recommended values:

Configuration UnitName

The configuration UnitName and its Domain object is used in several places:

1. As reference to resolve the SFS folder where the 'copy from SFTP to the SFS process' stored the images:
   1. /server/share/sites/inSPIRED-Site/units/inSPIRED/impex/src/images - resolved actual SFS folder
   2. ${IS_SHARE}/sites/${SITE_NAME}/units/${UNIT_NAME}/impex/src/images - possible, see Transport Framework Setup
2. As folder for the images uploaded to the Cloudinary CDN:
   1. https://res.cloudinary.com/<cloud-name>/image/upload/inSPIRED/<product-sku>_<image-view-id>.jpg
   2. https://res.cloudinary.com/intershop-cdn/image/upload/inSPIRED/<product-sku>_<image-view-id>.jpg
      1. example: https://res.cloudinary.com/intershop-cdn/image/upload/inSPIRED/2015784_front.jpg
3. As domain for the image product assignment.

SMC - Example Job Configuration

The following section shows an example configuration in Intershop System Management for both, organization and channel:

The Domain for Which the Job Must Run

Organization
Channel

Create a New Cloudinary Job

Configure Job Attributes

Job Description

The following section, describes what the job does exactly:

1. Read the configuration of TransportProcessID, TransportUnitName and UnitName
2. With the TransportProcessID and the TransportUnitName determine the transport framework configuration for this job, see Transport Framework Setup.
   1. If a configuration was found, execute it to get files from the SFTP server into the 'images' folder of the SFS.
   2. If not found go to the next step.
3. Based on the UnitName the root folder underneath the 'images' folder in the SFS is determined for all the following processes.
   1. Get the file extension of all files to be processed, see Job Creation for Import Cloudinary-CDN.
   2. Check the SFS 'images' folder for files matching the configured extensions.
   3. If the file is a ZIP archive:
      1. ZIP archives containing images will be automatically extracted into the same 'images' SFS folder and then processed like all other images.
      2. The order in which the files are processed can not be guaranteed. I.e. images in zip archives might replace image files with the same name. 
      3. The extracted ZIP archive will be deleted.
      4. Get the list of files to be processed again.
   4. If the file is an image: (The required file name pattern is described here.)
      
      1. From the image file name, extract the product SKU and the image view ID later required for the assignment (this step can be customized).
      2. If the product SKU or the image view ID could not be extracted or the image view ID does not exist:
         • Move the file which can not be processed into the 'images_ignored' SFS folder.
         • The next file is processed.
      3. If both exist and are valid:
         1. Upload image to Cloudinary CDN. Use UnitName as target folder for the image upload.
            The Cloudinary Service must be configured properly and enabled for this to work, see Cloudinary Managed Service Setup.
         2. If the Cloudinary CDN upload fails, the file is moved to the images_ignored SFS folder.
            The next file is processed. The image assignment step is skipped.
         3. Create an image product assignment with the product determined by the SKU, the image view determined by the ID and all view related 'Cloudinary' image types, see Image Management.
         4. Delete image file which was just successfully processed.
         5. The next file is processed.

Process Diagram

The following diagrams show a simplified process as well as the full process of Cloudinary uploads.

The processing of zip archives containing images is not shown in either diagram. However, they are extracted and the contained images are also processed.

Simplified Process Description Diagram

Full Process Description Diagram

This diagram also contains the error handling:

Configuring Different Image Location Scenarios (Organization vs. Channel)

This section provides in-depth configuration details for an individual setup.

This includes example parameters for the organization shared to all channels as well as individual configurations for each channel. 

Organization, Shared to Channel

This setup is to be used for the organization and shared and activated for the channels.

Configuration - Transport Framework

Configuration - Managed Service

Configuration - SMC Job

Folders Details

Individual Configurations for Channels

Configuration - Transport Framework

Configuration - Managed Service

Configuration - SMC Job

Folders Details