Document Tree
Document Properties
Kbid
K31179
Last Modified
14-Feb-2024
Added to KB
02-May-2024
Public Access
Everyone
Status
Online
Doc Type
Concepts
Product
ICM 11
Concept - Mass Data Cleanup

Introduction

Intershop Commerce Management 11 supports installations with large amounts of data, including products and users. Additionally, it allows for the creation and deletion of web shops at runtime. The document explains how the mass data of these web shops is removed from the database.

Asynchronous Cleanup: To avoid browser timeouts and error pages, mass data cannot be cleaned up during web request execution. Instead, jobs are used to asynchronously remove specific mass data objects. Asynchronous cleanup also allows for scheduling the cleanup job during low web traffic periods, reducing the load on the database.

Transaction Handling and Stored Procedures: Using the persistent API (ORM objects) to cleanup the data leads to a large amount of SQL statements resulting in a high load on the database server and poor runtime performance. Additionally, it is important to consider transaction handling and avoid large transactions in OLTP applications like web shops. Therefore, the cleanup jobs use PL/SQL stored procedures with accordingly optimized transaction handling.

Extensibility: The business objects are defined in one or more cartridges. Implementation partner can also define or extend the business objects, which reference domains, products or users in their own cartridges. Each cartridge is responsible for properly removing referencing objects.

Glossary

This glossary describes the terms in this document:

Term

Definition

PL/SQL/Stored Procedure

PL/SQL (Procedural Language/Structured Query Language) and Stored Procedure are procedural extensions of the language for Databases (SQL)

Implementation

Jobs

Each cartridge responsible for removing cross-cartridge business objects has a pipeline for triggering the removal process. This pipeline is triggered periodically by a job, also prepared by the cartridge.

Currently the following jobs for the root site are available:

  • DeleteDomainReferences

  • DeleteUserReferences

  • DeleteProductReferences

The trigger pipeline executes a deletion pipeline or handler (see below which one) in each started cartridge. The cartridges are processed in reverse order, following the last created, first removed principle.

Mass Data Deletion Jobs

The available jobs for mass data deletion with detailed job parameters are:

Type

Cartridge

Job Name

Job Interval
in minutes

Job
Enabled

Attributes

Pipeline

Handler

Description

Domain

core

DeleteDomainReferences

60

true


TriggerDeleteDomainReferences

Deprecated for removal pipeline calls:

  • PrepareDeleteDomain

  • DomainDeletionPipeline

  • RefreshCache

DomainReferenceDeletionHandler

Triggers deleted domain cleanup.

User

core

DeleteUserReferences

60

true

BlockSize = 200

TriggerDeleteUserReferences

Deprecated for removal pipeline calls:

  • UserDeletionPipeline

  • RefreshCache

UserReferenceDeletionHandler

Triggers deleted user cleanup.

Product

xcs

DeleteProductReferences

60

true

BlockSize = 200

TriggerDeleteProductReferences

N/A yet

Triggers deleted product cleanup. The maximum number of products which the
job can process can be set via the job parameter attribute "BlockSize". The default value is 200.

The System Management Console (SMC) displays the configured deletion jobs, navigate to Schedules | Scheduling within the root domain.

Cartridge Properties (Only Remaining for Type Product)

Each mass data cleanup job calls all pipelines in cartridges defining the corresponding property with the pipeline and start node name:

Example for ac_bmecat:

ac_bmecat.properties

intershop.cartridges.ac_bmecat.ProductDeletionPipeline = BMECAT_RemoveProductReferences-Start

Cartridge Module Multibinding (For Type Domain or User)

Each mass data cleanup job executes all handlers in the loaded cartridges, which include a handler binding for either the interface com.intershop.beehive.core.capi.domain.DomainReferenceDeletionHandler or com.intershop.beehive.core.capi.user.UserReferenceDeletionHandler with the Guice Multibinder:

Cartridge Module Multibinder

public class BcOrganizationNamingModule extends AbstractModule
{
    @Override
    protected void configure()
    {
		// Domain Reference Deletion Handler
        final Multibinder<DomainReferenceDeletionHandler> domainReferenceDeletionHandler = Multibinder.newSetBinder(binder(), DomainReferenceDeletionHandler.class);
        domainReferenceDeletionHandler.addBinding().to(YourDomainReferenceDeletionHandler.class);

		// User Reference Deletion Handler
        final Multibinder<UserReferenceDeletionHandler> userReferenceDeletionHandler = Multibinder.newSetBinder(binder(), UserReferenceDeletionHandler.class);
        userReferenceDeletionHandler.addBinding().to(YourUserReferenceDeletionHandler.class);
    }
}

Stored Procedures

All stored procedures that remove domains, users, and products referencing objects are named

Type

Stored Procedure Name

Domain

sp_delete*ByDomain

User

sp_delete*ForUser

Product

sp_delete*ForProduct

whereby * gives a hint about the objects, the stored procedure removes.

Replication Environments

Intershop Commerce Management 11 alters the table structure in replication environments. This means that each stored procedure must function in both replication and non-replication environments. According utility PL/SQL packages are available.

Deployment

The default location for stored procedures is <cartridge>/src/main/resources/resources/<cartridge>/dbinit/scripts.

All stored procedures removing domains are created and maintained within the database by DBPrepare.

The usage of preparers are:

  • Preparer (DBInit)

    • com.intershop.beehive.core.dbinit.preparer.database.SQLScriptPreparer resources/<cartridge>/dbinit/scripts/

      • <spSingleFile> or

      • spmainfile.ddl

  • MigrationPreparer (DBMigrate)

    • com.intershop.beehive.core.dbmigrate.preparer.database.ExecuteSQLScriptPreparer resources/<cartridge>/dbinit/scripts/

      • <spSingleFile> or

      • spmainfile.ddl

Usage

All stored procedures removing domains are triggered using JDBCUtils.executeStoredProcedureWithDbmsOutput(...) or by several executions of the pipelet ExecuteStoredProcedure.

Example

For example of sp_deleteFooBarsByDomain, see Sample Domain Deletion Stored Procedure.

Disclaimer
The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.
The Intershop Knowledge Portal uses only technically necessary cookies. We do not track visitors or have visitors tracked by 3rd parties. Please find further information on privacy in the Intershop Privacy Policy and Legal Notice.
Home
Knowledge Base
Product Releases
Log on to continue
This Knowledge Base document is reserved for registered customers.
Log on with your Intershop Entra ID to continue.
Write an email to supportadmin@intershop.de if you experience login issues,
or if you want to register as customer.