Introduction

A subscription is an agreement to place the same order several times. When the initial order was placed successfully, the placement of all further orders is automatically triggered within the given time frame depending on the selected recurrence interval.

General Workflow

The basic idea behind the implementation of the subscription feature is to attach recurrence information to an ordinary basket in order to mark it as recurring. Afterwards the marked basket will pass the standard checkout steps incl. approval (if required).

The marking of the basket can be done at any point during the checkout, but due to limitations of payment methods (see section Concept - Subscriptions (valid to 7.10)#Limitations) it is possible that the already selected payment method might be invalid. When the basket is marked as recurrent after the payment method has been selected the basket validation will raise an error in case of an invalid payment method. The order creation process will then be interrupted because of the validation errors.

When the order process was successful, the recurring basket will get a new type and status to preserve it for the whole life cycle of the subscription. Afterwards the Recurring Order microservice is called in order to create a recurring order for this basket. The recurrence information and the resource to access to subscription basket are transmitted. The Recurring Order microservice only stores a reference to the basket in the ICM. The Recurring Order microservice itself creates a new entry, and the Scheduling microservice creates a new schedule corresponding to the given recurrence information.

The Scheduling microservice does not have any knowledge about subscriptions or any other business scenario. It just gets schedule information and a callback URL. When an event has to be triggered the Scheduling service just calls the callback URL. The first execution is always triggered immediately after a new entry is created. The Scheduling microservice calculates the next regular execution date before every execution and delivers this information together with the number of previously failed executions to the Recurring Order microservice.

Accordingly the Recurring Order service is immediately called back from the Scheduling service to create a new order for the according subscription. The Recurring Order service itself forwards the request to the basket service (ICM) to create a new order for the subscription. As mentioned before the subscription basket is stored and will now be used as blueprint for the new basket. More precisely the subscription basket is cloned by a new BasketCloneService. It might be that the clone process will result in a different basket in terms of line items, prices etc. because products may not be available anymore or promotions expired and so on. That is why the BasketCloneService is able to compare the two baskets (original and cloned) and will collect and return differences. So the result of the basket request to create a new basket for the subscription is a new (cloned) basket and possibly a list of differences between the two baskets.

The Recurring Order microservice provides an interface which allows to add a custom handler that evaluates those results and decides whether the basket can be ordered or not. If the check was successful the Recurring Order microservice calls the basket service again to create an order. By default there is no such handler and therefore the check is skipped. If the order is successfully created it will be identified as an order from a subscription and will be linked to that very subscription. So the customer can see which orders belong to a subscription and which are one-time purchases.

There are some additional aspects which are explained in the next paragraphs. The whole workflow will be explained again in more detail and from a technical perspective in section Concept - Subscriptions (valid to 7.10)#Workflows and Concept - Subscriptions (valid to 7.10)#Implementation Details.

Feature Toggle

The subscription feature can be switched ON and OFF per channel.

See Cookbook - Subscriptions | 5 Recipe Enable/Disable the Subscriptions Feature (Toggle) for further details.

When the feature is disabled:

• All affected REST calls will return an error status 501 - Not implemented.
• All subscription related information will automatically disappear from all checkout pages in the storefront.

Fixed Prices

When a new recurring order (subscription) is created it can be decided whether the prices of the included products are fixed or may change over time.
If the fixed price option is selected then only the product prices for all included products are fixed during the whole life cycle of the subscription. Nevertheless the total price of the orders might change because of exceeding promotions, changed shipping costs etc.

If the subscription is created without the fixed price option then each basket will be calculated with the actual prices for the products at the time of the basket creation. This means the resulting orders may differ regarding their costs.

The Recurring Order microservice can handle both scenarios. The fixed price flag is part of the REST API. The flag is transmitted from the ICM depending on the channel preference FixedPriceRecurringOrders which determines whether the fixed price option is active or not for all subscriptions of the channel. See also Cookbook - Subscriptions | 4 Recipe Set Fixed Product Prices for Subscriptions to learn how to change this channel property.

Promotions

After basket clone process the newly created basket will be recalculated. The recalculation will also include recalculation of promotions. This means:

• All types of promotions are automatically applied on all orders created out of subscriptions; currently there is no possibility to exclude them.
• originally applied promotions that become somehow invalid or unavailable for the current customer (expire by date, change removal of customer segments, application assignment etc.) will be automatically removed from the created orders. In case the promotion is available again, it will automatically reappear.
• Fixed Price only refers to a fixed product price; but discounts from promotions will influence the total price anyway.
• Orders out of subscriptions also count for categories like "first order of month".

Payment

Change Payment Method of the Subscription

During the life time of a subscription it might be necessary to change the payment method e.g., because the credit card expired.Therefore the WebShop REST API has capabilities to manage payments of subscriptions:

Changing the payment method might lead to a different total price of the order because of possible payment costs, promotions on payment methods etc. This depends on the configuration of the system.

Fallback

During the order creation process it could happen that payment failures occur because e.g., the credit card is expired. It is possible to define a fallback payment method which is automatically used in these cases. The fallback payment method can be configured per channel and must not need additional information and must not be user-specific like bank accounts. Possible values are e.g., Invoice or Cash in Advance. If there is no fallback configured the order creation will immediately fail.

The original payment method will remain at the subscription basket and is only replaced with the fallback payment method in every cloned basket. If the original payment method becomes invalid for a certain period, it will be replaced by the fallback within this period and when the original payment method becomes valid again it will be used again for all orders without any fallback.

That behavior but can be changed following the the recipe: Cookbook - Subscriptions | 6 Recipe Configure Payment Fallback Method for Subscription.

Addresses

Orders that are created for a subscription use the invoice and shipping address that has been selected in the original order. The addresses are referenced via ID (URN). This means that changes to an existing address are automatically applied to all future orders.

If a used address is deleted then a fallback mechanism will be applied.

The following fallbacks are used:

• Invoice address:
  
  1. Use preferred Invoice address (if exists).
  2. Use address if the customer has only one invoice address left.
• Shipping address
  1. Use preferred shipping address (if exists).
  2. Use preferred invoice address (if exists and is also a shipping address).
  3. Use address if the customer has only one shipping address left.

The fallbacks are applied in the given order. If no fallback can be applied then no invoice or shipping address will be set. This will result in an incomplete subscription which will be automatically deactivated during the order creation process.

Order Creation Date/Time

The start date of the subscription defines the time at which the orders are triggered. Currently There is a difference between the storefront implementation and the implementation for REST.
Our current storefront implementation truncates the time information when submitting subscriptions. Therefore all orders out of subscriptions are created at 0:00 (midnight).

If the recurrence information is set via REST it is possible to specify a time stamp as start date which also includes the time. E.g., if the start date is set to 01/01/2017 07:00 am then all orders for this subscription will be triggered from 01/01 on (according to the schedule) at 07:00 am (plus a couple of seconds for latency).

Next Order Date

It might be interesting to know when the next order will be created according to regular plan. In general this information is exclusively known by the scheduling microservice. In order to make this information also available at the recurring order microservice the schedule service calculates and transmits this date every time the recurring order microservice is called to create a new order for a subscription. The date is then stored in the data base of the recurring order microservice and can be accessed via REST interface in order to accomplish further actions, e.g., inform the customer when the next regular order will be. This date the schedule service provides is only valid in the case that the current order can be successfully created.

If a recurring order starts in the future the schedule service didn't call the recurring order microservice yet. In that case the next order date is not set at recurring order microservice side. When the recurring order details are requested via REST the next order date will be directly fetched from the scheduling service and afterwards stored locally for further requests.

Notifications

A notification is triggered on a successful order of a subscription as well as on order creation errors. The notifications are implemented using Java extension points.The responsive storefront demo store contains a implementation of the Java extension point that sends an e-mail to the Shop Manager if the order creation for a subscription failed.

It is possible to implement custom handlers that can be registered for one of the events (Order success, Order failed). See Cookbook - Subscriptions | 7 Recipe Implement notification for Failed/Successful Orders for technical details.
If no extensions are wired only a basic debug logging is executed.

Order Count

When an order that is created out of a subscription is canceled afterwards it will not be removed (subtracted) from the order count and the number of wanted repetitions for the subscription.

Inactive Subscriptions

Subscriptions can be set to inactive either by the customer via Storefront/WebShop REST API or by the system when an error occurs. To distinguish both cases an additional data base field errorcode contains some error code when the subscription was deactivated as a result of a system error. When this field is empty, it indicates that the customer deactivated the subscription on his own. The field error code is also exposed in the WebShop REST API and processed in the Responsive demo store to display an adequate error message in the subscription details in the my account section.

When a subscription is deactivated with error code (by system) and is activated by the customer again the error code field is cleared.

Subscriptions that have been set to inactive will not create orders anymore. If the subscription is enabled again, depending on the flag executeMissedOrders, orders will be caught up which were missed during the time the subscription was not active, or not. If no orders are caught up the next order will be created at the next possible date regarding the recurrence. The flag executeMissedOrders is included in the recurrence information and is set when a basket is marked as recurrent. So it is possible to adjust this behavior per subscription. The flag is optional. If no value is specified, the default value true is used.

Logging

The logging capabilities are divided into business and technical logging.

There are two business log files:

• schedules.log
• recurringorder.log

which contain business information with a reduced technical scope. They should help to understand what the system does and if there are any problems.

Any technical issue is logged regarding the log level to additional log files:

• error.log
• debug.log
• sql.log
• http.log

The error log also contains the full stack trace of an entry in the business log if this was caused by an exception.

Due to a time based rolling policy all log files are created on a daily basis and old files are automatically zipped.

Resilience

For the subscription feature several microservices are working together. Currently the recurring order microservice, scheduling microservice and the basket microservice (ICM) communicate directly via REST interfaces. Especially the recurring order microservice acts as the leading part in that ecosystem because it controls and distributes the messages flow. If there are technical problems, interruptions, systems failures etc. the whole system should be able to cope with. Particularly the creation of orders of a subscription is one of the most vulnerable points.

That is why the recurring order microservice contains a configurable retry strategy on technical errors during the order creation.This is restricted to technical errors (e.g., ICM/microservice/Eureka not reachable or other unknown technical errors), business errors (e.g., basket not covered due to invalid payment, invalid line items etc.) are handled separately and lead in most cases to the deactivation of a subscription.

This retry strategy means that the failed operation is repeated (several times) after a certain time span. The number of retries and the time span in between is configurable.

Let us assume the following scenario: The configuration says that there are 4 retries on technical errors with an increasing time span (1 minute, 10 minutes, 1 hour, 4 hours), to address the different types of errors (short network interruptions, maintenance breaks etc). The schedule microservice triggers the recurring order microservice because a new order has to be placed for a subscription. Unfortunately in this moment the basket service of the ICM is offline for maintenance reasons, but this service is needed to create an order.

The following picture shows the principle. Since there are 4 configured retries, the total maximum number of attempts to call the services is 5 (initial plus number of retries). The schedule service counts the number of failed attempts (failedcount) and transmits this number in every request. The number is then evaluated by the recurring order microservice to decide whether another retry will be executed or the process will be canceled by deactivating the subscription and disabling the schedule. If the ICM basket service will remain offline during the whole time then place order will fail after the 5 attempts. If the systems recovers within the retry time span the operation can be successfully processed and no further attempts are necessary.

See Cookbook - Subscriptions | 8 Recipe Adapt Retry Strategy in Case of Technical Errors to see how the retry strategy can be configured.

Glossary

References

• Cookbook - Subscriptions (valid to 7.10)
• Concept - Microservice Registration and Discovery (valid to 7.10)
• Cookbook - Eureka Deployment and ICM Configuration (valid to 7.10)
• Concept - Dependency Injection and ObjectGraphs

• Cookbook - Dependency Injection and ObjectGraphs (Recipe: Replace a Manager/Provider/Service Implementation) for further details how to exchange a custom implementation.

• Google Guice Dependency Injection on GitHub
• Reference - WebShop REST API 7.8
• Swagger

Workflows

Mark Basket as Recurrent

Create Recurring Order

Place an Order

Get Subscription List

Get Subscription Details

Remove/Deactivate a Subscription

Before an order is placed, the recurring order service checks, if the subscription is expired. A subscription is expired, when:

• The subscription order count did reach the number of allowed repetitions/occurrences, or
• The end date is already passed

The subscription is NOT expired, when:

• No repetitions are set
• The end date is today

When trying to place an order for an expired subscription, the recurring order service will:

• Deactivate the subscription
• Return HTTP status 410 (Gone)  this will force the scheduling service to delete the schedule for this recurring order
• No order is placed

Implementation Details

Technical Overview

The microservices - except the legacy Basket service - are completely separated from the Intershop Commerce Management (ICM).

They are running in a different environment including different data sources. The microservices itself are independent from each other and therefore easily exchangeable. The communication between all microservices is a HTTP based communication based on the REST API, that every service has to provide. The documented REST API is the fix interface between the ICM and the microservices. Everything "behind" is exchangeable.

The Basket service is a legacy microservice that is still located within the ICM because it would be very complex to extract it due to high dependencies to other core functionalities. But the service behaves like any other microservice and can only be used via REST, too. Therefore the service may be extracted someday and the depending microservices do not have to adapt their communication with that service.

The microservice REST APIs are only accessible internally and thus cannot be called from the outside. Functionality that should be used from outside has to be implemented in the WebShop REST API which itself uses the internal microservice REST API to fulfill a request.

All microservices register themselves at the EUREKA Service Discovery Service. If a microservice wants to communicate with another microservice via REST then it asks the EUREKA Service Discovery to get its address. The microservice will also be informed if no appropriate microservice is available. See Concept - Microservice Registration and Discovery (valid to 7.10) for further details.

Recurring Order Microservice

The Recurring Order microservice contains functions to:

• Create a new recurring order based on given recurrence information
• Get a list of recurring orders for a given owner
• Enable/disable recurring orders
• Place a single order at the scheduled time for a recurring order (subscription)

The Recurring Order microservice interacts with other microservices to fulfill the incoming requests. Enabling/disabling a recurring order will also trigger the Scheduling service to enable/disable the schedule accordingly.
Placing a single order based on a recurring order calls the Basket service which is part of the ICM to create a cloned instance of the stored recurring basket. After cloning the basket a recalculation is triggered. The calculation might cause changes in the cloned basket, e.g., products that are not available anymore are removed, product prices may have changed, promotions might be exceed and so on. That is why the clone method of Basket service returns a list of relevant changes. Some changes might be acceptable for customers, others are not. The interface NextOrderValidator has been introduced to allow some custom implementation to do some validations on the returned basket differences. If the wired NextOrderValidator implementation returns a NextOrderValidationResult which returns isFailure()  == true then the Order creation is skipped, the subscription and the schedule will be disabled (with error code). Manual intervention is required to solve this issue.

See section Concept - Subscriptions (valid to 7.10)#NextOrderValidator for further details.

NextOrderValidator

NextOrderValidator is an interface which is injected via Google Guice Dependency Injection to allow partner/customers to implement a custom solution. For the given input parameter BasketCreatedRO, which contains the calculated differences between the original and the cloned basket, an implementation of this interface can decide whether or not the cloned basket can be processed to create an order for the subscription. Since all basic checks (are there line items, are the line items available etc.) are already done in the order creation process which starts afterwards there is currently no standard implementation of this interface. Partner/customers can implement a custom solution according to their requirements. See Cookbook - Subscriptions | 9 Recipe Implement a Custom Next Order Validator for further information on how to do it.

REST API

As mentioned before the microservices exclusively communicate via internal REST APIs. That is why these REST APIs are described here in detail as follows.

To access recurring orders via REST a repository is needed. There is a list and an item resource for repositories to create, delete, change, get details.
The path is "/repositories/{repositoryID}". The {repositoryID} can be any external ID that might be useful. For recurring orders the ID of the customer is used as repositoryID.

Below, the repository item resource identified by {repositoryID}, the list and item resource for recurring orders are located. All recurring orders are also identified by a unique {externalID} which is the ID of the underlying basket.
It could be any other ID as well. The path is /repositories/{repositoryID}/recurringorders/{externalID}.

The REST API provides functions to create, update, delete, get details of recurring orders. It is also possible to enable and disable recurring orders via separate resources /enable and /disable:
/repositories/{repositoryID}/recurringorders/{externalID}/enable and /repositories/{repositoryID}/recurringorders/{externalID}/disable.

For creating an order out of an recurring order (subscription) the sub-resource orders can be used. Path: /repositories/{repositoryID}/recurringorders/{externalID}/orders.

It is possible to get a complete list of all recurring orders over several repositories filtered by an additional field targetrepository which is stored at the recurring order entries. Per default this value is filled with the name of the channel the subscription exists in.
This is usually used for back office functionality, that is why it is implemented in a separate service called Recurring Order List Service (Backoffice).

The REST API documentation is generated directly out of the Java code using Swagger. It is available under http://<ms-ip>:<port>/api.

There is an overview of all REST calls:

Clicking on an entry will reveal detailed information:

It is even possible to immediately send demo requests.

Persistence

The persistence of the microservice is defined via JPA (Java Persistence API). This enables the customer to freely decide which data base system should be used. It is possible to use the Oracle of the ICM or an external data base like Postgres.
During the server start of the microservice all database tables are automatically created if they do not exist. Even changes of data base tables (extending table) are automatically executed without need to do manual steps.

Basket Service (ICM)

The Basket service (microservice) provides ICM functionality to be used by other microservices:

• Create a basket
  • Empty a basket
  • Clone an existing basket
    • Variable/fixed prices
    • Generate a list of differences between cloned and original basket
• Get basket details
• Edit recurrence information of a basket (change, remove)
• Create an order
  • Subscription or one-time purchase
  • Java extension points (in case of Success and Error order creation)
• Delete a basket

Most of the functionality in this Basket service uses existing artifacts like repositories and pipelines to fulfill the request. Only the possibility to clone a basket was introduced for the subscription feature.
This was implemented in a separate BasketCloneService. It might be that cloned basket might not contain less or changed data because, e.g., products are not available anymore or prices/promotions might have changed.
The BasketDifferenceService collects all relevant fields that differ from the original basket to the cloned one. The differences are returned as a result of the clone process and can be evaluated by the executor of the service.

For more details see the sections Concept - Subscriptions (valid to 7.10)#BasketCloneService and Concept - Subscriptions (valid to 7.10)#BasketDifferenceService.

BasketCloneService

BasketCloneService is an interface which is injected via Google Guice Dependency Injection to allow partner/customers to implement a custom solution.

The standard implementation in BasketCloneServiceImpl currently clones all required basket fields that are required to order the basket. The following aspects are cloned:

• Product line items
• Addresses
• Shipping method
• Payment method

At the end of the cloning process the cloned basket is invalidated and re-calculated.

Besides wiring a BasketCloneService there are two additional ways to adapt the functionality of the standard implementation.
The implementation class defines public method for every aspect like line items, shipping etc. These methods could be particularly overwritten by a custom solution which extends standard implementation class in order to change to implementation.

If some custom functionality should be added to the CloneHandler, a BasketCloneCustomHandler can be defined and wired. There the custom code that has to be done additionally can be placed.

BasketDifferenceService

BasketDifferenceService is an interface which is injected via Google Guice Dependency Injection to allow partner/customers to implement a custom solution.

The standard implementation in BasketDifferenceServiceImpl currently only evaluates a reduced set of attributes:

• Product line item count
• Grand total gross
• Grand total net

As already mentioned above, these attributes can be used by the NextOrderValidator in the Recurring Order microservice to decide whether an order can be created for subscription or not.

Basket Status & Type

The Java class BasketConstants (bts) defines constants for all used basket status and type values. These constants should be used instead of integer values.

Status:

0 - BasketConstants.BASKET_OPEN
21 - BasketConstants.BASKET_RECURRING

Type:

10 - BasketConstants.BASKETTYPE_REQUISITION
11 - BasketConstants.BASKETTYPE_RECURRING

REST API

The REST API of the Basket microservice provides the possibility to

• Create a basket
• Get Basket details
• Edit a basket (only limited to recurrence information)
• Delete a basket
• Create an order for a basket

Here is a detailed overview:

Intershop Commerce Management (ICM)

WebShop REST API

The WebShop REST API has been extended to offer the possibility to mark a basket recurrent. This is done by setting the recurrence information at the basket using the REST call REST API - Set basket related information (7.8) (PUT /baskets/<basketID>).
It is also possible to remove the recurrence information again which marks the basket as not recurrent.

A subscription will be created during the order creation without any further specific interaction.

When subscriptions are available they can be retrieved as list and with details with additional REST calls for B2C as well as for B2B customers. Additionally subscriptions can be disabled and enabled.

Extensions

There are two Business Object Extensions - one for BasketBO, one for OrderBO - to handle Recurring Order related topics.

BasketBORecurringExtension

• Get and set recurrence information from/at basket
• Delete recurrence
• Ask if basket is recurring
• Set status for original recurring order basket and cloned baskets

OrderBORecurringExtension

• Ask if order is from subscription
• Get-and set recurring order ID from/at the order

Communication With Microservices

As already said before the ICM communicates with all microservices always via REST after asking the Eureka service for the URI where the microservice is available. There is no direct communication based on Java classes and method calls etc.

In order to facilitate an easy access to the Basket and Recurring Order microservice from the ICM, service clients have been implemented.

BasketServiceClient to access to the Basket microservice.

RecurringOrderServiceClient to access to the Recurring Order microservice.

Limitations

Restrictions on Payment Methods

Currently there are limitations on the payment methods that can be used for subscriptions. This is required because orders out of subscriptions are created without any further user interaction. This way only payment methods without redirect and with unlimited tenders are allowed.

The basket validation will raise errors if an invalid payment method is selected for a subscription.

Incompatibility with Quotation Feature

The Subscription feature currently does not work with the Quotation functionality (see Concept - Quoting). This is true even if the product prices are fixed (see section Concept - Subscriptions (valid to 7.10)#Fixed Prices).
Further details are described in the next chapter.

B2B Approval

By default, every subscription has to be approved for B2B channels. This is done by a special order approval rule named SubscriptionApprovalRule that is wired via Component Framework. This approval rule for subscriptions can be adapted or removed in order to change the approval process. For more information refer to Cookbook - B2B Order Approval.

Subscriptions & Quoting

The Subscription feature currently does not work with the Quotation functionality (see Concept - Quoting). This is true even if the product prices are fixed (see section Concept - Subscriptions (valid to 7.10)#Fixed Prices).

There are multiple validators implemented to prevent that a:

• Cart can be marked as recurrent when a quote is already in the cart (1)
• Quote is put into the basket when the cart is already marked as recurrent (2)
• Cart that is marked as recurrent and accidentally contains a quote can be checked out (3)

(1) BasketRecurringValidator

(2) BasketQuoteValidator

(3) BasketValidationQuoteRecurringOrderHandler

The implementations for the validator interfaces (1) and (2) are bound using Google Guice. See Concept - Dependency Injection and ObjectGraphs for more information.
Validator (3) is included in the handler chain of the basket validation which uses Extension points to define the set of executed validation handlers. See Cookbook - Basket Validation (valid to 7.9) for more information.