WCS Dataload/Data Migration Approach

Introduction
Dataload is a backbone of any e-Commerce application which is not only required to kick start the application such as setting up online catalog navigation but also to address the intermittent incoming data requests such as setting up the users and buyer accounts on the fly.

Whether its a dataload or data migration we always need the ultimate source of data to load into the WCS DB. Specific Chunk of WCS data can also be migrated either from old version existing WCS DB or from some other legacy data repository using following steps-

1. Data Extraction/Data Retrieval 
2. Dataload Utility Customization (Optional)

3. Data Load using DataLoad Utility

4. Dataload Automation ( Optional )

1. Data Extraction/Data Retrieval
Depending upon the fact that either we are performing data migration or data load, we either need to extract the data from the source WCS DB or we need to retrieve the data from the ultimate source in the csv format that WCS understands.

Data can be extracted from the source DB be it WCS DB or non WCS DB in the form of CSV using the SQL queries. We need to plan the data extraction in such a way so that the extract data once loaded into the target WCS DB drives the functionality concerned. e.g while migrating the Orders from the legacy system we need to ensure that we extract all the related data such as orderitems, users, user roles to ensure that the migrated orders become functional in new setup.

2. Dataload Utility Customization (Optional)

Dataload utility is a business object based customizable utility which is very scalable to address the requirements of the custom dataload including load for new custom tables/custom columns and also for injecting the business rules in the data load.

PFB the four pillars of dataload utility-
DataReader- Reads the data source e.g. CSVReader, XMLReader
BusinessObjectBuilder- Populates data object e.g. TableObjectBuilder, BaseBusinessObjectBuilder
BusinessObjectMediator- Transforms data object into physical objects e.g. PersonMediator
DataWriter- Saves the physical objects in DB using JDBC e.g. JDBCDataWriter

WCS does provide OOB component mediators such as for catalog, inventory, and price etc which can further be customized to suit the business dataload requirements. On the other hand WCS also provides the OOB TableObjectMediator which is very handy to load the load into single table directly by mapping the csv columns into the purposed custom table columns.

So the customization start point relies on the fact which business object we are aiming to load and what's the OOB support being provided in terms of OOB component mediator as a base.

2A. Loading Data Utilizing Component Mediators

PFB the OOB component mediators being provided at the moment-

Catalog/ Commerce Composer/ Content/ Member/ Price/ Promotion/ StoreConfiguration

2B. Loading Data Utilizing TableObjectMediator
Few OOB table dataload such as Orders and the extension tables data load can be done through the table to CSV mapping using TableObjectBuilder. 

PFB few of the essentials being used within dataload- 

IDResolver
IDResolver helps resolving the ID based upon the unique index of the table, it can also generate the new ID if the unique index data is absent in the table.

<_config:Table name="ORDERS" excludeUnListedColumns="true" deleteKey="Delete" deleteValue="1">

<_config:Column name="ORGENTITY_ID" value="ORGENTITY_ID" valueFrom="IDResolve">
            <_config:IDResolve tableName="ORGENTITY" generateNewKey="false" primaryKeyColumnName="ORGENTITY_ID">
                <_config:UniqueIndexColumn name="LEGALID" value="orgLegalID" />
            </_config:IDResolve>
        </_config:Column>

</_config:Table>

ColumnHandler

It allows to work and manipulate on the business rule to load column value through the dataload utility.

Business Context Service
It helps retrieving the contextual information such as storeID, catalogID and so on.

3. Data Load using DataLoad Utility

One we are done with our customization we can setup and validate the utility which compromises of below configuration files-
a. Environment configuration file ( wc-dataload-env.xml )
b. Load order configuration file ( wc-dataload.xml )
c. Business object configuration file ( wc-loader-<object>.xml )

4. Dataload Automation (Optional)
Dataload can be automated on the server environments with the help of shell scripts so that there is no manual intervention required for execution.
- dataloadenv.sh can be used as a base for the automation 
- custom mediators java files can be bundled into the jar files
- customs jars can be added into the classpath of the utility.

LDAP Integration & Customization Points

Introduction
When the authentication mode is set to LDAP, LogonCmdImpl invokes the LDAPIntegrationCmdImpl which contains the main customization points when dealing with the user and organization data flow to and from the WCS and LDAP.


LDAPIntegrationCmdImpl provides methods that can be overridden to customize the LDAP synchronization. Since LDAP integration requires a site-wide implementation, only one implementation of this command can be registered by using store 0.

Organization Structure Mapping between WCS & LDAP
It is recommended that the organization structure and distinguished names on the LDAP server match what is in WCS database. Following LDAPIntegrationCmd command methods are used for mapping between LDAP and WCS DNs:
public String getLDAPDN(String astrCommerceDN) throws ECException;
public String getCommerceDN(String astrLDAPDN, DataObject adoMember) throws ECException;

Callout to do extra processing during Single sign on (SSO) and Logon
In LogonCmdImpl.java, when the authentication mode is set to LDAP, and the authentication is successful,
if (MemberHelper.isLDAPUsedForAuthentication())
{
        postLDAPAuthenticationProcessing();
}

protected void postLDAPAuthenticationProcessing() throws ECException
   {
    LDAPIntegrationCmd cmdLDAPIntegration =
       (LDAPIntegrationCmd)CommandFactory.createCommand(
      LDAPIntegrationCmd.NAME,
      new Integer(0));    
    cmdLDAPIntegration.postLogonProcessing(getUserObject());
}


The following method of LDAPIntegrationCmdImpl.java is called to allow further processing to be done:
- public void postLogonProcessing(UserAccessBean aUserAccessBean) throws ECException;

- UserSyncBean.findByMemberId(aUserAccessBean.getMemberId())
   Can be called to get the UserSyncBean.
- UserSyncBean.getLDAPMember()
   Returns a DataObject representing the object in LDAP, which can be used for any further processing.

After single sign on takes place, the following method is called to handle any additional processing:
public void postSingleSignOnProcessing(UserSyncBean aUserSyncBean) throws ECException;

Callout to do extra processing in SyncBean
The following method can be overridden to do extra processing whenever LDAP is being updated using a sync bean:
public void LDAPIntegrationCmd.postUpdateToLDAP (UserSyncBean userSyncBean) throws ECException;

The following method can be overridden to do extra processing whenever the WebSphere Commerce database is being updated by data from LDAP:
public void LDAPIntegrationCmd.postRefreshFromLDAP (UserSyncBean userSyncBean) throws ECException;


Synchronizing User/Organization Data between WCS & LDAP
The ldapentry.xml file is used to configure which attributes are synchronized between the LDAP server and the WebSphere Commerce database. Some commonly used attributes are specified by default. The actual synchronization is done whenever the synchronization beans are used: UserSyncBean for user profile data, and OrganizationSyncBean for organization data.

UserSyncBean and OrganizationSyncBean read and write data to the database as well as to LDAP. Each class reads and writes to a default set of WCS database tables. Each of these tables has a corresponding sync helper data object (DO) class that is used by the sync bean to read and write to the table:

UserSyncBean-
UserDO                         <->      USERS
UserRegistryDO            <->     USERREG
UserDemographicsDO  <->      USERDEMO
SelfAddressDO              <->     ADDRESS (SELF ADDRESS)
BusinessProfileDO         <->     BUSPROF
UserProfileDO               <->     USERPROF
MemberAttributesDO    <->     MBRATTRVAL

OrganizationSyncBean-
OrgEntityDO                <->     ORGENTITY
SelfAddressDO             <->    ADDRESS (SELF ADDRESS)
MemberAttributesDO  <->     MBRATTRVAL

The DO classes to include for each sync bean can be specified and changed from the default implementation to match custom needs. The task command can be extended, and more DO classes can be added if you want to synchronize with new custom user/organization tables.

Limiting Synchronization of Data
In some cases, you might not want the OrganizationSyncBean to update the LDAP server whenever an organization is created or update WebSphere Commerce. This exclusion can be specified in the ldapentry.xml file by using the "ldapToWcs" or "ldapOnly" flow options. In other case "bothDirections" can be used.

wimconfig.xml(VMM- Virtual Member Manager)
Add a property mapping to the wimconfig.xml to correctly use a LDAP repository with VMM. Member manager is a service which runs on the WAS.



To use WCS with LDAP, WAS Administrative Security must be enabled and configured to use Federated Repositories. The Federated Repositories feature is implemented using another WAS component called Virtual Member Manager (VMM). com.ibm.commerce.member.syncbeans.VMMProxy is used by WebSphere Commerce to call VMM APIs to interact with the LDAP server.

Commerce Composer Customizations

Introduction
PFB the four main components of the online catalog navigation pages designed by the commerce composer architecture-
1. Page
2. Template
3. Layout
4. Widget

1. Page
We need to choose the targeted or create the new page.


2. Template 
We can make use of the OOB templates built on fluid grid system to fit in the intended UI structure.
If not addressed by the OOB template then we can create our own template by following below steps-

a. Defining the storefront assets for Commerce Composer layout templates
b. Registering a Commerce Composer layout template
c. Testing a custom layout template in Management Center


3. Layout
By leveraging the OOB/custom template underneath, we can fit in the custom/OOB widgets on the layout.

4.Commerce Composer Widgets
Depending upon the need we might have to create a fresh widget or customize the OOB widget by creating a replica or using the pure OOB widget.

Please find below the high level steps to create a custom commerce composer widget-
Create Widget's Template base xml & base files generation
A. Create a temporary project which will hold the template assets for the newly creates widget,
B. Create a XML pattern input file in it with below format and supply the entries in it  as per the need-
e.g.
<pageLayout vendor="myCompany" targetFolder="NewWidgetProject">
  <widget>
    <widgetDef identifier="myNewContentRecommendationWidget" 
     UIObjectName="myNewContentRecommendationWidgetObject"
     displayName="My New Content Recommendation Widget"
     description="This widget displays marketing content, such as ads, on a store page"
     widgetDisplayGroups="AnyPage,SearchPage"
     widgetRestrictionGroups="AnyPage,SearchPage" >
      <property name="widgetOrientation"   
       displayName="Widget Orientation"/>
      <property name="numberOfContentToDisplay"   
       displayName="Maximum number of content to display"/>
    </widgetDef>
    </widget>
</pageLayout>

C. Select the modified xml and run the JET transfarmation on it. In the Transformation section, select com.ibm.commerce.toolkit.internal.pattern.pageLayout.
D. Store,LOBTools and Dataload related material is generated

Customize the generated widget JSP files
Copy the generated JSPs into Store project and customize the JSPs as per the need-
myNewContentRecommendationWidget_Data.jspf
myNewContentRecommendationWidget_UI.jspf
myNewContentRecommendationWidget.jsp
mywidgettext_en_US.properties
mywidgettext.properties

Register the Widget through Dataload
To register a widget, use the Data Load utility to load the widget identifier and definition XML into the PLWIDGETDEF database table. Copy the registerWidgetdef.csv and subscribeWidgetDef.csv and modify them accordingly for registering and subscribing to the widget through dataload-
registerWidgetdef.csv
subscribeWidgetDef.csv
wc-dataload-widget.xml
wc-dataload-env.xml   
wc-loader-registerWidgetdef.xml   
wc-loader-subscribeWidgetdef.xml

Add CMC support for the custom Widget
Update the Management Center framework to support displaying your custom widget in the Commerce Composer tool-
WidgetObjectDefinition.def
WidgetPropertiesView.def
mycompanyPageLayoutResourceBundle.lzx
PageLayoutLOB_en_US.properties
PageLayoutLOB.properties

Calculation Framework

Introduction
Calculation framework helps us determining the monetary amounts associated with each orderitems since every orderitem can have reference to the offer, shipping mode, fulfillment center and contract. Consolidated calculated orderitems amount is applied to the order eventually.

High Level Steps of Calculation Framework
--------------------------------------------------------
OrderCalculate interface invokes the class PromotionEngineOrderCalculateCmdImpl which in turn invokes the four major steps mentioned below-
1. InitializeCalculationUsage  ---- Clears up the existing adjustments on orderitems and order
2. ApplyCalculationUsage calls:  -- Calculates and applies the amounts
     a. CalculationCodeCombine calls:
               CalculationCodeQualify
      b. CalculationCodeCalculate calls:
                 CalculationRuleCombine calls:
                               CalculationRuleQualify
                 CalculationRuleCalculate calls:
                              CalculationScaleLookup
                              CalculationRange
       c. CalculationCodeApply
3. SummarizeCalculationUsage   --- Summarizes the total adjustments to the order
4. FinalizeCalculationUsage    --- Validates & finalizes the Promotion


Overall Flow
-----------------
STENCALUSG table entry for the calusage_id determines the business logic to be executed for the below four steps of calculation framework. Let us consider the example of -1 calcusage(i.e discount)-

1) InitializeCalculationUsage
First of all, initializeCalculationUsage() is invoked which invokes the InitializeAdjustmentCmd interface based upon the entry in STENCALUSG table for CALUSAGE_ID -1. Default class implementation for discount InitializeAdjustmentCmdImpl is invoked and resets the adjustments to zero.
InitializeAdjustmentCmdImpl performs below activities-

e.g. nOrderItemId=50001, original dOrderItemAdjustment=0.00000
initializing ORDERS.TOTALADJUSTMENT to zero, dOldOrderAdjustmentAmount=0.00000

2) ApplyCalculationUsage
This is the major step in terms of customizations. ApplyCalculationUsageCmd is invoked which further invokes the following methods inside ApplyCalculationUsageCmdImpl-

2A) CalculationCodeCombineCmd (Internally invokes CalculationCodeQualifyCmd, Tables- px_promotion,clcdpromo,calcode)

SELECT * FROM cmdreg c where c.INTERFACENAME like '%CalculationCodeCombineCmd%' gives me com.ibm.commerce.order.calculation.PromotionEngineDiscountCalculationCodeCombineCmdImpl
So CalculationCodeCombineCmd interface in turn invokes PromotionEngineDiscountCalculationCodeCombineCmdImpl command

This command collects the promotions- calcodes that are eligible for the given orderitems. The relationship between calculation code and promotion is defined in CLCDPROMO which is created during promotion creation.

More specifically, the applyCalculationUsage calls PromotionEngineCalculationCodeCombineCmdImpl version of the CalcodeCombineCmd which invokes promotion engine. Once the promotionEngine is invoked, this retrieves the associated promotion execution records which are converted into the format that the calculation framework can understand.

2B) CalculationCodeCalculateCmd
After the above step, the data read from the above is used to calculate the discounts(via calrule combine and calrule calculate methods).

2C) CalculationCodeApplyCmd
The calcodes determined in step a and adjustments calculated in step b are finally applied to the order items in this step via the DiscountCalculationCodeApplyCmdImpl implementation.
That is the ORDADJUST and ORDIADJUST tables are populated in this step.

We can customize any of these by overriding these with your version of method in CALMETHOD table or by overriding in CMDREG table.

3) SummarizeCalculationUsage
Default implementation SummarizeAdjustmentCmdImpl is invoked which summarizes the total adjustments to the order.

e.g. PFB the log snippet-
SummarizeAdjustmentCmdImpl.performExecute orderitem id=50001, status=P
SummarizeAdjustmentCmdImpl.performExecute nOrderItemId=50001, status=P, dOrderItemAdjustmentForUsage=-5.70000
SummarizeAdjustmentCmdImpl.performExecute summarizing ORDERS.TOTALADJUSTMENT to dOrderAdjustmentTotal=-5.70000
SummarizeAdjustmentCmdImpl.performExecute summarizing SUBORDERS.TOTALADJUSTMENT to dSubOrderAdjustmentTotal=-5.70000
SummarizeAdjustmentCmdImpl.performExecute Exit

4) finalizeCalculationUsages
Default implementation is FinalizeDiscountCalculationUsageCmdImpl command which validates the promotion code and finalizes the Promotion Argument.

e.g. PFB the log snippet-
com.ibm.commerce.order.calculation.CalculationHelper.finalizeCalculationUsages finalizeCalculationUsageMethodId(storeId=10152,usageId=-1)=-204
com.ibm.commerce.order.calculation.FinalizeDiscountCalculationUsageCmdImpl.performExecute Entry
com.ibm.commerce.order.calculation.FinalizeDiscountCalculationUsageCmdImpl.performExecute Exit

Note: The calculation framework is invoked when you call OrderCalculate or OrderPrepare which in turn invokes the promotion engine when we pass the usage -1 as we mentioned Or we can also invoke promotion engine by calling PromotionEngineOrderCalculateCmdImpl.
Therefore, we can also customize this command based upon the needs.

OrderItemAddCmd/OrderItemUpdate/OrderPrepare commands calls the first three steps of the calculation framework whereas OrderProcess command calls the final step i.e. finalizeCalculationUsages of the calculation framework.

Five Major Components
---------------------------
1. Calculation methods
2. Calculation usages (-1 discount,-2 shipping,-3 sales tax,-4 shipping tax,-5 coupon,-6 surcharge,-7 shipping adjustment)      
3. Calculation codes
4. Calculation rules
5. Calculation scales and calculation ranges

Sequence of activities
------------------------
Define the calculation codes.
Define the calculation rules.
Define the calculation scales.
Define the calculation ranges.
Define the look-up results for the calculation ranges.
Associate the calculation scales with the calculation rules.
Attach the calculation code with the all catalog entries.

Customizations
If we do not have the monetary amounts to be applied on orderitems in the WCS DB and are to be retrieved from elsewhere then we can write our custom java classes for the calmethods by associating them inside the stencalusg table.

The custom java classes can make the real time call to retrieve the monetary amounts from the ultimate source e.g. shipping charges can be retrieved from the third party by adding the real time call inside the custom java class MyApplyShippingCmdImpl and the same class entru need to make inside the stencalusg table for the calusage_id -2.

This is how the custom ApplyShippingCmdImpl command would look like-

@Override
    public void performExecute() throws ECException {
        super.performExecute();
       
        if (iItems != null && iItems.length > 0) {
           try {
               for (int index = 0; index < iItems.length; index++) {
                    Item iItem = iItems[index];
                       //Setting dummy price which can be retrieved from third party
                       iItem.setShippingTotal(new BigDecimal(111));
                       iItem.commit();
               }
           }catch (Exception e) {
               e.printStackTrace();
           }
        }
    }

@Override
    public void setItems(Item[] aItems) {
        iItems = aItems;
        super.setItems(aItems);
    }

Payment Flow

Overall OOB Payment Flow
------------------------------------ 
1. When the buyer lands on Shipping-Billing page, Payment section drop down lists the various payment options. All of these payment options are configured in WCS DB as payment business policies for the store.

Tables used-
Policy/Policydesc/Policycmd/Policytype/Policytyc

2. Once buyer fills the credit card details and click next, AjaxRestOrderPrepare & AjaxRestOrderPIAdd Ajax REST calls are made (after successfull server side validation of credit card details using CheckCCNumberCmdImpl) which results in OrderPrepare and PIAddCmdImpl invocation respectively. PIAddComImpl adds the payment instructions to the order i.e. in PPCPAYINST and PPCEXTDATA WCS tables.
    CheckoutPayments.js -> processCheckout -> processPIAndCheckout -> addPaymentInstructions

3. Once buyer submits the order, payment plugin will be invoked and entries are made in ORDPAYINFO, PPCPAYMENT and PPCPAYTRAN tables-
    CheckoutHelper.js -> checkoutOrder -> AjaxRESTOrderSubmit

Typical Punchout Payment Setup Steps

In a typical punchout payment configuration below steps are being followed-
- Configuring the new Payment method in the DB
- Configuring the new Payment method in the payment xml files 
- Create the payment method jsp and place it at intended location. 
- Creating a Payment Plug-in
- Customizing the Billing Page Storefront
- Customizing the CallBackCmd with action registration in struts-confog-ext.xml and ACP load
- Configuring merchant related information
- Configuring certificate for the handshake
- Testing the setup with PSP test account.

These steps are explained in detail below-

Configuring a new Payment/Refund method 
-------------------------------------------------------------
In order to add a new payment/refund method we would be required to-
a. Make entries in OOB WCS tables (POLICY/POLICYDESC/POLICYCMD with the Payment/ReturnPayment type)
b. Define the payment/refund actions/rules in XML files.

1. Configure Payment
toolkitDir/workspace/WC/xml/config/payments/ppc/plugins/PSP/PluginDeployment.xml
2. Configure Payment Method
PaymentMethodConfigurations.xml
3. Configure Action Rules
toolkitDir/workspace/WC/xml/config/payments/edp/groups/default/PSP/ with all the files copied from toolkitDir/workspace/WC/xml/config/payments/edp/groups/default/CreditCardOffline
4. Add Payment Mappings
toolkitDir/workspace/WC/xml/config/payments/edp/groups/default/PaymentMappings.xml
5. Map payment systems to payment plug-ins
PaymentSystemPluginMapping.xml

For new entry inside POLICY table, Business Policy Name must match the payment/refund method ID configured in payment rules (the paymentMethod value in the PaymentMethodConfigurations.xml file, or the refundMethod value in the RefundMethodConfigurations.xml file).

Merchant Feature
---------------------------
Merchant information is managed in below WCS tables-
MERCHANT, MERCHCONF, MERCHCONFINFO and STOREMERCH

Creating a Payment Plug-in
---------------------------------
Create an EJB stateless session bean.
Ensure your plug-in remote interface extends the Plugin or QueryablePlugin interface.
Create a plug-in deployment descriptor.
Create a JAR file (EJB module).

Payment Business flow: Extension Points
------------------------------------------------------
In a typical setup which is subject to customization-
On WCS order submission, payment is authorized with order total amount and order is transferred to the external system( subject to customization ). After order fulfilment by the external OMS, a shipment confirmation message is sent back to WCS to capture payment for the shipped order items in this package.

For Payment, PrimePaymentCmd is called at Order Submission ( ProcessOrderCmdImpl )
ReservePaymentCmd is called at ReleaseToFulfillment
FinalizePaymentCmd is called at shipment confirmation

Bypassing Payments : Void Implementations
----------------------------------------------------------
Though bypassing the OOB payment flow can be done through bypassing the ajax calls from shipping billing page to bypass the call to PIAddCmd however below OOB void commands can also be used-
e.g. com.ibm.commerce.edp.commands.PrimePaymentCmd                 com.ibm.commerce.edp.commands.PrimePaymentVoidCmdImpl

Payment life cycle
-----------------------
a. There are three stages to payment processing: validation, reservation, and finalization.

b. The payment life cycle is related to the order life cycle stages: order capture, release to fulfillment, and shipping.

c. Order capture- Payment validated
Release to fulfillment- Payment reserved
Order Shipping- Payment finalized

Customizing REST service over Solr to include custom table data in WCS 8 Mod Pack1

Introduction
--------------
PFB the steps to include the custom table XI_RATING data in Solr response and then in REST response in WCS 8-

1) Create the new DB table which will keep the ratings of the products loaded through third party system, populate it with some data for testing-
CREATE TABLE "WCSDB"."X_RATINGS"  (
"CATENTRY_ID" BIGINT NOT NULL ,
"RATING" BIGINT NOT NULL)  

2) Create the custom preprocess file to pick the data from the custom table X_ratings and populate the tem ptable XI_Ratings
File name - WCDE80\search\pre-processConfig\MC_10001\DB2\wc-dataimport-preprocess-custom.xml

<_config:DIHPreProcessConfig xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../xsd/wc-dataimport-preprocess.xsd ">
  <_config:data-processing-config processor="com.ibm.commerce.foundation.dataimport.preprocess.StaticAttributeDataPreProcessor" batchSize="500">
      <_config:table definition="CREATE TABLE XI_RATINGS ( CATENTRY_ID BIGINT NOT NULL, RATING BIGINT, PRIMARY KEY (CATENTRY_ID))" name="XI_RATINGS"/>
         
   <_config:query sql="SELECT DISTINCT CATENTRY.CATENTRY_ID,X_RATINGS.RATING FROM CATENTRY,X_RATINGS WHERE CATENTRY.CATENTRY_ID = X_RATINGS.CATENTRY_ID"/>
       <_config:mapping>
          <_config:key queryColumn="CATENTRY_ID" tableColumn="CATENTRY_ID"/>
             <_config:column-mapping>
                <_config:column-column-mapping>
                    <_config:column-column queryColumn="RATING" tableColumn="RATING" />
                </_config:column-column-mapping>
             </_config:column-mapping>
        </_config:mapping>   
  </_config:data-processing-config>
</_config:DIHPreProcessConfig>

3) Modify the Solr extension files as follow-
a)x-schema.xml-
<field name="rating" type="string" indexed="true" stored="true" required="false" multiValued="false"/>

b) x-data-config.xml-
<field column="RATING" name="rating"/>

c) solrcore.properties-
dataImporter.ext.querySelect=XI_RATINGS.RATING,
dataImporter.ext.queryFrom=LEFT OUTER JOIN XI_RATINGS ON (CATENTRY.CATENTRY_ID=XI_RATINGS.CATENTRY_ID)

4) Run preprocess with server stopped, and then start server and build Solr index
Newly included field can be validated using the Solr testing URL-
http://localhost/solr/MC_10001_CatalogEntry_en_US/select?q=catentry_id:<catentryId>

5) Make the new search profile in wc-search.xml by extending the existing search profile invoked in the REST service call under Search folder-
<_config:profile name="X_findCatalogEntry" extends="IBM_findProductByIds_Details">
   <_config:query inherits="true">
       <_config:postprocessor           classname="com.ibm.commerce.foundation.server.services.rest.search.postprocessor.solr.SolrRESTSearchCatalogEntryViewUserDataQueryPostprocessor" />
   </_config:query>
   <_config:result inherits="true">
           <_config:field name="rating" />
   </_config:result>  
</_config:profile>

6) Make the new wc-component.xml under Search folder ext folder- (It takes care of the mapping we used to have it under wc-business-object-mediator.xml prior to version 7 feature pack 7)

<?xml version="1.0" encoding="UTF-8"?>
<_config:DevelopmentComponentConfiguration
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-component.xsd "
    xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

    <_config:valuemappingservice>
        <_config:valuemapping externalName="CatalogEntryUserDataFieldNameMapping" internalName="CatalogEntryUserDataFieldNameMapping">
                <_config:valuemap externalValue="RATING" internalValue="rating" />
        </_config:valuemapping>
    </_config:valuemappingservice>
</_config:DevelopmentComponentConfiguration>

7) Make the new wc-rest-resourceconfig.xml file under /Search-Rest/WebContent/WEB-INF/config/com.ibm.commerce.rest-ext/wc-rest-resourceconfig.xml

<ResourceConfig>
    <Resource name="productview">
    <GetUri uri="store/{storeId}/productview/byId/{productId}"
            description="Get product by unique ID"                 searchProfile="X_findCatalogEntry,IBM_findProductByIds_Details,IBM_findProductByIdsWithAttributesAndAttachments,IBM_findProductByIds_Summary,IBM_findProductByIds_Summary_WithNoEntitlementCheck,IBM_Admin_findProductByIds"/>
</Resource>
</ResourceConfig>

8) Restart the server and access the new indexed fields in userdata section using rest call.

9) Testing
http://localhost/search/resources/store/10158/productview/byId/121892?responseFormat=xml&catalogId=10158&currency=USD

Customizing REST service over Solr in WCS 8 Mod Pack1

Introduction
---------------
There is a slight difference in the way custom fields of the OOB WCS tables are added in Solr response and ultimately in REST response. Solr extensions now needed to be put in x-schema.xml, x-data-config.xml and solrcore.properties instead of schema.xml and wc-data-config.xml

Steps to include custom data catentdesc.AUXDESCRIPTION1 column on PDP page REST call
-----------------------------------------------------------------------------------------------------------------
1) Go to Path- toolkit\search\solr\home\MC_10001\en_US\CatalogEntry\conf
Modify x-schema. xml,solrcore.properties and x-data-config.xml as follow-

x-schema.xml-
<field name="auxdescription1" type="wc_text" indexed="true" stored="true" required="false" multiValued="false"/>

x-data-config.xml-
<field column="AUXDESCRIPTION1" name="auxdescription1"/>

solrcore.properties-
dataImporter.ext.querySelect=CATENTDESC.AUXDESCRIPTION1,

2) Run preprocess with server stopped, and then start server and build Solr index
Newly included field can be validated using the Solr testing URL-
http://localhost/solr/MC_10001_CatalogEntry_en_US/select?q=catentry_id:<catentryId>

3) Make the new search profile in wc-search.xml by extending the existing search profile invoked in the REST service call under Search folder-
<_config:profile name="X_findCatalogEntry" extends="IBM_findProductByIds_Details">
       <_config:query inherits="true">
           <_config:postprocessor           classname="com.ibm.commerce.foundation.server.services.rest.search.postprocessor.solr.SolrRESTSearchCatalogEntryViewUserDataQueryPostprocessor" />
       </_config:query>
       <_config:result inherits="true">
               <_config:field name="auxdescription1" />
       </_config:result>   
    </_config:profile>

4) Make the new wc-component.xml under Search folder ext folder- (It takes care of the mapping we used to have it under wc-business-object-mediator.xml prior to version 7 feature pack 7)

<?xml version="1.0" encoding="UTF-8"?>
<_config:DevelopmentComponentConfiguration
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-component.xsd "
    xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

    <_config:valuemappingservice>
        <_config:valuemapping externalName="CatalogEntryUserDataFieldNameMapping" internalName="CatalogEntryUserDataFieldNameMapping">
                <_config:valuemap externalValue="AUXDESCRIPTION1" internalValue="auxdescription1" />
        </_config:valuemapping>
    </_config:valuemappingservice>
</_config:DevelopmentComponentConfiguration>


5) Make the new wc-rest-resourceconfig.xml file under /Search-Rest/WebContent/WEB-INF/config/com.ibm.commerce.rest-ext/wc-rest-resourceconfig.xml
 
<ResourceConfig>
    <Resource name="productview">
        <GetUri uri="store/{storeId}/productview/byId/{productId}"
                description="Get product by unique ID"                 searchProfile="X_findCatalogEntry,IBM_findProductByIds_Details,IBM_findProductByIdsWithAttributesAndAttachments,IBM_findProductByIds_Summary,IBM_findProductByIds_Summary_WithNoEntitlementCheck,IBM_Admin_findProductByIds"/>
    </Resource>
</ResourceConfig>

6) Restart the server and access the new indexed fields in userdata section using rest call with new search profile.

7) Testing
http://localhost/search/resources/store/10158/productview/byId/121892?responseFormat=xml&catalogId=10158&currency=USD