Customizing REST service over Solr to include custom table data in WCS 8 Mod Pack1

Introduction
--------------
PFB the steps to include the custom table XI_RATING data in Solr response and then in REST response in WCS 8-

1) Create the new DB table which will keep the ratings of the products loaded through third party system, populate it with some data for testing-
CREATE TABLE "WCSDB"."X_RATINGS"  (
"CATENTRY_ID" BIGINT NOT NULL ,
"RATING" BIGINT NOT NULL)  

2) Create the custom preprocess file to pick the data from the custom table X_ratings and populate the tem ptable XI_Ratings
File name - WCDE80\search\pre-processConfig\MC_10001\DB2\wc-dataimport-preprocess-custom.xml

<_config:DIHPreProcessConfig xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../xsd/wc-dataimport-preprocess.xsd ">
  <_config:data-processing-config processor="com.ibm.commerce.foundation.dataimport.preprocess.StaticAttributeDataPreProcessor" batchSize="500">
      <_config:table definition="CREATE TABLE XI_RATINGS ( CATENTRY_ID BIGINT NOT NULL, RATING BIGINT, PRIMARY KEY (CATENTRY_ID))" name="XI_RATINGS"/>
         
   <_config:query sql="SELECT DISTINCT CATENTRY.CATENTRY_ID,X_RATINGS.RATING FROM CATENTRY,X_RATINGS WHERE CATENTRY.CATENTRY_ID = X_RATINGS.CATENTRY_ID"/>
       <_config:mapping>
          <_config:key queryColumn="CATENTRY_ID" tableColumn="CATENTRY_ID"/>
             <_config:column-mapping>
                <_config:column-column-mapping>
                    <_config:column-column queryColumn="RATING" tableColumn="RATING" />
                </_config:column-column-mapping>
             </_config:column-mapping>
        </_config:mapping>   
  </_config:data-processing-config>
</_config:DIHPreProcessConfig>

3) Modify the Solr extension files as follow-
a)x-schema.xml-
<field name="rating" type="string" indexed="true" stored="true" required="false" multiValued="false"/>

b) x-data-config.xml-
<field column="RATING" name="rating"/>

c) solrcore.properties-
dataImporter.ext.querySelect=XI_RATINGS.RATING,
dataImporter.ext.queryFrom=LEFT OUTER JOIN XI_RATINGS ON (CATENTRY.CATENTRY_ID=XI_RATINGS.CATENTRY_ID)

4) Run preprocess with server stopped, and then start server and build Solr index
Newly included field can be validated using the Solr testing URL-
http://localhost/solr/MC_10001_CatalogEntry_en_US/select?q=catentry_id:<catentryId>

5) Make the new search profile in wc-search.xml by extending the existing search profile invoked in the REST service call under Search folder-
<_config:profile name="X_findCatalogEntry" extends="IBM_findProductByIds_Details">
   <_config:query inherits="true">
       <_config:postprocessor           classname="com.ibm.commerce.foundation.server.services.rest.search.postprocessor.solr.SolrRESTSearchCatalogEntryViewUserDataQueryPostprocessor" />
   </_config:query>
   <_config:result inherits="true">
           <_config:field name="rating" />
   </_config:result>  
</_config:profile>

6) Make the new wc-component.xml under Search folder ext folder- (It takes care of the mapping we used to have it under wc-business-object-mediator.xml prior to version 7 feature pack 7)

<?xml version="1.0" encoding="UTF-8"?>
<_config:DevelopmentComponentConfiguration
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-component.xsd "
    xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

    <_config:valuemappingservice>
        <_config:valuemapping externalName="CatalogEntryUserDataFieldNameMapping" internalName="CatalogEntryUserDataFieldNameMapping">
                <_config:valuemap externalValue="RATING" internalValue="rating" />
        </_config:valuemapping>
    </_config:valuemappingservice>
</_config:DevelopmentComponentConfiguration>

7) Make the new wc-rest-resourceconfig.xml file under /Search-Rest/WebContent/WEB-INF/config/com.ibm.commerce.rest-ext/wc-rest-resourceconfig.xml

<ResourceConfig>
    <Resource name="productview">
    <GetUri uri="store/{storeId}/productview/byId/{productId}"
            description="Get product by unique ID"                 searchProfile="X_findCatalogEntry,IBM_findProductByIds_Details,IBM_findProductByIdsWithAttributesAndAttachments,IBM_findProductByIds_Summary,IBM_findProductByIds_Summary_WithNoEntitlementCheck,IBM_Admin_findProductByIds"/>
</Resource>
</ResourceConfig>

8) Restart the server and access the new indexed fields in userdata section using rest call.

9) Testing
http://localhost/search/resources/store/10158/productview/byId/121892?responseFormat=xml&catalogId=10158&currency=USD

Customizing REST service over Solr in WCS 8 Mod Pack1

Introduction
---------------
There is a slight difference in the way custom fields of the OOB WCS tables are added in Solr response and ultimately in REST response. Solr extensions now needed to be put in x-schema.xml, x-data-config.xml and solrcore.properties instead of schema.xml and wc-data-config.xml

Steps to include custom data catentdesc.AUXDESCRIPTION1 column on PDP page REST call
-----------------------------------------------------------------------------------------------------------------
1) Go to Path- toolkit\search\solr\home\MC_10001\en_US\CatalogEntry\conf
Modify x-schema. xml,solrcore.properties and x-data-config.xml as follow-

x-schema.xml-
<field name="auxdescription1" type="wc_text" indexed="true" stored="true" required="false" multiValued="false"/>

x-data-config.xml-
<field column="AUXDESCRIPTION1" name="auxdescription1"/>

solrcore.properties-
dataImporter.ext.querySelect=CATENTDESC.AUXDESCRIPTION1,

2) Run preprocess with server stopped, and then start server and build Solr index
Newly included field can be validated using the Solr testing URL-
http://localhost/solr/MC_10001_CatalogEntry_en_US/select?q=catentry_id:<catentryId>

3) Make the new search profile in wc-search.xml by extending the existing search profile invoked in the REST service call under Search folder-
<_config:profile name="X_findCatalogEntry" extends="IBM_findProductByIds_Details">
       <_config:query inherits="true">
           <_config:postprocessor           classname="com.ibm.commerce.foundation.server.services.rest.search.postprocessor.solr.SolrRESTSearchCatalogEntryViewUserDataQueryPostprocessor" />
       </_config:query>
       <_config:result inherits="true">
               <_config:field name="auxdescription1" />
       </_config:result>   
    </_config:profile>

4) Make the new wc-component.xml under Search folder ext folder- (It takes care of the mapping we used to have it under wc-business-object-mediator.xml prior to version 7 feature pack 7)

<?xml version="1.0" encoding="UTF-8"?>
<_config:DevelopmentComponentConfiguration
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-component.xsd "
    xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

    <_config:valuemappingservice>
        <_config:valuemapping externalName="CatalogEntryUserDataFieldNameMapping" internalName="CatalogEntryUserDataFieldNameMapping">
                <_config:valuemap externalValue="AUXDESCRIPTION1" internalValue="auxdescription1" />
        </_config:valuemapping>
    </_config:valuemappingservice>
</_config:DevelopmentComponentConfiguration>


5) Make the new wc-rest-resourceconfig.xml file under /Search-Rest/WebContent/WEB-INF/config/com.ibm.commerce.rest-ext/wc-rest-resourceconfig.xml
 
<ResourceConfig>
    <Resource name="productview">
        <GetUri uri="store/{storeId}/productview/byId/{productId}"
                description="Get product by unique ID"                 searchProfile="X_findCatalogEntry,IBM_findProductByIds_Details,IBM_findProductByIdsWithAttributesAndAttachments,IBM_findProductByIds_Summary,IBM_findProductByIds_Summary_WithNoEntitlementCheck,IBM_Admin_findProductByIds"/>
    </Resource>
</ResourceConfig>

6) Restart the server and access the new indexed fields in userdata section using rest call with new search profile.

7) Testing
http://localhost/search/resources/store/10158/productview/byId/121892?responseFormat=xml&catalogId=10158&currency=USD

B2B Contracts

Contracts
A Contract helps defining the terms & conditions for the online customer. It becomes very crucial in setting up the different shopping experiences on the same online store for different buyer accounts in B2B setup.

These T&Cs modify the online customer's shopping experience including-
Products entitlement,
Price entitlement,
Shipping and payment options,
Permissible order limit.

1. A contract can either be assigned to Member group or Organization.

2. Buyer can have multiple contracts setup and active during the shopping session and he can even avail distinct contracts for distinct orderitems of the same order(OrderItems.TRADING_ID & TERMCOND_ID) provided that the contracts are compatible against each other(OrderItemBaseCmdIml-> validateTradingPaymentMethods()).

3. ContractSetInSessionCmdImpl command helps to setup a particular contract during shopping (out of various active contracts created for the buyer organization) which is ultimately saved inside EntitlementContext context of Ctxdata table

Types
1) The default contract
A store has a default contract which allows guest and unregistered shoppers to shop in the store
            Storedef -> Store default contract & default shipmode (Buyer default contract through Contract.Usage column 0 value)
2) A base contract
Can be shared by many contracts
3) A customer contract.
A customer contract is one that has a Buyer participant

Data Model
STOREENT(STOREENT_ID) -> ORGENTITY(STORE OWNER) -> ACCOUNT(ACCOUNT_ID) -> TRADING(TRADING_ID)  -> PARTICIPNT

Store Contracts (STORECNTR)
    Store -> Contracts
Business Accounts (ACCOUNT)
    Store -> Accounts
Participants (PARTICIPNT)
    Buyer organization -> Contracts
Terms and Conditions (TERMCOND)
    Contract -> Terms & Conditions


Storcntr- Contains all contracts deployed in the store
Contract- Base table for contract
Account- Represents a business account between a Buyer organization and a Seller organization
Participnt- Contains the Buyer organization and Contract relationship
Trading- Contract is a specific type of trading
TermCond- Contains all the terms and conditions associated with the contract such as Price rule, Catalog filter, Price list, Payment and Shipping method, Approval required etc.
Catfilter- Base table for catalog filter
Pricerule- Base table for the price rule
Expression- Contains the solr runtime expression for the contract.
Tradeposcn- Contains the price list details

Creating Contracts
1. Simplest way is to create a contract using accelerator,
2. We can also create a contract by extracting/loading the XML,
3. Creating a new version of a contract,
4. Creating a contract through custom dataload utility.


TermCond Mapping
TermCond table contains a number of generic columns used to store information about terms and condition with different terms and conditions use the columns differently. Each mapping shows how the information in the XML elements of the terms and conditions map to columns in the TERMCOND table-

PriceRuleTC
CatalogFilterTC
PriceTCMasterCatalogWithFiltering
ShippingTCShipToAddress
ShippingTCShippingMode
ShippingTCShippingCharge
OrderApprovalTC
RightToBuyTCByAmount
ObligationToBuyTCByAmount

Price List, Price Rule & Contracts
A very prominent usage of contracts in B2B system is to utilize it to setup the multiple pricing tier for the buyer aligned under different buyer organizations.

Price list can be created and assigned to the contract through price rule allowing the different set of buyers to avail different prices for the same products.
For a price rule to display offer prices on store pages, you must assign the price rule to a contract.

BUYER -> BUYER ORG -> CONTRACT <- PRICE RULE <- PRICE LIST <- PRICE

After you assign the price rule to a contract, customers shopping under the contract see and are entitled to prices from the price rule. Remember a price rule is one of many terms and conditions that a contract can have.

Tables-
PRICERULE: Each row of this table represents a price rule in a store. Price rule is a business rule which defines how to get a dynamic price under different business conditions
PRELEMENT: Each row of this table represents a price rule element.
PRELETEMPLATE: Each row of this table represents a price rule element template
PRELEMENTATTR: Each row of this table represents a price rule attribute in name value pair. P.S: This table maps the price rule of termcond table with actual price list in tradposcn table.

Data Flow-
CONTRACT -> TERMCOND(STRINGFIELD1) -> PRICERULE -> PRELEMENT -> PRELEMENTATTR -> TRADEPOSCN -> OFFER -> OFFERPRICE -> ACTUAL PRICE

CMC provides support for-
Price Rule,
Price List

UseCase: Applying Multiple Tier Pricing for different buyer orgs
Export an existing price list and then upload it again after modification as a new price list through CMC.
Now create a price rule to make use of this new price list
This price rule can be applied to a contract through accelerator
User logging into the buyer organization can avail the special pricing through the contract.

Catalog Filter and Contracts
Another prominent usage of contracts in B2B system is to utilize it to setup the multiple catalog tier for the buyer aligned under different buyer organizations.

Catalog filter can be created and assigned to the contract which allows the buyer to shop for the filtered items in the e-site.  
BUYER -> BUYER ORG -> CONTRACT <- CATALOG FILTER <- CATALOG ENTITY

Data Flow-
CONTRACT -> TERMCOND(BIGINTFIELD1) -> CATFILTER -> CFCATGROUP -> CFCONDGRP -> CFCOND -> CFCONDVAL

We can use catalog filters for the product entitlement, that is, to entitle customers to a subset of the catalog

Note: We can create catalog filters to filter objects for only our master catalog. We cannot create catalog filters for a sales catalog. So if we apply the catalog filer to the master catalog it eventually retrieves the entitled products from the sales catalog.

Catalog Filter Tables-
CATFILTER - Each row of this table represents a catalog filter.
CATFLTDSC- Each row of this table represents a language specific description of a certain catalog filter.
CFCATGROUP- Each row represents a selected catalog group in the catalog tree.
CFCONDGRP- Each row of this table represents a condition group in a certain catalog filter
CFCOND- Each row of this table represents a condition in a certain condition group
CFCONDVAL- Each row of this table represents the association between a condition and an attribute value.
PRODUCTSET- This table holds the definition of a product set. The actual representation of the product set is held in the PRSETCEREL table.
CFPRODUCTSET- Each row represents the relationship between a catalog filter and a product set used for filtering.

Unit Testing Contracts Filtering
Expression table entry helps building the solr expression for debugging  for catalog filter or productset-

e.g.
http://server/solr/MC_10001_CatalogEntry_en_US/select?q=*:*&fq=catalog_id:"11111"&fq=storeent_id:("22222" "33333")&fq=published:1&fq=((( +*:* -*:*) productset_id:("55555555") ) productset_id:("44444444") )

Steps to enable registered users to access all stores in the Extended Sites business model

WCS users can access storefronts by virtue of having the role of Registered Customer in the organization that owns the store, or in any organization above it. The roles a user has access to during registration are defined within the MemberRegistrationAttributes.xml file.

By default the MemberRegistrationAttributes.xml file is set up to give access to only the store where the user is registering.

By Adding the below snippet in the top of the file MemberRegistrationAttributes.xml under <UserRoles> section, we can allow the registered user of any of the esite of the extended site setup to access all stores as a registered user-

<User registrationType="UserRegistration" memberAncestor="o=Default Organization,o=Root Organization"
                        storeAncestor="o=Extended Sites Seller Organization,o=Root Organization">
                  <Role name="Registered Customer" roleContext="explicit" DN="o=Extended Sites Seller Organization,o=Root Organization"/>
                </User>

Customizing REST service over BOD Part-1 WCS 7 FEP 8

Introduction
REST services are built on top of OAGIS services. Each REST resource is mapped to a Business Object Document (BOD) noun which further follow either the SOI or SOA(BOD) paradigm.


High-Level Changes Required
BOD service can first itself be extended to include the additional attribute in userdata by modifying the BOD related configuration files such as wc-business-object-mediator.xml, wc-object-relational-metadata.xml, wc-component.xml and TPL files and then the additional BOD custom data can be mapped to REST response using rest-resourceName-clientobjects.xml file.

BOD Level Changes-
1. Extend the WCS DB Schema (in case of custom table/column)
2. Run the DSL wizard to generate the supporting BOD configuration files
3. Add the query template files.
4. Add the Access profiles.
4. Validate the changes (can be done through JUNIT)

REST Layer Changes-
1. wc-rest-resourceconfig.xml
2. rest-resourceName-clientobjects.xml
3. Resource Handler

e.g. rest-wishlist-clientobjects.xml is the mapping file used for the wishlist noun stored under the directory- Rest.war/WEB-INF/config/bodMapping

Details
We are considering the BOD pattern in this example so we will pick the wishlist noun to explain the flow and required files to be modified to add additional data


1. wc-rest-resourceconfig.xml is the main file which contains all the REST URLs being utilized for the nouns, This file should be extended if we want to add custom access profile or search profile for the noun-

<Resource name="wishlist">
    <GetUri uri="store/{storeId}/wishlist/@self" description="Get wish list data for a logged in user"
                accessProfile="IBM_Store_GiftListSummaryAndItems"/>
       <GetUri uri="store/{storeId}/wishlist/@default" description="Get default wish list for a logged in user"
         accessProfile="IBM_Store_GiftListSummaryAndItems"/>
      
    <GetUri uri="store/{storeId}/wishlist/{externalId}" description="Get wish list by external id"
                accessProfile="IBM_Store_GiftListSummaryAndItems"/>

   <GetUri uri="store/{storeId}/wishlist/{externalId}/item" description="Get wish list items by external id with paging support"  accessProfile="IBM_Store_GiftListItems"/>
</Resource>

2. rest-resourceName-clientobjects.xml
This file contains the mapping between REST data and BOD noun. Any customization to be done in this file requires a copy to be created under the structure-
Rest.war/WEB-INF/config/bodMapping-ext

e.g. below snippet provides support to add custom grgftreg data in userdata section-

<_config:URLParameter name="x_" nounElement="/UserData/UserDataField" key="false" return="true" type="UserData"/>


3. Resource Handler
Resource handler contains the RESTful services to manage the operations on the noun.
e.g. WishlistHandler class provides RESTful services to get and manage a shopper's wish lists.


4. wc-component.xml
It contains properties to configure various REST features.


5. wc-business-object-mediator.xml

It contains the mapping between physical and logical SDOs

e.g. below snippet maps the custom grgftreg fields-


<_config:property-mapping logicalType="com.ibm.commerce.giftcenter.facade.datatypes.GiftListType" physicalType="com.ibm.commerce.giftcenter.facade.server.entity.datatypes.Grgftreg">
             <_config:userDataProperty logicalPropertyName="field1" physicalPropertyName="field1"/>
            <_config:userDataProperty logicalPropertyName="field2" physicalPropertyName="field2"/>
            <_config:userDataProperty logicalPropertyName="field3" physicalPropertyName="field3"/>
            <_config:userDataProperty logicalPropertyName="field4" physicalPropertyName="field4"/>
            <_config:userDataProperty logicalPropertyName="field5" physicalPropertyName="field5"/>
         </_config:property-mapping>


6. wc-object-relational-metadata.xml

It contains the mapping between physical SDOs and the actual database tables.

e.g. below snippet maps the field1 column-


<_config:table name="GRGFTREG" occColumnName="OPTCOUNTER" propertyName="Grgftreg">
<_config:column name="FIELD1" nullable="true" primaryKey="false" propertyName="field1" type="INTEGER"/>
</_config:table>


7. TPL files

In the BOD paradigm, tpl files contains the queries( single step or two step ) to interact with the database.

e.g. the below section contains the second step of the query to be executed based upon the access profile-


BEGIN_PROFILE
   name=IBM_Store_GiftListSummaryAndItems
   BEGIN_ENTITY
     base_table=GRGFTREG
     associated_sql_statement=IBM_GR_Summary
     associated_sql_statement=IBM_GR_Registrant_Without_Address
     associated_sql_statement=IBM_GR_Events
     associated_sql_statement=IBM_GR_Items
    END_ENTITY
END_PROFILE

Pulishising E-Sites with B2B Store Model

IntroductionCreating the the extended sites setup depends upon many factors like the Store model /StorefrontAssetStore/CatalogAssetStore/Store currency/Inventory to be used.

We can refer the high level steps required to be performed to get e-sites published in B2B store model-
First of all we need to have the instance created in the server environment or WCS installation to be done if working in the local setup. Then we need to perform the below steps-

1) Log into WCS Admin Console and publish the ExtendedSitesOrganizationStructure.sar

2) Publishing the extended site hub using ExtendedSitesHub.sar.

3) Publish the Catalog Asset Store using ExtendedSitesCatalogAssetStore-FEP.sar

4) Publish the Storefront Asset Store using AuroraStorefrontAssetStore.sar.

5) Create the Buyer Organizations using WC Organization Admin Console

6) Create the Seller Organizations using WC Organization Admin Console

7) Create the required esites using WC Accelerator by providing values like storeIdentifier, default currency, store type and payment modes

8) Open the newly created stores using WC Acclerator

9) Create Sales Catalogs for Each Site as per the requirement

10) Load Catalog Data as per the requirement

11) SOLR Setup – SetupSearchIndex, Di-preprocess, Build Search Index

Note: By default SARs may not be available to be published in Admin console, in this case SARs need to be copied into the path defined in wc-server.xml.

Organization Structure

Introduction
All parts of the WCS business whether its actor or entities, including customers, administrators, stores, catalogs and distributors, must be owned by either an organization or organizational unit.

1) Basic WCS Organization Structure
This is how the basic organization structure looks like after the WCS instance creation, regardless of the business model being used-

Root Organization- It's right there at top with all WCS organizations acting as descendants of the root organization. The site administrators are owned by the root organization.

Default Organization- The default organization is owned by the root organization. And this is where all the B2C and guest users point to.

Note: B2C users under the default organization can be managed by an administrator in WCS Accelerator. Business users outside of the default organization e.g. B2B users can be managed in the Organization Administration Console. 

 

 

2) B2C Organization Structure

The B2C organization structure contains three organizations: root, default, and seller-

 

Seller Organization- A seller organization is created to own all the stores (Retailer) and the Seller administrators who maintain the store. 

B2C Seller Organization ( Intermediate and Optional )- An intermediate B2C organization is created to own a single store (Retailer) and Seller administrators. 

 

 3) B2B Direct Organization Structure

 

Buyer Organization- B2B Buyers are represented by a buyer organization in the B2B organization structure. 

4) Extended Sites Organization Structure
The extended sites organization structure consists of the root organization, default organization, extended sites organization, and extended sites seller organization-