ElasticPath: browsing catalog without specifying category

Greetings dear readers,
As promised in previous post, this time I’m about to explain how to make ElasticPath 6.1.2 allow browsing catalog without specifying a category. This is probably one of the more significant changes we did to core EP6.1.2 logic, so I believe some people might find this post very helpful. On the other hand, the amount of lines of code and configs for this change is actually quite small, so don’t let the significance mention scare you off.
Note: I can not and wish not to give any guarantees about the code/configs provided in this or any other post in my blog – so use at your own risk.

This particular change is pretty recent, not yet very thoroughly tested and didn’t pass the “test of time”, so some bugs or omissions might still be there. But in general it has been checked, does what it’s intended to, and no “big” bugs are expected to emerge.

Foreword: Searching and Browsing
If you’re familiar with EP API already, you probably know that there are two ways to obtain list of products from Store’s catalog: searching and browsing. There are search and browsing controllers, search and browsing services (implementations of which extend AbstractCatalogViewServiceImpl), search and browsing request classes for those service (that implement CatalogViewRequest interface). The difference between these two ways is that for search you must provide a keyword/keyphrase, and for browsing you must provide a category to browse.

But what if you just want to allow user to browse the whole store’s catalog, not just a particular category? And do it with possibility to specify price and/or attribute filters from filtered navigation (AKA intelligent browsing), just like you can normally do it when browsing within store category or searching with keyword.
Well, you’re out of luck. EP doesn’t provide such possibility, and official position of support is to create one parent category for all store categories, so that it would then contain all your products.

Missing feature: why is it important
Ok, so why not just create a parent category for all subcategories and get on with it? Well, for us it would mean we’d have to change structure of the catalog that is in production already, which could be problematic.
But more important were the questions about linked catalogs. In our linked catalogs for affiliate stores we need to have only a subset of products within main catalog, so we either have not to link this parent category, or unlink it’s “unwanted” subcategories afterwards (which still didn’t made it clear for us whether the products will remain in parent category or will be excluded too).
And browsing logic had to be affected, because we wanted user to get to this parent category right on index page.
All in all instead of clumsy workarounds it would be better to just allow browsing without category filter. It seemed feasible and not extremely hard, and, as existence of this post indicates, it really was.
Moreover, it all was done within one day, while just explaining our customers the need for parent category could cause longer delay because of all discussions and decision making.

Implementation: Changes to Browsing logic
Ok, less with watery explanation – let’s get into the “guts” (-:

*** BrowsingServiceImpl
First thing we need to do is change Browsing logic to not depend on specifying category (id) when browsing.
We go straight to BrowsingServiceImpl, find parts that depend on specifying category and “secure” them.

BrowsingServiceImpl – first:

if (category == null) {
	category = getCategoryService().load(browsingRequest.getCategoryUid());
}

result.setCategory(category);

// get featured product list first
searchAndSetFeaturedProducts(browsingRequest, result, shoppingCart, productLoadTuner, false);

becomes:

if (category == null && browsingRequest.getCategoryUid()>0) {
	category = getCategoryService().load(browsingRequest.getCategoryUid());
}

result.setCategory(category);

if(browsingRequest.getCategoryUid()>0) {
	// get featured product list first
	searchAndSetFeaturedProducts(browsingRequest, result, shoppingCart, productLoadTuner, false);
}

BrowsingServiceImpl – second:

searchCriteria.setOnlyWithinDirectCategory(!includeSubCategories);
searchCriteria.setDirectCategoryUid(browsingRequest.getCategoryUid());
searchCriteria.setStoreCode(getStoreConfig().getStoreCode());

// this is done elsewhere, but need to do this here for additional logic
if (includeSubCategories) {
	final Set<Long> categoryUids = new HashSet<Long>();
	categoryUids.add(browsingRequest.getCategoryUid());
	searchCriteria.setAncestorCategoryUids(categoryUids);
}

becomes:

if(browsingRequest.getCategoryUid()>0) {
	searchCriteria.setOnlyWithinDirectCategory(!includeSubCategories);
	searchCriteria.setDirectCategoryUid(browsingRequest.getCategoryUid());
} else searchCriteria.setOnlyWithinDirectCategory(false); 
searchCriteria.setStoreCode(getStoreConfig().getStoreCode());

// this is done elsewhere, but need to do this here for additional logic
if (browsingRequest.getCategoryUid()>0 && includeSubCategories) {
	final Set<Long> categoryUids = new HashSet<Long>();
	categoryUids.add(browsingRequest.getCategoryUid());
	searchCriteria.setAncestorCategoryUids(categoryUids);
}

(Sorry for not providing fully qualified class names and line numbers, but there is only one “BrowsingServiceImpl”, and no repeating parts of code in it, so short class names and code snippets should be sufficient. Same applies to rest of mentioned classes and provided snippets.)

*** ProductQueryComposerImpl
If you’d test it like this you’d notice that resulting SOLR/Lucene query contains after catalog and product-visibility conditions contains () part, which results in parsing exception. So category condition is empty, but still added to query in brackets – thus the empty brackets.
To fix this we go to ProductQueryComposerImpl and change this code:

} else {
	final BooleanQuery innerQuery = new BooleanQuery();
	if (categoryCode != null) {
		final String categoryFieldName = getIndexUtility().createProductCategoryFieldName(SolrIndexConstants.PRODUCT_CATEGORY, catalogCode);
		hasSomeCriteria |= addWholeFieldToQuery(categoryFieldName, categoryCode, null,
				searchConfig, innerQuery, Occur.SHOULD, true);
	}
	hasSomeCriteria |= addWholeFieldToQuery(SolrIndexConstants.PARENT_CATEGORY_CODES, ancestorCategoryCodes, null, searchConfig,
			innerQuery, Occur.SHOULD, true);
	
	booleanQuery.add(innerQuery, Occur.MUST);
}

to this:

} else {
	boolean innerQueryHasSomeCriteria = false;
	final BooleanQuery innerQuery = new BooleanQuery();
	if (categoryCode != null) {
		final String categoryFieldName = getIndexUtility().createProductCategoryFieldName(SolrIndexConstants.PRODUCT_CATEGORY, catalogCode);
		innerQueryHasSomeCriteria |= addWholeFieldToQuery(categoryFieldName, categoryCode, null,
				searchConfig, innerQuery, Occur.SHOULD, true);
	}
	innerQueryHasSomeCriteria |= addWholeFieldToQuery(SolrIndexConstants.PARENT_CATEGORY_CODES, ancestorCategoryCodes, null, searchConfig,
			innerQuery, Occur.SHOULD, true);
	
	if(innerQueryHasSomeCriteria) {
		booleanQuery.add(innerQuery, Occur.MUST);
		hasSomeCriteria |= innerQueryHasSomeCriteria;
	}
}

You can see we now check for innerQuery to have some criteria before adding it to query. So now when category criteria is not present, innerQuery isn’t added, and no empty brackets appear.

*** BrowsingRequestImpl
But we also get “Not initialized!” exception from BrowsingRequestImpl. What’s wrong? The sanity check method checks if category ID is provided, and if not, browsing query is considered to be wrong.
Reasonable, but we want to change this. So in BrowsingRequestImpl we find sanityCheck

@Override
protected void sanityCheck() {
	if (this.getCategoryUid() == 0) {
		throw new EpDomainException("Not initialized!");
	}
}

And comment out the innards (or you can just remove it if parent sanityCheck() is fine – I haven’t checked).

*** BrowsingControllerImpl
Ok, so we’re done? Not quite. It all starts from BrowsingControllerImpl that will do call to browsing service, so let’s check what category dependencies are there.
First check populateBrowsingRequest method. We’ve been changing it before, so in our case if category filter is not present it just leaves categoryUid == 0, but I’m not sure now if it was that way originally. This should be easy to do yourself.
Next is the dependency on category to determine which template to use (as categories of different types may be rendered by different Velocity templates in EP):
In BrowsingControllerImpl it’s this code:

final String categoryView = browsingResult.getCategory().getTemplateWithFallBack("categoryTemplate");

What we decided to do it just use “catalog/category/categoryTemplate.vm” in case category is not provided. So code changes to this:

final String categoryView; 
if(browsingResult.getCategory()!=null) {
	categoryView = browsingResult.getCategory().getTemplateWithFallBack("categoryTemplate");
} else {
	categoryView = "categoryTemplate";
}

Now we’re good! We can call /browse.ep and browse whole catalog using filtered navigation without specifying category (or keyword, since we “browse”). Hooray!
But, as soon as we turn on SEO (by setting ‘COMMERCE/STORE/seoEnabled’ system property to ‘true’ for our store) it all break down again – pagination links don’t work when no filters are present, and filtered navigation links aren’t working, except for category filters. What to do? Well, continue to the next section.

Implementation: Changes to SEO logic
There are two problems with SEO.
First – links that no longer contain category filter aren’t recognized by URL rewriter. This is fixed by correcting URL rewriter config (system setting ‘COMMERCE/SYSTEM/urlRewriteConfig’).
Second – pagination URLs for browsing without any filters look like “-pN.html” (where N is page number). EP considers “-” as filters separator, finds “empty filter” before page number filter, and considers URL to be wrong. And it just looks ugly. This problem is fixed by correcting FilterSeoUrl class to not include “-” in SEO URL before page number filter in case there are no other filters before it.

*** urlRewriteConfig
In URL Rewriter config, which is essentially an XML file contents stored in DB, we find <rule> element for category, which has this pattern:

<!-- Category url parsing. -->
<from>^.*/c[^/]*\.html[^/]*$</from>

The pattern inside “from” tag should be changed to “^.*/(c|at|pr|p)[^/]*\.html[^/]*$” in order to match category, attribute, price and page filters. But then it will also match product details page SEO URLs (that have “prod.html” part in them), thus the whole “rule” tag with it’s contents should be moved below the “rule” for Product url parsing.

Now we’ve got it all working, but pagination SEO URLs for no-filters case. Let’s fix those too.

*** FilterSeoUrl
This is a quick one. In FilterSeoUrl we find the “appendPageNumberFragment” method:

private void appendPageNumberFragment(final StringBuffer buffer) {
	if (pageNumber < 0) {
		return;
	}
	buffer.append(fieldSeparator).append(SeoConstants.PAGE_NUMBER_PREFIX).append(pageNumber);
}

and change it to be like this:

private void appendPageNumberFragment(final StringBuffer buffer) {
	if (pageNumber < 0) {
		return;
	}
	if(buffer.toString().trim().length()>0) {
		buffer.append(fieldSeparator);
	}
	buffer.append(SeoConstants.PAGE_NUMBER_PREFIX).append(pageNumber);
}

So in case buffer is empty, we don’t add the fieldSeparator, which will keep the redundant “-” away.
The check “buffer.toString().trim().length()>0” is a bit superficial – theoretically no whitespace should be in the buffer, or else URL with whitespace might cause problems elsewhere, but it feels more secure (in case whitespace will be there once, but will be stripped elsewhere etc – not very probable, but who knows).
If you believe you can win some performance here, because the method is called quite often per request, you may just use “buffer.length()>0”.

Now we’re done for sure!

Afterword
There’s no better reading for a developer than a source code!
Whatever books and articles on software development you read, they lack practical values if they don’t have at least some code snippets. So working with opensource products may actually save company and developer money on studying, because going through working code of commercial (or even enterprise if you will) level solutions is very rewarding.
And of course another good part of it is that if you lack some feature you need, you can always add it yourself, as we just did.
That’s why we like ElasticPath a lot (-:

And I hope this post was a useful and rewarding reading for you!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s