28th July 2021

It’s Not Just a Website – Beginner’s Advice for Testing an Optimizely Site

Testing an Optimizely CMS or Commerce site requires both traditional web testing skills as well as an extra set of CMS-specific approaches to ensure your implementation is doing what it does best – delivering great digital experiences that connect with your customers.

author image

read by Bob Beck

So, you’ve tested many a website and consider yourself pretty darn good at the old testing thing huh? And now you want to step in and take responsibility for the quality of an Optimizely CMS site? Well stop right there, we have a few things to go over…


Catalog entries and Pages and Blocks, Oh My!

It is important not just to verify that the pages being delivered match the current wishes of the client. Yes, right now they may want their Contact Us page to look like that. You must confirm the look and feel, font styles, fields, and controls as you normally would but that won’t be enough here.

The whole point of a content management system is to empower the user to manage their own content within whatever limits are defined by the requirements. Like Leg…well… like certain brands of Scandinavian toy blocks. There are a finite number of types of building block but an almost unlimited number of combinations of those types of blocks.

Optimizely CMS also has blocks – content blocks. Reusable bits of content that the user can add to a page. There are also different page types – these are like template pages which allow certain pieces of content (and content Blocks) to be added as properties. You can think of the page types as those large, flat pieces you build other things on top of.

As a QA you must ensure that each block type and each page type function correctly on its own, accepting only the values and including only the properties defined. Then you must also ensure that these fit together too – to stretch the analogy – to ensure that everything can be attached to everything else without falling off or getting stuck (or in our case, throwing errors and breaking the page.)

Furthermore, if your Optimizely implementation also includes Commerce, then things included in the Commerce catalog can also be used in the content blocks and page types. Yes – you have to test those too!


The World Page Tree

Much like all the realms of Norse cosmology (the Scandinavians again…) all the content in your Optimizely CMS site is ordered and connected by a tree – the Page tree.

This can be customized to work in different ways so I can’t generalize too much here about how your pages should be structured but it’s vital that you make sure to test adding different page types in different parts of the Page tree, both as they’re designed to be ordered according to the requirements and in other ways a user might try and build out their site content.

Structuring page types in ways they weren’t intended can break your site quicker than a Viking axe could break a Lego house (I swear I’ll stop now.)


You’re Not Supposed to Be in Here

If you’re used to testing websites, you are probably familiar with pages a site visitor can and cannot navigate to directly. This is the same in your Optimizely site.

A user can access a Login page and gain access to the Edit View where they manage all the content mentioned previously. A user who is not logged in should NOT be able to access the dashboard or navigation top bar.

Furthermore, your Administrator can likely create different User Groups as well as allocating different permissions to individual users. Again, these combinations should be tested so that each type of user only has the access they are supposed to have.


SE-Oh right!

Another important aspect of managing website content via Optimizely CMS is the control over things like analytics, meta titles, meta descriptions and canonical URLs. I can’t go into depth on SEO here but if you’re not writing your customers website pages from scratch then they will be managing these things as a CMS Editor.

That means your job as a QA is to ensure not only that these elements are correctly built into the web pages but that these are also manageable via the Optimizely CMS as specified in the requirements and that when changes are published that these are also reflected in the web page’s HTML. Get familiar with Ctrl+U.


And everything else…

It should be becoming clear now that testing an Optimizely CMS site includes everything that a web tester needs to do on any other website. You still need to test all the functionality as a site visitor. You need to test around security and performance. You must test across different browsers and screen sizes (and don’t forget tablets - always an area where front-end issues can creep in between the more common desktop and mobile screens.)

But on top of all these types of tests you also have to test the ability to change most of these things as well.

Yes, you will need to check permission access for each user type but also test reconfiguring and creating new user types as well and ensuring they have appropriate access.

Yes, you will need to check that the site conforms to the designs agreed with the client but that these designs are adhered to regardless of what content blocks the user uses on each page type.

Yes, you will need to check that fields and controls act appropriately on each form but also the ability to add new controls, and even whole new forms, as well.

This is only the tip of the iceberg when it comes to the array of functionality offered by Optimizely to enable your clients to boost their digital offering. To ensure the power is in their hands, it’s your job as a QA to make sure that the implementation is working as intended – not only as delivered - but however the client wants to use it in the future too.