Test strategy[Live]

Open

Terms, Abbreviations, and Definitions 

Objectives 

The current document describes the testing methodologies and testing flow which will be applied during all stages of Design system implementation.  

The key objectives are as follows: 

• To establish and involve testing processes in the project 

• Determine project implementation 

• Determine testing types that will be applicable to each testing task 

• Identify needed tools, determine testing phases during the project 

• Identify testing metrics 

• Identify what will be tested and what will be out of the testing scope 

The document doesn’t describe detailed functional requirements and design for the site. This information will be provided in the separate documentation  - product specification (Specification on Confluence)  and detailed design (Design in Figma). 

This document is the live one and should be adjusted promptly to conform to the current state of the application requirements. 

Testing Scope 

Features to be tested 

• Container 

• Grid 

• Section 

• Typography  

• Image 

• Icon 

• Heading Block 

• Link 

• Button 

• CTA Block 

• Accordion Item 

• Accordion List 

• Page Header 

• Text box 

• Navigation Link 

• Navigation 

• Logo 

• Basic Header (i.e. excl. Site Search) 

• Subtle Button variant  

• Footer 

• iFrame  

• Carousel 

• Product Header 

• Overview 

• Product Card 

• Product Lists 

• Picto Card 

• Picto List (aka Ingredients/Benefits section) 

• Image + Text 

• Article Header 

• Article Decoration (e.g. Pull Quotes, etc) (Image Carousel only in Figma) 

• Embedded Video  

• Page Card 

• Page List 

• Teaser 

• Sitemap 

• Product How To + Safety Block (How To block, reference below, seems a bit complicated for markets to use it) 

• Promo Strip 

 Testing types in scope 

• Code review - peer review and acceptance of developed code by the RB side 

• Unit testing (automated) - developers team responsibility with coverage of at least 80 % 

• UI compatibility and cross-browser testing in a manual way; 

• Smoke testing on a preview build of each new version that is deployed; 

• Accessibility testing  (automated and manual) 

• Visual Regression testing in a semi-auto way[Probably will be changed] 

• Functional testing 

 Testing types out of scope 

• Performance testing 

• Loading testing of server-side is out of scope  

• Testing of content - The DA Team will copy and port content provided by the customer/brand team. So the correctness of the provided content is on the customer/brand team  

• Testing of the functionality of third parties, only the testing of  integration is in our scope  

 Test Approach 

Testing flow 

All test activities will be divided into four phases:  

• Analysis & Planning Phase  

• Test Phase 

• Stabilization Phase 

• UAT 

*This flow will be implemented for every component 

 Analysis & Planning Phase 

The purpose of this phase is to cover requirements provided by the customer with test documentation such as checklists, which will be used in the test phase and during the Stabilization testing phase. 

List of activities are to be performed during this phase: 

• Requirements clarification for and checklists creation; 

• Consideration of requirements testability and request for development support if needed; 

• Test documentation creation / change / support functional and non-functional checklist's 

 Test Phase 

The purpose of this phase is to validate features against requirements using the test documentation created in the elaboration phase. 

List of activities to be done during this phase: 

• Unit testing (automated) - developers team responsibility; 

• UI compatibility and cross-browser testing in a manual way  

• Accessibility testing; 

• Bugs verification 

• Reporting 

• Test documentation support 

Stabilization Phase 

The purpose of the phase is to ensure that all set of components works together as expected according to requirements, any change doesn’t impact existing functionality. This phase includes regression testing and the final smoke testing (before delivering the application to the customer for acceptance). 

It is supposed that regression mostly will be done with developed manual UI tests.  

UAT + Go-live phase 

Test activities will be done during the phase: 

• Review issues reported during the UAT by the customer 

• Verification of the fixes and changes that will be made during the UAT 

• Performing the accessibility assessment of the library 

• Performing the smoke testing of the components before it will be shipped to the production (on UAT env) 

• Performing the final smoke testing of the application after the project is delivered to a stakeholder 

Each component must get sign-off and approved by the responsible people: 

• Nathan McKean 

• Katarzyna Lewczyk 

This sign-off should be tracked in JIRA in the specific story for each component. 

Test Types and Levels 

Unit testing 

The tests will be developed by the DataArt dev team. These tests will be included in CI/CD pipeline and will be executed as a part of the build process. Failed unit tests will prevent the creation of build artifacts (application build). 

Requirements: 

Test coverage for unit tests > 80%. 

Tools to be used: 

• Jest Testing Library  

Cross-browser and compatibility testing 

The purpose of the testing is to ensure that all components look in accordance with the approved design in all agreed browsers. Such as the application is to be developed in a mobile-first approach, it is highly important to verify the application on different mobile devices that are agreed with the customer. 

The verification firstly will be done manually to detect and eliminate all layout issues. 

Requirements: 

The layouts should be responsive. The following systems, browsers, and resolution the layout should be matched: 

• Desktop (more than 1440px viewport width) 

• Windows: 

• Chrome (the latest version for Apr-20 2022) 

• MacOS: 

• Safari (the latest version for Apr-20 2022) 

• Chrome (the latest version for Apr-20 2022) 

• Mobile (more than or equal 360px viewport width) 

• Android (at least ver. 8 – i.e. Samsung Galaxy S9/S10) 

• Chrome (the latest version for Apr-20 2022) 

• iOS (at least ver. 10 – i.e. iPhone 8/XR/11 Pro) 

• Safari (the latest version for Apr-20 2022) 

Tools to be used: 

• Browser DevTools  

• Percy/Chromatic 

*Tools must be approved 

 Accessibility testing 

The purpose of the accessibility testing is to make sure that components conform to WCAG 2.0 standards.  The accessibility testing is to be started in the design review step -  all colors from the approved design will be checked on conformance with color contrast ratio by WCAG 2.0 standards. This testing type will be performed manually and using accessibility automation tools.  

Tools to be used: 

• StoryBook 

• AxeCore 

UI testing and Visual Regression testing 

The purpose of UI testing is to check that created UI Library matches the design  

• Percy/Chromatic 

Smoke testing 

The goal of the smoke testing is to execute a limited suite of the checks which verify main functionalities and UI to make sure that a deployed build is not broken at all. This testing will be performed manually. 

Functional testing 

The purpose of functional testing is to make sure that all modules work correctly according to requirements. Functional testing will be performed manually.  

Tools to be used: 

• StoryBook 

Test Environment 

It is supposed that during the application implementation and releases the following environments will be used 

• Storybook development env  

Reporting 

This section enlists the reports to be created during the testing activities. 

UI checklist 

The report is provided by the test team for the sprint. UI checklist is expected to contain the following data: 

• List of modules(atoms/molecules/organism)  

• Confirmation status according to the design of this module 

• Browsers and devices on which the module was tested 

• The person who reviewed the module 

The report is planned to be delivered as a document added to the Confluence. The recipients are the Customer’s side, PM, and management personnel. 

Accessibility checklist 

The report is provided by the test team for the sprint. The accessibility checklist is expected to contain the following data: 

• List of modules(atoms/molecules/organism)  

• Confirmation status according to Accessibility testing 

• Browsers and devices on which the module was tested 

• The person who reviewed the module 

The report is planned to be delivered as a document added to the Confluence. The recipients are the Customer’s side, PM, and management personnel. 

Visual Regression report 

The report is provided by the test team for the sprint. It’s planned to include the following information in the report: 

• Comparing screenshots of a created module and its original designs  

• Auto-generated report 

• Com 

The report is planned to be created in the Visual regression tool. 

Functionality checklist 

The report is provided by the test team for the sprint. The functionality checklist is expected to contain the following data: 

• List of modules(atoms/molecules/organism) that belong to the sprint 

• Confirmation status according to functional testing 

• Browsers and devices on which the module was tested 

• The person who reviewed the module 

The report is planned to be delivered as a document added to the Confluence. The recipients are the Customer’s side, PM, and management personnel. 

Issue and test documentation management  

Issues management 

It is planned to store and track all issues(Epics, Stories, Bugs, Tasks) in Jira. 

All features will be organized into Stories. Subtask for development test cases development, automation tests implementation will be created under each story. 

All found bugs during the testing will be linked to a relevant Epic and Story. All detected issues will be marked with the relevant bug type.  

The following bug types are going to be used: 

Issues priority definition:  

Checklist management 

For the checklist creating Confluence will be used. 

Accessibility checkpoints management 

The accessibility checklist will contain only checkpoints that have to be executed manually. These checks will be stored in Jira as Test case issue type and marked with the ‘Accessibility’ label  

Suspension and Resumption Criteria 

Testing of Test Items will be suspended if: 

 Suspension criteria: 

Non-working environment or blocker 

 Resumption requirement: 

The bug was fixed, environment working properly 

 Entry criteria: 

Requirements and design for the project are ready 

Access to tools for testing and development 

 Exit criteria: 

All duties of the DataArt team are done 

Test Deliverables 

The following test artifacts are planned to be created within the project: 

• Checklists ( UI, Accessibility, Functionality)  

• Bug reports 

• Visual Regression Testing reports  

Detailed information about each document type is described below along with approaches and examples. 

Check Lists 

Set of functional and non-functional tests documented as a simplified list of checks without strict steps to reproduce, pre-requisites, etc. 

Requirements coverage by tests is to be traced by mapping all requirements from specification to appropriate checks in the Check List document. 

Bug Reports 

The report describes the application’s flow, error, failure, or fault that causes it to produce an incorrect or unexpected result or behave in an unintended way. 

All bug reports are to be posted to the project bug tracking system (Jira) with appropriate severity for further processing by the Customer, Project Manager, and Development team. 

Visual Regression Testing reports 

 Reports showing d differences or similarities between modules and their original design. The report should be automatically generated after every commit 

Risks 

 Approvals 

The following people are required to approve the Test Strategy: