/
NAC Deployment guideline

NAC Deployment guideline

Owned by Wiktor Wiśniewski
Dec 07, 2021
11 min read

Deployment prerequisites

In order to be able to run the successful deployment, fix issues happening on the fly, or face additional needs occurring during the sprint, you should have already some knowledge about GIT and Shopify. This document describes the standard deployment process, but according to our experience in 1 of 2 deployments, you will run into a problem that needs to be resolved under time pressure. Because there are plenty of possible challenges, it's not possible to describe all of them in a short guide. Make sure that you have:

  1. Good understanding of GIT

  2. Good understanding of Shopify theme kit and Shopify CLI

A simplified version of the git-flow branching model 

We use a simplified version of the git-flow branching model. During the sprint, a build moves from the development stage through various testing stages (staging – integration stage and pre-prod environment) to production. In each stage, the build is checked against certain quality criteria. If these are fulfilled, the build is promoted to the next stage.

In a typical iteration, developers work on multiple features and create a feature branch from develop branch (1) whenever they start to develop a new feature or bugfix. They commit their changes to according the feature branch. When the developer finishes his work, he should pass his change to the QA Team for testing and when it’s approved he creates a new pull request typically pointing to the development branch. A reviewer should pay attention to the code quality and readability, double-check the solution in the preview mode and run automatic validations and then decide on approval. Depending on Pull Request some changes can be requested or declined if the number of changes would be too big. If the pull request was approved, the developer before merging should sync branch. Then the merge happens (2a, 3, 5).  Merge to the staging branch can happen at any time during the Sprint. Other developers can pull these changes and apply them to their feature branch (4).

 

At the end of the iteration, before the review with the customer, the team needs to release the current state of the software by deploying the changes of the development branch to the preprod environment. After testing is done (6) and business approvals are acquired, the production deployment process can start.

All merges to the master branch are maintained by the release manager only.

After merge to master is done, all merged feature branches should be closed. If given features require a change and were already deployed to the production, a new ticket and branch should be created.

While this use of branches complicates the collaboration, because developers now need to consider multiple branches and the according to merge procedures, it also increases the flexibility. Developers can easily introduce new features or revert them if a developer e.g. does not understand all requirements behind a feature or if he is unsure about his concrete implementation.

Common challenges

·      Reverting a feature from the staging branch.

·      Merge conflicts on staging < - > master PR (because of the sync branch or hotfixes)

·      Deploying a single feature in the middle of the sprint –> skipping the staging branch. The challenge here is that feature branches are created at any time from the staging branch, which can contain not approved yet features. The release manager is responsible for spotting additional changes that should not go to the production environment.

·      Cherry picking commits

·      Dealing with too many commits

·      Finding the commit that broke something after a merge

·      Fixing issues that are the result of other developer mistakes (e.g. wrong merge destination)

·      Deployment errors after the merge to master was done.

·      … and many uncommon challenges ;)

Pro tips

·      Use VSC extensions or GIT tools wisely. On one button click, 10 git commands can be executed. Do not use invasive actions if you do not understand those tools well.

·      Above tools can help visualize the current git state and work well for managing merge conflicts

·      Terminal is always better.

·      Git sometimes is very challenging. If you are not familiar with a given command it’s better to take your time and try to create a new repository to simulate the procedure you are about to do.

·      Do not execute commands that you do not understand!

·      Do not overwrite repository history!

·      Dealing with git issues takes time. Do not start working on these at the end of the day. Most likely you will have to work over hours.

Bitbucket pipelines

Bitbucket Pipelines is an integrated CI/CD service built into Bitbucket. It allows you to automatically build, test, and even deploy your code based on a configuration file in your repository. Essentially, containers are created in the cloud. Inside these containers, commands are run with all the advantages of a fresh system, customized and configured for our needs. Pipelines are defined using a YAML file called bitbucket-pipelines.yml, which is located at the root of the repository. For more information on configuring a YAML file, refer to https://support.atlassian.com/bitbucket-cloud/docs/configure-bitbucket-pipelinesyml/.

In some cases, you will need to look into these files to find the answers or modify them.

Running pipelines

  1. Go to pipelines menu

  2. Find run pipeline button

  3. Select a branch – this step is mandatory, even if no branch is needed for a given action

  4. Set up other fields

  5. Click run pipeline

Notes:

  1. A pipeline can be terminated any time by clicking cancel/abort button

  2. Closing a tab/window does not terminate the pipeline

Available pipelines

create-new-theme

Creates a new theme and a branch in the repository. We do not use this functionality

get-theme-by-id-staging / get-theme-by-id-production

Creates a new branch and downloads all files from the given theme ID into it. Staging stands for development store and Production for the live store.

The main purpose of this pipeline is to check the difference between repository and theme files on the store. Although this pipeline is not creating a pull request (bug) you should pick a correct branch and type BRANCH_NAME_TO_PULL_REQUEST. THEME_ID is a theme id of a theme that you want to get files from and TICKET_ID is the issue number from Jira. Because of some repositories policy, a ticket number is required for creating a commit. In this case, the ticket number would be the same as the ID of the Jira issue used for managing the deployment process. 

This pipeline will create a new branch. You need to create a pull request that points to the branch that you want to check:

Usually, you will merge this pull request for backup purposes. Remember to check differences and confirm what changes are intentional.

get-themes-list-staging / get-themes-list-production

Returns a list of themes in a given environment. This pipeline helps to find a live theme ID. This pipeline does not change any file and does not create/change branches.

  1. Run the pipeline

  2. Find completed flow in the history or wait till the process finish.

  3. Open

  4. In the right dark panel, find the theme get –list … command. Expand it.

  5. Example: [126205952181][live] Release -> 126205952181 is the ID of the published theme.

deploy-to-staging / deploy-to-production

 Deploys branch to the given environment. OPTIONAL_SKIP_IGNORING_FILES does nothing because it was not fully implemented. Ignore files are listed in the bitbucket config file at the root of the repository.

Branch: branch to be deployed

THEME_ID: id of the theme that branch is deployed into

Deployment without pipelines

 Sometimes pipelines are not available. You should use the terminal instead. Available options are – theme kit for old themes and Shopify CLI for 2.0 themes.

The process is the same however you need to create a diff/sync branch on your own. Then you need to manually pull the files from the Shopify Theme into that branch and create a pull request.

You should also remember about ignore files list. Ignore config will be different on each environment so make sure you have the correct one. Default ignore list for all environments:

 

development: store: password: theme_id: bucket_size: 40 refill_rate: 2 timeout: 60s ignore_files: - "*.swp" - "*~" - "config/settings_data.json" - "node_modules" - "locales/*" - "ordergroove*.liquid"

DO NOT deploy locales, app-generated files, or settings_data file. Those are shared with all teams across the market and those teams are excluded from the need of managing repository. Treat them as user files. Be careful with ordergroove files

Deploy command for themekit:

https://shopify.dev/themes/tools/theme-kit/command-reference#deploy

Communication & Jira Ticket

During each sprint special issue should be created by one of Release Managers, called Deployment DTC Sprint X:

 

A few days before pre-prod deployment (usually, it’s EOD of code freeze day which is Friday) you should discuss with a team if there are any Husky or DEP deployment dependencies. There are other teams working on different parts of the platform and some features are linked to those platforms. That means deployment time depends on other teams' deployment time. It’s recommended to start deployment around your morning time (so you have room to fix any occurring issues) and nighttime of the given market (so customers don’t encounter 404 or 401). You should also take into consideration the timezone of team members because deployment is a result of teamwork.

Checklist

In Jira, you can create a to-do list and import a checklist from other tickets. The main intention of this checklist is to guide the release manager through the deployment process and additionally inform other team members what the current state of deployment is. This checklist must reflect the current state of the deployment just in case if you cannot finish the deployment and somebody else has to complete it.

It’s important to follow these steps in that order.

Communication

During deployment preparations and process, communication is the most important factor. You should must act proactive rather than reactive. Remember that missing communication affects mostly release manager by causing unnecessary delays. Your main objective should be to start deployment on time. Deployment time in 95% of cases is fixed and cannot be postponed.

It’s also your responsibility to find features that are about to create a huge delay. Some issues cannot be moved to the next release. Those should be always on your watch list. Experience shows that developers usually communicate problems with a delivery a few minutes before delivery time is reached. This is one of the top 5 delay causes. Acting proactive means finding feature/bug fixes that cannot be postponed/moved to the next sprint and checking if there are any risks related to a delivery time.

As a good practice communicate if a ticket is blocking deployment, a plan related to it is unknown or should be passed to the next stage and yet is not.

Again, the cost of not communicating those issues for you as a Release manager is probably the biggest.

The form of communication is free, but the example above shows the cleanest way of notifying problems. You should also let other team members know if deployment to a given stage was done or ask what the current state is. For a better visibility, it is better to communicate on Jira.

 

Schedule and statuses:

https://rb-digital.atlassian.net/wiki/spaces/RES/pages/2047574332/Jira+Guide#Schedule

Exceptions

  1. If the change was submitted to a Code Review on Friday (code freeze day), it still can be checked by Monday morning and deployed to staging

  2. If issues were found on staging  on Monday, they can be fixed by submitting a new PR (if possible, please confirm with QA team what’s their capacity)

  3. The same goes for preprod environment

 Good practices 

·      Avoid late deployments. It’s better to postpone it than work late

·      Never deploy on Friday or the day before the Holiday

·      Never start without a backup

·      Always have a backup plan in the back of your head

Backup 

Make sure that you can deploy a backup theme first. Some apps are linked to the theme ID and publishing another theme may cause more issues. Try to find a source of the issue first and if possible try to fix it with the backup files, and then start reverting the buggy feature or start the hotfix process.

You can also redeploy a theme by reverting staging -> master PR or moving to the older commit (recommended). There are plenty of strategies that could be used here.

In case of an emergency or huge impact, ask to lock a store with a password and plan the restore.

Rules

  • The release manager is not responsible for handling staging/4th env deployments

  • Developers cannot deploy to pre-prod/production without notice or permission

  • Each developer is responsible for configuring the theme on 4 stages on his own (locales, settings) unless it was agreed otherwise  

Step by step process

  1. Create a deployment task

  2. Check the status of all tasks, find the risks

  3. Discuss the deployment times with team members

  4. Testing on staging happens. When it’s done regression is being made. Wait for confirmation from QA team.

  5. By the EOD Monday or by Tuesday morning, deployment to preprod should be made if all tasks are assigned to the release manager and are in In UAT state.

  6. Confirm successful deployment. On Tuesday preprod testing should happen.

  7. A few hours before deployment get differences between live and repository master branch theme. Find uncommitted stages, report them, decide on adding them to repository or not, backup user data files.

  8. Create a backup theme on the production store.

  9. By now all tickets should be assigned to Release manager in Pending Tech Approval state.

10.  Run deployment

11.  Confirm successful deployment

12.  Deploy Script Editor scripts or set up the launchpad. Confirm deployment or set up.

13.  Close all deployed feature branches.

 

Script Editor

The script editor is responsible for running advanced campaigns, hiding shipping options and payment gateways. The impact of wrong timing or disabling all scripts might be huge.

 Code can be found in the separate repository:

https://bitbucket.org/rbdigital/shopify-scripts-enfamil-us

 https://bitbucket.org/rbdigital/shopify-scripts-enfamil-ca  

Those repositories also follow a simplified git-flow approach.

Script Editor is an app that can be found on the installed app list:

There are 3 types of scripts:

  1. Line item scripts

  2. Shipping scripts

  3. Payment scripts

Only 1 of each type can be published at the given moment:

In order to publish a new one, open one of the published scripts and duplicate it, replace the code and change the name. The naming convention is: [version] Sprint name + features labels added in the given sprint.

Some scripts must be deployed on a certain date. You can automate this process in the launchpad App:

Scripts can be published forever or for a given time period. Click create event in order to schedule script deployment:

Note: Payment scripts cannot be scheduled

Important: confirm the schedule with QA or Tech Manager

Related content