/
NUTRINTG Google Analytics data integration

NUTRINTG Google Analytics data integration

Owned by Sebastian Baltruszewicz
Aug 16, 2023

This section describes how data from Google Analytics are uploaded to the CDP.

Customer Data

Flow consists of three parts

Web → Middleware

#StepDescription
1User enters the website
  • Formulars may be available on various platform - Shopify, Husky and others
2GA check if ClientID exist - if no creats one and put into COOKIE
  • GA cookie is per domain, example: one user visits 2 domains → user has two ClientIDs
3User fills the form and send
4Shop back-end receive the form with ClientID included in the cookie header
  • ClientID from Google Analytics cookie is sent in Cookie header, example:
POST request example 
:authority: www.durex.de
:method: POST
:path: /account
:scheme: https
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
accept-encoding: gzip, deflate, br
accept-language: en-GB,en-US;q=0.9,en;q=0.8
cache-control: max-age=0
content-length: 186
content-type: application/x-www-form-urlencoded
cookie: snize_session=bha9c44ms; __cfduid=d85a67e9c27932717404f6c77bef126031570000195; _shopify_y=4a063ae3-5bda-4b23-8172-6ac50fe6d735; secure_customer_sig=; _y=4a063ae3-5bda-4b23-8172-6ac50fe6d735; _shopify_fs=2019-10-02T07%3A09%3A56.048Z; optimizelyEndUserId=oeu1570000196122r0.6458797381018733; _gcl_au=1.1.1785747956.1570000196; _ga=GA1.2.422434676.1570000196; _fbp=fb.1.1570000196499.1872653321; _evidon_consent_cookie={"consent_date":"2019-10-02T07:10:07.354Z"}; snize_uid=rgjkv8rze; _sp_id.de86=a100ce1cab311bcb.1570000251.1.1570000256.1570000251; cart_currency=EUR; _orig_referrer=https%3A%2F%2Fwww.google.com%2F; _shopify_country=Poland; _landing_page=%2F; cart_sig=; visited=yes; 

_gid=GA1.2.1029947871.1571404361; 

snize-recommendation=erobub6nq5; _secure_session_id=fdfdf3b689218e8c8c648f8aa8909c6a; snize_session=drnqlvnp4; _s=df37ddd5-713A-4D98-5B3B-9B21750B9F5E; _shopify_s=df37ddd5-713A-4D98-5B3B-9B21750B9F5E; _shopify_sa_p=; _shopify_sa_t=2019-10-18T14%3A14%3A08.743Z; RT="z=1&dm=durex.de&si=6mynsbq3g4&ss=k1w7rwkf&sl=3&tt=372&next_page_url=123d615f2b2607ec5ed72dde0493c6db&ld=67d&cl=1fez&ul=1ff3"
origin: https://www.durex.de
referer: https://www.durex.de/account/register
sec-fetch-mode: navigate
sec-fetch-site: same-origin
sec-fetch-user: ?1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36
5Back-end extracts the ClientID from header and send form-data together with Client ID
  • Each backend needs to develop a script to extract ClientID from the header and sent it together with Profile data.
  • GA ClientID will be put as "GA_ClientID" in the "to MW" API request section "UnmappedAttributes": {
      "GA_ClientID": "example_ClientID",
      "MyProperty2": " Property2",
      "MyProperty3": {
        "MyProperty4": " Property4"

Full profile payload schema, here: NUTRINTG Profile API

Middleware

#StepDescription
6MW  Profile Service collects and validates data
  • Validation relevant to each platform:
1. Shopify - validation in accordance with Customer data validation, here: NUTRINTG Shopify to CDP Integration
6aCommunication
  • The user receives relevant messages.
7Queue exchange (duplicate request and  send a request to two flow)Data from Google Analytics will be uploaded via Profile Update. Queue Exchange sends a request to standard Mule Queue and GA loop queue. As a result profile without GA data is created in CDP and after MW gathers data from GA, send Profile request to the Mule queue.
8MW aggregates the data in the Mule queue. Requests will be accumulated in the dedicated Mulesoft queue.
9MW send User Activity API to Google Analytics with specified ViewID, ClientID

curl --request POST \
'https://analyticsreporting.googleapis.com/v4/userActivity:search?key=[YOUR_API_KEY]' \
--header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data '{"viewId":"153292692","user":{"type":"CLIENT_ID","userId":"1584310179.1570190773"}}' \
--compressed


  • example request:
POST https://analyticsreporting.googleapis.com/v4/userActivity:search

{
    "viewId": "9999999",
    "user": {
        "type": "CLIENT_ID",
        "userId": "1034600000.76425000000"
    },
    "dateRange": {
        "startDate": "2018-01-01",
        "endDate": "2018-12-31",
    }
}


10GA responds with JSON. API data have up to 4h delay in Analytics360
  • It is possible that there will be multiple session in the response, for example:
multisession_response_GA 
{
  "sessions": [
    {
      "sessionId": "1570191379",
      "deviceCategory": "desktop",
      "platform": "Macintosh",
      "dataSource": "web",
      "activities": [
        {
          "activityTime": "2019-10-04T14:16:21.830388Z",
          "source": "test",
          "medium": "test",
          "channelGrouping": "(Other)",
          "campaign": "test",
          "keyword": "(not set)",
          "hostname": "onion.solutions",
          "landingPagePath": "/",
          "activityType": "PAGEVIEW",
          "customDimension": [
            {
              "index": 1
            },
            {
              "index": 2
            }
          ],
          "pageview": {
            "pagePath": "/",
            "pageTitle": "Onion Solutions"
          }
        },
        {
          "activityTime": "2019-10-04T14:16:19.387922Z",
          "source": "test",
          "medium": "test",
          "channelGrouping": "(Other)",
          "campaign": "test",
          "keyword": "(not set)",
          "hostname": "onion.solutions",
          "landingPagePath": "/",
          "activityType": "PAGEVIEW",
          "customDimension": [
            {
              "index": 1
            },
            {
              "index": 2
            }
          ],
          "pageview": {
            "pagePath": "/",
            "pageTitle": "Onion Solutions"
          }
        }
      ],
      "sessionDate": "2019-10-04"
    },
    {
      "sessionId": "1570190772",
      "deviceCategory": "desktop",
      "platform": "Macintosh",
      "dataSource": "web",
      "activities": [
        {
          "activityTime": "2019-10-04T14:06:12.709250Z",
          "source": "(direct)",
          "medium": "(none)",
          "channelGrouping": "Direct",
          "campaign": "(not set)",
          "keyword": "(not set)",
          "hostname": "onion.solutions",
          "landingPagePath": "/",
          "activityType": "PAGEVIEW",
          "customDimension": [
            {
              "index": 1
            },
            {
              "index": 2
            }
          ],
          "pageview": {
            "pagePath": "/",
            "pageTitle": "Onion Solutions"
          }
        }
      ],
      "sessionDate": "2019-10-04"
    }
  ],
  "totalRows": 3,
  "sampleRate": 1
}


  • In that case, the assumption about attribution is needed. Our assumption is to take the earliest session (activity_time)
  • Example response with a single session:
    {
            "sessions": [{
                "sessionId": "1539184430",
                "deviceCategory": "desktop",
                "platform": "Windows",
                "dataSource": "web",
                "sessionDate": "2018-10-10"
                "activities": [{
                    "activityTime": "2018-10-10T08:13:50.555105Z",
                    "source": "(direct)",
                    "medium": "(none)",
                    "channelGrouping": "Direct",
                    "campaign": "(not set)",
                    "keyword": "(not set)",
                    "hostname": "shop.googlemerchandisestore.com",
                    "landingPagePath": "/gpsmap",
                    "activityType": "PAGEVIEW",
                    "customDimension": [{
                        "index": 1,
                        "value": "(not set)"
                    }, ...]
                }, ...]
            }, ...]
            "totalRows": 1000,
            "nextPageToken": "A6JGY3",
            "sampleRate": 1
    }
  • Google Analytics 360 may have up to 4h delay, till customer data is available via API
  • Standard Google Analytics may have up to 24h delay
  • More about GA limits and SLA:
- https://support.google.com/analytics/answer/1070983?hl=en
- https://www.google.com/intl/en_us/ga360suite/sla.html
11MW merge profile data from a form with GA API data based on ClientID as a key
  • UTM parameters are extracted from GA response
  • Relevant Profile Payload parameters are added to Profile Request
12MW puts profile to the Mule queue
  • Profile request goes to Mulesft queue used for standard profile upload, described here: 
NUTRINTG Shopify to CDP Integration
13Profile Mule Queue for CDP
14Profile adapter

Middleware → CDP → SFMC

#StepDescription
15Standard flow to SF
16SF downloads data to extended for UTM data extensionNUTRINTG CDP to SFMC Integration