NUTRINTG (5) File Zipper (DEPRECATED/OUTDATED DOCUMENTATION)

File zipper is a generic process responsible for creating archives from specific s3 files. Following there is a high level overview:

Open

Main process steps

1. Process polls for messages with fixed interval (process.fixed-delay)

2. All files for specific brandOrgCode are zipped together to file with the following pattern name: deltaDate + "_" + brandOrgCode + "_CDP_Extract.zip" if files are not zipped already (it is checked by presence of the expected zip file on s3). The output zip is created in the same directory in which were input files.

3. If all files for each brand is zipped correctly the result message is sent to the next response queue (aws.sqs.outbound-fifo-queue-url)

4. The ProcessedDelta information is stored in the database (to omit processing two times the same delta)

5. Lastly the message from the inbound queue is deleted and the result email is sent.

Spring integration channels graph and error handling

Below there is presented a channel graph of the file-zipper:

Open

Processing the message

1. The received message is split by brandOrgCode first

2. All messages are processed in zip-files channel (each brand has only one output file)

3. All messages are aggregated in AggregatorChannel and sent as one aggregated result to the result channel where the ProcessingDelta is saved, Inbound queue message is removed and successful email is sent.

Error handling

It is worth to notice that there are two more channels in the message channel graph:

• General Error Channel - responsible for handling exceptions in the main flow, it sends email message with the details of the exception (ex. JSON format exception after receiving the message)

• Aggregation Compensation Channel - handles all errors occurred in processing the one split message. The channel is responsible for sending the compensation message to the aggregation channel. It is required because all messages that were produced in the 1) Splitter have to be received all in aggregation channel to proceed the aggregation operation (ex. one upload failed). It is because split step produces messages which contains correlationId, sequenceNumber, sequenceSize headers and aggregator handler triggers only if all sequence messages are collected. The compensation channel gets the original message headers and adds it to the error message.

Most important properties