Release Process

Carefully review this release candidate report.
If you are not satisfied, try to identify a different release candidate.
Ensure that:
- There is no "Change Freeze" in place.
- Fellow team member is on hand in case something goes terribly wrong.
- Both the approver and has co-deployer commit to being in the office for 2 hours after clicking the approval button.
Click the approval button.

Release Notes

The release notes for the completed tickets should help the target audience understand what this change is about. Are there examples and / or links to the documentation? When you have any doubts talk to the person who worked on the ticket and / or review the commits.

Completed

TICKET-2931: Dreaded Bug [B2B customers]
Fixes the dreaded bug that would violate the users expectations.
- 2015-07-17 11:19:00 +0200 Joe Developer 59a7ff0 TICKET-2931 Finally fix the bug
- 2015-07-17 09:21:13 +0200 Joe Developer 8ae0846 TICKET-2931 Refactor a little

TICKET-2988: Internal Server Error [Internal stakeholders]
Fixes the internal server error that was preventing us from returning a response.

The tickets that are in play require extra attention. Could the ongoing work break existing functionality? When in doubt speak to the person who worked on it and review the commits.

In Play

TICKET-2955: New Feature [Internal customers]
- 2015-07-17 14:36:01 +0200 Pete Programmer 48304e8 TICKET-2955: Create that feature

TICKET-2864: Simplified Deployment [Internal stakeholders]
- 2015-07-23 10:43:46 -0500 Joe Developer 208c46c TICKET-2864 Removing waste

Load Test

The load test replays production log data. The average response time as well as CPU load over the 20 minutes of the test should be simlar to the current productions version. To accomodate for changes in traffic we run a pattern from the previous as well as one from the current month. If there are major discrepancies review the console ouput of the respective builds or try a different build candidate.

This Month's Log

production                                        |    candidate
============================                           ============================
LOG REPLAY LOAD TEST RESULT                            LOG REPLAY LOAD TEST RESULT
SAMPLE FROM --date 2015-07-01                          SAMPLE FROM --date 2015-07-01
============================                           ============================
       Total requests: 25280.00                               Total requests: 25280.00
Average response time:     197.61 ms              |    Average response time:     195.97 ms

timestamp                 average                      timestamp                 average
---------                 -------                      ---------                 -------
2015-07-17T10:05:00.000Z  35.25%                  |    2015-07-24T14:30:00.000Z  41.67%
2015-07-17T10:06:00.000Z  63.33%                  |    2015-07-24T14:31:00.000Z  61.31%
2015-07-17T10:07:00.000Z  60.00%                  |    2015-07-24T14:32:00.000Z  61.69%
2015-07-17T10:08:00.000Z  58.03%                  |    2015-07-24T14:33:00.000Z  52.46%
2015-07-17T10:09:00.000Z  49.83%                  |    2015-07-24T14:34:00.000Z  50.51%
2015-07-17T10:10:00.000Z  49.33%                  |    2015-07-24T14:35:00.000Z  47.21%
2015-07-17T10:11:00.000Z  49.00%                  |    2015-07-24T14:36:00.000Z  47.80%
2015-07-17T10:12:00.000Z  48.33%                  |    2015-07-24T14:37:00.000Z  50.00%
2015-07-17T10:13:00.000Z  50.00%                  |    2015-07-24T14:38:00.000Z  50.33%
2015-07-17T10:14:00.000Z  49.33%                  |    2015-07-24T14:39:00.000Z  48.00%
2015-07-17T10:15:00.000Z  50.00%                  |    2015-07-24T14:40:00.000Z  51.33%
2015-07-17T10:16:00.000Z  44.59%                  |    2015-07-24T14:41:00.000Z  43.00%
2015-07-17T10:17:00.000Z  45.08%                  |    2015-07-24T14:42:00.000Z  47.67%
2015-07-17T10:18:00.000Z  44.59%                  |    2015-07-24T14:43:00.000Z  42.00%
2015-07-17T10:19:00.000Z  42.37%                  |    2015-07-24T14:44:00.000Z  44.00%
2015-07-17T10:20:00.000Z  42.33%                  |    2015-07-24T14:45:00.000Z  42.67%
2015-07-17T10:21:00.000Z  44.92%                  |    2015-07-24T14:46:00.000Z  40.98%
2015-07-17T10:22:00.000Z  42.71%                  |    2015-07-24T14:47:00.000Z  45.76%
2015-07-17T10:23:00.000Z  48.20%                  |    2015-07-24T14:48:00.000Z  46.00%
2015-07-17T10:24:00.000Z  44.07%                  <

Last Month's Log

production                                        |   candidate
============================                           ============================
LOG REPLAY LOAD TEST RESULT                            LOG REPLAY LOAD TEST RESULT
SAMPLE FROM --date 2015-06-01                          SAMPLE FROM --date 2015-06-01
============================                           ============================
       Total requests: 20145.00                              Total requests: 20145.00
Average response time:     262.98 ms              |    Average response time:     263.69 ms

timestamp                 average                      timestamp                 average
---------                 -------                      ---------                 -------
2015-07-17T10:06:00.000Z  40.98%                  |    2015-07-24T14:30:00.000Z  40.67%
2015-07-17T10:07:00.000Z  55.59%                  |    2015-07-24T14:31:00.000Z  57.70%
2015-07-17T10:08:00.000Z  52.33%                  |    2015-07-24T14:32:00.000Z  52.88%
2015-07-17T10:09:00.000Z  44.92%                  |    2015-07-24T14:33:00.000Z  44.67%
2015-07-17T10:10:00.000Z  46.00%                  |    2015-07-24T14:34:00.000Z  44.92%
2015-07-17T10:11:00.000Z  45.76%                  |    2015-07-24T14:35:00.000Z  42.03%
2015-07-17T10:12:00.000Z  44.92%                  |    2015-07-24T14:36:00.000Z  42.33%
2015-07-17T10:13:00.000Z  43.39%                  |    2015-07-24T14:37:00.000Z  42.67%
2015-07-17T10:14:00.000Z  48.00%                  |    2015-07-24T14:38:00.000Z  47.00%
2015-07-17T10:15:00.000Z  47.21%                  |    2015-07-24T14:39:00.000Z  48.33%
2015-07-17T10:16:00.000Z  45.08%                  |    2015-07-24T14:40:00.000Z  41.64%
2015-07-17T10:17:00.000Z  43.33%                  |    2015-07-24T14:41:00.000Z  43.73%
2015-07-17T10:18:00.000Z  40.66%                  |    2015-07-24T14:42:00.000Z  44.33%
2015-07-17T10:19:00.000Z  47.80%                  |    2015-07-24T14:43:00.000Z  46.00%
2015-07-17T10:20:00.000Z  40.33%                  |    2015-07-24T14:44:00.000Z  39.00%
2015-07-17T10:21:00.000Z  40.67%                  |    2015-07-24T14:45:00.000Z  40.00%
2015-07-17T10:22:00.000Z  40.67%                  |    2015-07-24T14:46:00.000Z  39.67%
2015-07-17T10:23:00.000Z  41.67%                  |    2015-07-24T14:47:00.000Z  43.33%
2015-07-17T10:24:00.000Z  40.66%                  |    2015-07-24T14:48:00.000Z  41.33%
2015-07-17T10:25:00.000Z  40.34%                  |    2015-07-24T14:49:00.000Z  39.34%

Stress Test

The stress test tries to bring the system to the breaking point by increasing the request rate. The graphs show latency and error rate over request rate. Again the candidate shouldn't be too different from the current production system. Due to virtualisation we expect fluctuactions. The behaviour beyond the rated load is not as important as we ideally scale up before that point is reached.

Average Latency

Error Rate

Log Replay Diff

This step compares the responses that production and candidate system give to randomly sampled user requests. The status codes should be the same, i.e. on the downward diagonal. Deviations need to be investigated, because we generally guarantee backwards compatibility. The json responses are all diffed and the summary is also printed. The changes should be individually reviewed. Ideally they are linked to the things you read it the release notes above.

            Running replay of production logs against
            a: https://product.production.dev.company.com
            b: https://product.candidate.dev.company.com
            Number of requests: 2000


            Status Code Statistics

            Exp\Act    200   302
                200   1996     0
                302      0     4
            PASSED

            Response Differences

            Number of responses with json nodes that changed: 1
            Number of responses with json nodes were added:   0
            Number of responses with json nodes were removed: 1

Review differences...

Frontpage Screenshot

Review our public facing front page for layout problems such as misplaced or missing elements.

Candidate Frontpage

Production Frontpage

Differences

Release

If you are still happy with the candidate should be suitable for release using the approval button: product-dev-deploy/11848

Release Candidate Build 11848 product-dev-deploy/11848