Cloud Services & Cloud Evidence Locker Uptime and Maintenance Bulletin

Follow

This bulletin is to notify all users of any known issues real-time, it will detail when issues arise and when they have been resolved. Please ensure you "follow" this article for automated e-mail updates.

Date Description

14th March 2024

Our Azure Kubernetes Service is retiring - specifically versions: v1.26.x on 31st March 2024.  In anticipation of this we will be upgrading our production deployments to 1.27.9 between 2200-0000hrs GMT on 14/03/24.  Every production tenancy will be impacted with momentary downtime of up to 10 minutes.  

20th November 2023

There is an upgrade planned across multiple Cloud Services and Cloud Evidence Locker sites today, Monday 20th November.

The update should only take a few minutes. If you are trying to use Cloud Evidence Locker or other Cloud Services during this time you may see an error. If this happens please wait a few minutes and try again.

16th November 2023

Redaction services are currently unavailable. We are working on a resolution and will update this bulletin with further news.

16th November 2023 09:38am

 

UPDATE - sites not upgraded today will be planned for upgrade from Monday 20th November. 

 

16th November 2023

10am GMT, Thursday 16th November - There is an upgrade planned across multiple Cloud Services and Cloud Evidence Locker sites this morning, commencing at approximately 10:00am GMT. 

The update should only take a few minutes. If you are trying to use Cloud Evidence Locker or other Cloud Services during this time you may see an error. If this happens please wait a few minutes and try again.

29th September 2023

The upgrade on 28th September completed successfully.

27th September 2023

7pm BST, Thursday 28th September - there will be an upgrade to our back-end Kubernetes services to improve security. There are no user-facing changes. This upgrade will take approximately 30 minutes.

There should be minimal disruption to services during this upgrade. In case of any errors, please log out and try again in a few minutes.

7th September 2023

We will be upgrading our redaction services for Cloud Evidence Locker week commencing 11th and 18th September. During this time the redaction functionality may be unavailable. If you have persistent issues with redaction during this time please contact your local support team.

29th June 2023 10:46

The planned maintenance upgrade is completed and all services are operational.

27th June 2023

There is a maintenance upgrade planned for Thursday 29th June at approximately 10:00am BST. This is an update to the backend IoT device management; there are no user-facing changes.

The update should only take a few minutes. If you are trying to use Cloud Services during this time you may see an error. If this happens please wait a few minutes and try again.

15th June 2023

Planned maintenance upgrade completed successfully

13th June 2023 11:35

There is a maintenance upgrade planned for Thursday 15th June at approx 11:30am BST. This is a database update only; there are no user-facing changes.

The update should only take a few minutes. If you are trying to use Cloud Services during this time you may see an error. If this happens please wait a few minutes and try again.

26th May 2023 15:37

Email notifications are being received OK. 

26th May 2023

We've received reports that some users are not receiving email notifications. We are investigating the root cause.

15th May 2023

There are two maintenance upgrades planned this week:

10am BST, Tuesday 16th May - Synectics Cloud Media Services maintenance upgrade

7pm BST, Thursday 18th May - Kubernetes upgrade to improve security. This upgrade will take approximately 15 minutes.

There should be minimal disruption to services for both these upgrades.

21st March 2023

A maintenance update for T2000 health data will be carried out from 8am to 2pm (GMT) on 21st March 2023. Any downtime will be minimal and should not be notice-able to most users.

20th March 2023

A maintenance update for T2000 health data will be carried out this afternoon (GMT). Any downtime will be minimal and should not be notice-able to most users.

16th March 2023

All cloud services have been updated and are online.

14th March 2023

Regular maintenance upgrades are scheduled for Synectics Cloud Services and Cloud Evidence Locker from 08:00 GMT on Wednesday 15th March and Thursday 16th March. Loss of access to Cloud Services during this time should be minimal and operations may be slower during this period.

8th February 2023

Update 12:11 GMT: Maps are back online in Cloud Services (e.g. Telematics Mapping)

8th February 2023

Some sites are seeing a message "This page can't load Google Maps correctly" when trying to view maps. We are actively investigating the root cause and will update this bulletin with further info.

4th November 2022

Regular maintenance upgrades are scheduled for Synectics Cloud Services from 18:00 GMT on Tuesday 8th November. There will be 2 planned maintenance windows of 2 hours each; loss of access to Cloud Services during this time should be minimal and operations may be slower during this period.

3rd November 2022

Regular maintenance upgrades are scheduled for Synectics Cloud Services from 18:00 GMT on Thursday 3rd November. There will be 2 planned maintenance windows of 2 hours each; loss of access to Cloud Services during this time should be minimal and operations may be slower during this period.

27th October 2022

Update: As of 17:30, the issues have been resolved. Cloud Services are back to being operational.

27th October 2022

A problem has been identified with a recent update for optimising data ingestion. As of 10:00, we have identified the issue and are in the process of rolling back the upgrade.

22nd June 2022

Update: email services are fully operational

21st June 2022 10:44

Update: The 'SendGrid' email component is now operational. We are assessing the impact to emailing from Cloud Evidence Locker.

20th June 2022

An issue was reported with the 'SendGrid' component used to send emails within our Cloud Services including Cloud Evidence Locker. This meant emails generated by Cloud Evidence Locker from 6am this morning were not sent. We are tracking the issue with the provider of SendGrid.

9th March 2022

(09:00 - 09:44)

Update: Overnight back-end upgrades caused an issue with dependent front-end cloud services. We identified the dependency and rolled-back the upgrade.

Mitigation: the inter-dependency has been identified and steps taken to ensure future updates will not impact front-end services.

Cloud Services back online 09:44

23rd December 2021 (13:09)

A recent Azure upgrade to the AKS for the Cloud Evidence Locker cluster reset our VM types back to a basic level.  We have changed the VM types back to the higher specification level.   We are implementing process improvements to ensure this never happens again.

Cloud Services back online.

23rd December 2021 (09:00) We have identified that all of our Cloud tenancies are offline. We are working to resolve this as soon as possible.

14th December 2021

(16:00)

Update: Microsoft Azure have determined that a recent deployment task impacted a backend service configuration which caused devices to disconnect and fail to reconnect to IoT Hub.

Mitigation: We rolled back the deployment task to mitigate this issue.

Cloud Services back online.

13th December 2021

(23:41)

We have been made aware by Microsoft that there is an ongoing issue, starting 23:41GMT on the 13th December, specifically the UK West region IoT Hub.  This is affecting our Cloud Services platform with no vehicles being able to connect to  our platform.

Current status from Microsoft: We are aware of the issue and are actively investigating. Next update will be provided in 60 minutes or as events warrant.

6th October 2021 (11:18)

The issue was due to the expiry of a legacy certificate, as a result access to services failed.  There were complications with the existing certificate manager which have now been resolved; we are now acquiring certificates that validated automatically.

Cloud Services back online.

5th October 2021 (12:18) We are aware of some instances of our Cloud Services including Cloud Evidence Locker being offline. We are working to resolve this as soon as possible.
25th May 2021 (15:20)

Our hosting provider has narrowed down possible causes for this system outage and is working as a priority on identifying the root cause.

Cloud Services back online.

24th May 2021 (10:00) We are aware of some instances of our Cloud Services including Cloud Evidence Locker being offline. We are working to resolve this as soon as possible.
Was this article helpful?
1 out of 1 found this helpful