Status Updates

Data Center 1 Network Issues

Our hosting provider is investigating an issue affecting data center 1. This is impacting a number of our customers. We will be providing updates as more information becomes available.

UPDATE: Our hosting provider reported that Data Center 1 is experiencing power failures. We are watching over this closely, but have started our disaster recovery process, to restore the service to a different location.

UPDATE 2: Power is restored, but our host is not online yet. We are hoping it will be back online shortly. Meanwhile our disaster recovery process is underway in case service is not restored quickly enough.

UPDATE 3: Service is restored. We are still running some tests to make sure everything is stable.

Data Center 2 Temporary Unavailable

Update (15:21 UTC) – Data center up and running.

Looks like we’re having a misunderstanding with our hosting provider… we’ll be up and running again in about 15 minutes. Updates will be posted accordingly.
We apologize for this temporary inconvenience.

Suspected network issues in Europe

UPDATE: Our hosting provider informed us that those issues are now resolved.

We have been reported of some network issues affecting Data Center 3 and Data Center 5 in Europe. We are not observing any significant packet drops or errors currently, but European customers might experience slowness or lost connections. We are keeping a close eye over this and take action if necessary.

Data Center 2 Connectivity Issues

Update: We have moved the server to another location and it is now up and running. We will seek further clarification from our hosting provider.

We are currently investigating an issue connecting to data center 2 (trial servers only) with our hosting provider. We will post updates when more information is available. All other data centers (production) are fully operational currently.

Data Center 1 Connectivity Issues

Update: Our hosting provider indicated the connectivity issues were caused by a large scale denial-of-service attack targeting another host on the same data center. They have further informed us that they have taken the necessary measures to mitigate the attack and connectivity is now stable again.

Update: Connectivity appears to have been restored. We will continue to monitor the situation and seek further clarification from our hosting provider.

Update: It appears that connectivity is only available from certain locations. Our hosting provider is investigating and trying to resolve this issue.

We are currently investigating an issue connecting to data center 1 with our hosting provider. We will post updates when more information is available. All other data centers are fully operational currently.

Pingdom – false alarm

We have received a false alarm from Pingdom about all our data centers. However, our independent monitoring shows no issues in any of our data centers. We have informed pingdom and until they resolve the problem would keep an open eye on our running services using our internal monitoring tools. All data centers are up and running as usual.

Connectivity loss to data center 2

UPDATE: Connectivity appears to have been restored. We keep monitoring in case there are further issues.

UPDATE: Our hosting provider reported wide scale networking affecting the whole data center, resulting from network backbone issues. We will be monitoring the situation and take action to move to a different data center if the situation continues for much longer.

There is currently no connectivity to data center 2. This is being investigated with the hosting provider. Our trial service is affected as a result.

Connectivity issues data center 2

Our hosting provider has reported a Distributed Denial of Service (DDOS) attack on its infrastructure, affecting the service to our trial users. Data center 2 was unavailable for approximately 17 minutes. Service is now restored and back to normal.

RESOLVED – Connectivity issues to data centers 3 & 5

UPDATE: Whilst not observing any connectivity issues since the last update, we were made aware of some further work by our hosting provider to replace some network equipment. This might cause further intermittent issues. We are carefully monitoring the situation and will take action if necessary.

Our hosting provided reported that the problem was caused by a routing loop that disabled some network ports temporarily. Our monitoring systems only observed a brief period of connectivity drops. All services are fully operational at this point.

Connectivity issues to data centers 3 & 5

One of our hosting providers reported connectivity issues which may affect data center 3 and 5. We are awaiting further updates whilst investigating. Service is still available, however high latency and packet loss issues have been observed.