[SUCS] Your non-weekly newsletter

Imran Hussain imranh at sucs.org
Mon Oct 13 09:54:03 BST 2014


It saddens me that the first news post I make of this academic year is 
about a data corruption incident.

On Friday the 10th of October some of you may have noticed that silver 
(our main server) went away for a bit, an so did your files. What 
happened was that sometime during Wednesday the 8th of October the part 
of the server where we keep your (and our) files got corrupted, we 
didn't notice. Maybe if we had noticed then this wouldn't have been so 
bad, maybe. However for the next 2 days or so, random files and data 
were being erased, overwrite or just not being written.

When we finally noticed on the morning of Friday the 10th we took the 
server offline, after examining the damage we decided the best course of 
action was to wipe everything and restore from a backup. Due to the fact 
that files were being randomly written to etc.. the last backup that we 
had and that we also trusted was taken on the morning of the 8th. That 
has been restored and you should all now have the vast majority of your 
files back.

The file restoration process is still going on and at a later date we 
shall make the backups from the 9th and the 10th available to you all, 
these are to be used at your own risk as we cannot guarantee that they 
are what you think they are. Instructions on how to access them will be 
posted later, watch this space.

We have taken steps to prevent data corruption potentially becoming so 
bad in the future, upon detecting an error, file systems will be 
remounted as read only and only after an admin has examined the 
situation will they become available for writing again.

In the occurrence of unexpected downtime places like facebook 
(https://www.facebook.com/SwanseaUniversityComputerSociety) and twitter 
(https://twitter.com/sucsexec) are where we will post updates.

I would also like to remind that you should have anything important 
backed up in multiple places.

Thank you.

Imran Hussain
President & Systems Administrator
Swansea University Computer Society

More information about the Members mailing list