What a pleasure to talk about what a wonderful system was implemented, how the costs were reduced, how equipment was modernized or what a cool server room was built. It’s nice when the big bosses com to cut the ribbons and many talk about everything. It’s nice when the news about this is printed in newspapers. It’s nice, incredible, amazing, surprising.
But we are living in Russia – we are not looking for easy ways. Maybe in other countries as well as with us, but maybe not – I don’t know, I didn’t see, didn’t implement. But I have seen a lot of different companies in Russia from inside. Here on such case will be discussed.
Thus, a large company, more than thousand people of employees, huge production spaces, a large staff of accountants. It is located somewhere in the north. The leaders communicate directly with the leadership of the region. Money is available.
In this company I helped to implement the system of payroll accounting and the system of bookkeeping. The workload was very large. Primary data entered in system was also too much. About 60 accountants were sitting with primary documentation. A heap of mindsets on my part. The typical configuration was fitted with different ways. It has acquired a bunch of treatments, new program modules and other features. The system however work stridently, with tambourine, with F-bomb and tea deciliters.
It was my cross. Every day for lunch I cam to this company and worked with numbers, data, primary documentation, accountants and system. Hardly, but the system began stable after all and I could come in a day, once a week, once every two weeks.
And then, one fine day, the leaders decided to create a cool server room. Like on the picture in the booklet. Well, a good initiative. I’m also curious. I had a number of claims to an existing server infrastructure.
Done! Like in the picture. microclimate, the most cool servers, power supply, the cables in beautiful ducts under floor. Mmmmm, nice to look at. Everything is fantastic.
Next, the red ribbons, beautiful speeches, the big bosses, administration officials. And weekdays have come. Inspiring passed – everything got back to usual routine. Databases were downloaded on the servers, the users were connected and continued to work. Frankly, work became much more pleasant.
And here the terrible thing happened. The storm. A bunch of trees was brought down, the windows were smashed in some houses, a few roofs were blown off. Including from building of company. Probably, in heaven there are also the creatures with humor, the roof was blow off above the server room.
What was next – you can guess. All servers were burned, the bases on them too. Server room ceased to exist. As it’s said – in less than six months.
But this is not the most terrible – together with working databases their backups were burned too. Backups though were created every day on another server, but in the same room. Shock. Panic. Hope. Anger. Fatigue.
What to do? The last survived backup is one year ago. When the previous major accountant was leaving, we had made with her the database backup. And the discs were putted in a safe. Just in case. There were no flash drives then. There were the discs. And database had 2 Gb of data. Of course the discs were about 5 pieces.
The old servers were launched. A singled backup was downloaded and the bookkeepers were seated to recover all accounting for the last year. That was really something. They worked seven days a week, 12 hours a day. More 50 accountants were sent for help from other organizations.
Database was restored hardly, tearfully, with F-bomb. Of course there were the losses, but plus of minus 2-3 millions were no longer considered as a critical error. Further it was already not so interesting. Life settled down and this case was still remembered for a long time and used to scare the new employees and system administrators.
But I always tell this case in the topic «Safety engineering» on my courses. The funniest thing in this story is that some accountants went to maternity leave after system was restored – to have a rest. Whole departments. And a lot of cute kids were born.
So, to raise demographic situation it is necessary to create the local crisis. Or to unroof.