Skip to main content

All,

As you know, we experienced a lengthy outage of Auctiva.com over the weekend just past. Service has now been fully restored, and the Auctiva.com site is back on-line and fully functional. We sincerely regret the inconvenience this caused our users.

I want to take a moment to explain in more detail what happened, what we did, and what we are doing about it going forward.

This past weekend, we experienced a sudden, unexpected and serious failure of the communications front-end hardware that Auctiva.com relies on to communicate over the internet. The specific hardware components that failed have a fully redundant backup system in place, and to our dismay, that backup system also failed. For most of the outage duration, we had no connectivity to the internet from our data center, with the result that we were unable to transmit the ‘site down’ page that we usually use.

All Auctiva staff who possess the technical knowledge required to assist with a hardware failure of this type were engaged for the entire duration of the outage in correcting the issue. In addition, they consulted with outside experts.

Unfortunately, this was the type of outage where the vast majority of time was consumed in diagnosing and correcting multiple issues that were in play. Additionally, we could not give any estimates of time remaining as the only thing we could be sure of is that such estimates would likely prove to be wrong.

Further complicating matters was the fact that our main means of communicating to Auctiva users, the site itself, was not working. Our priority at that time was, and had to be, restoring the site to full service as quickly as possible.

Now that the problem has been resolved, we are looking at measures we can take to prevent situations this one from occurring in the future. As a part of this discussion, we will consider how we can improve our communication with the user community about our status in such situations as this one.

Our goal remains for Auctiva.com to be the most reliable site possible, and in this instance we fell short of that goal. Again, let me re-state that we sincerely regret any and all inconvenience this outage caused.
Auctiva Tony M. Sr. Product Manager, Auctiva.com
Original Post

Replies sorted oldest to newest

Well,
No matter how you think every eventually is covered there is always a possibility of an unforseen event.
We suffered from the weekend outage but...we still have a life.
Hopefully a post mortem of the problem(s) will lead to a more secure site in future. I am guessing a "site down" page from a server in a different geographical location could be a high priority and a designated person be appointed quickly, for any major problem, to communicate with customers (could be anyone with the right attitude and doesn't need technical knowhow).

Anyway, many thanks for posting an explanation of the problem.

Jeff
Tony, thanks for the explanation. Personally, I know that NO system is 100% failsafe, and that these things will happen (hopefully not too frequently!).

My only concern with Saturday's events was the lack of communication. A post in the forums or on Facebook or Twitter just saying you all knew you were having problems and were working on them would have been appreciated. I do understand that everyone was working on fixing the issue, but your end users need to be kept in the loop.

Thanks again for the update and for the hard work - I am a loyal Auctiva customer and will continue to be one.
Observations:
1.. it appears that the Forum server must be housed off site than as it was up the entire time
2.. someone not critical to the restoration of service could have communicated earlier and more often than was done. That would be Robert Green. One post in all the time the issues were being worked on is abysmal
3.. for those of us in the Auctiva community that would understand a technical explanation what has been given is minimal at best
We agree that we need to communicate better in situations like this. No argument from us there; we understand we should have communicated better, and we are taking steps to do better in the future.

As to your question about why no one used the forums for so long, our forums run on servers provided by a third party, not on servers in our data center. Our team in our data center had no internet access...that was in fact the problem they were trying to fix.
Right, we did not expect an IT person to quit working and head over to forumville. I would assume there were people manned there that were not IT personnel that would have a cell phone that they could post from, or at least make a phone call to non-site personnel to post or a wireless card, something that they could access a forum on.

We did eventually get notification through the forums, so how did they get on at that point, as we were not back up for many hours after that posting? I know for myself I have had internet outages before but took the time to get myself to the library so that I could use the internet to communicate with the people so kind to buy from me. Have gone to neighbors and IN desperation have pulled up at hotel and used their network.

Could we be enlightened at some point on what the new communication system will be. How we will know that you are aware there is a problem. Some found information on facebook and twitter. . frankly I do not find having to sign up for those services and be friends to be a solution, but is better than nothing.

I just do not understand what the problem with posting on forums is if everyone could get on it and eventually auctiva did get on but the problem still existed.

Not trying to beat a dead horse, but I would like to know that next round I will know where to go and get information. Kind of like a fire drill, we need to be prepared and know where to go.

I would like to think this outage would NEVER ever occur again, but not one soul can guarantee that so we need to be prepared in case.


For some reason I would guess that no one even thought about communicating the information and now we are filibustering rather than just saying, it never crossed our mind. Could very well be wrong. . but since someone did eventually get on . sure makes me wonder. Normally a note or familiar name on the forums is reassuring. . . but in this case I had no idea who was writing and did not feel overly reassured by the writer.

I am sorry if you feel you are being beat to death with this, but I find the answers about future issues and plans very vague. . very vague and while you can't change the past we need to plan for the future.

Thanks.
lookandbuyme you said it so much better than I could. That's why I've worked in IT for 38 plus years you don't need to speak in full sentences to a computer.

You comment about vague is spot on. The explanation about what happened was just as vague. I understand "stuff" happens but from an IT perspective I do have to question why there weren't certain disaster recovery things in place that could have lessened the time to recover
Ed. . again I feel like I am beating a dead horse. I guess the thought is that we will just go away, maybe that is the wish who knows?

Auctiva, where do we look for information IF this site crashes suffers an outage tomorrow??? I am sure the hope is that will not happen, but obviously auctiva nor anyone else can guarantee that, ANY site can go crash or suffer outage. I believe in recent years ebay has, amazon and paypal, so noone is immune.

We need to know what plan is for information moving foward. I do not think that is too much to ask. I just do not want to be asking these same questions the next time it happens.

Also, I would like to know what the plan is to inform people who may not forum and get on say this Sunday to see their listings in total disarray. (this could be any of us). As I have friends who are clueless that their listings did not post this weekend. IF I came in to find that my listings failed and I spend hours trying to figure out why I would be FURIOUS! That is time wasted by the seller which to me is not fair. (not that life is fair). It is common courtesy to let people know their listings failed at minimum.

Thanks
Last edited by lookandbuyme

Add Reply

Copyright © 1999-2018 Auctiva.com. All rights reserved.
×
×
×
×
Link copied to your clipboard.
×