Skip to main content

Closed6 replies

Auctiva Hosted Images Missing from Accounts/Listings 7/20/2008

A auctivamikedAuctiva Customer Support Manager
Hi Community,

As many have been discussing on our forum today, we are currently experiencing a technical issue that is causing Auctiva hosted images not to show up in our user's accounts and listings. The root cause is actually on the end of a third party company that is involved in our image hosting system and our technical team brought this to their attention as soon as our image hosting went down. We haven't been provided any sort of ETA as to when this issue may be resolved, but, if we do receive any such information, I’ll be sure to post it here.

I’ll be posting further updates on this situation to this thread as additional information becomes available. We sincerely apologize for the inconvenience this issue is causing and we appreciate your understanding during this difficult time.

-Mike
Original Post

Replies sorted oldest to newest

A auctivajeffAuctiva CEO
Amazon, our storage provider, is having a service interruption that's causing problems serving some of your photos.

Here are the updates from their service dashboard. Amazon S3 is the name for their file storage which we use for our image hosting:

9:05 AM PDT We are currently experiencing elevated error rates with S3. We are investigating.
9:26 AM PDT We're investigating an issue affecting requests. We'll continue to post updates here.
9:48 AM PDT Just wanted to provide an update that we are currently pursuing several paths of corrective action.
10:12 AM PDT We are continuing to pursue corrective action.
10:32 AM PDT A quick update that we believe this is an issue with the communication between several Amazon S3 internal components. We do not have an ETA at this time but will continue to keep you updated.
11:01 AM PDT We're currently in the process of testing a potential solution.
11:22 AM PDT Testing is still in progress. We're working very hard to restore service to our customers.
11:45 AM PDT We are still in the process of testing a series of configuration changes aimed at bringing the service back online.
12:05 PM PDT We have now restored communication between a small subset of hosts. We are working on restoring internal communication across the rest of the fleet. Once communication is fully restored, then we will work to restore request processing.

You can find those updates here:
http://status.aws.amazon.com/
It's the "Amazon Simple Storage Service (US)" link. S3 is short for "Simple Storage Service."
A auctivamikedAuctiva Customer Support Manager
Hi Community,

Update - A number of customers have expressed concern over whether or not the images that cannot currently be seen in their accounts and listings will be visible once again when this issue has been resolved. Fortunately, the images that cannot be seen now due to this problem will be visible once again when Amazon gets this service interruption taken care of. I'll post again to this thread when additional information on this situation becomes available.

-Mike
A auctivajeffAuctiva CEO
More updates from Amazon:

12:25 PM PDT We have restored communication between additional hosts and are continuing this work across the rest of the fleet. Thank you for your continued patience.
12:51 PM PDT The restored hosts are stable and we are moving forward in restoring communication between additional hosts.
1:17 PM PDT We continue to make incremental progress and communication between additional hosts has been restored. We are continuing with the plan to restore communication across Amazon S3's large fleet of hosts.
1:38 PM PDT At this point, we are accelerating progress on restoring internal communication as all signs continue to look good.
2:03 PM PDT We have restored all internal communication between hosts in the EU and we are continuing to make progress in the US. Once all internal communication has been restored, we will start a multi-step process to begin accepting requests across Amazon S3 locations.
2:19 PM PDT A quick update to let you know that we have now also restored all internal communication between hosts in our West Coast facilities in the US.
2:36 PM PDT We have restored all internal communication across Amazon S3 hosts. We have started the multi-step process to begin accepting requests across Amazon S3 locations.
3:07 PM PDT We are attempting to bring EU back up now, followed by our US locations. EU will be first due to the smaller number of hosts. No data has been lost during this incident.
3:23 PM PDT EU service has been fully restored. We have been working on the US in parallel but restoration will take longer due to fleet size.
A auctivajeffAuctiva CEO
Images were restored between 4:15pm and 5pm:

4:22 PM PDT US service has been partially restored. We continue to work to fully restore the service.
4:42 PM PDT We continue to restore US service. We expect that request processing will be fully restored within 15 minutes.
5:00 PM PDT Amazon S3 service has been restored and is returning to normal. We will continue to monitor closely.

Amazon has been a valuable partner helping to serve images for Auctiva for almost 2 years now with a very high degree of reliability. We'll be working with Amazon over the next few days to make sure this issue is fully addressed and doesn't happen again.

Thank you for your patience and we apologize for the inconvenience. We're aware of how critical a part of your listings images are and especially so on Sundays.
A auctivajeffAuctiva CEO
Another company that stores images on Amazon, that I personally also use said things quite well. SmugMug and Auctiva are two of Amazon S3's larger users:

http://smugmug.wordpress.com/2008/07/20/amazon-s3-outag...uses-smugmug-outage/

quote:
Amazon’s S3 service, SmugMug’s primary storage provider, is currently experiencing problems. As a result, a large portion of the photos and videos stored on SmugMug are currently offline.

Historically, Amazon has been very stable. We’ve seen three of these in our entire history with Amazon (>2 years), including this one. I expect, like the last two, that service will be restored shortly. You can keep track of their efforts over on their own Status Dashboard.

Our faith in Amazon, and the care they take of your priceless memories, hasn’t been shaken. Your photos and videos are safe - which is our #1 concern. Since problems in this industry are inevitable, and Amazon’s performance over the last two years has been so exceptional, we’ve been afraid an outage like this. I’m sure there will be more over the next few years, too.

The important thing is that they’re few and far between, short, and handled properly. Every component SmugMug has ever used, whether it’s networking providers, datacenter providers, software, servers, storage, or even people, has let us down at one point or another.

It’s the name of the game, and our job is to handle these problems and outages as best we can. We’ve already spent the last few months investigating additional resources we can combine with Amazon to cover outage scenarios like the one we’re experiencing today, and have some promising prospects.

Please continue to check here and Amazon’s Dashboard for future updates. Thank you for your patience - we realize this is no fun, and we’re truly sorry it’s happening.

UPDATE: Amazon continues to make progress, we hope they’ll be online again soon, at which point we’re ready to re-enable SmugMug. A few of our newer customers have asked, so I’d like to point out that I believe this our first unplanned long (more than a minute or two) site-wide outage in well over a year, and we’re definitely going to take steps to reduce the likelihood of this recurring.
Post
Copyright © 1999-2018 Auctiva.com. All rights reserved.

Membership Required

We're sorry. You must be signed in to continue. Sign In or Register
×
Are you sure you want to remove from your Block List?
×
Loading...
×
When you block a person, they can no longer invite you to a private message or post to your profile wall. Replies and comments they make will be collapsed/hidden by default. Finally, you'll never receive email notifications about content they create or likes they designate for your content.
Note: if you proceed, you will no longer be following .
×
Advanced Search
Advanced Search
Link copied to your clipboard.
×