More Antivirus Warnings on Auctiva Website (2/21/09)

Hi Community,

Update - I just heard from our IT team and am pleased to report that users should not longer receive warnings from their antivirus software while on the Auctiva website. However, the site is still running on fewer servers than normal so it may be slower then usual for a little while longer.

We are working on getting some additional servers configured and we'll be adding those back to the rotation as soon as they are ready. As those are added back throughout the rest of the day, we expect the speed of the site to continue to improve.

I will post further updates on this situation to this thread as additional information becomes available. Thanks again for bearing with us.

-Mike

Hi Community,

[ I have removed a portion of this post that I do not feel was an appropriate action to take for our users. At all times, I recommend keeping your browser security settings as is and not disabling any warnings. -Kevin ]

We have removed what was causing that warning to be displayed from our systems but that warning is still showing up because we need to be rescanned by Google, which we are going to do as soon as we finish up the work we're doing.

If you have any questions, please feel free to contact our customer support team using the web form on the following page of our site: http://www.auctiva.com/help/requesthelp.aspx. I'll post further updates to this thread as additional information becomes available.

-Mike

H- Community,

Update - While we have made some considerable progress since I posted last to this thread yesterday, I would like to reassure everyone that we are still aware of several issues that are causing the Auctiva site to be slower than usual and that our IT team is continuing to work on getting these issues resolved as soon as possible. Unfortunately, we're still not sure exactly how long it will take.

I will post further updates on this situation to the following thread as additional information becomes available. We apologize for the inconveniences these issues may be causing and we appreciate your patience while we deal with this difficult situation.

-Mike

Hi Community,

Update - Our engineering team is still investigating this situation but, at this point, it appears the reason these virus alert warnings started showing up on our site is because some of our machines were injected with malware originating in China. The malware we believe to be at fault has also hit a number of other high profile websites over the past 6 months.

If our current suspicions about what happened are correct, we know some things we can do to prevent this from happening again, but some additional investigation will be required before we reach a conclusive determination.

The affected machines are no longer in our rotation so it is currently safe to navigate the Auctiva website, however, if you did visit our site between Thursday evening and Saturday afternoon at about 2 PM PT, as a precautionary measure, we recommend taking the following actions to ensure that your computers are not infected:

1) Clear your browser cache, delete ALL temporary internet files, and restart your browser.
2) If using a Windows machine, make sure you are updated with all the current Microsoft updates and patches.
3) Make sure you are running some reputable antivirus software (AVG is available for free at http://free.avg.com and is known to catch this malware)
4) Use the Firefox browser if possible, as it has been shown to be less susceptible to this sort of malware than Internet Explorer.

We will post further updates on this situation to this thread as additional information becomes available. If you have any additional questions, please feel free to contact our customer support team using the appropriate link on our help page: http://www.auctiva.com/help/requesthelp.aspx

-Mike

Since we detected a virus was put on our servers we have been working 24/7 to resolve this problem.

We have put up new, clean servers in place of the old and are closely monitoring to make sure no additional problems occur.

As of 5pm on Sunday, we have initiated a request with Google to have our site cleared from being reported as a malicious site. I currently do not know how long it will take Google to clear us.

As we have updates from either Google or our IT team, we will update the community.

Sometime during the night, Google cleared us. This is very good news!

A couple other updates:

1. Our edu site is currently offline. We believe the vuneralbility was within this part of our site, which is open to the public regardless as to if you are logged in or not. Our developers are working to fix this, so we can bring this part of the site back up.

2. We are continuing to go through the entire site and our servers to monitor things and look for any other potential vuneralbilities.

3. We don't have a full capacity of Auctiva.com webservers up at this point. Although we have enough so that the site should be nearly as fast as normal. We should have all servers back up sometime today.

4. This only affected our Auctiva.com site. None of our other sites were affected, auctivacommerce, sellathon, buyshield, etc.

I have seen reports of some users reporting that they are still getting infected when visiting Auctiva.com Monday morning.

Auctiva.com is NOT still infected.

However, we did not have versioning in place for some javascript files. What this means is the file on our server is clean and the correct file that should be there. But your home computer will use a cached version of the file that is malicious.

Two things here:

1. We are making sure to put out versioning on all servers for all js files. (This is a normal part of the process we do, but some versioning did not occur due to the frantic nature of getting clean machines out the door).

2. Make sure to clear your cache on your browser. I'll have support give detailed info on how to do this.

Hi Community,

I would just like to follow up on Kevin's post above to let all of you know how to clear your cache in case you aren't familiar with that process already. The instructions below detail how you can clear your cache for each of the browsers supported by the Auctiva website:

Firefox 2.0+:

In order to clear your cache in your Firefox web browser, please select "Clear Private Data" from the "Tools" menu, ensure the boxes next to "cache" and "cookies" are checked, and then click the "Clear Private Data Now" button.

Internet Explorer 6.0:

In order to clear your cache in your Internet Explorer 6.0 web browser, please select “Internet Options” from the “Tools” menu. Then, on the ensuing page click the “Delete Files” and “Delete Cookies” buttons, followed by “OK”. Once this process has been completed, close and re-open your Internet Explorer 6.0 web browser for the changes to take effect.

Internet Explorer 7.0+:

In order to clear your cache in your Internet Explorer 7.0 web browser, please select “Internet Options” from the “Tools” menu. Then, under the “General” tab, click the “Delete” button. Click the “Delete Cookies” and “Delete Files” buttons on the ensuing page, followed by “Close” then “OK”.

Internet Explorer 8.0 (BETA)

In order to clear your cache in your Internet Explorer 8.0BETA web browser, please select “Internet Options” from the “Tools” menu. Then, under the “General” tab, click the “Delete” button. Click the box next to the “Cookies” and “Temporary Internet Files” buttons on the ensuing page, followed by “Delete” then “OK”.

Once you have completed this process, close and re-open your web browser for the changes to take effect. Also, please to ensure that only one browser window is open when you are doing this or your cache may not empty properly.

-Mike

Checkout update:

Our checkout servers were never infected throughout the entire time we have been dealing with this virus.

No buyers will have received any type of infection from going through Auctiva checkout.

However, buyers using certain browsers, like Google Chrome, will have seen a warning when going to checkout over the weekend. This was because of the flag on the Auctiva.com domain name by Google. This has since been lifted and buyers are no longer receiving any type of warning.

ALERT: Due to additional monitoring we have been doing, we are going to have to take additional auctiva.com webservers offline as a precautionary measure.

We have taken the Auctiva.com website down.

This is to ensure that we do not expose our users to any virus threats.

A message has been put up on the site with more info explaining this.

Your existing eBay listings will still function as normal. Images are not affected by this and will display in your auctions. Checkout is working on seperate servers and is working as normal.

It looks like after many hard and dedicated hours of work by our IT and development team, we have pinpointed the root cause of the original virus. By doing this, we can now feel secure that our servers can come back online and are safe for our users to use.

To make certain we are clean, we are putting out new servers that have fresh installs of our website. We plan to have this completed by 2am Pacific Time. At that point, our website will be back up online and open to use as you normally do.

If anything changes in our plan, we will update this thread.

Thanks for your continued patience and understanding during this difficult time.

-Kevin

Our timeframe for coming back online remains about the same. It may delay a bit past 2am Pacific time, but I feel we're still within an hour or so of that original time estimate.

It's been a long couple of days and a lot of people have put in a tremendous amount of work to make this happen.

I'd also like to point out that eBay has worked with us to resolve this problem, including lending a hand with their IT staff to help narrow down the problem.

Thanks are due to so many people right now, our customers, our IT staff and eBay for working with us on this issue. I know we're all looking forward to moving past this and getting back to running our respective businesses.

Auctiva.com is up and running as of around 5am this morning. We are on a more segregated network with increased security.

We feel confident the root issue of our virus was indentified and we're moving forward with the necessary protection to prevent this from happening again.

Thanks again to all those involved in making this happen.

There are a few minor issues with the live site still that we are resolving. These are no longer related to the virus, but rather issues in bringing the site back up on new servers. Our Release Engineering team is working with IT to get things back to complete. The sign-up form and education are two of these areas.

Hi Community,

Update - I would just like to let you all know that our engineering and IT teams, as well as some people from eBay's technical team, have been constantly monitoring our systems since they were brought back up early this morning and all the signs we have seen thus far have been good ones. We have not seen any evidence that we may have been reinfected at this time.

The Auctiva website is still running on less servers than normal and, while we haven't heard of any major issues with the speed of the site, it is expected to be a little slower than usual. Our IT team is in the process of building some additional servers and we expect to be back to full capacity at some point this week.

We will post further updates on this situation to this thread as additional information becomes available. We're hopeful that there will be nothing but good news to report on this thread from this point on.

-Mike

Kevin recently sent out an email update that has some new information. I thought I would share that email here.

SUBJECT: UPDATE - Information regarding Auctivas Site Warning
February 24, 2009
5:30 PM PST

After notable efforts by our IT and Development teams, as well as assistance from eBay, we were able to bring Auctiva.com back online as of 5 am, PST. Our site is safe to navigate, as verified by Google.

We identified the root issue of the malware and we're moving forward with the necessary protection to prevent this from happening again.

We are on a more segregated network with increased security and are performing on-going virus scans. Additionally, eBay is currently running a vulnerability scan to ensure the integrity of our database. There are still a few minor issues with the live site that we are resolving. These are no longer related to the malware, but rather issues in bringing the site back up on new servers. Our Release Engineering team is working with IT to get things back to complete.

What happened?

The virus malware was injected via a third third-party plug-in. Once in the file directory, the virus malware executed malicious script that gained access to files. Once access was gained, the perpetrators used that access to place low-level malicious script into files that were distributed to some of our users.

What can you do now?

• As a matter of good practice everyone should be regularly scanning their computer with antivirus software. If you already have antivirus software then you should check to be sure you have the latest virus definitions update for the software and run routine scans of your computer.
  
• If you don't already have antivirus software eBay recommended that users try Microsoft's OneCare antivirus scanner for home users. http://onecare.live.com/standa.../install/install.htm .
  
• If you are using a Windows machine, make sure you are updated with all the current Microsoft updates and patches.

Thanks are due to so many people, our customers, our IT staff and eBay for working with us on this issue. I know we're all looking forward to moving past this and getting back to business as usual. We will continue to post regular updates on our Community Forums https://community.auctiva.com/e...s/a/frm/f/1081020411.

Sincerely,
Kevin Kinell
VP, Engineering