in

CLE02WEB09 (WEB30) RAID Array Maintenance

Last post 01-13-2010 1:30 PM by kschleeper. 12 replies.
Page 1 of 1 (13 items)
Sort Posts: Previous Next
  • 01-05-2010 9:20 AM

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    CLE02WEB09 (WEB30) RAID Array Maintenance

    Early this morning Tuesday Jan 5th, we have detected a degraded RAID array on CLE02WEB09 (WEB30) IIS server. Our engineers elected to continue operating this server until service team arrives at the datacenter to replace a failed hard drive later this afternoon. This server will have to be taken offline for diagnostics and repair. We will post further announcement related to the RAID rebuild that will take place today.

     

    • Post Points: 0
  • 01-05-2010 11:12 AM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    Service team has been dispatched to the datacenter to replace failed hard disk.

    • Post Points: 0
  • 01-05-2010 1:05 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    Server will be taken offline shortly for maintenance. We will update ETA as soon as more information is available

    • Post Points: 0
  • 01-05-2010 2:44 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    Services has been restored. Unfortunately the RAID controller is returning errors which indicates hardware problems with the MB chip-set. For now services are operational, however we will be moving all sites hosted on this server to a different physical server in the near future.

    We appreciate your patience, additional information will be posted prior to moving hardware.

    • Post Points: 0
  • 01-05-2010 7:32 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    We have made changes in the RAID BIOS and are currently attempting to rebuild RAID array in windows safe mode. This process should take 45 to 60 minutes to complete.

      Repairing existing controller in place is much faster than restoring all sites on a spare server. We appreciate your patience and understanding, we are working hard to resolve this issue as soon as possible.

     

    • Post Points: 0
  • 01-05-2010 10:16 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    Our team is currently at the data center preparing spare standby server to recover sites currently hosted on WEB30. If we cannot correct hardware issues on the existing server with in next couple hours, we will begin restoring all sites on a new server.

     We will be posting updates as the situation develops. Thank you for you patience.

    • Post Points: 0
  • 01-05-2010 11:52 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    We were able to replace RAID controller and rebuild RAID array on WEB30 server. As of 11:45AM EST server is operational. We have flushed out the .NET assembly cache, as the result ASP.NET based sites may experience initial load delay due to JIT compiler process running in the background. The page load speed will improve over next several hours as bulk of assemblies are JITed and CPU I/O normalizes.

    Also, the standby spare server is ready to be pushed in production should further hardware problems continue to occur with this server.

     

    Thank you for your patience and understanding.

    • Post Points: 0
  • 01-06-2010 10:08 AM In reply to

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    Our technicians detected additional errors in the functionality of the RAID controller momentarily ago.  Currently our technicians are rebuilding the RAID array and running full diagnostics offline, there forth sites which are hosted on this server are currently unavailable.  Our technicians are estimating it will take 2 hours to complete the RAID rebuild and restore services.  We will continue to post updates as new information is determined.  We appreciate your continued patience in regards to this issue.

    • Post Points: 0
  • 01-06-2010 11:24 AM In reply to

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    Our technicians have completed the RAID rebuild and diagnostic checks on this server's hard disks.  Windows has been booted on this server and web sites are starting to spool up.  Performance maybe delayed on sites for a while since all web sites on this server must complete JIT compilation when loading.  We appreciate your continued patience in regards to this issue.

    • Post Points: 0
  • 01-07-2010 7:34 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    We will be shutting down this server in approximately 5 minutes to replace "A" side RAID disk as part of this preventative maintenance effort. Server should resume regular operation approximately in 1 hour.

    • Post Points: 0
  • 01-07-2010 8:48 PM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    The "A" side disk replaced and RAID rebuild completed, server is fully operational. As a precautionary measure we are replacing all member disk drives with brand new units. The "B" side rebuild will be scheduled for next week.

    • Post Points: 0
  • 01-13-2010 9:48 AM In reply to

    • skozyuk
    • Top 10 Contributor
    • Joined on 10-28-2007
    • Westlake, OH
    • Posts 328
    • Points 1,081

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    This afternoon we are going to be replacing disks on the B side of the RAID array. You may experience outage lasting 45 minutes to an hour while we swap and rebuild raid on the WEB30 server.

    • Post Points: 0
  • 01-13-2010 1:30 PM In reply to

    Re: CLE02WEB09 (WEB30) RAID Array Maintenance

    The disk replacement and RAID rebuild has been completed.  Services have been started on our WEB30 server and we believe most sites should be available now.  Due to the high volume of sites loading on the server, extensive JIT compilation is occurring, which may temporarily cause some delays when loading sites hosted on this server.

    • Post Points: 0
Page 1 of 1 (13 items)
2004 - 2008 IHOST, LLC