RKS Server Update Thread

[PY004]

Re: RKS Server Downtime...

For those wondering what happened to the UK servers... The entire server machine crashed. I blame it on the chinese rasin bran that i've been feeding it.

 

[PY004]

Re: RKS Server Downtime...

Texas server has crashed at approximately 11:25PM EST Jan 11,2011. Automatic reboot kicked in at 11:26PM, but failed to revive the server. A manual reboot was initiated at 11:35PM, but the server still did not start up as of 11:40PM. A support ticket was sent into the datacenter to resolve this issue.

Time line
11:25 - Server crash
11:26 - Automatic Reboot attempted
11:27 - First report of crash received via Keldorn
11:28 - Report of crash received via e-mail from the monitoring system
11:35 - Manual reboot attempted
11:43 - Support ticked sent to datacenter
11:44 - Support Dispatch responds... "We are working on resolving your issue and have prioritized your ticket about server down as a Priority 2".
11-48 - Support Dispatch responds... "We have confirmed that the server is not responding to remote access/reboot and are passing it over to the appropriate department to be addressed"
00:04 - Server back online.
00:23 - Datacenter responds... "Server was unable to boot to OS after PXE. Operating System not found. Rebooted server, and checked BIOS to make sure that HDD was being recognized which it was. Rebooted server agian, this time server booted back into OS without incident. Server is back online and is RDC accessible"

 

[PY004]

Re: RKS Server Downtime...

UK Server is Offline for Updates and Maintenance.

Timeline
00:18 - Windows Updates Installed. Rebooting...
00:24 - Reboot complete

 

[OpsMiller]

Re: RKS Server Downtime...

all seriousness aside..
when you came back n found out what he did.. did he troll face u? or ask if umadbro?

 

[Dejiko]

Re: RKS Server Downtime...

its an US central server, who cares? :roll:

Europe servers are the thing tbh

I care, considering I live in the US central zone and the European servers are the worst for me.

 

[Thothie]

Re: RKS Server Downtime...

Buggers seem to be going down a bit more often... I'm gonna start making a log of this, and the maps they come back on.

My Favorites list has become useless for them... Might be the new windows install though - probably gotta delete my blob files.

 

[PY004]

Re: RKS Server Downtime...

I dont know why but this update seems to have made the servers disk intensive. It's using like 70 IO/s (when all servers are active) and sometimes up to 120-150MB/s on the RAID array... Weird thing is it's fine when only one server is on the box.

 

[FER]

Re: RKS Server Downtime...

Have you tough about removing 1 or 2 server if that helps at all?

Also would be nice if you hosted one server for low level players

 

[zeus9860]

Re: RKS Server Downtime...

Have you tough about removing 1 or 2 server if that helps at all?

As long as these aren't mostly european servers...

 

[Thothie]

Re: RKS Server Downtime...

I dont know why but this update seems to have made the servers disk intensive. It's using like 70 IO/s (when all servers are active) and sometimes up to 120-150MB/s on the RAID array... Weird thing is it's fine when only one server is on the box.

This last winderz update, or the MSC update? We do have a buggerup that may cause HD thrashing on the client, but the server's thrashing should actually be down some.

 

[PY004]

Re: RKS Server Downtime...

@ Thothie (Thothie first since he's awesome, the rest of you line up behind him!)
MSC update. It happens for like 5-10 minutes after a map change, but strangely enough, doesnt happen when only one server instance is running on the box. I might look into this issue more to see if there's something weird going on with the box.

@FER
Heavy HDD seems to affect the servers during map changes mostly and do not seem to affect it much once the server's actually up. Given that neither the processor usage nor the memory usage reach more than 50% at any given time (usually it stays much below this) I would consider the server balanced. As for the low level server, is it really necessary? I would imagine that there's more than enough server already for players to simply join an empty one if they really want to be alone.

@zeus9860
Wait until April Fools Day to read this....
All UK Servers will shut down forever on April 4th, 2011.

 

[J-M v2.5.5]

Re: RKS Server Downtime...

What's going on with the RKS servers these days? All of them appear laggier and the time between map changes is easily five times as long as it was before.

 

[Keldorn]

Re: RKS Server Downtime...

It might be because the servers don't restart on their own anymore due to a bug.

I'll start restarting them every now and then when they're empty.

 

[PY004]

Re: RKS Server Downtime...

Can't help it. The new MSC servers are just using that much more resources. The RAID array just can't handle the amount of I/O demanded. I think it has to do something with the server OS and something that was changed in MSC. Short of putting MSC on a ramdisk, there isn't much I can do.

 

[Thothie]

Re: RKS Server Downtime...

Meh, nothing major's changed since DEC2010. Indeed, the monster spawns have been slightly optimized, so I'm thinking it's just the restart bug.

It's a major pain to go in, edit the crashed.cfg, then restart the empty server... I'm tempted to put out a little server-side patch for it. I *think* the SC.DLL I have now is still compatible with FEB2011a. Should really test it a bit - but if so, it should fix the restart problem. The servers should be cleaner then, allowing maps to turn over more quickly. (Plus it really sucks when a server empties out on a gauntlet map, preventing anyone from joining.)

 

[Keldorn]

Re: RKS Server Downtime...

That'd be good.

I've just been restarting/joining empty servers and votemapping edana.

Think people can just votemap out of gauntlet maps even without being able to join legitimately, but they avoid them anyway.

 

[Thothie]

Re: RKS Server Downtime...

Done... Very minor patch linked yonder:
viewtopic.php?f=34&t=1496&p=131071#p131071

Think we'll depend on RKS to actually apply it though... *Might* be possible through the web interface, but tricky and tedious, no doubt.

 

[PY004]

Servers going down today to install Service Pack 1. Check first post on this thread for details (if there is one).

Also take note of the new IP for TX and UK Servers

 

[PY004]

All maintenance work completed.

 

[Snow Wolf]

I don't really know who you are..
but you're p sexy.

 

[Jelly]

Happy birthday, PY004.

 

[J-M v2.5.5]

Yeah congrats

Edit: Argh, mother****ing Steam randomly deleted all my favorites again.
*runs to page 1*

 

[PY004]

Thank you everyone!

 

[PY004]

There will be an approximately 15 minute downtime sometime during March 16th thru the 17th, 2011 at 11:00pm-05:00am EST on the RKS East (VA) Servers. This includes the RKS Master Server, RKS Web server, MSC East servers, and RKS FN VM. This is due to a emergency network maintenance being performed by my server host. The following is the e-mail received by me.

!! THIS IS AN URGENT MESSAGE CONCERNING YOUR SERVER !!
!! PLEASE READ CAREFULLY !!

Dear PY004

server01.rkszone.net
server02.rkszone.net

Please be advised that on March 16th thru the 17th, 2011 at 11:00pm-05:00am EST, we will be conducting network upgrades and maintenance which will impact your server within our IAD Datacenter. This work is part of our planned ServerBeach Network Upgrades that will provide you with a more robust and scalable network and enable us to better server your needs.

This maintenance will last approximately 6 hours, during which 15 minutes of network downtime will occur as the switch your server is located on is upgraded.
This maintenance will ensure that the networking systems are standardized and will allow us to use more supported commands for configuring and investigating issues at the networking level.

If you have multiple services throughout the data center, please keep in mind which services will be impacted for this event. We will be performing the data center upgrades over three days, during which time you would see different groups of services not responding on each day. At the beginning of this message, you will see that we have provided you with the services that will be affected on the dates listed above.

If you have any questions or concerns about this maintenance, please feel free to contact us at 800.741.9939 x1, or through our ticketing system at my.serverbeach.com.

As always, thank you for being a ServerBeach customer!

Sincerely,
Facil Yohannes
Support Manager
ServerBeach | By Geeks For Geeks
PEER 1 Hosting

 

[MrJohnson]

I always wondered who runs and manages all these servers.

A very big "thank you" to you Sir!