• Welcome to the Two Wheeled Texans community! Feel free to hang out and lurk as long as you like. However, we would like to encourage you to register so that you can join the community and use the numerous features on the site. After registering, don't forget to post up an introduction!

Server Busy Errors

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
Howdy,

:tab We are aware of the "Server Busy" messages and of the site simply not responding at all. We're working on tracking down the cause. We're sorry for the interruption of your addiction. For the latest info on what is happening, find the end of this thread ;-)
 
Last edited:
Joined
Apr 21, 2004
Messages
2,685
Location
Bryan, TX!!
Any word on the cause? I have been getting these for a while. More today. I have wondered if TWT was just too big for the setup you have had going. (All that new stuff still isn't sorted out and it's still fickle.) :doh:
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
:tab Well, there are several issues. Right now, we are sharing the server with ST-Owners.com. Their site is about the same size as TWT. Even with both sites, the hardware should not be having a problem. So we are thinking it has to be some kind of configuration snag. So I have gone to the VB support site and gotten the recommended settings and changes to the server. However, it is beyond my abilities so I will be paying the hosting service techs to do it for me. Also, ST-Owners is likely to be moving off to their own dedicated server soon. This will double my hosting costs, but it means that TWT and my own personal sites will be the only sites on the machine. That gives us a little more flexibility with regards to making changes and upgrading stuff because I don't have to check with anyone regarding timing, costs, etc,... Hopefully, all of the above will be sorted out before the end of this month.

:tab Is there any particular time of day you are getting these messages? I am on most of the day from noon until about 2:00am and I pretty much never see the message my self. However, I do get the occasional email from the site letting me know the server stopped responding for a moment or two. I can also monitor the server load live and every now and then it will just spike for no apparent reason... :ponder: Very annoying. We are still working on it though.
 
Joined
Apr 21, 2004
Messages
2,685
Location
Bryan, TX!!
I got a few just before I posted here. It took about 20 minutes before I could get a page to load.

I have gotten them at various times ranging from 6-7am, 1-4pm, 6-11pm. So pretty much anytime. My visits can be a bit sporadic.
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
Hmmmm, before people go to work, lunch break time, and after they get home. Makes sense to me!;-)
I thought the same thing, but when I monitor the server loads, it will often spike with very few people on the site :shrug: It will spike for a few minutes and then drop back to normal. Like I said, I am on pretty much noon to 2:00am and I cannot recall the last time I saw it :scratch:
 
Joined
Dec 19, 2004
Messages
3,403
Location
Austin Texas
First Name
david
Last Name
day
I'm in a java/apache environment. We see periods where apache consumes more cpu than java apps, which should never/rarely happen, so I bounce apache. I've also seen far too many sql calls being made from an app that appears to have little load, so we've done a lot of work looking at sql and driver updates.

Its not always load related. Interesting little buggers these things are. I assume someone is perusing forums, and its likely someone else is seeing the same thing in their environ. Or it will flush with your next update.
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
Well, once we get the server sharing ironed out, I will be providing my tech guys a list of software updates for Apache, MySQL and some other stuff, in addition to tweaking settings. Hopefully that will take care of it. The hardware is a dual 3G Pentium, 2GB Ram, couple hundred Gig hard drive, so it really should be more than enough for a site the size of TWT.
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
:tab So we got a HUGE load spike just a few minutes ago (just after noon). A normal server load number should be 5 or less. I was monitoring the load and it spiked up to about 275. People start getting the server busy message when the load hits 10... During that spike, the server memory was pegged at 100% usage and the CPU loads were bouncing around from zero to 100%. As the memory usage finally started to come back down, the server load number dropped. This is strange because we have 2GB RAM in the server which all the tech types are telling me should be plenty for the number of people we are getting on the site :scratch: I may try sticking in another GB of RAM just to see if that helps. I'll keep you posted.
 
Joined
Jan 13, 2007
Messages
3,389
Location
Gravel Switch, KY
I thought the same thing, but when I monitor the server loads, it will often spike with very few people on the site :shrug: It will spike for a few minutes and then drop back to normal. Like I said, I am on pretty much noon to 2:00am and I cannot recall the last time I saw it :scratch:
I was wondering, when you're talking about how many people are on, is this how many members are logged in or how many people, total, including guests are viewing the site?
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
I was wondering, when you're talking about how many people are on, is this how many members are logged in or how many people, total, including guests are viewing the site?
It is the number of total users that matters. Also, even if we have hundreds of people on at one time, not all of them are actually doing anything at any given moment. Someone that is just reading a post is not using any server resources. Someone doing a bunch of searches really hammers the server resources.

The interesting thing is that the other night I watched the load spike really high and there were maybe 15 people logged in and maybe 15-20 guests. Yet at other times, we might have over 100 people logged in and over 100 guests and the server load is nice and low. :shrug:
 
Joined
May 28, 2007
Messages
2,776
Location
Midlothian, TX
First Name
Mike
Last Name
Fletcher
Does the database have any scheduled maintenance tasks? index reorgs... cleanups... I don't know which one is there, but from the "sound" of the symptoms, it makes sense that something large is getting kicked off periodically... you've probably already thought of that.
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
Yeah, most of that stuff happens during off hours. The most intensive one starts every night at 4:00am and runs for maybe a few minutes.
 
Joined
Apr 21, 2004
Messages
2,685
Location
Bryan, TX!!
I still see the messages about once or twice a week. I quit posting about it since I know Scott is already on top of it. I look forward to the solution arriving.
 

Tracker

Forum Supporter
Joined
Dec 25, 2005
Messages
11,224
Location
down in a holler, Catawba County, NC
First Name
Gary
:tab So we got a HUGE load spike just a few minutes ago (just after noon). A normal server load number should be 5 or less. I was monitoring the load and it spiked up to about 275. People start getting the server busy message when the load hits 10... During that spike, the server memory was pegged at 100% usage and the CPU loads were bouncing around from zero to 100%. As the memory usage finally started to come back down, the server load number dropped. This is strange because we have 2GB RAM in the server which all the tech types are telling me should be plenty for the number of people we are getting on the site :scratch: I may try sticking in another GB of RAM just to see if that helps. I'll keep you posted.
Can you capture what process is creating the spikes? Sounds like a bad process if it's spiking memory and CPU to 100%. You're on linux/mySQL, right?
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
So I watched from about 4:30pm-5:30pm today and never saw it spike above 5 despite having nearly 200 people on the site during that time.

Gary, I never really see any one process that goes wild. Each user is listed as a process and those will run up to 100% on the CPU sometimes for a brief moment or two, but they seldom go nuts on the memory. I will be continuing to monitor it. The problem is that I am not always sitting at my computer to watch when a spike hits.
 

wczimmerman

Forum Supporter
Joined
Apr 1, 2004
Messages
1,204
Location
Renton, WA
Have you checked your mysql config? At the shop, we've run into problems with mysql where the number of database connections available to the application was too low for the load causing, obviously, failures to connect. The other thing that comes to mind is some sort of table contention with the DB.

From a system standpoint, check the system log (/var/log/messages depending on your Linux distribution) and run the command dmesg to get console messages. These would point out any hardware errors or software crashes that may be happening.
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
:nono:

I am NOT a sysadmin!! I am a user struggling in unfamiliar territory :lol2:

What I have done is get some recommendations from the VB tech heads about appropriate mysql config settings which I am forwarding to the hosting tech heads so they can implement. Also, the will be updating Linux, Apache, MySQL, and a few other software thingies to the latest greatest versions. So hopefully that will clear up the issue.
 

wczimmerman

Forum Supporter
Joined
Apr 1, 2004
Messages
1,204
Location
Renton, WA
:nono:

I am NOT a sysadmin!! I am a user struggling in unfamiliar territory :lol2:

What I have done is get some recommendations from the VB tech heads about appropriate mysql config settings which I am forwarding to the hosting tech heads so they can implement. Also, the will be updating Linux, Apache, MySQL, and a few other software thingies to the latest greatest versions. So hopefully that will clear up the issue.
Sorry. I AM a SysAdmin (Linux/AIX) so sometimes my work face comes out. :giveup:
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
The deal is, I don't have full root access to the server so I am not able to make all the changes anyway. Nor am I even remotely familiar with the environment having spent all my years in DOS/OS2 before finally coming over to Winbloze. I have never had any need/reason to take up Linux prior to doing this site. For the first few years of the site's existence, I had a fantastic tech head that just made everything happen with no fuss ;-) No I am just another customer among many when I call tech support :doh:
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
:tab Okay, so with the help of Wczimmerman, we think it may be a kernel issue with the version of Linux running our server. I am going to check with the techs at the hosting service to see if I can get it upgraded to the latest version. Ideally, this will only require that the site be down for a few minutes while the machine reboots.
 
T

Texfire

Kernel? Does this mean we have too many posts about corn in the ethanol thread? ;)
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
:tab:tab:tab:thumb:
:tab I wouldn't even notice.
:tab It has been locking up more frequently for longer periods of time than a reboot. :mrgreen:
:tab You make an interesting observation that I have been musing over lately. Nothing has really changed in terms of the actual usage of the server or in the software and yet the load spikes seem to be getting more frequent and lasting longer each time! :angryfire: I contacted my tech heads at the hosting service yesterday to see what can be done in terms of getting the server OS updated to the most recent kernel version. Supposedly, this will fix the problem. I am waiting on them to respond to my latest inquiries.
 

Tourmeister

Keeper of the Asylum
Admin
Joined
Feb 28, 2003
Messages
46,175
Location
Huntsville
First Name
Scott
Last Name
Friday
Okay. We now have the latest greatest version of the kernel for the operating system. I will continue to monitor the server loads. Hopefully, this will have resolved the issues. Thanks for your patience!
 
Top