increacing random slow page load times

[BigBensDad]

My wife and I run a really great forum that is hosted by Godaddy we have slow page loads at times and they seem to be getting worse. We get a http 400 errors and other times we get just really slow load times. Most times mine are 1.6 seconds but other times it is 200 seconds..we are losing members and godaddy says they see no problem and it must be a scripting problem....my question is what do i do to fix it as simply as possible...and I am sorry if this problem is addressed somewhere else please give me the link if so.

thanks for any help you can give

Richard
mastiffonlinecommunity.net

[Helter]

if your on one of godaddy's $7 plans your on a shared server with hundreds of others accounts and your at the mercy of the other accounts activity. I noticed mostly 1.5 sec page loads there, but when it got bad (18 sec) I noticed the same slow load on the login page as the forums. That tels me that it is probably not your site causing the slow load. You seem to have a pretty busy site. it may be time for a VPS type server. If godady wont take the time to help you solve it, check out http://www.lunarpages.com/solutions.php
Ive heard good things about their Virtual Private Servers.

[.QUACK.Major.Pain]

I tried your site and on the first attempt, it took 1.7s.
On the second attempt, it took 148.8s.
Third was 48.9s.

I did notice 19 users online, but unless they were uploading or downloading all together, it shouldn't be slow.
Like Helter mentioned, shared servers can kill.

You can look in your cPanel and on the left side, click on Server Status if available.
It will show the server load, disk capacity, and other stats.

If like my host, they run their hard drives at 98-99% almost all the time trying to profit to the max.
When my site slows, my server load is only 2-12%, so this would tell me it's not my site, but the other 100-1000 other sites sharing my server.

I may be off on the number of sites. Not sure what range of sites can occupy a server.

If your donations can sustain a VPS, it may be worth it to upgrade.

[BigBensDad]

I tried your site and on the first attempt, it took 1.7s.
On the second attempt, it took 148.8s.
Third was 48.9s.

I did notice 19 users online, but unless they were uploading or downloading all together, it shouldn't be slow.
Like Helter mentioned, shared servers can kill.

You can look in your cPanel and on the left side, click on Server Status if available.
It will show the server load, disk capacity, and other stats.

If like my host, they run their hard drives at 98-99% almost all the time trying to profit to the max.
When my site slows, my server load is only 2-12%, so this would tell me it's not my site, but the other 100-1000 other sites sharing my server.

I may be off on the number of sites. Not sure what range of sites can occupy a server.

If your donations can sustain a VPS, it may be worth it to upgrade.

Thanks for your reply I know your busy, but I do have the $$ plan from godaddy always have but on the slow page loads you will see a %php be low and the other one high does that mean anything...they are telling me it is a scripting problem and I have no clue you all are the only ones that i have access to that could help

[.QUACK.Major.Pain]

[Page generation time: 1.4917s (PHP: 63% | SQL: 37%) | SQL queries: 856 | GZIP enabled | Debug on]

Stats I got today.
I don't know what php or sql percentages are good or bad.
Helter can explain them better when he reads this thread again.

[BigBensDad]

here is a more thought out reply....(my wives <img> )

We are on a Virtual Dedicated Server with GoDaddy… “premium planâ€

[Helter]

what SEO mod are you using?

[BigBensMom]

Hi! I am "the wife" <img>

...I'm not sure what SEO we are using. I don't know for sure that we are using one ?
How would I find it?

We have the IntegraMod 141... I don't think the guy that set it up for us added much to it. But, if you tell me where to look I'll get that info on the SEO.

Thanks!!!

[Helter]

I was just asking because of this statement by your husband
[quote=""BigBensDad""]
“The "SEO" options or scripts that is used by most forums or content management systems is not functioning properly or is using a misconfigured redirect
The script is not properly parsing the URLâ€

[Helter]

I dont think this is causing your trouble, though your board needs the ability to send mail for several of its functions to work properly, such as digests and post notifications.[attachment=0:2pipayjk]mx.jpg[/attachment:2pipayjk]

[BigBensMom]

We continue to have problems with our site.... slow page loads and "page cannot be found" errors.... This is the info that GoDaddy (we have a virtual dedicated pkg with them) gave to us..... Can someone explain what this means and what we can do to fix our problem? (we are limited in our computer skills and our main tech person is no longer helping due to a busy schedule) .... do we need to upgrade to a more recent version of IntegraMod??? or move site to a Dedicated server??? ...or something else??? What will help this problem?

(thanks!!!!)

*****************

Upon reviewing your server, we found that the MySQL process was using 100% of the CPU. During our visits to the forums on mastiffonlinecommunity.net, we saw the CPU usage spike to well over 100% and this did not come back down until we went to the home page of the site or exited the site entirely.

We found the below SQL query being logged to your server's MySQL slow query log several times, often a minute apart during the time that MySQL process was using 100% of the CPU:

SELECT this.topic_id AS thistopic
, next.topic_id AS nexttopic
, prev.topic_id AS prevtopic
FROM mastiffonlinecommunity_topics this
LEFT JOIN mastiffonlinecommunity_topics next
ON ( next.forum_id = this.forum_id
AND next.topic_last_post_id > this.topic_last_post_id )
LEFT JOIN mastiffonlinecommunity_topics prev
ON ( prev.forum_id = this.forum_id
AND prev.topic_last_post_id < this.topic_last_post_id )
WHERE this.topic_id = 6681 <----------- note: this value is not always the same
ORDER BY next.topic_last_post_id ASC
, prev.topic_last_post_id DESC
LIMIT 1;

We see that this query has taken anywhere between 11 seconds to complete, to 86 seconds to finish. This would explain the slowness you have been experiencing.

We are not able to determine which version of phpBB you are using so if you are using a version that is older than the one that is currently available for download, we suggest downloading the current version of the forum software and applying it to your site to see if this helps. You may also want to get in touch with the developers of phpBB to see if any optimization can be done to this query.

[.QUACK.Major.Pain]

Something I did notice, but Helter would have to say if it matters or not, but the stats at the bottom of the page of your site compared to mine are:

Yours: [Page generation time: 1.4033s (PHP: 69% | SQL: 31%) | SQL queries: 903 | GZIP enabled | Debug on]

Mine: [Page generation time: 1.3072s (PHP: 68% | SQL: 32%) | SQL queries: 136 | GZIP enabled | Debug on]

Your forum page loaded fast several times.
The stats compare to be close, other than your SQL queries: 903
I tried several times and your site is always 903-907.
Is this normal to be that high Helter? Is this his issue.

Just a thought or a learning point for us both. LOL

[.QUACK.Major.Pain]

Something else you can check is:
ACP>CrackerTracker>Maintenance & Tests

The Automatic System Tests should be 4 Active and 6 OK
The Security Test section may have some SAFE & CAUTION listed in this area.

Maybe you could post this section or a screenshot of this page.
Might help with finding if there is a problem or if everything is OK.

On mine, the PHP version is out of date, and PHP Global is ON. I can't change these because of shared server.
I did notice that my CrackerTracker is out of date. I guess Helter never updated the master download file.

[Helter]

that is too high. We had another user with the same problem and he elected to go to a dedicated server with a dual core cpu and 4 gigs of ram. This solved his page load issues, but this should definitely NOT be necessary.
If Bens parents are agreeable to this, I will ask one of my integraMOD3 developers to take a look. He has already shaved a ton of redundant and unnecessary queries off of phpBB3/Stargate Portal.
To do this he would need FTP and mysql access to the server. I am currious myself because I spent nearly 6 hours attempting to solve this issue for another user and was relatively unsuccessful.

[.QUACK.Major.Pain]

Glad to hear we found somewhere to start.
It's a learning process for me too, and enjoy learning some of the ins and outs.
I'm extremely basic when it comes to coding, but picking up a little here and there.

Fun stuff <img>

[BigBensDad]

helter we are begging for help and if you or the person you mentioned need access let us know who and when by email <a>richard@reynoldsproperties.net</a>

[Helter]

Ill contact him now and ask him to reply to this thread

[livewirestu]

Hi Guys, I just got Helter's PM.
I got:
[Page generation time: 5.0666s (PHP: 20% | SQL: 80%) | SQL queries: 882 | GZIP enabled | Debug on]
on your site which seems like a very excessive query count!
I am not familiar with phpbb2 or IM141 but I can give it a try if you like. If you could please PM me with FTP and phpMyAdmin details so I can take a local copy of your site, hopefully I will be able to debug a little easier.

Cheers,
Stu

[Helter]

thank you Stu. I know that the queries vary depending on what options/functions are in use. They also are higher in php5 than in php4, but a healthy 141 forum in heavy use should still not have much more than 170 querries max and will usually average around 50-90

[BigBensMom]

Thank you very much, Stu! (and Helter!)

Stu - I sent you a PM ...let me know if there's more info needed.

Thanks!!!
Mary

[livewirestu]

Ok, I found something potentially very wasteful...
In page_header.php (which, incedentally is called on EVERY page), there is some code to work out who has visited in the last 24 hours. There is also an option to display who has NOT visited in that time. However, even with that option disabled, a query is still executed on every non-visiting user - I.e., 1 sql query for every user in the database! On Mary's site, that is about 850 members.
I have made a slight alteration so that the non-visitors are not processed if this option is disabled. That reduced the query count by about 700 (will vary on a day to day basis depending on the amount of visitors).
The forum page now loads like a bullet - However, the portal page is still somewhat slow, although I can't seem to reproduce the 60+ secs load times.
So, the fix to reduce that count... (I have already done this on your live site Mary)
includes/page_header.php
Find:

Code: Select all

// ############ Edit below ############
// #
$display_not_day_userlist = 0; // change to 1 here if you also want the list of the users who didn't visit to be displayed
$users_list_delay = 24; // change here to the number of hours wanted for the list
// #
// ############ Edit above ############
$sql = "SELECT user_id, username, user_allow_viewonline, user_level, user_session_time
      FROM ".USERS_TABLE."
      WHERE user_id > 0
      ORDER BY IF(user_level=1,3,user_level) DESC, username ASC";

Replace with:

Code: Select all

// ############ Edit below ############
$display_not_day_userlist = 0; // change to 1 here if you also want the list of the users who didn't visit to be displayed
$users_list_delay = 24; // change here to the number of hours wanted for the list
$time_min = time() - ($users_list_delay * 3600);
// ############ Edit above ############

$sql = "SELECT user_id, username, user_allow_viewonline, user_level, user_session_time
        FROM ".USERS_TABLE."
        WHERE user_id > 0 "
        . ($display_not_day_userlist ? '' : ("AND user_session_time >= " . $time_min)) .
        " ORDER BY IF(user_level=1,3,user_level) DESC, username ASC";

So now I'm off to try to find what is still causing slow-motion on the portal page.
Hopefully be back soon with some good results <img>

Stu

[BigBensMom]

AWESOME!!!!!

I think I've told you "thank you" about a million times --- but get ready to hear it again!
THANK YOU!!!!
thanks sooooooo much, Stu!

Mary

[livewirestu]

You're welcome <img>
I am actually getting sub-second page loads on your live site now! Had a couple around the 0.6 seconds mark on the portal page. So maybe this has fixed it afterall?

I can see you have quite a few images hotlinked from photobucket and a few other sites on your portal page. Depending on the traffic on those sites at any given time, the page load times on your site may be degraded.
For now it looks pretty fast, but lets keep an eye on it for a while.

What sort of load times are everyone else seeing at the moment?

Stu

[BigBensMom]

I can't get the portal to load.... I'm getting the "page cannot be found" error.

[livewirestu]

Really? It's working for me.
Are you using firefox? Occasionally it reports that a page cant be found if you are downloading something or otherwise have a reasonable amount of traffic coming through your port.
Normally a couple of refreshes does the trick.

[BigBensMom]

ok... portal just loaded for me:

[Page generation time: 1.1613s (PHP: 63% | SQL: 37%) | SQL queries: 180 | GZIP enabled | Debug on]

...a random forum page:

[Page generation time: 1.4398s (PHP: 56% | SQL: 44%) | SQL queries: 207 | GZIP enabled | Debug on]


1.03 seconds
0.8 seconds

[.QUACK.Major.Pain]

Good to see your getting some improvement.

My question is, why is this happening to you, when everyone else has the same type of site, and myself, with 2000 members, only average 170 sql queries.

Why is this site acting different to others?

[Helter]

there have been several others with this problem. There are so many different setup combination's for servers and forums, it is difficult to find the common denominator, but this has already helped the one ive been working on for months.
Thx Stu! let me know if you find anything else.

[BigBensMom]

not sure if this is related to the original problem, or the fix... or something new... but our site has been down more than it's been up today <img> I've been off line, but our admins have emailed that the site's been down a lot.... same thing --- random times, the page doesn't load and gives the HTTP 400 Bad Request / Page Cannot Be Found" error.

We've contacted GoDaddy and they said they'd "look into it" and get back with us. :/

I guess it's happening a LOT more.... the problem is very bad right now. I don't know if this has anything to do with sql queries or not. If GoDaddy gives us any info, I'll post it here.

Thanks
Mary

[BigBensDad]

Well just checkin in on things, Stu did you have a chance to look at the error logs? we are still getting 400 and 404 errors and don't get it....Godaddy tells us they have looked into it and we are not having problems....weird it seems so real and the 800 other members are having the same illusion i'm guessing that they are trying to mislead us... <img> but tonight we could not get on the server while we were having "problems" thanks for all the help you guys are great...

[BigBensMom]

We pruned our users... deleted the accounts of anyone that has never posted (244 fit that descripton). And we are working on pruning a few of our fourms that are not that important to the purpose of the site.... so, hoping that will reduce the database some. ?

What we've discovered is that EVERY time the forum has problems the PHP % is very very low --- and the SQL % is very very high. We are looking at the bottom of the page where it gives the "page generation" information....

If the PHP% and SQL% are more equal -- like 50/50 or so... the forum works great.... if the PHP goes to 2% or even 0% --- the SQL is then 98% or 100% ---- then the forum will not load. I do not see any pattern with the SQL Queries... they vary from 200 to 500, but have no pattern... it's the % that seems to be showing the pattern related to when the forum won't load.

Please keep in mind that I am "learning as I go" and while this is probably something very "basic" -- I don't know what it means.

Can anyone explain it? Is there something I can do to make the % stay more "even"?

Thanks!!!! <img>

[BigBensDad]

I just want to thank you guys for your help, I have dealt with some not so helpful folks but you guys.....you fixed it.....and you don't even have mastiffs....but 800 mastiff owners are enjoying pictures tonight and haveing a good ol time...You guys can move this to the solved area...Helter thanks for your help and stu if you guys ever need a house built in missouri.....I'm you man....thanks again...

[Helter]

glad you got it worked out <img>

[DjPorkchop]

I got pruned too Glad you all got it sorted out. You guys have a great site and it would be a shame to see it down all the time. Happy posting to all your mastiff owner members!

[BigBensDad]

I got pruned too Glad you all got it sorted out. You guys have a great site and it would be a shame to see it down all the time. Happy posting to all your mastiff owner members!

thanks goes to you too.....thanks for the help