Why Your Website Fails When You Run Traffic To It
December 10, 2022 | By David Selden-Treiman | Filed in: hosting, Website Reliability.The Problem with Hosting & How to Fix It
Issue
If you’re wondering why your site is failing when you run traffic to your site, this is why.
Your issue is likely related to your website’s hosting. If you’re like many of our clients, you’re running a WordPress website through a hosting provider. That provider uses Apache to serve your website to visitors.
If you’re experiencing loading issues when you send traffic to your site, chances are that you’re exceeding the limit your provider has set for you.
Solutions
The two best solutions we’ve found are:
- to change the configuration of your server to improve efficiency and increase limits; or
- to cache results, saving everything so you don’t have to keep generating the same page over and over again.
Need Website Help?
We have a lot of experience working with server configuration and website hosting. If you need any help, please contact us using the form below. Please send us your contact info, what your website is, and what issues you’re experiencing.
Q&A
What setting do I change to increase the Apache connections limit?
To increase the Apache connections limit, you will need to increase 2 sets of settings.
- If you’re using Apache prefork, you’ll need to increase both the MaxRequestWorkers setting and the ServerLimit setting.
- If you’re using “event” or “worker” Apache, you’ll need to increase the MaxRequestWorkers setting and either/or the ServerLimit and the ThreadsPerChild. These last 2 values are multiplied together, e.g. each server has a number of children threads.
I increased the MaxRequestWorkers setting. Why isn’t Apache handling more connections?
There are 2 sets of settings depending on how your Apache version is configured that need to be changed in order to increase the connection limit. See “What setting do I change to increase the Apache connections limit?”.
What other issues could be causing my website’s timeout?
There are a number of other issues that could be causing your website to timeout. In our experience, these are less common than the Apache limit.
- Firstly, your database could be really slow. If your site is running enough database queries or the queries take too long, your site could time out waiting to complete. To resolve this issue, you can either work to optimize your database (usually by adding table indexes), or by using database caching through something like memcached.
- Secondly, there could be network throttling or a network configuration error. Sometimes with lower-quality hosting, we’ve seen some sites be unable to load due to restrictions placed by the hosting provider. Alternatively, we’ve also seen some bad network configurations where the site will simply refuse to load due to some sort of internal network issue. This was most common for us with GoDaddy’s shared hosting.
- Thirdly, the server’s CPU could just be throttling. If your site requires either a lot of database queries, or isn’t large enough to handle the traffic, your web server’s limits could be exceeded.
- Fourthly, your server could be running out of available RAM. This will happen if Apache is misconfigured to allow PHP to use up too much memory, or if there is something else running on your server (like a cryptocurrency miner installed by a hacker).
It’s worth noting though that the CPU/RAM issues are usually due to a configuration error exceeding the limits of your server (perhaps trying to accommodate more traffic than the server is prepared to handle), and the network issues are more related to bad hosting providers.
Full Transcript
Scenario
Imagine this scenario.
You’re running your site and it works great. You’ve done all the work. The design’s done, everything’s configured, and you’re all good to go.
You head on over to your ad provider, and start running the ads. Everything’s fine at first. Then disaster strikes. As visitors start going to your site more and more, your page gets slower… and slower… and slower. Eventually it gets to the point where your site won’t load at all.
So, in a rush to not lose your ad money, you head on over to the ad console and click the stop button.
Why did everything fail?
Start of The Issue
Some of our clients have been through this issue. When you’re just starting out you’ll be going to hosting providers with so-called unlimited bandwidth. Based on all of the research you may have done this sounds like they will allow as many visitors to your site as you could possibly get. Unfortunately that’s not quite how this works.
While the bandwidth itself may be unmetered, what’s more important is how your server is configured.
If you’re receiving a lot of visitors at the same time, you need to not only be concerned with the amount of data you can use, but the number of visitors that your hosting can handle at the same time. A lot of hosting providers don’t mention this since it’s not relevant to the majority of their users.
Technical Background
Here’s where the issue is coming from.
Most of the providers will be using a web server program called Apache. Especially if you’re running WordPress as the back end of your site, this is by far the most popular software for hosting websites. The problem is, that by default, Apache can only handle 150 simultaneous connections.
To be clear this doesn’t mean 150 simultaneous visitors. This means connections, and most browsers will initiate two to three connections at a time.
In general webpages are made up of a bunch of different files. In order to show the page, your server has to provide all of these, not just the website’s main HTML.
So, while an individual HTML page may load in under a second, if all of the images, CSS, Javascript, etc. take 5 seconds to load, this means that two connections will be taken up for the entire 5 seconds.
The end result is that you’re limited to around 50 to 100 simultaneous visitors at any one time.
If your site is really, really fast this may not be an issue. But, if you’ve made it to this video chances are this is already a problem.
Solutions
There are a few direct solutions to this.
Increase Apache Limit
You could just increase the Apache connection limit, but depending on your hosting provider this may or may not even be possible.
If it is possible, then you have to worry about resource usage. The more simultaneous connections your server is configured to handle, the more resources your server will need in order to run all of them.
The big limit here is cost. If you’re solely trying to run your hosting on Apache, your hosting costs may skyrocket with large amounts of traffic. A big store with many products, especially if your site is inefficient, could easily reach the limits of your server. You could end up using a large server just to handle a few hundred visitors at the same time. This is especially true when your site is running WordPress with a plug-in like WooCommerce.
Caching
Another solution is extensive caching.
Decreasing the resource-intensive traffic will get around this issue. If you don’t have to constantly keep generating the same page over and over again, the load on your server will be significantly less.
There are a number of ways to handle this caching. The ideal is when you have a server like Nginx that can handle thousands of simultaneous requests.
Basically, a properly configured Nginx server will save a copy of your site. When a visitor goes to your site, instead of constantly querying your Apache server and database, asking what page to serve, it will simply provide the saved file.
This is how large sites can end up having entire page load times under 2 seconds with thousands of visitors. It’s all about optimizing the resources you have and how you can use them.
The problem is that this kind of setup can be rather complicated to configure yourself. At Potent Pages, we have spent years trying to eliminate all of these sorts of issues. The config files for a server can end up being thousands of lines long and it can become a mess unless you know exactly what you’re doing.
Cost
There are providers that offer similar services. The problem with these is that they tend to cost a pretty penny. Generally you’re looking at around $100-200 per site, minimum.
There are some sub-$100 providers that will offer a similar type of service. The problem with these though is that usually the cost is indirectly based on the number of visitors. So, if you are trying to handle a large quantity of visitors, you’re still going to have to pay for more traffic regardless of what happens. WP Engine is arguably the most popular of these.
If you’re in this position and need help, please feel free to contact us. We are used to helping clients that have gone from hosting to hosting only to find that none of them work very well. By the time people come to us they’re usually in a position where their sites are getting enough traffic that these providers just aren’t going to work.
David Selden-Treiman is Director of Operations and a project manager at Potent Pages. He specializes in custom web crawler development, website optimization, server management, web application development, and custom programming. Working at Potent Pages since 2012 and programming since 2003, David has extensive expertise solving problems using programming for dozens of clients. He also has extensive experience managing and optimizing servers, managing dozens of servers for both Potent Pages and other clients.
Comments are closed here.