Archives: PHP7
![Web Crawler Development Techniques](https://potentpages.com/wp-content/uploads/2019/07/web-crawler-development-techniques.png)
Web Crawler Development Techniques
July 15, 2019 | By David Selden-TreimanLooking for some quick code to build your web crawler in PHP? Here's some code we use a lot here at Potent Pages to make our development a lot easier!
![How to create a polite PHP web crawler using robot.txt.](https://potentpages.com/wp-content/uploads/2018/05/3.png)
Creating a Polite PHP Web Crawler: Checking robots.txt
May 31, 2018 | By David Selden-TreimanIn this tutorial, we create a PHP website spider that uses the robots.txt file to know which pages we're allowed to download. We continue from our previous tutorials to create a robust web spider and expand on it to check for download crawling permissions.
![How to create a simple PHP web crawler to download a website](https://potentpages.com/wp-content/uploads/2018/05/2.png)
Creating a Simple PHP Web Crawler
May 24, 2018 | By David Selden-TreimanLooking to download a site or multiple webpages? Interested in examining all of the titles and descriptions for a site? We created a quick tutorial on building a script to do this in PHP. Learn how to download webpages and follow links to download an entire website.
![How to Download a Webpage using PHP and cURL](https://potentpages.com/wp-content/uploads/2018/05/1-6.png)
Downloading a Webpage using PHP and cURL
May 24, 2018 | By David Selden-TreimanLooking to automatically download webpages? Here's how to download a page using PHP and cURL.
![How to make your website load in under 2 seconds using Apache, Nginx, Redis, PHP7, MySQL, and WordPress](https://potentpages.com/wp-content/uploads/2018/04/banner.png)
Load Times Under 2s: MySQL, Apache, PHP7 & Composer
May 6, 2018 | By David Selden-TreimanInstalling MySQL Next we need to install MySQL. This will be our database of choice however you can easily use • Read More »