Give us a call: (800) 252-6164

Archives: PHP7

Web Crawler Development Techniques

Web Crawler Development Techniques

July 15, 2019 | By David Selden-Treiman

Looking for some quick code to build your web crawler in PHP? Here's some code we use a lot here at Potent Pages to make our development a lot easier!

How to create a polite PHP web crawler using robot.txt.

Creating a Polite PHP Web Crawler: Checking robots.txt

May 31, 2018 | By David Selden-Treiman

In this tutorial, we create a PHP website spider that uses the robots.txt file to know which pages we're allowed to download. We continue from our previous tutorials to create a robust web spider and expand on it to check for download crawling permissions.

How to create a simple PHP web crawler to download a website

Creating a Simple PHP Web Crawler

May 24, 2018 | By David Selden-Treiman

Looking to download a site or multiple webpages? Interested in examining all of the titles and descriptions for a site? We created a quick tutorial on building a script to do this in PHP. Learn how to download webpages and follow links to download an entire website.

How to Download a Webpage using PHP and cURL

Downloading a Webpage using PHP and cURL

May 24, 2018 | By David Selden-Treiman

Looking to automatically download webpages? Here's how to download a page using PHP and cURL.

How to make your website load in under 2 seconds using Apache, Nginx, Redis, PHP7, MySQL, and WordPress

Load Times Under 2s: MySQL, Apache, PHP7 & Composer

May 6, 2018 | By David Selden-Treiman

Installing MySQL Next we need to install MySQL. This will be our database of choice however you can easily use • Read More »

Scroll To Top