Give us a call: (800) 252-6164

Installing Selenium & Docker on a Virtual Private Server in 2024

September 21, 2023 | By David Selden-Treiman | Filed in: hosting.

If you’re looking to install Selenium to run either a web crawler or set up a testing environment, this guide’s for you.

How to install Selenium grid and docker-compose on a new Ubuntu virtual private server.

Start By Getting a Virtual Private Server

To begin with, we will need a virtual private server. Personally, I recommend going with either Digital Ocean, or Hetzner (affiliate links). At Potent Pages, we have dozens of VPS’s, and we have had the best experience with these two providers.

To set up your VPS, head on over to your hosting provider. Follow their instructions to add in a new virtual private server.

When you select your operating system, we’re going with Ubuntu for this guide. We won’t need a very big VPS, just large enough to run a web browser for the most part. Think 1 gig of RAM and one CPU core. You can get one that’s larger of course, but this is the minimum.

As far as the other settings go, you will need an IP address that you can connect to. We will be using an IPv4 address, but theoretically you could also use an IPv6 one instead.

Firewall

When it’s set up, you’ll also need to enable some ports on your firewall. Make sure port 22 is enabled for SSH. You’ll also want to enable port 4444 with inbound access permitted on your IP address that you’ll be connecting from.

SSH Into Your VPS and Update the Server

Now that that’s set up, make sure that you can SSH into your machine. The exact instructions may depend on whatever hosting provider you’re using. However, on digital ocean and hetzner, you upload your public SSH key when creating your vps. Then, you connect using the root user at the IP address they give you.

You may be asked if the SSH key is ok the first time you log in. If that’s the case, type yes, and you should be able to log in.

Now that that’s done, you’re going to want to make sure that the machine is up to date. To do this, you’ll want to run:

sudo apt update
sudo apt upgrade -y

This might take a bit of time to complete.

Installing Docker

To install Docker, we’ll want to use the following instructions for setting up docker compose. The official instructions are here: https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository .

I recommend literally copying and pasting these instructions into your terminal (usually it’s CTRL+SHIFT+V for pasting).

To start with, we’ll install a couple packages to download and install the Docker GPG key.

# Add Docker's official GPG key:
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

Next we’ll want to add the Docker repository to our apt sources. This will allow for easy installation and updating:

# Add the repository to Apt sources:
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

Finally, we can install the Docker packages:

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose

You can test the Docker installation by running:

sudo docker run hello-world

Installing Selenium Grid Using Docker Compose

We’re going to install Selenium grid next using Docker compose. You’re going to want to create a new folder for your docker-compose.yml file. I usually put mine in this folder (/opt/selenium). You can create this folder with the following command:

mkdir -p /opt/selenium;

Next, you’ll want to copy and paste the following text into a docker-compose.yml file in the selenium folder. You can change the settings as you’d like. For example, if you want more sessions available, you can change the max sessions setting.

# To execute this docker-compose yml file use `docker-compose -f docker-compose-v2.yml up`
# Add the `-d` flag at the end for detached execution
# To stop the execution, hit Ctrl+C, and then `docker-compose -f docker-compose-v2.yml down`
version: '2'
services:
  chrome:
    image: selenium/node-chrome:4.2.2-20220609
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=1
      - SE_NODE_OVERRIDE_MAX_SESSIONS=true
    ports:
      - "6900:5900"

  edge:
    image: selenium/node-edge:4.2.2-20220609
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=1
      - SE_NODE_OVERRIDE_MAX_SESSIONS=true
    ports:
      - "6901:5900"

  firefox:
    image: selenium/node-firefox:4.2.2-20220609
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=1
      - SE_NODE_OVERRIDE_MAX_SESSIONS=true
    ports:
      - "6902:5900"

  selenium-hub:
    image: selenium/hub:4.2.2-20220609
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

You’ll want to create a file with your favorite terminal-based text editor. I’ll be using nano here. To do this, run:

cd /opt/selenium/ && nano docker-compose.yml;

Copy and paste in the docker-compose file (again, that’s CTRL or CMD + SHIFT + V). 

Next, we can start selenium grid. We can run this command to start the Selenium grid containers.

docker-compose up -d

Running this command will show you the state of your selenium grid containers:

docker-compose ps

Testing

At this point, if you go to [your VPS’s IP address]:4444 in your browser, you should see a screen that looks something like this.

A working Selenium Grid control page.

Complete

At this point, you should be all good to go. You can point your Selenium settings to your new VPS.

David Selden-Treiman, Director of Operations at Potent Pages.

David Selden-Treiman is Director of Operations and a project manager at Potent Pages. He specializes in custom web crawler development, website optimization, server management, web application development, and custom programming. Working at Potent Pages since 2012 and programming since 2003, David has extensive expertise solving problems using programming for dozens of clients. He also has extensive experience managing and optimizing servers, managing dozens of servers for both Potent Pages and other clients.


Tags:

Comments are closed here.

What Is The Best Web Hosting Provider?

Finding the best web hosting provider for your needs is an important step in optimizing your website. There's a lot to consider. Here are our basic recommendations:

Simple Websites

For simple websites, you have a lot of options. Most web hosts will do acceptably for a simple small-business website or blog.

That said, we recommend avoiding website builders so that you maintain control of your website.

VPS Hosting

If you just need a simple VPS, most providers will work well. Different providers have different downtimes, but the big differentiators are cost.

Providers like AWS and Google Cloud tend to be much more expensive than more specialized providers.

We recommend Digital Ocean and Hetzner if you're looking for a good VPS provider at a good price (it's what we use.)

High Performance Hosting

If you're looking for high performance web hosting, you're going to need something more specialized.

You can't just expect a simple cPanel host to give you what you'll need. You need a custom configuration.

Generally, you'll need either a managed host, or you'll need to get your servers configured with custom configurations.

If you're looking for a high performance hosting provider, we offer hosting designed for high-availability and high-traffic.

WordPress Hosting

What WordPress Hosting Should You Get?

There are many considerations when getting a WordPress hosting provider. Focus on the performance needs of your website.

WordPress Hosting Setup

When setting up your WordPress hosting, or switching hosts, there are a number of steps to complete. These include:

WordPress & Security

There are a number of WordPress security threats to contend with. We recommend using a plugin like WordFence to help secure your site.

WordPress Backups

Make sure to also back-up your site. It's absolutely essential, and ideally use an off-site backup provider that's different from your hosting provider.

WordPress Speed Improvements

There are a number of ways to improve the speed of your WordPress site on its hosting.

There are a number of plugins that can help improve your site's speed.

DNS

DNS Records

There are many different types of records, each with their own purpose. These include: SOA, A, TXT, CNAME, PTR (reverse DNS), and more. On some servers, you can also set up wildcard records.

The records you need will depend on what you're doing; WordPress sites require different records than mail servers, for example.

Propagation

The process of your records transmitting to DNS servers around the world is called propagation. It normally takes 48 hours, but you can speed it up a bit with some planning.

Testing

To test your DNS records, there are 2 main tools: dig and nslookup. Each is very helpful in its own specialty.

Reliability & Security

There are a number of ways to improve your DNS reliability and security.

  • Split Horizon allows you to separate networks, either for intranets or for separating by geographic region.
  • GeoDNS allows you to give different records to different locations based on the requesting IP address. This allows you to create your own CDN, speeding up your site.
  • DNS over QUIC speeds up your DNS requests and gives you better DNS security by encrypting your DNS connection.
  • DNSSEC allows you to sign and encrypt your DNS connection, ensuring that nobody is changing your records.
  • DNS over HTTPS allows your visitors to request your DNS records over an encrypted connection.

Internationalized Domains

Internationalized domain names allow character encodings other than Latin characters. They have their own methods for backward compatibility.

Scroll To Top