PostgreSQL^® 12 performance on GCP and AWS [Benchmark]

Check out the updated article for 2021: Aiven for PostgreSQL 13 performance on GCP, AWS and Azure

At Aiven, we’ve run different PostgreSQL cloud performance benchmarks since 2016. In this latest post, we’ll consider four different benchmarks that compare write performance for PostgreSQL 12.1 between Amazon Web Services (AWS) and Google Cloud Platform (GCP).

We not only measured the performance in terms of speed and cost variations across the two clouds, but also examined the effects of adding CPUs while adjusting RAM and storage upward to write performance.

We generated load using pgbench, which we ran on another VM in the same cloud. The load generator commands for each plan were as follows:

``$$ pgbench --client=<startup_plan> --jobs=2 --report-latencies --time=3600 --log --aggregate-interval=1 keepalives='1' keepalives_count='12' keepalives_idle='180' keepalives_interval='10' sslmode='require'```

In this case the --client switch was the variable adjusted geometrically to correspond to the startup plan under test. Possible values are 2, 4, 8, 16, for each of the four startup plans, respectively. Additionally:

• The load generator had only 2 cores and 2GB RAM when tests ran Aiven startup-4 or startup-8 plans,
• The load generator had 4 cores and 4GB RAM for the other plans.

As in a production PG setup, the local SSDs running PG used LUKS full-disk encryption with WAL archiving enabled. The PG instances under test were initialized as follows (shown here without connection parameters):

``$$ pgbench --initialize --scale=125 --quiet keepalives='1' keepalives_count='12' keepalives_idle='180' keepalives_interval='10' sslmode='require'```

Here, the --scale parameter was adjusted to accommodate each plan; in this case, the DB size was roughly double the size of RAM, with the possible values of 125, 500, 2000, and 4000.

Thus, our four benchmarks measured and compared performance of PostgreSQL 12.1 running on Amazon Web Services (AWS) and Google Cloud Platform in each of the following four Aiven managed PostgreSQL plan configurations:

To minimize the effects of variable latency on our benchmark, we used these comparable regions for each provider, respectively:

AWS regions:

• eu-west-1
• eu-west-2
• us-east-1
• us-west-1

GCP regions:

• europe-west1
• europe-west2
• us-east4
• us-west1

Cost calculation comparison

Finally, we concluded the benchmark by calculating the average mTPS/$ per PG plan.

Transactions Per Second: Aiven Startup-4 plans

Aiven startup-4 plan tests consisted of the following configurations:

Load generator: 2 cores and 2GB RAM

System under test:

• 1 dedicated VM
• 1 core
• 4 GB RAM
• 80 GB storage

On Aiven for PostgreSQL 12.1 startup-4 plans, the difference was 483 to 413 respectively, with AWS leading by 16.76 percent.

Transactions Per Second: Aiven Startup-8 plans

Aiven startup-8 plan tests consisted of the following configurations:

Load generator:

• one VM deployed in the same region as the DB
• 2 cores
• 2 GB RAM

System under test:

• 1 dedicated VM
• 2 cores
• 8 GB RAM
• 175 GB storage

On this, as on all plans tested subsequently, GCP was the leader: with 940 TPS to AWS’s 645 TPS (a 31.34% difference).

Transactions Per Second: Aiven Startup-16 plans

Aiven startup-16 plan tests consisted of the following configurations:

Load generator:

• one VM deployed on the same region as the DB
• 4 cores
• 4 GB RAM

System under test:

• 1 dedicated VM
• 4 cores
• 15 GB RAM
• 350 GB storage

Here, the performance gap between GCP and AWS widened to 45.72% (GCP’s 1881 vs. AWS’s 1021 TPS).

Transactions Per Second: Aiven Startup-32 plans

Aiven startup-32 plan tests consisted of the following configurations:

Load generator:

• one VM deployed on the same region as the DB
• 4 cores
• 4 GB RAM

System under test:

• 1 dedicated VM
• 8 cores
• 30 GB RAM
• 700 GB storage

Finally, on Aiven’s largest startup plan, the performance gap between the clouds narrowed slightly to 42.73% (with GCP at 3802 and AWS at 2177 TPS).

mTPS/$ average per service version

Performance value was calculated as an average of millions of transactions per second per dollar or (mTPS/$) for each cloud provider. We found that, overall, the performance/cost difference between cloud providers quite substantially favored Google, although as before with the exception only of Startup-4 plans:

Conclusions

When averaging all the data across regions and plans for either cloud provider, overall the better cost performance boils down to better performance generally on GCP.
Google was hands down the better performer, with 1742 TPS vs. AWS’s 1081 TPS (a difference of 61.09%) for all Aiven plans on PG 12.1

Of course, many factors affect PostgreSQL performance, including latency differences between the different regions, which we used but didn’t differentiate for in this benchmark. There is also a difference to consider with variably-sized workloads.

We always recommend that standalone PostgreSQL users identify performance bottlenecks and tune their database configuration, workload or hardware configuration to optimize. However, in our managed PostgreSQL services, PostgreSQL system parameters come preconfigured for typical workloads, and you can try out different hardware configurations by choosing different Aiven for PostgreSQL plans.

We’ll be publishing more benchmarks like these moving forward, so stay tuned. In the meantime, check out our no commitment, 30-day trial, or find out more on the product page!

Many thanks go out to Felipe Morato for his hard work on these benchmarks!