My Community

Before and after block storage

fitkoh

  • Administrator
  • Newbie
  • *****
    • Posts: 66
    • Karma: +1/-0
    • View Profile
Buyvm.net sells kvm vps "slices" that can be upgraded with attached storage "slabs" - advertised as having near local performance.

In this thread, we will see how well advertisements hold up. Here are bencharks from a 1GB slice and 2GB slice, before attached storage is mounted. I will hopefully be able to get the storage properly mounted after work tonight and update this thread.

# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Sat Aug  7 10:41:54 PDT 2021

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 991.1 MiB
Swap       : 1024.0 MiB
Disk       : 18.6 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 124.42 MB/s  (31.1k) | 1.02 GB/s    (15.9k)
Write      | 124.75 MB/s  (31.1k) | 1.02 GB/s    (16.0k)
Total      | 249.18 MB/s  (62.2k) | 2.04 GB/s    (31.9k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 1.32 GB/s     (2.5k) | 1.26 GB/s     (1.2k)
Write      | 1.39 GB/s     (2.7k) | 1.34 GB/s     (1.3k)
Total      | 2.71 GB/s     (5.2k) | 2.60 GB/s     (2.5k)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed     
                |                           |                 |               
Clouvider       | London, UK (10G)          | 1.05 Gbits/sec  | 176 Mbits/sec 
Online.net      | Paris, FR (10G)           | 1.06 Gbits/sec  | 144 Mbits/sec 
WorldStream     | The Netherlands (10G)     | 1.04 Gbits/sec  | 185 Mbits/sec 
Biznet          | Jakarta, Indonesia (1G)   | 83.5 Mbits/sec  | 115 Mbits/sec 
Clouvider       | NYC, NY, US (10G)         | 1.12 Gbits/sec  | 720 Mbits/sec 
Velocity Online | Tallahassee, FL, US (10G) | 970 Mbits/sec   | 226 Mbits/sec 
Clouvider       | Los Angeles, CA, US (10G) | 1.05 Gbits/sec  | 213 Mbits/sec 
Iveloz Telecom  | Sao Paulo, BR (2G)        | 194 Mbits/sec   | 65.0 Mbits/sec

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value                         
                |                               
Single Core     | 686                           
Multi Core      | 703                           
Full Test       | https://browser.geekbench.com/v5/cpu/9210629

# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Sat Aug  7 10:51:01 PDT 2021

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 1.8 GiB
Swap       : 2.0 GiB
Disk       : 37.3 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 192.49 MB/s  (48.1k) | 1.07 GB/s    (16.7k)
Write      | 193.00 MB/s  (48.2k) | 1.07 GB/s    (16.8k)
Total      | 385.50 MB/s  (96.3k) | 2.15 GB/s    (33.6k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 1.19 GB/s     (2.3k) | 1.20 GB/s     (1.1k)
Write      | 1.25 GB/s     (2.4k) | 1.28 GB/s     (1.2k)
Total      | 2.45 GB/s     (4.7k) | 2.49 GB/s     (2.4k)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed     
                |                           |                 |               
Clouvider       | London, UK (10G)          | 1.05 Gbits/sec  | 132 Mbits/sec 
Online.net      | Paris, FR (10G)           | 1.04 Gbits/sec  | 155 Mbits/sec 
WorldStream     | The Netherlands (10G)     | 1.03 Gbits/sec  | 143 Mbits/sec 
Biznet          | Jakarta, Indonesia (1G)   | 129 Mbits/sec   | 101 Mbits/sec 
Clouvider       | NYC, NY, US (10G)         | 1.12 Gbits/sec  | 760 Mbits/sec 
Velocity Online | Tallahassee, FL, US (10G) | 1.09 Gbits/sec  | 224 Mbits/sec 
Clouvider       | Los Angeles, CA, US (10G) | 1.03 Gbits/sec  | 203 Mbits/sec 
Iveloz Telecom  | Sao Paulo, BR (2G)        | 170 Mbits/sec   | 46.2 Mbits/sec

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value                         
                |                               
Single Core     | 735                           
Multi Core      | 730                           
Full Test       | https://browser.geekbench.com/v5/cpu/9210733



fitkoh

  • Administrator
  • Newbie
  • *****
    • Posts: 66
    • Karma: +1/-0
    • View Profile
Here is the next set of benchemarks for after block storage attached.

First, the file server:
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Sun Aug  8 19:41:42 PDT 2021

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 991.1 MiB
Swap       : 1024.0 MiB
Disk       : 270.6 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 259.72 MB/s  (64.9k) | 1.04 GB/s    (16.2k)
Write      | 260.40 MB/s  (65.1k) | 1.04 GB/s    (16.3k)
Total      | 520.12 MB/s (130.0k) | 2.08 GB/s    (32.5k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 1.16 GB/s     (2.2k) | 1.28 GB/s     (1.2k)
Write      | 1.22 GB/s     (2.3k) | 1.36 GB/s     (1.3k)
Total      | 2.38 GB/s     (4.6k) | 2.64 GB/s     (2.5k)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed     
                |                           |                 |               
Clouvider       | London, UK (10G)          | 1.06 Gbits/sec  | 234 Mbits/sec 
Online.net      | Paris, FR (10G)           | 1.05 Gbits/sec  | 136 Mbits/sec 
WorldStream     | The Netherlands (10G)     | 1.01 Gbits/sec  | 215 Mbits/sec 
Biznet          | Jakarta, Indonesia (1G)   | 374 Mbits/sec   | 187 Mbits/sec 
Clouvider       | NYC, NY, US (10G)         | 1.12 Gbits/sec  | 853 Mbits/sec 
Performing IPv4 iperf3 recv test from Velocity Online (Attempt #1 of 5).Velocity Online | Tallahassee, FL, US (10G) | 1.08 Gbits/sec  | 255 Mbits/sec 
Clouvider       | Los Angeles, CA, US (10G) | 1.06 Gbits/sec  | 167 Mbits/sec 
Performing IPv4 iperf3 recv test from Iveloz Telecom (Attempt #1 of 5)..Iveloz Telecom  | Sao Paulo, BR (2G)        | 146 Mbits/sec   | 77.7 Mbits/sec

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value                         
                |                               
Single Core     | 789                           
Multi Core      | 749                           
Full Test       | https://browser.geekbench.com/v5/cpu/9231471


In this test it seems the block storage actually improved disk speed slightly; although I could attribute this to lack of load on the host node, since these benchmarks were run days apart.Block storage is mounted to /home, which is currently empty.

Next, the database server:

# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Sun Aug  8 19:54:13 PDT 2021

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 1.8 GiB
Swap       : 2.0 GiB
Disk       : 289.3 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 182.99 MB/s  (45.7k) | 1.06 GB/s    (16.6k)
Write      | 183.47 MB/s  (45.8k) | 1.07 GB/s    (16.7k)
Total      | 366.47 MB/s  (91.6k) | 2.13 GB/s    (33.4k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 1.27 GB/s     (2.4k) | 1.31 GB/s     (1.2k)
Write      | 1.34 GB/s     (2.6k) | 1.39 GB/s     (1.3k)
Total      | 2.61 GB/s     (5.1k) | 2.70 GB/s     (2.6k)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed     
                |                           |                 |               
Clouvider       | London, UK (10G)          | 1.07 Gbits/sec  | 177 Mbits/sec 
Online.net      | Paris, FR (10G)           | 1.04 Gbits/sec  | 149 Mbits/sec 
WorldStream     | The Netherlands (10G)     | 1.05 Gbits/sec  | 231 Mbits/sec 
Biznet          | Jakarta, Indonesia (1G)   | 413 Mbits/sec   | 141 Mbits/sec 
Clouvider       | NYC, NY, US (10G)         | 1.12 Gbits/sec  | 776 Mbits/sec 
Velocity Online | Tallahassee, FL, US (10G) | 1.04 Gbits/sec  | 269 Mbits/sec 
Clouvider       | Los Angeles, CA, US (10G) | 1.01 Gbits/sec  | 194 Mbits/sec 
Iveloz Telecom  | Sao Paulo, BR (2G)        | 74.7 Mbits/sec  | 71.8 Mbits/sec

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value                         
                |                               
Single Core     | 763                           
Multi Core      | 748                           
Full Test       | https://browser.geekbench.com/v5/cpu/9231581

This test registered just slightly lower on disk i/os; but such a small difference I doubt it'd be very noticeable.

I'm happy to say that fran has delivered exactly what is advertised: attached block storage with near local performance.


fChk

  • Hosted Member
  • Newbie
  • **
    • Posts: 30
    • Karma: +2/-0
    • View Profile
@fitkoh
A couple of questions:
1-> What type of block storage are you adding NVMe SSD, SATA SSD or HDD ?
-> Most probably the answer is HDD given the size of the block.
2-> Where did you run the fio tests in the last couple of benchs: in the same folder as in the first one or somewhere inside the /home folder (which represents the mount point of the new block storage as I gathered from what you said above) ?
-> Most probably the answer is in the same location as in the first (as I assume that the additional block is HDD and testing them will yield more modest results.)

The first benchs make sense to me giving that the second bench has more RAM than the first which helps a lot when doing I/O. Indeed, not quite a lot in random read/write (as compared with the sequential ones) but still it does show up in the benchs.

For the second tests, I would think the conditions aren't the same system load-wise (ie less RAM available for caches etc..)


fitkoh

  • Administrator
  • Newbie
  • *****
    • Posts: 66
    • Karma: +1/-0
    • View Profile
These are great questions fChk.

1. Here's what buyvm says about their block storage slabs:

"Our high performance is possible through the use of multiple cache layers, including large NVMe read and write buffers. All communications are done through a InfiniBand RDMA based Storage Fabric giving near locally attached storage performance."

It's probably HDD at the end, given the price (5$USD / TB); but the performance I think hope is much better.

2. This question really interests me. You're saying the benchmarks will yield different results if I change directories to one underneath the block storage. I had never considered such a thing. Let's test it out!

Tests from server 1:
/home/admin/public_html# curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh | bash
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Thu 12 Aug 2021 06:58:20 AM PDT

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 987.3 MiB
Swap       : 1024.0 MiB
Disk       : 270.6 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 15.79 MB/s    (3.9k) | 33.45 MB/s     (522)
Write      | 15.81 MB/s    (3.9k) | 33.75 MB/s     (527)
Total      | 31.60 MB/s    (7.9k) | 67.21 MB/s    (1.0k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 100.35 MB/s    (196) | 83.58 MB/s      (81)
Write      | 105.68 MB/s    (206) | 89.15 MB/s      (87)
Total      | 206.03 MB/s    (402) | 172.73 MB/s    (168)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed     
                |                           |                 |               
Clouvider       | London, UK (10G)          | busy            | busy           
Online.net      | Paris, FR (10G)           | busy            | busy           
WorldStream     | The Netherlands (10G)     | busy            | busy           
Biznet          | Jakarta, Indonesia (1G)   | busy            | busy           
Clouvider       | NYC, NY, US (10G)         | busy            | busy           
Performing IPv4 iperf3 recv test from Velocity Online (Attempt #1 of 5).Performing IPv4 iperf3 recv test from Velocity Online (Attempt #2 of 5).Performing IPv4 iperf3 recv test from Velocity Online (Attempt #3 of 5).Performing IPv4 iperf3 recv test from Velocity Online (Attempt #4 of 5).Performing IPv4 iperf3 recv test from Velocity Online (Attempt #5 of 5).Velocity Online | Tallahassee, FL, US (10G) | busy            | busy           
Clouvider       | Los Angeles, CA, US (10G) | busy            | busy           
Performing IPv4 iperf3 recv test from Iveloz Telecom (Attempt #1 of 5)..Performing IPv4 iperf3 recv test from Iveloz Telecom (Attempt #2 of 5)..Performing IPv4 iperf3 recv test from Iveloz Telecom (Attempt #3 of 5)..Performing IPv4 iperf3 recv test from Iveloz Telecom (Attempt #4 of 5)..Performing IPv4 iperf3 recv test from Iveloz Telecom (Attempt #5 of 5)..Iveloz Telecom  | Sao Paulo, BR (2G)        | busy            | busy           

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value                         
                |                               
Single Core     | 729                           
Multi Core      | 695                           
Full Test       | https://browser.geekbench.com/v5/cpu/9286799

These results worry me a bit. If I'm interpreting this correctly disk hang is consuming ram and causing problems to networking. And, it turns out you're right. The disks are much slower than when run above the attached storage directory.

Here's the results from the database server:
/var/lib/mysql# curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh | bash
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Thu 12 Aug 2021 07:08:26 AM PDT

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 1.9 GiB
Swap       : 2.0 GiB
Disk       : 289.3 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 16.67 MB/s    (4.1k) | 64.62 MB/s    (1.0k)
Write      | 16.68 MB/s    (4.1k) | 65.06 MB/s    (1.0k)
Total      | 33.36 MB/s    (8.3k) | 129.68 MB/s   (2.0k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 302.29 MB/s    (590) | 335.52 MB/s    (327)
Write      | 318.35 MB/s    (621) | 357.86 MB/s    (349)
Total      | 620.64 MB/s   (1.2k) | 693.38 MB/s    (676)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed     
                |                           |                 |               
Clouvider       | London, UK (10G)          | 1.04 Gbits/sec  | 182 Mbits/sec 
Online.net      | Paris, FR (10G)           | 1.03 Gbits/sec  | 181 Mbits/sec 
WorldStream     | The Netherlands (10G)     | 1.01 Gbits/sec  | 164 Mbits/sec 
Biznet          | Jakarta, Indonesia (1G)   | 322 Mbits/sec   | 93.7 Mbits/sec
Clouvider       | NYC, NY, US (10G)         | 1.12 Gbits/sec  | 593 Mbits/sec 
Velocity Online | Tallahassee, FL, US (10G) | 1.09 Gbits/sec  | 202 Mbits/sec 
Clouvider       | Los Angeles, CA, US (10G) | 1.05 Gbits/sec  | 134 Mbits/sec 
Iveloz Telecom  | Sao Paulo, BR (2G)        | 129 Mbits/sec   | 190 Mbits/sec 

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value                         
                |                               
Single Core     | 785                           
Multi Core      | 788                           
Full Test       | https://browser.geekbench.com/v5/cpu/9286916

These results are much less bothersome. The same properties of a proper demonstration of the attached storage performance are here, but the networking is great. I need to consider some changes to the services. I could either 1. Upgrade the web server to a plan with more ram or 2. Consolidate the services and go for a single server setup with better resources.

Before I decide I want to do some more live testing. I probably won't make a move until I'm nearing my next invoice.
« Last Edit: August 12, 2021, 07:53:37 am by fitkoh »


fChk

  • Hosted Member
  • Newbie
  • **
    • Posts: 30
    • Karma: +2/-0
    • View Profile
2. This question really interests me. You're saying the benchmarks will yield different results if I change directories to one underneath the block storage. I had never considered such a thing. Let's test it out!
Yes indeed!.. The location of the I/O test means setting the target block storage that you're testing (for its iops and bandwidth specs.)

Also If you're just interested in fio tests, better just run yabs with the ig flags:
Code: [Select]
./yabs.sh -ig.. to improve readability of the results and save time.

Tests from server 1:
/home/admin/public_html# curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh | bash
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Thu 12 Aug 2021 06:58:20 AM PDT

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 987.3 MiB
Swap       : 1024.0 MiB
Disk       : 270.6 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 15.79 MB/s    (3.9k) | 33.45 MB/s     (522)
Write      | 15.81 MB/s    (3.9k) | 33.75 MB/s     (527)
Total      | 31.60 MB/s    (7.9k) | 67.21 MB/s    (1.0k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 100.35 MB/s    (196) | 83.58 MB/s      (81)
Write      | 105.68 MB/s    (206) | 89.15 MB/s      (87)
Total      | 206.03 MB/s    (402) | 172.73 MB/s    (168)
......................

These results worry me a bit. If I'm interpreting this correctly disk hang is consuming ram and causing problems to networking. And, it turns out you're right. The disks are much slower than when run above the attached storage directory.
Those are just slightly bumped up HDD performance through the providers networked storage via the advertised InfiniBand RDMA based Storage Fabric.

 If you had more RAM on the system, the speed will improve slightly but not much. The proof?.. see the next test performance: roughly the double of this one (except for 4K where they are roughly the same), given that he has twice as much RAM as the first.

Here's the results from the database server:
/var/lib/mysql# curl -s https://raw.githubusercontent.com/masonr/yet-another-bench-script/master/yabs.sh | bash
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2021-06-05                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Thu 12 Aug 2021 07:08:26 AM PDT

Basic System Information:
---------------------------------
Processor  : AMD Ryzen 9 3900X 12-Core Processor
CPU cores  : 1 @ 3792.872 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 1.9 GiB
Swap       : 2.0 GiB
Disk       : 289.3 GiB

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 16.67 MB/s    (4.1k) | 64.62 MB/s    (1.0k)
Write      | 16.68 MB/s    (4.1k) | 65.06 MB/s    (1.0k)
Total      | 33.36 MB/s    (8.3k) | 129.68 MB/s   (2.0k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 302.29 MB/s    (590) | 335.52 MB/s    (327)
Write      | 318.35 MB/s    (621) | 357.86 MB/s    (349)
Total      | 620.64 MB/s   (1.2k) | 693.38 MB/s    (676)
.....

Before I decide I want to do some more live testing. I probably won't make a move until I'm nearing my next invoice.
It's always a good idea to do a lot of testing before making an 'enlightened' decision..

Final thought: main take aways
The most important thing here I/O-wise is that:
 > the database has its storage in the NVMe block storage part (ie /var/lib/mysql) which will largely improve all the database queries response time.
 > the web server response time will be impacted with the HDD nature of the add-on block storage given that it has to serve contents from the users /home directories.