root@storage:/test# fio --filename=test --sync=1 --rw=randwrite --bs=4k --numjobs=20 --iodepth=4 --group_reporting --name=test --filesize=1G --runtime=300 && rm test
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=4
...
fio-3.1
Starting 20 processes
Jobs: 20 (f=20): [w(20)][98.3%][r=0KiB/s,w=80KiB/s][r=0,w=20 IOPS][eta 00m:05s]
Jobs: 20 (f=20): [w(20)][100.0%][r=0KiB/s,w=160KiB/s][r=0,w=40 IOPS][eta 00m:00s]
test: (groupid=0, jobs=20): err= 0: pid=17550: Mon Sep 21 10:38:16 2020
write: IOPS=265, BW=1062KiB/s (1088kB/s)(311MiB/300080msec)
clat (msec): min=12, max=1981, avg=75.30, stdev=205.76
lat (msec): min=12, max=1981, avg=75.30, stdev=205.76
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 15], 10.00th=[ 15], 20.00th=[ 15],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 23], 60.00th=[ 29],
| 70.00th=[ 31], 80.00th=[ 42], 90.00th=[ 65], 95.00th=[ 435],
| 99.00th=[ 1083], 99.50th=[ 1250], 99.90th=[ 1687], 99.95th=[ 1737],
| 99.99th=[ 1989]
bw ( KiB/s): min= 7, max= 281, per=6.83%, avg=72.53, stdev=78.85, samples=8785
iops : min= 1, max= 70, avg=18.11, stdev=19.71, samples=8785
lat (msec) : 20=49.09%, 50=35.15%, 100=9.09%, 250=0.20%, 500=2.36%
lat (msec) : 750=0.50%, 1000=2.06%, 2000=1.56%
cpu : usr=0.03%, sys=0.28%, ctx=166562, majf=0, minf=187
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,79687,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
Run status group 0 (all jobs):
WRITE: bw=1062KiB/s (1088kB/s), 1062KiB/s-1062KiB/s (1088kB/s-1088kB/s), io=311MiB (326MB), run=300080-300080msec