Web lists-archives.com

Re: Speed Problem Copying Files




Am 09.05.2019 um 11:51 schrieb Kevin DAGNEAUX:
> Le 09/05/2019 à 11:46, Lothar Schilling a écrit :
>> Am 09.05.2019 um 11:14 schrieb Jonas Smedegaard:
>>> Quoting Lothar Schilling (2019-05-09 10:49:32)
>>>> for years I have used CentOS for our server landscape. Now I decided
>>>> to give Debian a try.
>>> Welcome to Debian!
>>>
>>> I sincerely hope you will appreciate Debian.
>>>
>>>
>>>> I just set up a Stretch 9.8 system supposed to become our main backup
>>>> server. So I set up a backup job wih rsync. But the going is really
>>>> very very slooooow. Trying to figure out what's happening:
>>>>
>>>>    * iperf -c [host] => bandwith almost 1000 Mbit, that's fine.
>>>>    * dd if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct =>
>>>>      10737418240 Bytes (11 GB, 10 GiB) kopiert, 40,4992 s, 265
>>>> MB/s, so
>>>>      that's fine as well
>>>>
>>>> But whenever I try to rsync, cp or scp - whether copy files on the
>>>> local
>>>> hard disk only or over the network - speed goes down to 500 kB/s.
>>>>
>>>> Filesystem is ext4.
>>>>
>>>> I don't have any clue about what's going on. Any kind of help would be
>>>> appreciated, thank you!
>>> Is _only_ transfer speed affected?
>>>
>>> I am no expert in this, but imagine that if you rsync massive amounts
>>> involving hardlinks then memory becomes a problem too.
>>>
>>> Perhaps run atop to monitor bottlenecks live
>>>
>>>
>>>   - Jonas
>>>
>> It is most definitely not a memory or cpu problem. It's a HP Proliant
>> with 32 GB RAM and a Intel Xeon CPU 2.40GHz with 4 cores. Also the
>> problem stays the same if I just copy one large file.
>
> Check if it's related to the disk speed (hdparm and/or iotop).
>
> Kevin
>

hdparm -tT /dev/sda
/dev/sda:
 Timing cached reads:   13348 MB in  2.00 seconds = 6683.42 MB/sec
 Timing buffered disk reads: 1014 MB in  3.00 seconds = 337.72 MB/sec

iotop -o (for rsync and cp)
Total DISK READ :       0.00 B/s | Total DISK WRITE :     476.15 K/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:     487.86 K/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
19531 be/4 root        0.00 B/s  476.15 K/s  0.00 % 99.24 % rsync
--info=progress2 /daten/testfile /daten/testfile2

iotop -o (for dd)
Total DISK READ :       0.00 B/s | Total DISK WRITE :     297.68 M/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:     297.68 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
19557 be/4 root        0.00 B/s  297.68 M/s  0.00 % 99.99 % dd
if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct