Copy

Directory Copy

  • cp:

    $ cp -a fromdir todir

    -a, --archive: same as -dR --preserve=all.

  • cpio:

    $ cd fromdir
    $ find . | cpio -pdumv todir
  • tar:

    $ cd fromdir; tar cf - . | (cd todir; tar xfp -)

    tar with compression and ssh tunnel:

    $ tar cvf - fromdir | gzip -9c | ssh user@host 'cd todir; gzip -cd | tar xpf -'

File Copy

  • dd and aiocp performance compare:

    # time dd if=/dev/zero of=/dev/sda3 bs=1M count=30k oflag=direct
    30720+0 records in
    30720+0 records out
    32212254720 bytes (32 GB) copied, 167.305 seconds, 193 MB/s
    
    real    2m47.320s
    user    0m0.019s
    sys     0m4.176s
    
    # time ./aiocp -s 30G -w -b 1M -n 4 -f DIRECT /dev/sda3
    
    real 1m59.322s
    user 0m0.013s
    sys 0m3.337s

    Why so much faster? Because dd reads 1MB, writes 1MB, waits, reads 1MB, writes 1MB, waits, etc. Even though the read from /dev/zero is basically just a memset in the kernel, it introduces enough latency in the process that the write stream stalls.

None: Copy (last edited 2010-01-27 02:14:17 by ZhigangWang)