Two ways to speed up the copying speed of Linux files

In the copying of Linux files, the copying speed is very slow when there are many files. So, is there any way to speed up the copying of files? The following small series will introduce you to the method of speeding up the copying of Linux files. When

The first method

First of all, whether local or remote, you need to move or copy files and not much more, with the cp command mv command and inefficient You can use the tar tool to package/compress the content to be copied/moved, then copy/move it, and then unpack/unzip it.

In addition, it is also a very important skill, that is, you do not have to copy, unpack/decompress after tar is packaged/compressed, and you can perform package copy/solution by packing/compressing the other side through the pipeline. compression.

For example, the tar command can be combined with the nc command to quickly transfer files and directories between two machines:

B machine:

The code is as follows:

"/p" "p" nc -l 5555 | Tar -C /tmp/test/-xf - "/p" "p"

A machine:

The code is as follows:

"/p" "p" tar Cf - /tmp/test/| Nc B‘IP 5555 "/p" "p"

The above steps copy the contents of the A machine /tmp/test/to the directory corresponding to the B machine, where tar cf - /tmp/test/| Nc B & rsquo;IP 5555 package content side by pipe and nc command to be transferred to the B machine by the corresponding IP address and 5555 port, nc -l 5555 | Tar -C /tmp/test/-xf - Listens to the 555 port of the machine and unpacks the received content to the specified directory (the -C parameter specifies the target directory)

The second method
>

Combine tar with the scp and ssh commands:

After the A machine is packaged, copy it to the B machine and unpack it.

The code is as follows:

"/p "p" tar -cf - /tmp/test |  Ssh B‘IP “cd /tmp; tar -xf -”"/p" "p"

Packing in machine A and copying the packaged file to machine B

The code is as follows:

"/p" "p" tar -cf - /tmp/test |  Ssh B’IP “cd /tmp; cat - 》 test.tar”"/p" "p" tar -cf - /tmp/test |  Scp - "a href=“mailto:B‘USER@B’IP:/tmp”"B‘USER@B’IP:/tmp"/a"/p"p"

The package file of the A machine is copied to the B machine and unpacked.

The code is as follows:

"/p" "p" zcat test.tar |  Ssh B‘IP “cd /tmp; tar -xf -”"/p" "p"

can also be used directly locally:

The code is as follows:

"/p" "p" cd /tmp/test1 "/p" "p" tar -cf -. |  (cd /tmp/test2 ; tar -xvpf -) "/p" "p"

But some people have come to the conclusion that it is faster to use cp directly.

In addition to copying a single file when copying The directory will also be copied, sometimes with the properties of the file/directory. You can use the -R parameter to recursively copy the directory in the cp command, and use the -p parameter to copy the file retention attribute (the default is: mode, ownership, timestamps can also specify the attributes to be reserved by --preserve[=ATTR_LIST] such as: context. Links, xattr, all), use the -d parameter to copy the file to retain the connection. Or simply use the -a parameter (equivalent to using -dR --preserve=all)

If you want to see the progress of copying a large number of small files, you can write a simple little script:

As follows:

"/p" "p" cd /tmp/test "/p" "p" for i in * "/p" "p" do "/p" "p" cp $i target Directory "/p" "p" echo $i is ok. . . "/p" "p" done "/p" "p"

Finally add a trick that is not skill: before using a tool to complete a task, think about whether the tool currently used is the most suitable tool? Is there a better tool or method? If the tool is really suitable for the current task, is there any special technique to improve productivity when using the tool? (Usually, viewing help files can be a surprise.)

The above is the method to speed up the copying speed of the file. If you want to copy more and more files, use the method described above to speed up the copying speed. Try it.