Remove duplicate data from file

  

1. Sort the contents of the file first: cat 1.txt | Sort > 2.txt Save to 2.txt

2. Use vim to open 2.txt to execute the replacement command:

Replace spaces: %s///g

Replace tab: %s/^I//g (Note: ^I is the tab key that directly presses the keyboard)

Replace $: %s/^M//g (Note: ^M is pressed Ctrl+v then press Enter)

3.cat 2.txt |  Uniq > 3.txt

Copyright © Windows knowledge All Rights Reserved