Tip: Remove Duplicate Lines in File Using Perl

Rai    Linux     code, duplicate, file, i/o, Linux, perl, process, syntax, tip

Here is little nifty code I found to snip or remove duplicate lines in a file using Perl. An example of a file that has duplicate lines looks like this:

/home/index.php
/home/links.php
/home/index.php

The result we want to achieve is this:

/home/index.php
/home/links.php

And here goes the code:

$ftmp = 'file.txt';
my %match = ();
{
local @ARGV = ($ftmp);
local $^I = '.tmp';
while(<>){
$match{$_}++;
next if $match{$_} > 1;
print;
}
}

The original file, file.txt, has its duplicate lines removed.

Linux File Directory Structure

Rai    Linux     directory, file, hierarchy, Linux, navigation, structure, windows

When I am introducing Linux to new users like students, I noticed that it takes time for the new user to get used to Linux, particularly the directory structure. Since most are Windows users, they know how the directory structure of Windows looks like, and expects that it will be the same with Linux. To properly illustrate the Linux directory structure, linuxconfig.org has an illustration of the Linux File Hierarchy for easy explanation:

Understanding the directory hierarchy of Linux will result in ease of navigation while running the OS, regardless of the distro (Ubuntu, Red Hat, Fedora, etc.).