đŸŒżđŸ’» The Packets Garden

My Linux Upskill Challenge - Day 11

đŸŒ± đŸȘŽ ⏱ 7 min read

My Linux Upskill Challenge - Day 11

These are my notes following the Lesson No. 11 from the Linux Upskill Challenge

Table of Contents

Introduction

Following this lesson, I had the opportunity to review the main tools that we can use to find files and content within those files throughout the Linux file system.

This is a very special skill for a system administrator when we are looking for configuration files and settings within those files.

It’s also very useful when we want to explore log files and troubleshoot devices — at the end, on Linux, everything is a file!

locate

  • locate is a tool used to quickly find the paths where a file or a directory is located on Linux.

  • My Ubuntu Linux VM didn’t have ‘locate installed, so I had to install it with:

    sudo apt install locate

  • locate use a prebuilt index or database to quickly find the paths.

  • If you like me, had to install locate before use it, it’s probably that the database was not build already, you need to use this command:

    sudo updatedb

Examples

  • For instance, if you would like to find all the paths where a file called access.log is located, you could use:

    ❯ locate access.log
/var/log/apache2/access.log
/var/log/apache2/access.log.1
/var/log/apache2/access.log.2.gz
/var/log/apache2/access.log.3.gz
/var/log/apache2/access.log.4.gz
/var/log/apache2/other_vhosts_access.log
/var/log/apache2/sample_access.log
/var/log/apache2/sample_access.log.1

  • Let said you want to find where are located all the files *.conf

    locate .conf

find

  • With find you specify the directory where you want to search down, (that means it will begin search in the directory and all of its sub-directories)
  • You also specify “what” to search for using various criterias, such as:
    • file name
    • file size
    • last modification time (mtime)
    • file type (e.g. regular file, directory)
    • permissions
    • and more..

Examples

  • To search recursively inside/varfor any file with name access.log I used:

    find /var -name access.log

  • However I received a lot of Permissions denied warnings in the output, which means that my user does not have permissions to enter and inspect those directories, so I rerun find as root:

    ❯ sudo find /var -name access.log
/var/log/apache2/access.log

  • I received the only actual match found: the standard Apache log file that tracks web access requests.

  • To search for any file under /home that was modified in the last 3 days, I used

    find /home -mtime 3

NOTE

This searchs will take longer than locate did, because they scan thorugh the filsystem directly rather than using an index.

  • You can also filter the results from findusing grep like this:

    find /var -name access.log 2>&1 | grep -vi "Permission denied"

grep -R

  • We can also sue grep to search recursively throught a whole directory structure for a text within any text file.

  • For instance, you know that “PermitRootLogin” is a ssh parameter that should be shomewhere ona config file under /etc, but you canÂŽt recall exactly on which path and wich file. You can use grepto search inside the /etc/ folder, like this:

grep -R -i "PermitRootLogin" /etc/*

  • The -Roption tells grep to search throught the directory recursively.

  • The -ioption makes the sarch case-insensitive

  • This tool is specially usefull for the /etcand /var/log folders because it only works on plain text files.

which

  • Sometimes we are not sure where a command is running from, for instance when you run nano, how do we know where nanobinary is actually located?

  • To see where nano comes from, use:

which nano
  • By default, the system searches for binaries in the directories listed in your PATH environment variable. You can see those directories with:
echo $PATH

Additional Tools: zless and zgrep

  • lessand grep works only for plain text files, however sometimes we might need to search under compressed files, in these cases we can use zlessor zgrep commands.

Examples

  • The apache2 access logs files are stored under /etc/log/apache2/, in Linux, regularly, these files are compressed on daily basis to a .gz format, as we can see on this search:
❯ find /var/log/apache2 -name "access*"
/var/log/apache2/access.log.4.gz
/var/log/apache2/access.log.3.gz
/var/log/apache2/access.log
/var/log/apache2/access.log.1
/var/log/apache2/access.log.2.gz
  • Let said we would like to look inside one of the compressed files, we can use zless
zless /var/log/apache2/access.log2.gz
  • We also can search for an specific text inside a compressed file using zgrep
 zgrep "Chrome" /var/log/apache2/access.log.2.gz

More tools: -exec option in find

  • With -exec we can execute a command for each file in the find results

Basic Syntax

find [path] [conditions] -exec [command] {} \;
  • {} is replaced by each file name in the output of find
  • \; tell findwhere the command ends.

Examples

  • Delete all .tmp files in /tmp
find /tmp -type f -name "*.tmp" -exec rm {} \;

This deletes each .tmp file individually.

  • List files over 10MB
find . -type f -size +10M -exec ls -lh {} \ss;

This lists the size and details of each file found.

  • Change permissions of .sh files
find . -type f -name "*.sh" -exec chmod +x {} \;

Makes all .sh files in current directory executable.

  • Copy files to another directory
find . -type f -name "*.log" -exec cp {} /backup/logs/ \;

  • Bonus: + instead of \;

  • You can use + instead of \; to run the command on multiple files at once, improving the performance:

find . -type f -name "*.log" -exec rm {} +

This deletes in batches instead of one by one.

Finding who is using a file

lsof

  • lsofstands ofr List Open Files

  • With this command we can display all open files and the process that opened them

  • This is a powerful diagnostic tool, given that in Linux, everythin is a file - this includes regular files, directories, sockets, pipes, and even network connections.

  • lsof helps to inspect which processes are using those resources.

Examples

  • Show all open files by all processes
lsof
  • See which process is using a specific file
lsof /var/log/syslog
  • See all open files for a user
lsof -u carlos
  • Check which process is using a TCP port
sudo lsof -i :80
  • List network connections
sudo lsof -i
  • Filter by protocol or port
sudo lsof -i TCP:22

fuser

  • Similarly to lsof , fuser identifies which processes are using a file, a directory, or a network port.
  • It’s a simpler tool than lsof, but very handy for quickly identifying resources usage.
  • It returns the PIDs (Process IDs) of processes using the specified resource.

Basic Examples

  • Show processes using a file or directory
fuser /var/log/syslog

This tells you which processes are currently accessing the file.

  • Show usernames of processes
fuser -u /var/log/syslog

The -u flag also shows the user owning each process.

  • Kill all processes using a file
fuser -k /mnt/usb

Forcefully unmount a busy USB drive, for example. Be careful: this kills processes.

  • See what’s using a TCP or UDP port
sudo fuser -n tcp 80

Shows which PID is using TCP port 80.

  • List all users/processes using a mount point
fuser -m /home

Great for checking what is blocking an unmount.

External Resources

Graph View

Backlinks