July 18, 2021

Basic Process information via /proc

When you need information about a process in Linux, there are tons of command lines tools, like ps, htop, lsof etc. I often do not remember the flags to get the information I want. Luckily, Linux has the /proc file system which gives details on every process. Most Unix-like operation systems have it, but the formats differ. I only take a look at the Linux version here.

proc fs knows tons about running processes
Figure 1. proc fs knows tons about running processes

Process info /proc/{pid}

For every process running there is a /proc/{pid} directory. The directory contains file and sub-directories with all kinds of information about the running process:

roman@roman-mibex /p/20452> ls -l /proc/20452
total 0
-r--r... arch_status
dr-xr... attr
-rw-r... autogroup
-r---... auxv
-r--r... cgroup             --> Often hints to the Systemd service or Docker container,
                                Because both use cgroups to manage processes.
--w--... clear_refs
-r--r... cmdline            --> Command line which started the process.
-rw-r... comm
-rw-r... coredump_filter
-r--r... cpu_resctrl_groups
-r--r... cpuset
lrwxr... cwd -> /proc       --> Current working directory
-r---... environ            --> Environment variables
lrwxr... exe -> /usr/bin/bb --> The executable binary
dr-x-... fd                 --> List of open file decriptors
dr-x-... fdinfo
-rw-r... gid_map
-r---... io                  --> Amount of IO done so far
-r--r... latency
-r--r... limits              --> Current limits on the process
-rw-r... loginuid
dr-x-... map_files           --> Memory mapped files.
-r--r... maps                --> Memory regions allocated.
-rw--... mem
-r--r... mountinfo
-r--r... mounts
-r---... mountstats
dr-xr... net
dr-x-... ns
-r--r... numa_maps
-rw-r... oom_adj
-r--r... oom_score
-rw-r... oom_score_adj
-r---... pagemap
-r---... personality
-rw-r... projid_map
lrwxr... root -> /           --> Root directory of process / chroot
-rw-r... sched
-r--r... schedstat
-r--r... sessionid
-rw-r... setgroups
-r--r... smaps
-r--r... smaps_rollup
-r---... stack
-r--r... stat
-r--r... statm
-r--r... status
-r---... syscall
dr-xr... task                --> List of threads. Each /proc/{pid}/task/{thread-id}
                                 Then has again tons of information.
-rw-r... timens_offsets
-r--r... timers
-rw-r... timerslack_ns
-rw-r... uid_map
-r--r... wchan

Many utilities use the /proc under the hood to collect their information as well. The detailed docs are as in the man pages.

Here are some neat tricks:

Access File via /proc

You can access all files the process has open. This is especially useful when the process is using a deleted file, so you can’t reach the file via the regular file system:

roman@roman-mibex /proc> ls -l /proc/36644/fd
  total 0
  lr-x------ 1 roman roman 64 Jul  4 09:08 0 -> 'pipe:[306334]'
  l-wx------ 1 roman roman 64 Jul  4 09:08 1 -> 'pipe:[306335]'
  l-wx------ 1 roman roman 64 Jul  4 09:08 10 -> '/tmp/already-deleted.txt (deleted)'
  l-wx------ 1 roman roman 64 Jul  4 09:08 2 -> 'pipe:[306336]'
  lr-x------ 1 roman roman 64 Jul  4 09:08 3 -> /usr/lib/jvm/java-15-openjdk/lib/modules
  ....
  roman@roman-mibex /proc> cat /proc/36644/fd/10
  Time is 2021-07-04T07:08:33.207689169Z
  Time is 2021-07-04T07:08:35.233163910Z
  Time is 2021-07-04T07:08:37.234176107Z
  Time is 2021-07-04T07:08:39.234891225Z
  Time is 2021-07-04T07:08:41.235896408Z

Execute The Binary

You can execute the binary of an existing process with the /proc/{pid}/exe. This can be combined with the trick above. For example, if the binary got replaced, you can still run the old version. Well to some extent, I assume it picks up the new version of shared libraries =(

roman@roman-mibex /proc> ls -l /proc/40247/exe
  lrwxrwxrwx 1 roman roman 0 Jul  4 09:21 /proc/40247/exe -> '/tmp/hello (deleted)'
roman@roman-mibex /proc> /tmp/hello --version
  Version 42.2
roman@roman-mibex /proc [SIGINT]> /proc/40247/exe --version
  Version 42.0

Get the environment variables

The /proc/{pid}/environ gives the environment variables separated with a 0 character. That is not usable in a shell. You can split it up with xargs:

roman@roman-mibex /proc> xargs -0 -L1 -a /proc/40247/environ
  DESKTOP_SESSION=plasma
  XCURSOR_SIZE=48
  GDK_DPI_SCALE=0.5
  GTK_MODULES=canberra-gtk-module
  GTK_USE_PORTAL=1
  QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1
  LANG=en_US.UTF-8
  PWD=/tmp
  KONSOLE_DBUS_WINDOW=/Windows/1
  SESSION_MANAGER=local/roman-mibex:@/tmp/.ICE-unix/2007,unix/roman-mibex:/tmp/.ICE-unix/2007
  XDG_SESSION_CLASS=user
  XDG_SESSION_ID=2
  PAM_KWALLET5_LOGIN=/run/user/1000/kwallet5.socket
  XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session2
  XDG_DATA_DIRS=/home/roman/.local/share/flatpak/exports/share:/var/lib/fl
  ...

Self introspection with /proc/self:

There is a special entry /proc/self which is the current process information. Each process sees itself in that part of the file system.

You can combine this with the tricks above. For example, if an API only accepts a file path, but you want to pass in standard in/out, then you can pass a file pointing to standard in/out via /proc/{pid}/fd/{file-descriptor-number} Of course, there are limitations to it. It won’t work if the library tries to do advanced IO or tries memory mapping. Here’s an example in Java:

    public static void main(String[] args) throws Exception {
        // We want to use standard out.
        // So, we /proc/self to reference to the running process
        // And there to /proc/self/fd/1 to reference our stdout file descriptor
        String stdOutViaFile = "/proc/self/fd/1";
        libraryWhichOnlyAcceptFiles(stdOutViaFile);
        System.out.println("Done");
    }

    // A library which insist on files. (AAAARRRG)
    public static void libraryWhichOnlyAcceptFiles(String file) throws Exception {
        RandomAccessFile fs = new RandomAccessFile(file,"rw");
        fs.write("Hi from the library, totally writing to a file ;)\n".getBytes(StandardCharsets.UTF_8));
        fs.close();
    }

    // Output:
    //  Hi from the library, totally writing to a file ;)
    //  Done

There is also a /proc/thread-self which will point to the current threads /proc/{pid}/task/{thread-id} information.

/proc Global Information

Besides the /proc/{pid} there are many top level entries on the /proc directory. It has useful information about the running system. A few examples:

roman@roman-mibex /proc> ls -l /proc/
total 0
... // all the process pid entries
-r-... cmdline      --> Boot parameters,
-r-... cpuinfo      --> Details about the CPU, cores etc.
-r-... vmstat

Summary

The /procs/ file system offers tons of information, even without using any tool. Keep it in the back of your mind, it can become handy when troubleshooting.

Tags: Unix Development