Linux Logs

Linux Log Files and where to find them:

*nix systems store logs in a variety of locations but most of them are aggregated in one central folder:

/var/log

OK but what is *nix?

💡Linux is an open-source operating system that is freely available for anyone, the term *nix usually means unix like operating systems and many fall into that category - e.g. MacOS, unix and linux. They all behave similarly and many have compatible programs (e.g. bash on macos and linux is very similar

GREAT! but umm… is this universal?

Do all unix-like systems store the same log files?

Is the content always the same across distributions (e.g. ubuntu and kali linux)?

NO.

If you compare e.g. Debian and Debian-based ubuntu 22.04, you will see some differences:

But now comes the tricky part…

If you compare ubuntu 22.04 and ubuntu 23.04…

well… They also differ

WHAT THE FUDGE?!

Exactly - Apparently, ubuntu 23.04 added a new file called apport.log (it holds information about program crashes) which ubuntu 22.04 does not have

Huh, interesting.

Well, but it also makes sense - software evolves and changes - just like Operating Systems ;) (they are software)

Ok cool, but which files do I need to know when I want to become a Security/SOC Analyst?

There are at least 5 you should have heard of (probably more)

Wanna look at them?

OK HERE WE GO!

The first candidate is auth.log, located in /var/log/auth.log and it is one of the important logs that every SOC Analyst needs to know

auth.log holds

logins

user changes

group changes

system reboot timestamps

sudo command history

… more

Wanna look at an example?

Here we go! 🚀

Syslog

Next up on our list is syslog

The ultimate log, cut directly from Yggdrasil and forged in the hell fire protected by Kerberos (or something like that 🤣) - the holy grail of logs

The systemlog or syslog is a combination of multiple different log information - e.g. authentication events, kernel events and something called “locales”

a local is basically a configurable channel that you can fill with almost any log of your choice

Have a web server that you want to monitor closely? Put it on a local

Need to know when your Database throws up? Put it on a local

You get the idea 💡

Cool but what does syslog look like?

e.g. like this:

Wonderful, but I want to know where I can look for brute-force attacks.

Smart!

You could find those in two specialized logs - wtmp and btmp

These two are a little special though, they are binary logs.

Binary what?

They hold 0s and 1s? almost. They are in a format that is not plain text but needs a special command to be readable

PFFF I dont believe you.

You dont?

Well.. that makes me sad.

... but luckily, I can prove it!

try opening wtmp with a text editor:

WOAAAAAHHHHH ITS TOOO BRIGHT - MY EYES

HEAAAAAAALP

MAKE IT STOP - ahh ok much better.

How would I open those to be readable in dark mode though?!

use the last and lastb commands:

last reads the wtmp file which holds all the current logon sessions (who is using the computer)

while lastb needs sudo rights to execute and it holds failed logon attempts

check it out:

sudo lastb

oh. It's empty.

ok ok ok - we can easily generate some failed logins - either use hydra for brute-force or just type your password wrong at least 2x

Just like I do every time… … 😅🫠

Can we also see when users logged in the last time?

SURE! That is where lastlog comes in - it's a command that allows you to see who logged into the computer at which time.

if e.g. a hacker would login as root because they escalated their privileges and then uploaded their ssh key into the root .ssh folder

you can see that and start the investigation 🔬👀 🕵️

and then CATCH THE BAD GUYS MUHAHAHAHAH

CLICK CLACK IS THE SOUND OF THE HAND CUFFS (kinda)

The last one for today is called journalctl - it is a good starting point to check if services/software had errors during their installation / runtime

typically you would use

journalctl -xe

to see the end of the file (-e), the most recent part, and more metadata (-x)

Just like MAGIC! 🪄✨

I hope you learned something today - See you in the next module for even more knowledge drops