5 Minutes to Build a Basic Monitoring and Alerting System for New Subdomains

I spent a very long time automating my recon for bug bounties. I collaborated with a couple of friends for about 12 months to build out an automation beast. We had a custom framework, and constant recon scanning with good distribution, at times we scaled up to 100+ servers. We stored data on millions of targets and had Slack notifications for vulnerability detection. It was the third iteration of our automation and we thought it was great. I mean, it was pretty great, and it definitely helped us earn some cash on a few popular bounty programs. I learned a lot while I was coding this but ultimately, looking back I wish I had done things differently. Namely, I wish I had decoupled each component of the automation into small chainable tools instead of building everything as one giant framework. In other words, I wish I had followed the Unix philosophy. What's the Unix philosophy? Wikipedia puts it best: The Unix philosophy emphasizes building simple, short, clear, modular, and extensible code that can be easily maintained and repurposed by developers other than its creators. In this blog I'm going to show you how you can use three basic, free, open source tools to implement continuous monitoring for new subdomains in just five minutes. The three tools are: Anew by tomnomnom. Haktrails by me, you could use any subdomain enumeration tool here, like Subfinder or OWASP-Amass. Notify by ProjectDiscovery. Setting up your environment Let's see what the necessary steps are to build a basic monitoring and alerting system for detecting new subdomains. Picking a VPS This is the kind of automation that you probably want to run 24, 7, 365. The easiest way to do this is to set up a VPS. You can use whatever Linux distribution but I tend to use Ubuntu because that's where I feel at home. You can set up an Ubuntu VPS on any popular hosting provider. Linode and Digital Ocean have them starting at $5 per month. If you're a new AWS customer you could also use a free tier EC2 instance which will be free for the first 12 months. Installing Golang and Tmux Once you've set up your VPS, you'll need to install Golang. If you're using Ubuntu, this should be as simple as: sudo apt update && sudo apt install golang. You probably also want Tmux so that you can leave the automation running without stopping every time your SSH session ends. apt install tmux. Installing the tools Now that you have Golang installed, you can use the built-in Go package manager to install the tools that we'll be using. These commands work at the time that this blog was written, but installation instructions for these tools may change so it's always best to check the installation instructions on the repositories directly. GO111MODULE=on go get -v github.com/projectdiscovery/notify/cmd/notify. go get -u github.com/hakluke/haktrails. go get -u github.com/tomnomnom/anew. Adding GOBIN to your $PATH In order to run these new tools by typing the name of the tool: haktrails Instead of the full path: /go/bin/haktrails. We need to modify our bash $PATH to include ~/go/bin/. We can do this by adding the following line to our ~/.bashrc file : export PATH="$PATH:~/go/bin/". Then restart your terminal, or run: source ~/.bashrc. Setting up your config files To use haktrails, you'll need to set up your config file containing your Securitytrails API key. You can find instructions to do that here. To use notify, you'll need to set up your config file with your Discord, Slack, Telegram webhook(s). You can find instructions here. Gathering existing subdomains For the purpose of this blog, let's monitor securitytrails.com. The first step is to gather the existing subdomains for that domain and pop them into a file. You can achieve this by running the following command: echo securitytrails.com , haktrails subdomains , tee subdomains.txt Setting up constant monitoring There are a bunch of ways to do this. My personal preference would be to use hakcron, but you can easily achieve the same thing with s...

Om Podcasten

Listen to all the articles we release on our blog while commuting, while working or in bed.