DirBuster is a well-known scanner that comes in handy whenever you need certain information about a webpage whether it is up or not. Technically every http request occures between client-side and server-side based on http status code. First, client sends a get request then server receives the request and answers the client. When the requested page is available we do not normally see an error in the browser. Code 200 means that everything went well between client and server side without any mishap. Another example would be one of the ones we all know which is 404 (not found) error. This code indicates that there is no page found named whatever you requested. You can find further information about these codes on wikipedia.
Let's get back to DirBuster. This tool was written in Java by James Fisher and is not currently being developed anymore unfortunately. It has a GUI and easy-to-use interface. As to how DirBuster works, it first asks the website if the website has a file or directory with the name asked. After server receives the request it responds to the client with an http code. DirBuster finally determines based on the given http code by the server whether such directory or file exists or not. It steadily asks every words one by one that you predefine in a wordlist. The bigger your wordlist is, the more accurate results you have. You can either use one of the built-in wordlists that is in DirBuster's directory (/usr/share/dirbuster/wordlist) or find a more extensive one on the Internet. By the way, it would be pretty time-consuming depending on the size of wordlist to fully process, considering you run DirBuster with recursive mode.
DirBuster welcomes you with a main interface which should be divided into four section. At the top we have URL section where we enter the target site. It can crawl on both http (port 80) and https (port 443). Second section is where we customize work method and threads setting. Auto switch is preset to turn on and I think it should be applicable in most cases. You could go ahead and find more information about the difference between GET and HEAD methods in the link here. It also lets you adjust thread number. You can increase the number of threads way up to 500. However this might cause the server a bit of workload. Third section is about wordlist. We have two options here. Pure Brute Force method basically lets you create your own subdirectories. I didn't personally try out Pure Brute Force. To me It doesn't seem to be efficient either. Second scanning type is through wordlists. You should have a look at those by clicking List Info button. And finally in the last part you see some options ticked off by default. When you need to scan both directories and files at the same time, you should pick both of them. 'Be recursive' option as the name of it says itself is a method which it runs recursively for every directory. So it scans every subdirectories one by one. There is also a convenience to directly dig into particular subdirectories without wasting any time. Just stop the scan, get back to the main menu and put the name of subdirectory you want to dig into where it says 'Dir to start with' then start the scan again.
I scanned a website that I randomly picked for this tutorial. I set thread number 71 and picked medium size wordlist. As you can see it found some subdirectories and pages in the main directory. I secondly focused solely on the directory named '2005' and it shows every file and page it found with the response code for each one in a table. Meanwhile, it is also possible to run your scans through proxy and use your credentials for a specific target website. You can find this two options in the Advanced Options window.
That is pretty much how it maps a whole website. It'd be helpful to find admin login pages and ftp directories as well. Consequently, DirBuster is very practical and gives us a simple idea of what kind of hidden pages and directories our target has.
