I’m not sure if it’s a good thing or a bad thing, but one of the personally rewarding aspects of being a security consultant is observing new and unique scenarios that can make for a fun story to reflect on. When most people think about insider threats, they think about the accountant who was manipulated into volunteering sensitive information, or a disgruntled employee who maliciously attacks an environment because they were upset with management. Today, I’m going to share with you one of the most surprising Insider Threat scenarios we ran into, and how our team had to shift our mindset in order to get to the root of the problem.
One of our clients was suffering from an active, ongoing attack against their public-facing web application. The site kept getting knocked offline, so we sent one of our consultants on-site to figure out what was going on. The brief we received explained that an attack of some kind was originating from the public WAN, crashing the application, and would recur intermittently once the server was back up. The client could not figure out what was going wrong. We love a good challenge, so we were eager to take a peek ourselves.
Our consultant was greeted by two of our main points of contact as well as the system admin who was helping us with the investigation. They posted up in the client’s server room, which was a small, 8x15, air-conditioned, noisy room with a server rack in the center dividing the space into two areas. It wasn’t long before we got to see the server knocked offline in real time. Our consultant poked around, captured all the traffic sent to the server, waited for a crash, and then examined traffic to see what had happened. The intermittent crashes didn’t seem to have a pattern or root cause that was easily identifiable. Even so, we were able to isolate a handful of packets that must have been carrying the payload (or at least causing the crash if not intentional). We were also able to recreate the crash by replaying the capture payloads to the server, helping us isolate and confirm the issue even further.
Still, even with the replayed payloads, the crashes were not deterministic. This was frustrating. It makes things much harder to debug. This went on for 2-3 days with our consultant and the sys admin holed up in this server room, digging into code, capturing crash payloads, replaying crash payloads, reviewing log files, intermittent results, and so on. It seemed like nothing they attempted had an identifiable impact on the outcome. None of the variables in play seemed to have anything to do with it. But eventually, our persistence paid off.
Our consultant noticed one unusual pattern. When he would not mention that he was replaying the payload and kept that to himself, there was no crash. He also began verbalizing, “okay, I’m going to re-run the payload” but would secretly not do so, and sure enough…CRASH. He tried this a few more times and was able to verify that whenever the sys admin (who was sitting in the room with him and working alongside him), was on the other side of the server rack and not within eyesight, the server would go down whenever he announced that he was running the payload.
He was debugging the sys admin.
Turns out, the sys admin had been behind it all along. In some twisted approach to maintaining job security, he was causing the server to crash, and thought he could get away with this while our consultant was in the server room with him! Our consultant, normally focused on identifying much more typical vulnerabilities through ethical hacking techniques, now had to find and tell this guy’s boss that he thinks the sys admin is behind the attacks and to demonstrate it to them in secret. But even though this discovery presents an uncomfortable truth, our job is to identify the problem and fortunately our clientwas willing to listen. They watched the secret demonstration without the sys admin knowing, and our consultant successfully showed he was manipulating the engagement.
Sometimes, the call is coming from inside the house. While perhaps this specific scenario isn’t something we run into every day, when we perform a vulnerability analysis, we have to consider what an insider might do with their rights and make sure every environment has the security in place to mitigate these threats. Vigilence and thoroughness are necessary when performing the right kind of vulnerability assessment or penetration test. If you haven’t assessed how an insider threat can impact your organization, speak to our team of ethical hackers today to find out how ISE can help!
