17
Apr 19

How Not to Acknowledge a Data Breach

I’m not a huge fan of stories about stories, or those that explore the ins and outs of reporting a breach. But occasionally I feel obligated to publish such accounts when companies respond to a breach report in such a way that it’s crystal clear they wouldn’t know what to do with a data breach if it bit them in the nose, let alone festered unmolested in some dark corner of their operations.

And yet, here I am again writing the second story this week about a possibly serious security breach at an Indian company that provides IT support and outsourcing for a ridiculous number of major U.S. corporations (spoiler alert: the second half of this story actually contains quite a bit of news about the breach investigation).

On Monday, KrebsOnSecurity broke the news that multiple sources were reporting a cybersecurity breach at Wipro, the third-largest IT services provider in India and a major trusted vendor of IT outsourcing for U.S. companies. The story cited reports from multiple anonymous sources who said Wipro’s trusted networks and systems were being used to launch cyberattacks against the company’s customers.

Wipro asked me to give them several days to investigate the request and formulate a public comment. Three days after I reached out, the quote I ultimately got from them didn’t acknowledge any of the concerns raised by my sources. Nor did the statement even acknowledge a security incident.

Six hours after my story ran saying Wipro was in the throes of responding to a breach, the company was quoted in an Indian daily newspaper acknowledging a phishing incident. The company’s statement claimed its sophisticated systems detected the breach internally and identified the affected employees, and that it had hired an outside digital forensics firm to investigate further.

Less than 24 hours after my story ran, Wipro executives were asked on a quarterly investor conference call to respond to my reporting. Wipro Chief Operating Officer Bhanu Ballapuram told investors that many of the details in my story were in error, and implied that the breach was limited to a few employees who got phished. The matter was characterized as handled, and other journalists on the call moved on to different topics.

At this point, I added a question to the queue on the earnings conference call and was afforded the opportunity to ask Wipro’s executives what portion(s) of my story was inaccurate. A Wipro executive then proceeded to read bits of a written statement about their response to the incident, and the company’s chief operating officer agreed to have a one-on-one call with KrebsOnSecurity to address the stated grievances about my story. Security reporter Graham Cluley was kind enough to record that bit of the call and post it on Twitter.

In the follow-up call with Wipro, Ballapuram took issue with my characterization that the breach had lasted “months,” saying it had only been a matter of weeks since employees at the company had been successfully phished by the attackers. I then asked when the company believed the phishing attacks began, and Ballapuram said he could not confirm the approximate start date of the attacks beyond “weeks.”

Ballapuram also claimed that his corporation was hit by a “zero-day” attack. Actual zero-day vulnerabilities involve somewhat infrequent and quite dangerous weaknesses in software and/or hardware that not even the maker of the product in question understands before the vulnerability is discovered and exploited by attackers for private gain.

Because zero-day flaws usually refer to software that is widely in use, it’s generally considered good form if one experiences such an attack to share any available details with the rest of the world about how the attack appears to work — in much the same way you might hope a sick patient suffering from some unknown, highly infectious disease might nonetheless choose to help doctors diagnose how the infection could have been caught and spread.

Wipro has so far ignored specific questions about the supposed zero-day, other than to say “based on our interim investigation, we have shared the relevant information of the zero-day with our AV [antivirus] provider and they have released the necessary signatures for us.”

My guess is that what Wipro means by “zero-day” is a malicious email attachment that went undetected by all commercial antivirus tools before it infected Wipro employee systems with malware.

Ballapuram added that Wipro has gathered and disseminated to affected clients a set of “indicators of compromise,” telltale clues about tactics, tools and procedures used by the bad guys that might signify an attempted or successful intrusion.

Hours after that call with Ballapuram, I heard from a major U.S. company that is partnering with Wipro (at least for now). The source said his employer opted to sever all online access to Wipro employees within days of discovering that these Wipro accounts were being used to target his company’s operations.

The source said the indicators of compromise that Wipro shared with its customers came from a Wipro customer who was targeted by the attackers, but that Wipro was sending those indicators to customers as if they were something Wipro’s security team had put together on its own.

So let’s recap Wipro’s public response so far:

-Ignore reporter’s questions for days and then pick nits in his story during a public investor conference call.
-Question the stated timing of breach, but refuse to provide an alternative timeline.
-Downplay the severity of the incident and characterize it as handled, even when they’ve only just hired an outside forensics firm.
-Say the intruders deployed a “zero-day attack,” and then refuse to discuss details of said zero-day.
-Claim the IoCs you’re sharing with affected clients were discovered by you when they weren’t.

WHAT DID THE ATTACKERS DO?

The criminals responsible for breaching Wipro appear to be after anything they can turn into cash fairly quickly. A source I spoke with at a large retailer and Wipro customer said the crooks who broke into Wipro used their access to perpetrate gift card fraud at the retailer’s stores.

I suppose that’s something of a silver lining for Wipro at least, if not also its customers: An intruder that was more focused on extracting intellectual property or other more strategic assets from Wipro’s customers probably could have gone undetected for a much longer period.

A source close to the investigation who asked not to be identified because he was not authorized to speak to the news media said the company hired by Wipro to investigate the breach dated the first phishing attacks back to March 11, when a single employee was phished.

The source said a subsequent phishing campaign between March 16 and 19 netted 22 additional Wipro employees, and that the vendor investigating the incident has so far discovered more than 100 Wipro endpoints that were seeded with ScreenConnect, a legitimate remote access tool sold by Connectwise.com. Investigators believe the intruders were using the ScreenConnect software on the hacked Wipro systems to connect remotely to Wipro client systems, which were then used to leverage further access into Wipro customer networks.

Additionally, investigators found at least one of the compromised endpoints was attacked with Mimikatz, an open source tool that can dump passwords stored in the temporary memory cache of a Microsoft Windows device.

The source also said the vendor is still discovering newly-hacked systems, suggesting that Wipro’s systems are still compromised, and that additional hacked endpoints may still be undiscovered within Wipro.

Wipro has not yet responded to follow-up requests for comment.

I’m sure there are smart, well-meaning and capable people who care about security and happen to work at Wipro, but I’m not convinced any of those individuals are employed in leadership roles at the company. Perhaps Wipro’s actions in the wake of this incident merely reflect the reality that India currently has no laws requiring data owners or processors to notify individuals in the event of a breach.

Overall, I’m willing to chalk this entire episode up to a complete lack of training in how to deal with the news media, but if I were a customer of Wipro I’d be more than a little concerned about the tone-deaf nature of the company’s response thus far.

As one follower on Twitter remarked, “openness and transparency speaks of integrity and a willingness to learn from mistakes. Doing the exact opposite smacks of something else entirely.”

In the interests of openness, here are some indicators of compromise that Wipro customers are distributing about this incident (I had to get these from one of Wipro’s partners as the company declined to share the IoCs directly with KrebsOnSecurity).

Tags: ,

112 comments

  1. Well, what do they have to fear? Even if they were a U.S. based company like Equifax, no action will be taken by the government besides a few questions from incompetent senators who solely use such a breach to push their own agenda.

    Until companies like Equifax, Target, Linkedin, Facebook and so on will be shut down for good and their executives get life sentences in prison, nothing will change. People register it and sadly move on with their life as if nothing happened.

    Same for cyber criminals – there need to be specialized anti-terrorism organizations that fight and bring to justice any of these criminals. But I assume we are at least 10-20 years away from that.

    Do you know of anybody who stopped buying at Target? or by now, anybody who remembers what happened at Target? I don’t, and I ask at every event that I speak at…

    • Why do you mix in terrorism here? Nothing to do with it, no need for scaremongering.

      • The claim from “sources” and stated in the article is that “state” sponsored actors were involved. That’s why there a reference to cyber-terrorism.
        But honestly. … I don’t think the “state” would be interested in retail vouchers

        • Gift card fraud is quite lucrative. Easily could be black ops funding for a nation state actor.

    • You’re actually saying put a major retailer out of business due to one data breach? So one can only make one mistake in your world? This is absolute lunacy. Habitual offenders should have tougher requirements to meet before being allowed to process customer data. That’s what PCI standards are about and enforcement for that is through the credit card companies that set PCI up years ago.

    • I am laughing my head off at the notion that executives should get prison sentences let along life prison sentences when their companies are victimized by cyber criminals.

      If someone crashes a car into their lobby should they get the death penalty? How about we put them on the rack if they fall victim to a wire fraud email?

      Maybe we should just murder everyone in business school proactively just in case they may some day become a senior executive? Can’t be too careful.

    • They fear of losing business. And we can now see it’s impact on it’s business ranking. It has moved to 4th position, HCL replaced Wipro to be the 3rd largest Indian outsourcing firm.

  2. I used to work as an IT leader for a retailer that used WiPro to build, support, and maintain it’s e-commerce site (as well as most of the rest of it’s IT technology).
    I can tell you that if an attacker has gained access into a WiPro network, they pretty much have the keys to the castle. I regularly had to chastise WiPro employees for storing credentials in text files, checking config files with passwords into source control, storing private certificate files in public folders, storing customer data exports in file shares or sending then in plain text emails, and a whole host of poor practices.
    The amount of information that may have potentially leaked is staggering and any company that used them should be in a mad scramble to not only cut off their access but also to rotate every password and certificate in use. Of course that may not help if a valid cert for your domain was stolen and someone decides to use it to impersonate your site; you know… after logging into your DNS system with a stolen password and creating a new entry pointing to their server.
    But this could be just what the world needs… An exposure of all of the egregious practices that have been used inside of these major corporations for years. The companies might try to point the finger at WiPro, but let’s face it, those companies should have had vaults, policies, and mechanisms in place that prevented that type of information from being readily accessible in the first place. I asked for it… and every time the retailer said it wasn’t a priority and / or they didn’t have the money. Well… you pay for it one way or the other and I think the piper might be getting paid soon.

  3. I’m not condoning the laid-back arrogance that Wipro execs seem to be exhibiting. Way to try and minimise the impact to their earnings. How do you not have your ciso fully prepared and at the ready to answer questions from journalists or the public when you’ve just found out you were in a high-profile cyber attack?

    On the other hand it’s really annoying seeing all the comments blaming the practice of outsourcing to Indian companies. Like all American and European IT service providers are immune to attacks and have great ethical standards. As if none of them have ever been hacked and even if they have, were humble, graceful and quick to respond with honesty and compassion to affected customers.

    Good on Krebs for steering clear of implying any such thing.

  4. Brian, wanted to thank you for putting this across. Undoubtedly that Wipro could have done much better than they did. Their long tenured CISO- who was well known and respected leader in industry left couple of months back and then this report on breach and then media interaction and public statements in such a poor way.

    By looking at Wipro’s statement and other details together, it is very safe to put across that theory of Zero Day would not be entirely true but another version of response would be getting ready for next media interaction.

    Any individual who have worked with Wipro in past and in same Cyber Security space of business would have laughed when they blamed their lack of phishing awareness to a breach caused from a zero-day. Nothing about that story or threat actor seemed highly skilled beyond any common phishing campaign, let alone a targeted zero-day attack. They did not realize that number of ex-Wipro are in industry who know much more to counter this theory. Thankfully, there ethical practitioner but what if even one goes rouge…. And no-wonder if an insider with malicious intent involved in whole; how easy it always is to extract “content of choice” for any personnel; not just person in Cyber of Technology but be it in finance & sales- do they really not know? Theory of ScreenConnect would also not go long…..why a bad actor would use something else when you get handful stuff and ways straight !

    No second thought, there are good. smart and capable people around to deal such issues and situations both but at times did not give due attention to known vulnerable areas that they were always aware of, because of reason best known to them.

    Moreover, If it really was a Zero Day, why not to put it across and let at least your customers know, if not on public forum. If mimikatz where are details of C&C and IOCs- published by Wipro? Any of Wipro’s customer would have not and still cannot (in most of scenario) inspect all communications with Wipro network. They are MSSP and it is one of their responsibility to update their customers; they sell Threat Intelligence- but I doubt if this would have been shared with customers who subscribed such from them.

  5. I see this often and it is managed by the executives in the ‘customer’ portion as they seem to have failed to have engaged with a ‘WiPro’ type of customer. The CIO of the ‘customer’ company has a lot to explain for as well. Equifax CIO defense was that he was ‘removed’ six levels away from patch management.

  6. These phishing attacks scream out to me for one big security measure . . . 2-factor authentication.

    I’m sure that 2FA is not a panacea, but it sure can help. Phish (and catch) all you want. If you don’t have the token, you aren’t getting in.

    I am thankful that my bank has enabled it.

  7. Based on the IOCs shared and the pDNS lookups (passive DNS) : it seems Wipro issint the only one targeted.

    What are your views on this ?

    PS: I have DM’ed the partial pDNS on Twitter

    • I’ll have more in a piece about that today. As much as it pains me to publish something when the Mueller report is coming out, which will suck all of the oxygen out of the media cycle today.

  8. Add to that wipro has a business model that is antithetical to simple concepts like IAM and PAM and activley discourage them where they are the primary access management team.

  9. Hi Brian,

    Let me begin with appreciating your efforts in putting up this story on public forum.

    Also, before i share my views on the entire episode thus far, let me assure you and your followers that inputs given by me are solely my views and opinion and are not by any means influenced by any sponsored/un-sponsored channels, mediums or organisations.

    I completely acknowledge and agree with you that Wipro IT network certainly had open gateways for exploitation, and that, Wipro’s officials could have done a better job in handling the media communication and response to reputed personal like you.

    But, as much as i appreciate yours and your respected followers comments and sentiments, i personally get a feeling that the sentiments, opinions and/or views are very judgmental in nature. “Its easier said than done” is the only phrase that hits my mind.

    Wipro is a billion dollar company standing strong across the globe catering its fortune customers with an army of 160,000 employees armed with varied skillsets best suitable to demand of its customers business. This too with an understanding and acknowledgment that they are not perfect, as one may expect, but they are constantly evolving for the good of their services. And, an individual and an organization cannot evolve,grow or improve if there are no room left for errors. This stands true not just for brand like Wipro but for many million, billion and trillion dollar companies that exists and faced hacks, data breaches etc at one point of time or the other. I, unlike few, do not believe in name shaming, hence will avoid taking names of individuals and/or organisation just to prove my stand. Thus, now if we keep an open mind, let me share my views on this incident.

    Wipro as a Cyber Security & Enterprise Risk service provider is undoubtedly responsible for data security, privacy and reputation safety of its customers, but all of it is subjected to the limitations of the security controls adopted by it. Which means irrespective of the strength or gartner status of the security technologies an organisation may implement, irrespective of the processes and policies put in place for governance of same – i personally do not believe that any professional and ethical security service provider company can ever guarantee 100% security. One may gurantee 100% containtment of security incident, but not occurance of incident itself.

    Unfortunately, it seems the little window that went unnoticed is what has costed Wipro this reputation damage. When we refer to IT Security, Wipro as we know is a Service company and not a product company. Thus, its own network security too is dependent on more or less same technology, process and policies available for any organisation in the market. Thus, if a Security service leader like Wipro engages or leverages an extra layer of expert brains from market and/or its reputed partners, one should not immediately question or doubt Wipro’s own security services capabilities. And, if we do, we should remind ourselves of it being known as one of the leaders in the industry for a reason.

    Sources will always give you unverified or unofficial content, hence termed as “stories”. A story can only be validated and termed “based on true events” once we have facts and data to see for ourselves. Did any of your sources inform you with supporting data the level of investigations completed or in-progress ? Whether what level of investiagion is carried out by Wipro’s own team of experts or third party ? Did any of your sources confirm to you that the IOC or for that reason any findings so far are as a result of investiagtion by Wipro in house experts or third party engagements ? Did your sources confirm you the individual or group and their motive behind this targeted attack on Wipro ? Did your sources confirm you the names or even the geographic location of the Wipro office and/or its customers affected by this attack ? Are your sources aware of the level of damage done post this attack? – I am sure the answer to many of these questions will be grey, or a direct NO.

    I agree that Zero Day exploit and Phishing attack do not fit the equation, but we dont know whether Phishing attack was just one stage of exploitation and Zero day other. For all we know is Wipro is a victim of targeted Phishing attack, and that not all of its 160000 employees are security experts, or understand security the way a tech savy person would, and not everyone is sharp enough to differentiate between a genuine and a suspicious emails.

    Having said that i or even you do not have a confirmation that the investigation is complete, for all that we know it may be far or near to its final stage. Thus, until the sufferer comes out vets out its pain, lets be mature enough not to speculate and wait for official communication and acknowledgement from Wipro based and supported by facts.

    The best we can do at this stage is FIRST : have an open mind that Security breaches can happen to any organisation, of any strength, of varied nature, and to companies based out of any piece of land on the earth. Thus, it does not matter where Wipro’s customer base is, where they outsource their business and where Wipro holds its root from. And, SECOND : share some out of the box suggestions and solutions to safeguard against such attacks and ensure security in connected world.

    Thank You.

    • TLDR

      • The response is based on two stories with ~100 comments, thus is expected to be lengthy 🙂

        Anyways cant expect more when we would slang a sentence into 4 letter word.

    • The nature, length, detail and content of your reply leads me to believe you are affiliated in some way with WiPro, perhaps a member of their PR team?

      • My response is irrespective of my association with Wipro. One of the intent of my response is to make believe that “a story is mere a story unless authenticated and/or supported by facts otherwise”. But unfortunately all we could pick up for believe is my association 😐

    • Wipro has always based it’s reputation on it’s high competency. A level of competency that makes extremely few errors. This incident shows Wipro’s competency to be paper thin.

      It’s true that a lot is still unknown. But what is known is pretty bad.

      • The other point of view may say, its not competency, but ones eager to jump to conclusion & mindset that’s paper thin 🙂

    • Are you WiPro’s PR flack?

      You’re also forcing me to point out that for “And, SECOND : share some out of the box suggestions and solutions to safeguard against such attacks and ensure security in connected world.” that’s specifically one of the things that Wipro didn’t do.
      Mutual aid is part and parcel of the IT/Security load…and Wipro didn’t do their share.

    • i totally subscribed to your submission Manish Gupta. Kreb approach is a bit judgemental.

    • Question, concern, Brian’s follow up blog or comments from people- do not talk more than just acknowledgment in right way and voicing it out in public.
      They were aware of it for a week (As per their own statement- given to ET and IT, ToI)) but they did not acknowledge to Brian.

      It was Zero Day and yet Anti-Virus vendors involved and that Zero Day never came out in market…it is still Zero Day only because Wipro knows this….

      Any company can get breached and it is understood in industry but then it should be voiced out as well and that’s the whole point and article all about !

    • Cybersecurity isn’t easy, but Wipro’s customers and competitors are MUCH better at it. The amount of lying, incompetence, and hubris displayed here is far above the norm. No amount of vague flowery language will gloss this over.

      • Right on, Uncnvinced.

        WiPro and it’s management have demonstrated only that their profit-line and executive bonuses are more important than their customers…

    • Thank you, the length of your reply and the absence of any useful information is truly culturally appropriate and very endearing.

  10. Really great work Brian,

    Safe to assume Wipro is a Forcepoint AV customer as only FP has all the IOC malicious domains flagged.
    Rather scary that 90% of the AV companies out there still haven’t flagged the malicious domains as such. Sophos had a couple, Fortinet and Kaspersky each had 1-2.

    What is also interesting is one group of the domains was purchased at the same time from the same registrar (Namecheap) and one domain was registered to a RUS entity dating back over a year. The rest aren’t visible.

  11. This explains why Wipro hurriedly deployed multi factor authentication on their email system last week. They botched that also, leaving thousands of their employees without email access.

  12. Calling into the meeting. Glorious.

  13. Since Wipro outsources so much to the US, I’m curious if the NSA gets involved and tries to push more regulations and accountability for both the US clients and the outsourcing.

  14. “I’m sure there are smart, well-meaning and capable people who care about security and happen to work at Wipro, but I’m not convinced any of those individuals are employed in leadership roles at the company.”

    Actually, I know you’re completely right that there are leaders within the company that have zero clues with information security. Wipro maintains a cybersecurity consultant group, but they are client facing and not focused on security for Wipro. Wipro has systemic problems with delivery, customer satisfaction, and as a true body shop, they can’t maintain people, especially great people. If there are great leaders, they aren’t afforded the ability to provide true mentorship to their teams because they have to remain billable too. The whole company is centered around growth, revenue generation, and winning business but less on delivery, fulfillment, and client satisfaction. The breach, its impact, and how Wipro leaders responded to the issue is not a surprise in the least.

    Being ethical is not Wipro’s strongest suit regardless if it’s a client, employee within Wipro, or their leadership and they have now shown the world.

    Wipro? It’s a large cruise liner full of absolute clowns, proud of their 8 billion dollar revenue marker, boasting about their abilities, and yet falling flat on their faces when it comes time to fully deliver to a client. Of course, they spit and sputter out their usual garbage about how other firms are no different being Deloitte, E&Y, Accenture, … – but, those firms actually have more decency, commitment, integrity, and follow-through than Wipro ever will.

    The data breach is just the icing, there is plenty more wrong with the company than just what’s happened lately. This is just what is known to the public. Do not fall for Wipro’s smoke and mirror routine. If you’re a US candidate, look elsewhere. There is no future for you there, ever.

  15. The news of this breach (and its poor handling by Wipro) should stand as an example for what NOT to do. Ultimately security vulnerabilities will happen, and it’s how we respond to times of crisis that show our true character. Let this be a wakeup call to Wipro and any firms which outsource IT work to them… if you don’t control your entire IT supply chain – and who does these days – then it pays to know how secure your business partners are, and how they respond to security incidents.

  16. I call Wipro’s responce as “Clueless and Running Scared”

  17. This article and the whole Wipro kerfuffle was discussed at our monthly HIPAA meeting this morning. It was mainly about outsourcing, fourth-party IT vendors, and security assessments.

  18. Find a way to contact me. I have the answers on its true origin. Please