Synoptek, a California business that provides cloud hosting and IT management services to more than a thousand customers nationwide, suffered a ransomware attack this week that has disrupted operations for many of its clients, according to sources. The company has reportedly paid a ransom demand in a bid to restore operations as quickly as possible.
Irvine, Calif.-based Synoptek is a managed service provider that maintains a variety of cloud-based services for more than 1,100 customers across a broad spectrum of industries, including state and local governments, financial services, healthcare, manufacturing, media, retail and software. The company has nearly a thousand employees and brought in more than $100 million in revenue in the past year, according to their Web site.
A now-deleted Tweet from Synoptek on Dec. 20 warned against the dangers of phishing-based cyberattacks, less than three days prior to their (apparently phishing-based) Sodinokibi ransomware infestation.
News of the incident first surfaced on Reddit, which lit up on Christmas Eve with posts from people working at companies affected by the outage. The only official statement about any kind of incident came late Friday evening from the company’s Twitter page, which said that on Dec. 23 it experienced a “credential compromise which has been contained,” and that Synoptek “took immediate action and have been working diligently with customers to remediate the situation.”
Synoptek has not yet responded to multiple requests for comment. But two sources who work at the company have now confirmed their employer was hit by Sodinokibi, a potent ransomware strain also known as “rEvil” that encrypts data and demands a cryptocurrency payment in return for a digital key that unlocks access to infected systems. Those sources also say the company paid their extortionists an unverified sum in exchange for decryption keys.
Sources also confirm that both the State of California and the U.S. Department of Homeland Security have been reaching out to state and local entities potentially affected by the attack. One Synoptek customer briefed on the attack who asked to remain anonymous said that once inside Synoptek’s systems, the intruders used a remote management tool to install the ransomware on client systems.
Much like other ransomware gangs operating today, the crooks behind Sodiniokibi seem to focus on targeting IT providers. And it’s not hard to see why: With each passing day of an attack, customers affected by it vent their anger and frustration on social media, which places increased pressure on the provider to simply pay up.
A Sodinokibi attack earlier this month on Colorado-based IT services firm Complete Technology Solutions resulted in ransomware being installed on computers at more than 100 dentistry practices that relied on the company. In August, Wisconsin-based IT provider PerCSoft was hit by Sodinokibi, causing outages for more than 400 clients.
To put added pressure on victims to negotiate payment, the purveyors of Sodinokibi recently stated that they plan to publish data stolen from companies infected with their malware who elect to rebuild their operations instead of paying the ransom.
In addition, the group behind the Maze Ransomware malware strain recently began following through on a similar threat, erecting a site on the public Internet that lists victims by name and includes samples of sensitive documents stolen from victims who have opted not to pay. When the site was first set up on Dec. 14, it listed just eight victims; as of today, there are more than two dozen companies named.
How do those who own businesses protect their data from ransom, Brian? For a long time, I’ve been considering and researching how to start a nonprofit organization which would provide free computers and free technical support to a specific population of individuals in my city. I have considered the option of allowing individuals to self-certify their source of income and just showing their driver’s license or state identification card to the person doing the intake, without ever making physical copies of this information. Because I recognize that the expense, hassle and liability of dealing with clients’ stolen personal data just isn’t worth the headaches.
Backups! Backups! Backups!
Unfortunately with the new strategy of releasing the contents of encrypted data if the victim doesn’t pay, I think the calculus has changed. Even having backups doesn’t completely protect you from harm anymore.
Oh believe me, I am obsessed with backing up data offline. As relatives and friend’s grandmothers pass away, I am regularly getting back external hard drives that I stored under other people’s beds and in the back of their closets from the years when I was co-owner of a used bookstore that eventually went online via Amazon. But if I started this nonprofit and my client’s data was hacked, it wouldn’t matter that I had everything backed up offline – the ransomers are now posting the data online if you refuse to pay their hostage fees. So I am seriously considering that being completely offline with client data is the best option. My target clients would be those who live in public housing, aka housing projects owned by the government, so it wouldn’t be unreasonable to just accept their stated income as fact without ever making copies of their bank statements and government benefit letters – most of these individuals are genuinely low-income.
It’s called “encryption” so it can’t be read even if released.
There is no guaranty that even encrypted data is safe. Most of these attacks are persistent and target credentials for user and service accounts that need unencrypted read-access to the data.
That’s what I’m doing at my company. Windows systems with client and our data have no internet access (blocked with firewall rules at router). It’s a small company though.
For internet access I’m running Linux in Virtualbox with Firefox / Thunderbird and PDF additionally being sandboxed.
Not impossible to get compromised but sets a much higher bar and is not an easy target.
Andre
There are no longer any easy answers. And at this point in time, I find anyone using the ‘Backups!’ mantra to be a prime representative of very shallow thinking.
Any approach needs to answer multiple different questions including what you’re trying to protect against and how recovery works in various different cases. Deciding which of these questions and recovery scenarios are important to you makes the ‘Backups!’ mantra useless.
Beyond what to types of problems to protect against and how to recover, even deciding what data to protect can be a real hassle in terms of investment into means and resources. How people and businesses use their computers is critical to the entire data integrity question.
While I’m retired now, I still do some part time infra-structure support for a few local businesses and family friends. This support includes making sure they can (eventually) recover from malicious activities.
One way to help do this is to separate out the backup procedure from the computer or device being backed up. The theory here is that you’re backing up the user’s/client’s data in-the-clear to a system where there is no easy access by malicious actors other than through the data copy process itself.
I define ‘separating out the backup process’ to mean not using the computer hosting the data to create the back up as would normally be done through a disk copy, tape back up or real time transaction logging. The primary assumption here is there’s no point in this type of data copy if the back up itself is subject to malicious encryption.
Instead, assuming your computer has been infiltrated by malicious actors and they’ve subverted your traditional backup processes but have not yet triggered blocking you from access, you can still make a current copy of the infected computer’s data. You do this while you still have access to the data by sending all this data in-the-clear to another computer.
To make this concept be useful, you need to effectively have both an internally (read: traditional) generated backup and an externally generated (don’t mistake this for cloud based) back up.
The externally generated back up is fundamentally based on sending the data to be backed up in-the-clear on a regular basis to an external host. To do this, I set up these data copies to occur during working hours while users are clearly using their devices so that we guarantee that the data received was accessible at the time it was sent. This is the single, underlying concept behind doing this kind of data preservation.
There are obvious limitations behind this kind of approach. Making copies of data in the clear is pretty obvious on the system it’s implemented. Malicious actors who notice it will obviously try to stop it.
Remember, one of the reasons why malicious actors wait before triggering their ransom demands is to be sure they’ve exceeded your traditional backup retention period and rendered your backups useless.
Interrupting daily in-the-clear data copies means the last previously successful data copy is still good and useful. If you have a matching recovery plan with actual resources allocated to it (hint: many companies still talk ‘disaster recovery plans’ as a good thing but don’t fully invest in them. These same concepts apply to protecting friends’ data too.) reasonable recovery is feasible. How fast you can recover is a function of how much you invest beforehand and the nature of what you’ve decided to originally protect.
In other words, you get notified the same day your daily data copies get interrupted that your systems have been infiltrated subverted. You still have a chance at recovering up to the prior day’s data rather than relying on possibly subverted traditional backups.
Note that I’m trying to clearly differentiate between in-the-clear data copies and traditional backups. While both ultimately consist of copies of data where data integrity is desired, they are not the same thing and they are use to protect against completely unrelated problems.
Traditional backups are for protection against traditional data loss problems such as hardware failures, user mistakes, natural disasters and some malicious actions. As such, traditional backups are well understood and good implementations and associated recovery procedures are available. Because traditional backups are designed to run un-noticed at off hours, once you’ve been infiltrated, it’s easy for third parties to corrupt and subvert them over long periods of time.
In-the-clear data copies are one possible tactic to address the problem of malicious data encryption and system subversion. They are immediately noticeable when subverted and your last successful copy is where you start recovery from.
Like any data protection scheme, you have to decide what you’re willing to protect, what additional resources you need, how you plan to recover, what order recovery takes place, which recovery resources must be kept on-hand, which recovery resources can be put off as possible future emergency purchases and so on and so forth. It’s no longer as easy as walking a friend’s house or small business and backing up to an external drive or memory stick.
One possible approach:
The approach I currently take assumes 1) only user accounts/data needs to be protected, 2) recovery assumes abandoning the user’s current PC and replacing it with a new one, 3) new installs of any needed software and apps, 4) reasonable speedy Internet access.
I know a lot of people will be immediately offended or think the above is really stupid. In today’s environment of web browsers as the primary interface and portable, downloadable apps, backing up the entire computer is no longer critical. This is one of those areas where it pays to abandon the “80s mindset of ‘Backups!'”. Really.
What’s important is maintaining data integrity for actual user data and some recovery plan to regain access to said user data and user processes. In most cases, it’s simply easier and cheaper to abandon infected PCs and cheaply replace them. Such infected PCs can be investigated later at leisure for forensic clues without making people or businesses wait until forensics are complete before they are back up and working.
Also of note: in-the-clear data copies can be retroactively scanned before recovery for the presence of malicious code.
Yes, this philosophy doesn’t apply to specialty software or specialty hardware. Don’t make the exceptions be your rule for how to handle everything. This is a mindset no one needs.
I used to be responsible for a 250 plus PC network. More than 230 of the PCs only needed user accounts backed up. Most of them didn’t even need any backup at all. User data was stored externally on servers (think: email). The servers and PCs with specialty hardware were (and should always be) treated as exceptions requiring localized procedures. Most small business and personal PCs are the same. It really saves time, effort and money to decide beforehand what’s important to protect.
Data integrity only applies to data which cannot be easily re-created. A lot of people forget this and just blindly say “Back up everything!”
Installed operating systems, purchased software, downloaded entertainment media can all be re-installed, re-purchased and re-downloaded. User unique data and user configuration data are what’s important to protect. Web browsers and portable apps makes this even more true today than in the past.
Protecting only user accounts/data keeps the total amount of data to copied to be as small as possible. Sticking to web browser and portable apps keeps the tie-in to specific hardware light. This approach makes it possible to get most people/small businesses back up within a single day using daily in-the-clear data copies.
The resource investment I’ve made includes:
An offsite data copy target accessible through the Internet. Yes, this is a risk. What kind of data copy repository you build and use is up to you. My repository consists of a linux based high density server CEPH cluster. We still are using the ISP provided gateway which I monitor frequently. UPnP is turned off. WPA2 is the only accepted wireless protocol. All unused functions are turned off. To be honest, I’m not happy with the gateway (including firmware updates) being under the control of the ISP, but I’ve chosen to live with it.
For individual PCs which are almost always Windows based be they friends or the small businesses I assist in support, I install cygwin. This is the first step outside of most malware vectors. The point is to have software which is currently not susceptible to most automated malware that’s in the wild. This is not protection against directed, personal attacks. This is simply frustrates most automated attacks.
Installing cygwin allows me to install rsync. This allows me to set up a daily automated data copy from the PC to the secured repository. The communications link is secured through rsync using ssh.
When I set user end on a Windows based PC, there are a number of things I insist on.
1) Users are never permitted to use Internet Explorer or Edge to access the Internet. Ever. No excuses. Chrome, Firefox, Opera – any other alternative but IE or Edge. Of course, Adobe Flash is also not allowed.
2) Users with cloud based applications are required to make local copies of all cloud based data. If the application doesn’t support local data copies automatically, they either cannot use the application or accept they are on their own as far as their data in that application is concerned. This is an important education experience for them. They need to understand they have no control over data they store in the cloud. A lot of users naively believe that they always have access to their data when ever they want where ever they are.
3) Users must install a password manager if they don’t already have one installed. I also watch over their shoulder as I have them learn how to use a password manager by having them go to each online account they have and having them change each password to a randomly generated password. This is another important education experience. The haveibeenpawned web site is a useful assist for this.
4) Users and especially small businesses that make WiFi access available as a courtesy need to understand Internet of Things (IoT) issues including UPnP, IoT firmware updates, gateway logs etc.
My point of the above is that if these users/small businesses want your help in protecting their data, then there is work, time and effort involved. If they can’t accept this, then I know I can’t help them. I warn them and then don’t get involved.
I take this attitude regardless of what kind of relationship I have with them. This last week, my brother manually changed 43 online passwords because he still hasn’t learned how to use a password manager. He’s agreed to install one next week and use it to assign new, randomly generated passwords to all 43 online accounts (again) next week. Finally!!
Lastly, in-the-clear data copies and traditional backups are the very last lines of defense in maintaining data integrity and availability. Protecting against malicious attacks comes first. This is something I try to make clear to each person I assist.
Sticking a memory stick into a USB port or doing a back up to the cloud is based on old ’80s based mindsets just using modern tools. The people you want to help need to understand there is no single solution and protecting / backing up their data requires a certain investment on their part.
By the way, on my internal server I use as the target for external data copies, I’ve set up a CEPH cluster using SE Linux. I point this out because of obverse of protecting only what you need to protect on the user end is that you can’t cut corners in your protected repository. I follow all good security practices (no root login, chrooted users, no turned on but unused services, daily monitoring, intrusion detection, 2FA yada-yada-yada).
Finally, keep in mind this is a retiree’s personal hobby set up. I have no disaster plan for a major disaster like a fire. A flood or power outage I can handle but not a destructive disaster.
In terms of malicious attacks, I am reasonably confident that I’m resilient against most of whatever is in the wild. I don’t have any Internet facing website services. Most of the IoT devices in the house simply aren’t allowed on our internal network. The only exceptions are things like printers with manually assigned IP addresses. All WiFi is required to follow WPA2. No other protocol connection is allowed.
If I were still doing this kind of thing for a real business (more than 5 employees), there would be much more I would be doing including traditional offsite backups of my CEPH cluster.
If you’re going to assist people as a non-profit in their data protection, keep in mind what you can do and what you can’t do. I used to re-purpose PCs coming off of their hardware refresh intervals for my mother’s friends and other less than computer savvy non-profits. I especially set up refurbished PCs for people with vision impairments.
Generally speaking, the first step was to refurbish the old PCs by getting rid of Windows. Most of the target user group only needed web access and light gaming (for their grand kids). Regardless of how one feels about the ‘fix Windows by using Linux’ debate, if it’s an end user (rather than a webserver) linux based computer, even today, there isn’t very much to worry about from automated malicious attacks. You also don’t need to worry about software licensing.
Of course, this also make data integrity, back ups and etc easier.
Good luck with whatever approach you decide on. I applaud your desire to take set up a non-profit approach to performing this kind of support.
I hope my approach and comments help spark ideas for you.
Backups were the solutions 2 years ago. The problem is now the threat actors are staying within your environment for months before they kick off the ransomware. I have a client that was impacted by this action. Their backups are no good as the threat actor also infected their backups.
From what I have read many companies are compromised through fishing or password reuse. one the bad guys gain access to the network they take there time mapping out the network and seeing how many computers they can access so when they strike maximum damage is done.
The best way to avoid trouble is not reusing password, enabling multi-factor and not using the same login to access all your systems.
Make sure employees are aware of what phishing is and ways to spot phishing emails. Also make sure they know who to report possible incidents to.
Another thing if you have the mentality of not IF im gonna get hacked but WHEN will I get hacked. Also having the tools and system in place to detect unusual activity in your system so when you get hacked you are aware of it within days if the initial hack instead of getting that dreaded call from Brian and finding out you got hacked months ago.
1) DO NOT run as administrator unless you are installing software that you received from a trusted source.
2) Turn off macros in MS Office. You should never get a Word document via email that needs macros – never!
3) Disable email and internet for every Administrator account.
4) Use a good web proxy. Use a service if you don’t want to purchase and maintain an on premise device. These will block access to the command and control networks required for malware to work.
5) Do not use any software that requires you “run as administrator” if you already have this type of software, isolate it or find a work-around. Do not let your software vendor tell you otherwise!
6) Patch your systems.
Note that with the exception of the web proxy – these are all free to do!
** Note: running as “Administrator” means any account with administrator privileges – this also includes putting accounts in the “Power Users” group. Any accounts in these groups must be reserved for system administration only.
At my last contract, we had a policy that NO ONE had administrative permissions, and not even the actual administrators were allowed to operate as such. Only the IT staff was able to login to local administrator levels to do serious IT work. I rarely had to do it, as many things are doable without full rights. We were under HIPAA so even the CEO did not argue with us on this policy.
The real problem was users who were “power users” in older operating system environments, who always seemed to think they needed privileges to adjust their environment so suit every whim, and we had a war with them right away. – They lost.
This advice, while concise, has been around for at least 10 years and has gotten a bit worn out, and how’s it working? Not so well but it’s always a good way to thump the clueless users, eh?
I’m now retired after ~24 years in IT starting with lowly Desktop Support and ending as a Level 3 SME with responsibilities in 3 technologies. Throughout it’s been a struggle to get other IT staff onboard with security awareness, until the hammer comes down. Even then, if heads roll, it’s rarely the ones’ who allocate and manage the resources, nor is it ever the vendors or manufacturers selling their shiny wares to them.
So you can recite these tenets (with their subliminal insults to people like my Aunt Judy who gets all her news from FacePlant) till you’re blue in the face but it does little toward securing our networks.
Start with the well-gristled class, the middle managers all to willing to “yup” their way up the ladder. The problem isn’t so much an IT problem as it is a managing problem.
Serious question; if this particular strain of ransomeware has been in the wild for over a month why are the AV are software programs not able to catch and stop it?Is it because the various strains of ransomeware are different enough that you can’t make a signature to stop it? Seriously I just don’t get why this keeps happening.
Many strains from these manually operated attacks can be detected, but the truth of the matter is these attacks are being conducted by hand. Which means the criminals can afford to make changes and test whether they are evading defenses before deployment. In cases like this one, they likely disabled any security products before infecting the victim. This started with stolen credentials of a company entrusted to manage IT on behalf of their clients. If the criminals had control of valid credentials without 2FA, they can just disable all anti-virus or any other security measures before deploying the malware.
Thank you very much Chester. That explains alot. That said,If I follow you correctly most of these ransomware attack’s on high-value targets need to be bespoke projects in order for them to succeed. Is that correct?
Can the average small business or home user reliably rely on their antivirus protection to stop most of the other attacks? Since they are not individually targeted.
Small businesses are still targetted by the likes of Dharma, but these attacks are generally less organised.
Generally speaking, up to date patches and antivirus will help a lot.
I recommend the ACSC Essential Eight mitigations, or as much of these as you can manage.
As a maintainer of an open-source application I’ve learnt that virus scanners (as malware tries to look entirely different every few days) have started to try to find out what applications do (along with trying to guess if they look familiar). Every time an auto completion feature scans a directory for possible file names it risks its action being interpreted as the first step of an ransomware attack.
One thing that currently helps much against ransomware (along with installing the newest security patches, keeping the virus scanner up-to-date, making frequent backups on more than one media and not installing shady software that cracks licenses and computer games for you) is to tell Windows not to hide the filename extensions and never to believe anyone that it is necessary to enable macros in word documents: If ransomware isn’t installed by yourself following a fake Microsoft security expert that calls you on the landline it typically is contained in a .PDF.exe, .PDF.zip file or in a document that requires macros for installing the ransomware. If you don’t fall for that and if you aren’t a worthy target a real human being is feeling it being worthwhile to trick into something stupid using social engineering you are much safer than the average internet user.
Thank you Gunter. Presently I follow all of your outlined practices. So that makes me feel like I will have some success in avoiding this scourge.
As always, “Windows.” Is that really the core of the issue?
Yes they target windows but almost all businesses have settled on MS office. I know there are many options. I just have not seen one that is 100% compatible with the rest of the world.
Actually the problem isn’t just Windows, the real problem is the fact that many companies simply are not patching appropriately, have significantly cut their support budget, and are not paying for their maintenance contracts (typically an “expense” item and cutting expenses are the way that many companies show growth when sales do not meet expectations).
A crypt/pack doesn’t cost much. It doesn’t matter if the ransomeware strain is 3 years old or 1 day. Getting past the latest AV is trivial. To be honest, these guys probably use their own packer with a dedicated team to handle detections.
In many cases, the MSP will instruct (configure) AV softwares to ignore their tooling. If there is a vulnerability exploited in the MSP tooling or configurations, the hacker can use the MSP tools to control, or infect, the customer systems and the AV software won’t see it because it was instructed to ignore it.
In this particular case the attackers used their access to disabled anti virus on the clients machines before pushing out a package containing the ransomware.
I thought you where not suppose to pay the miscreants behind these types of ransomware? By paying out, this just incentivizes them to continue to do criminal activities .
Gents… your questions lead one to believe you think ransomware is petty theft carried out by small-time crooks. Au contraire. Ransomware is big business; the weekly payouts are in the millions of $’s. The targets are not Jack and Jill, they are vital organizations’ vital information that cannot afford the loss of access to data nor the consequential failure of data-dependent processes and products. Literally in some cases worth the lives of people and in other cases the capital value of corporations (hundreds of millions of $). This is a major financial and social crime and behind it are oligarchs, rogue nations, and well-capitalized, encrypted investment schemes involving people and places you might find hard to imagine. That is why its dangerous to hunt them.
Like all forms of war have ever been?
Good point Stillwater. I tell people that all the time, that these attacks are coming from well organized and substantially funded organizations and/or groups, not your average Joe for certain. There is much at stake here folks. We just can’t sit idle and watching the wheels go by. These acts of crime have serious implications on our government, and businesses both private and public. It’s only a matter of time when we as individuals will be targeted as well.
PS … you will never hear about most Ransomware attacks. Just like kidnapping: never tell anyone about it and never admit to paying.
I spot “evil” links in my email all the time. I feed them to virustotal.com and generally only 5 to 10 AVs detect the malware. I think you are kidding yourself if you depend on AV software to save you.
I wish there was some easy way to score the winners from the virustotal.com test. I notice Sophos is nearly always on the detected group, but who knows. It might have a high positive rate.
The best anti-virus is between your ears. Look at the actual link before you click on anything. I must get three fake PayPal emails a week. Easy to spot since they don’t know my name, but then again I’m not on social media under my name.
Thanks for the link!
Don’t rely on phishing emails NOT having your real name on it. There are too many breach dumps that put your email and your name together, so that they can introduce many emails with such accurate information on them in wide spread “spear phishing” attacks, only they can attack everyone in whatever dump they are gathering the data from.
I received such an email that was clever about avoiding the junk mail filter by putting only inactive images in composing the email and only one active link, that managed to fool the filter into letting it go to my inbox. Because of my name being on it, and the low instance of hyperlinks, it fooled me into clicking on it. Everyone thinks they can get away with it just this once; but nope! The only thing that saved me was my password manager, as it didn’t recognize the URL and wouldn’t fill in the user name or password. Believe me, I was really embarrassed!! I forwarded the phishing email to PayPal security, and took heart in this lesson! They emailed me back and confirmed it was some tricky work for a crook. The email looked completely official using every PayPal guideline and appearance, and no misspelled words.
Needless to say, I especially don’t trust email that are in my junk box, just because the images display is not an indication of safety either. And of course I don’t follow emails to my accounts at all. I do it all outside of email alerts, no matter how convincing they look.
Friends, Romans, InfoSec People…
AV is just another preventative measure, in my opinion, to be implemented as a last step to secure systems.
Is order to “reduce” the likelihood of a ransomware attack, you have to start with strong hardening policy, to remove all the non-essential services/programs (smb, netbios, remote registry etc). AV, if managed/configured centrally and depending on the Product (e.g McAfee ePO + ENS), can be set not to allow to be stopped/disabled.
Secondly, nowadays there is no reason why most organizations can’t enable MFA (2 step verification) for their Primary network authentication Directory System.
Lastly, the MSP must deliver IT Services in line with your Company’s Information Security Policy and connect to your networks through a “jumpbox” that also implements MFA.
It is the nature of capitalism to “cut corners” for large, or marginal, increases in profit. Even very smart and highly educated people do this – affecting staffing quality, and the time given to efforts (say, hardening a system) that don’t seem to pay off (increase profits) in any obvious way.
Given the comment about capitalism – the only fix may be regulations – people die from these attacks.
@PattiM, I do not follow your comments relative to “capitalism” and sound infosec/cybersec advise, that everyone is contributing. Are you perhaps implying that Product Vendors may be involved in the ransomware agenda, to drive their interest to drive/maintain sales/revenue streams?
The solutions you listed are considered “legacy” solutions are are inferior. Companies which are not leveraging a solution that uses deep learning/AI are going to find themselves on the receiving end of these kinds of attacks more and more as legacy AV solutions cannot and will not prevent against them.
The solutions you listed are considered “legacy” solutions which are inferior. Companies which are not leveraging a solution that uses deep learning/AI are going to find themselves on the receiving end of these kinds of attacks more and more as legacy AV solutions cannot and will not prevent against them.
Is it all about money on the victim’s end?
Where are the backups? Storage is so relatively cheap these days, why not pull down the backups so the loss is minimized?
Is one of the reasons they try to hide it not only publicity, but the fact that they are not doing their job in the first place with backups and too lazy to use 2FA on everything. They are custodians of other people’s property, but they don’t seem to take it seriously.
Hey Mike, from what I’ve experienced is a lot of companies do not have instantaneous or nightly backups running. For a company that grosses in the hundreds of millions, and needs to be 24×7, even an hour of downtime costs them more than the ransom. But, then again, OneDrive is so cheap to redirect files to, or implementing a similar automatic cloud redirect, I really see your point. Then there’s the added bonus of the Mean Time to Recovery. You have initial outage, time to detection, then the time to recovery. That can take much longer than some
Presidents and C-level’s agree with. Which is why a previous comment mentioning a strong security policy and BCP/DRP is absolutely key. If written, updated, and managed properly, these policies should have all the answers on what to do in a situation like this. Let’s be real though 3 hours to pull down backups and lose in the hundreds of thousands, or pay some crook 1 BTC for immediate restoration? Keeping costs at a minimum is the goal of any company.
With regards to cost, large companies (like Synoptek) should expect a ransom of 10+ Bitcoin or a similar amount in another cryptocurrency. That is still likely to be cheaper than spending 1+ hour restoring the systems if they have multiple customers relying on the servers being live.
In this case the target was a cloud service provider, so them making regular backups is expected. But reading between the lines makes it seem like their backups were not stored safely on a different server configuration. Or Synoptek was just not willing to risk having all their customer’s data exposed (with the potential legal backlash that would cause) if they didn’t pay up.
The exposure part of the current threats is not an insignificant risk if it hits a company like this. Synoptek would likely lose far more in legal fees and fines if a small fraction of their customers decided to sue than they’d have to pay the criminals.
Unfortunately if the criminals actually did grab a lot of data, Synoptek’s customers are now potential targets for the next level of attacks. Whether through direct extortion or intrusion via compromised credentials.
Unencrypting computer after computer hoping that the files being encrypted at different points in time won’t leave the system in an inconsistent state sounds more complicated to me than just restoring a known-good backup hoping that the backups aren’t encrypted, as well…
Even buying the decryption keys, and even if they work well, it can still be hours or days to completely decrypt- depends on how much data, and also the files- some you will need to say “ok” or affirm the actions on some files, depending on the ransomware strain. Imagine having to say ok to decrypt each of 100,000 files.
Twitter bug discovered and remediated. Phone number can be uploaded in Twitter. Read full arrival. https://t.co/1W7bjSP60e
…URL shorteners are still used by hackers!
I work for a small (<200 total employees) business that was victimized in the Sidnokibi attack mentioned in the article above that accessed our data through Complete Technology Solutions in Colorado. We were under contract with and paying CTS to protect us and engineer appropriate backups for our data. We followed their recommendations and protocols but but the systems and backups they established for us were compromised and all our files and data were encrypted in the attack. Frustrating to pay a MSP for advice and protection yet still be victimized in an attack where they were the gateway for the attackers to your data!
Post attack, CTS offered minimal assistance to us with recovery as they were overwhelmed and understaffed. They even refused to provide information as to their insurance carrier ("based on advice from their legal council") so we could submit a claim. A frustrating lesson in misplaced trust.
Unregulated capitalism in action – they reap profits and work contracts until it happens and bad faith is revealed.
@PattiM, I do not follow your comments relative to “capitalism” and sound infosec/cybersec advise, that everyone is contributing. Are you perhaps implying that Product Vendors may be involved in the ransomware agenda, to drive their interest to drive/maintain sales/revenue streams?
Stop trolling.
From everything I read about the latest ransomware, the attack can find everything that is attached to the server or network, even cloud services can be hit automatically without manual attack methods. I’m not a shill for Carbonite, but there are cloud storage services that have successfully insulated them selves against such attacks.
It might not hurt the check with the cloud service you were using and make sure there wasn’t a recovery method available. Some services have found other simple avenues to recover files outside the infection dates.
Backups are the only way to restore, as long they are not compromised. However, restoring from backups provide ridiculous RPO and RTO. You need a solution that provides immutable backups and snaps directly from primary storage for Zero RTO and minimal RPO. Those should also exist in multiple places. Datrium has been helping companies recover from ransomware attacks in a matter of seconds. I know this can be seen as a sales ad, but it can really save the day if you are under attack and you should know about it.
Unfortunately over many decades Sysops have unwittingly allowed themselves to be hoodwinked into accepting responsibility and liability for defective vendor software that uses legacy authentication routines from the 1960s that were never designed for wide area networks. The weaknesses in this authentication is now trivial to exploit and the attacks are so sophisticated that they cannot be detected by observation and they cannot be protected by second factors without opening a network to the risk of permanent lockout which the ransomware attackers are exploiting. The attacks are tested against malware scanners before they are launched if the attack is broad scale but if as others here have pointed out the attack is targeted they are simply disabled. It is so tiresome to continue to hear about the heartache being caused to well meaning people who have been brainwashed into believing the mantra of the cyber security industry who are taking advantage of this for their own benefit without actually fixing the root cause of attacks which is the defective authentication which needs to be fixed in operating systems at kernal level.
We need to remember, phishing is not a virus infected email so no Antivirus app will trap it. Phishing is a method to disguise, in an email, a link to bait the recipient into clicking and entering some credential (account log on details). If a large company has thousands of employees it’s relativity easy to get one to take the bait. Even if the company has all the AV tools and employee phish training etc, the weakest link in any business is the human. A company, of any size, needs multiple layers of protection. Consult a local MSP or security service provider for ideas on improving your layers. The best forms of protection block the attack at the door, do not depend on tools that ‘fix’ an attack after it is on your computers.
It may seem overly simplistic, however, separate your hosting org from your platform management functions. That way, if the hosting facility gets exploited, having access to your data becomes a lot more difficult. Also, good identity and access management practices (read: MFA everywhere) can reduce your vulnerability to situations like this.
More expensive, yes, but much cheaper than in the long run of getting whacked like this.
Ask your hosting facility what their protections are against a situation like this. If they can’t document it, move on. Other credible organizations will show it to you.
A better solution than anti-virus would be to white list your software. It takes a lot more effort and time, including updating the white list when upgrades/patches are made, but if done properly, it can prevent a lot more malware from running for the simple reason that you won’t be adding malware (including ransomware) to the list of approved software. And a proper white list will include CRC checks on the executable, so a compromised executable can’t be used, either. Do a web search on “white list software”.
Application or software whitelisting used to be included in the Parental Controls of Microsoft Vista Professional. I tested it in my honey pot lab, and it worked amazingly well! I never found a malware that could do an end run on it. Also, it wasn’t that much of a pain to maintain, as it only went off the first time a new vector was activated, and would identify what it was clearly, so you could tell if it was legitimate or not. I definitely missed this feature when it was dropped when Windows 7 came out. IIRC, Win7 still had Parental Controls but the program controls were changed and no longer worked at all, or at least for me it didn’t.
Thought..I try to teach all my partners to isolate the products. It appears everybody want sit under one window pane..you are just looking for problems vs just click on another link.
Select you product and keep them separate .
There is a product that runs a virtual (auto containment)
Which provides a verdict withing 45 seconds for 95% of files submitted.
Thanks for your hard work sir!
Congrats on your 10 year anniversary ! I appreciate your dedication and drive in providing us a trusted source for cybercrime news.
thank you
Scott Schober
http://www.ScottSchober.com
And a Happy New Year Mr. Krebbs. Thanks for your steady (and I am sure, often exhausting) work finding, exploring, and explaining, myriad risky discoveries out there.
Congrats Brian, and please keep up the great work.
The threat of releasing data is very curious. Ransomware is so effective in part because it relies on a level of trust that criminals can achieve. You know they will decrypt your files if you pay because they have reputations they’ve built. It also has a clear end point, kind of like kidnapping.
But the level of trust necessary to know that a blackmailer has been sufficiently paid off is inherently impossible to achieve. They can’t provide a sufficient certificate of destruction. You can’t trust their security. No honor among thieves means at any time one of their collective can abscond with your precious data. Many criminals get out while they’re ahead, so you have no guarantee that they won’t just sell your data in a few years anyways once they decide to close shop.
The risk calculus for blackmail is radically different. It isn’t worth paying off a blackmailer unless if they have catastrophic material. The painful acceptance of whatever failure occurred is often the cheaper long run option because you just have to assume the blackmailer will extort you eternally. I don’t think these people understand that they aren’t in the ransomware business anymore.
I believe the threat of releasing data is a bid to raise the level of fear for future victims, not for present ones.
The threat of disclosing GDPR violations is pretty potent as even some of the most diligent companies have no idea where they stand in that grey area. However outside of multinationals and EU based companies this threat narrows effective target selection, but it does increase payout chance from that slice.
This does seem to serve as something of a hint towards who they’re going to focus on. Good or bad news depending on regional operation. Scary to think about how they’re inching closer to exploiting the “crime a day” principle as a means for extortion.
As an MSP/MSSP, I predicted this type of “island hopping” attack would occur sooner or later…and here it is happening in full swing with not just the naysayers getting hit, but unfortunately their customers. If Brian remembers, we spoke about this at a McAfee conference a couple of years ago. I told you to get ready as it would happen and sure enough, even the big guys like HPE, IBM, Kaseya, and others have all been hit. In just the 4th quarter of 2019 we’ve received reports of at least 18 service providers being hit which impacted their customers, and those are just he ones who reported it.
Back then I told you how my company was configured and now I’ll (sort of) let your readers know how we are different. I configured our systems to prevent this type of thing from happening through layered security solutions and isolated system containment. I was called crazy, and made fun of for being so “paranoid”. Well my friends…we have yet to be breached and none of our customers who have followed our security framework and allowed us to handle their day-to-day security administration have had any issues. In the words of Bruce Schneier “Security is a process…not a product”. Yes we need the products, but just having them installed isn’t enough. There has to be ongoing maintenance, policy tuning, configuration changes, and even some common sense. Most companies don’t have the internal resources to allocate for this required daily maintenance…that’s where we step in. It isn’t rocket science and we definitely don’t charge the IBM/HPE rates.
Beyond all of the above…what sickens me most is how some of the service providers think it’s okay to downplay this. They’ll report they had a “minor issue” or as Krebs stated in the article regarding Synoptek: “credential compromise which has been contained”. The most aggravating piece is that this is the only external communication they made. They didn’t make any additional statements until external resources starting reporting it and held their feet to the fire. I’m sorry, you have hundreds if not thousands of customers under your watchful eye and all you do is release a tweet? Sad doesn’t cover it, that is just pathetic.
Ransomware is avoidable and preventable. Perhaps not 100%, but nothing is 100%. And anyone who tells you differently is fooling themselves. While we here at London Security have had a phenomenal success rate for over a decade, we understand it’s only a matter of time until something happens, but when it does we’ll be ready since we’ve practiced and rehearsed our responses. If it never happens then we’ll be the statistical anomaly. But hey, 67.3% of all statistics are made up anyway 😉
Yep, mate i have been called paranoid, tin foil hat wearer, that creepy dude who thinks the NSA is after him, you name it but like you, we have yet to be hit while nearly every client we deal with has been hit and some hit hard.
When someone says “Watch out for that car” when you walk across the road, they mean it so when i say “Dont click any links in emails” i bloody well mean it.
I have been a long time reader of Brian and Bruce along with Troy Hunt and if it wasnt for these guys being on the ball along with great articles from WeliveSecurity (ESET) then it might be a different story for my end users.
Yes its frustrating to do a simply updates etc or install software but there is a reason for the madness.
I have always told people that “The internet is like the old james bond movies, you are on the plant of wood over a pool and the internet is the pool full of sharks”
Phishing, spammers etc are trying to get you to get in the pool and then expect that every shark wants to eat you and thats just how dangerous the internet is.
Scare people, make them turn white with fear because thats what they need besides a slap in the face to keep them secure.
/rant
Happy Birthday! I’ve enjoyed your work and know that you’ve made a difference. Keep up the good work.