PerCSoft, a Wisconsin-based company that manages a remote data backup service relied upon by hundreds of dental offices across the country, is struggling to restore access to client systems after falling victim to a ransomware attack.
West Allis, Wis.-based PerCSoft is a cloud management provider for Digital Dental Record (DDR), which operates an online data backup service called DDS Safe that archives medical records, charts, insurance documents and other personal information for various dental offices across the United States.
The ransomware attack hit PerCSoft on the morning of Monday, Aug. 26, and encrypted dental records for some — but not all — of the practices that rely on DDS Safe.
PercSoft did not respond to requests for comment. But Brenna Sadler, director of communications for the Wisconsin Dental Association, said the ransomware encrypted files for approximate 400 dental practices, and that somewhere between 80-100 of those clients have now had their files restored.
Sadler said she did not know whether PerCSoft and/or DDR had paid the ransom demand, what ransomware strain was involved, or how much the attackers had demanded.
But updates to PerCSoft’s Facebook page and statements published by both PerCSoft and DDR suggest someone may have paid up: The statements note that both companies worked with a third party software company and were able to obtain a decryptor to help clients regain access to files that were locked by the ransomware.
Update: Several sources are now reporting that PerCSoft did pay the ransom, although it is not clear how much was paid. One member of a private Facebook group dedicated to IT professionals serving the dental industry shared the following screenshot, which is purportedly from a conversation between PerCSoft and an affected dental office, indicating the cloud provider was planning to pay the ransom:
Another image shared by members of that Facebook group indicates the ransomware that attacked PerCSoft is an extremely advanced and fairly recent strain known variously as REvil and Sodinokibi.
However, some affected dental offices have reported that the decryptor did not work to unlock at least some of the files encrypted by the ransomware. Meanwhile, several affected dentistry practices said they feared they might be unable to process payroll payments this week as a result of the attack.
Cloud data and backup services are a prime target of cybercriminals who deploy ransomware. In July, attackers hit QuickBooks cloud hosting firm iNSYNQ, holding data hostage for many of the company’s clients. In February, cloud payroll data provider Apex Human Capital Management was knocked offline for three days following a ransomware infestation.
On Christmas Eve 2018, cloud hosting provider Dataresolution.net took its systems offline in response to a ransomware outbreak on its internal networks. The company was adamant that it would not pay the ransom demand, but it ended up taking several weeks for customers to fully regain access to their data.
The FBI and multiple security firms have advised victims not to pay any ransom demands, as doing so just encourages the attackers and in any case may not result in actually regaining access to encrypted files. In practice, however, many cybersecurity consulting firms are quietly urging their customers that paying up is the fastest route back to business-as-usual.
It remains unclear whether PerCSoft or DDR — or perhaps their insurance provider — paid the ransom demand in this attack. But new reporting from independent news outlet ProPublica this week sheds light on another possible explanation why so many victims are simply coughing up the money: Their insurance providers will cover the cost — minus a deductible that is usually far less than the total ransom demanded by the attackers.
More to the point, ProPublica found, such attacks may be great for business if you’re in the insurance industry.
“More often than not, paying the ransom is a lot cheaper for insurers than the loss of revenue they have to cover otherwise,” said Minhee Cho, public relations director of ProPublica, in an email to KrebsOnSecurity. “But, by rewarding hackers, these companies have created a perverted cycle that encourages more ransomware attacks, which in turn frighten more businesses and government agencies into buying policies.”
“In fact, it seems hackers are specifically extorting American companies that they know have cyber insurance,” Cho continued. “After one small insurer highlighted the names of some of its cyber policyholders on its website, three of them were attacked by ransomware.”
Read the full ProPublica piece here. And if you haven’t already done so, check out this outstanding related reporting by ProPublica from earlier this year on how security firms that help companies respond to ransomware attacks also may be enabling and emboldening attackers.
FIRST!!!!!! Good article Brian
The obvious Q: does solving the source of this malware effectively take a bite out of crime? It was too easy to “say” that.
Ransomware is roughly a crime of opportunity. Just because you can lock up one gang doesn’t mean you’ve locked up all gangs.
I think better backup systems (offsite reduced privilege with unrelated login credentials, not running Windows) is probably the way to go.
If I were using Windows, I’d probably look into using Carbonite .
Here’s another obvious question: what is the solution to these malicious attacks? Further, if there is one, why aren’t more of these companies using it? Otherwise, they are just leaning over and saying “Enter Here.”
Its not difficult at all, these companies just don’t want to spend the money to implement basic protections or they hire incompetent people that fail to do the right thing. I have been doing this professionally for over 25 years and its been the same since day one, companies just don’t care. You want to protect against ransomware then remove admin rights for users, implement a decent firewall with content filtering, CnC protection, malware scanning and sandboxing, install halfway decent A/V on the systems (Malwarebytes and Defender works fine) and configure it properly, use least privilege for your shares and services, turn on auto patching for ALL systems and 3rd party software, enable 2FA in Office 365 or whatever productivity system you use and do a couple other small things that don’t cost money and you will stop pretty much all attacks. Sure there will always be extreme outliers depending on your risk profile but those are much smaller populations of companies. But again it takes commitment that most don’t want to do or are just not smart enough.
By the way I am speaking of small to mid market companies, large enterprise is mostly the same but add on a few other things that cost a little more. Still very easily preventable. I could go to hundreds of companies and count on 1 hand the number who have implemented basic CIS benchmarks configurations which by the way cost nothing other than IT staff work, but they are too busy watching cat videos.
I don’t think you understand what you just said. You started by saying it’s “not difficult at all” to prevent ransomware. Then you go on to list the following controls to prevent ransomware:
–remove admin rights for users
–implement a decent firewall with content filtering
–malware scanning and sandboxing
–install halfway decent A/V on the systems
–use least privilege for your shares and services
–turn on auto patching for ALL systems and 3rd party software
–do a couple other small things that don’t cost money
None of those are easy. They all take time to plan, configure, and implement in the network. Also, “turn on auto patching for ALL systems and 3rd party software”… I hate to say this, but I question that you have 25 years of experience as you say. There’s no way that you can tell me that you can just patch ANY and ALL systems in the environment. There are exceptions. There are outdated systems that cannot be patched. There may be embedded systems that you can’t just flip a switch and patch. You don’t have the same control over 3rd party assets than you do with ones you own yourselves. You can’t just blindly patch systems without first testing.
I also see no process-related controls, i.e., backups. This should be #1. Also, there was no mention of email controls–the vast majority of ransomware IoC’s initiate via email. Those two controls should trump every other on that list. To say that preventing ransomware is “not difficult at all” is a stretch. A very large stretch.
Have you ever personally done just 2FA / MFA for a company? Or been involved with the planning? That is not an easy task by any means. That’s not something you can plan on a napkin at an Applebee’s. That takes many months from start to finish. There are a lot of assets you have to consider. Where will it apply? Will there be exceptions? What factor(s) will we use? How will those factors be implemented? My company is in process of it right now, and let me tell you, this was not “not difficult at all”. This was extremely challenging and takes a LOT of planning, developing, assurance, testing… you get the point.
He is only saying there are things you can do. The solutions are largely already available… is it simple? Probably not, but the answers exist out there. (and have been for many years)
Spot on Travis. Redundant and secure backups are priority 1.
As someone who has done some of these things, it’s not that difficult. It isn’t any harder than getting proper wiring in a building. It takes time and money. Nobody wants to spend on security though so they get bit.
Not true Robert. Not all are too busy watching cat videos. Some of us are too busy reading articles and comments on krebsonsecurity.com
I would probably list backups before all those other things, but regardless, it should be part of your solution.
My post was about preventing ransomware, not restoring from it after you have been hit. That’s a different scenario.
“auto patch for ALL” begs the question “How do we know that the ‘auto-patch’ systems including the uSoft monthly, are not compromised? Surely ‘auto patch’, the most intimate access to a system, is the Holy Grail for super hackers. Could a human factor be the weak point there too, as always?
Trust has to leave off at some point if you are using someone else’s stuff. How do I know Azure or AWS isn’t fully compromised and attackers don’t own every hypervisor via a CPU backdoor? It can run deep, but 99.99% of people are going be protected by handling just the basics as I listed.
Backups! Backups! Backups!
All these companies need to do is basic IT knowledge, The 3-2-1 Backup Rule.
Take 3 Copies of your data, store 2 of those copies on different devices or storage media (Tapes), and keep at least 1 backup off site. Then when you are compromised by ransomware you can just revert to your offsite backup in the worse case scenario. Sadly most companies don’t want to fork over the money or properly test these types of backup solutions.
After an attacker has established wide access to your systems and likely installed back doors and other malware, do you think you can flip a switch and restore from back-ups? That is an uniformed myth. In reality, with the latest wave of ransom attacks, it can take days or weeks to wipe/rebuild/restart systems after a ransomware attack, even with back-ups.
There is more than one method of backing up; in the old days we used to have dated media storage with limited backups – time your choice – all separated physically, in other words not on a network, and clearly marked with the dates and times. Most used tape backups, and although it seems complicated, it really isn’t in practice.
The only problem is coming up with a plan on restoring, and what sequence you want to restore. Obviously several weeks or months may have gone by with the malware backed up with the data as well. If you are brave and want to save time, you can restore all the way minus the last month or so – then either scan the last backups for the malware, and or start restoring slowly each recursive backup until you reach the estimated attack date. It takes lot of guts, but that was how it was done. Malware wasn’t as damaging then – and they didn’t have such sophisticated ways of sleeping in the files and avoiding detection with special techniques.
The way I figure it – somewhere in the time line the malware attack package lost it zero day significance, and can be found and then it is anybodies guess what management wants to do with the remaining backups. My experience bears this out – but never were these attacks as sophisticated as they are now. Robert’s advice is good, and should have been implemented before the attack in the 1st place – of course.
I just saw a recent ransom attack. It was very sophisticated, but the good old Microsoft Server Backup was untouched–even on the currently connected USB drive (by design).
Boot from CDROM, do Cold Metal Restore, and all better same day.
Worse case you didn’t choose an old enough backup and have to go to an older one–a good malware scan after Restore might fix that.
Obviously, any newer data files won’t be in older backup, which is why you have a cloud backup (with versioning) of the data files that get changed, for restore of that smaller data set.
Thanks for posting! Folks might find that instructive! I am surprised the malware didn’t jump the USB connection; because I keep reading that the bug can see anything attached, even some types of cloud based storage. I also hear that some cloud providers like Carbonite, or similar are bullet proof backup can provide a clean restore backup complete. Hope that is correct – it would sure be easier to do it that way – especially if you have a fiber high speed connection and don’t have to wait forever for the data retrieval.
Me – I don’t have anything to lose, so I like playing with freeware like Cryptoprevent and/or the pay version of Malwarebytes Anti-Ransomware. Bigger businesses would need Enterprise solutions, of course. So far no ransomware has defeated Cryptoprevent in my lab- but it changed hands, and I quit testing it.
Actually, it did jump the other USB drives I looked at (plugged in to workstations). In fact it had no problem encrypting the supposed backup for Windows 10: File History.
I did find, on one of these other drives, an undamaged old backup that was made in Windows 7. Thank Microsoft for depreciating that nice backup feature in later versions.
As it pertains to backups…. 3-2-1 is a good start, but take a look at 3-2-1-1-0. It starts the same…
– 3 copies of your data – Originals/live/production, local backups, and off-site backups are typically how it goes.
– 2 different storage mediums – Tape, optical, different OS+file system, etc… the #2 is always hotly debated.. but at least it’s agreed that it’s not on the same drive as the source (Yes, I’ve unfortunately seen backups being sent to a second partition on the same drive that is being backed up!)
– 1 copy of your data is off-site.
Next… the extra 1 & 0….
– 1 copy of your data is off-line or otherwise inaccessible. People often think immediately of Tape, but if you are using a tape drive that’s connected 24/7 to your system.. it’s still accessible. Many off-site solutions are done with a VPN connection which is also maintained connected 24/7… which allows it to be hit by a spreading virus. So, use something that’s only available on-demand or connected as needed. Rotating drives, cloud services that are connected/disconnected as needed, etc.
– 0 errors in your backups. AKA… you know you can actually do a restore from them. So many people “take backups”… but never “verify backups.” You should periodically do tests to ensure your backups are not corrupt and can actually be used to conduct a restore. If all you are doing is file level backups… periodically try to recover a file from it. If you are doing a full system backup, try to do a full system restore from it. It’s amazing how many people think they can restore a full machine only to find out that they didn’t configure the back up to capture the master boot record or something equally silly. Or, that they had the encryption key stored safely on the same drive as what they are attempting to recover and is not therefore inaccessible.
Full Disclosure + Shameless plug… I work in IT and run a consulting business as a side job. One service is doing managed backups. I use Veeam which is a top notch product and in full disclosure is available to all of you for FREE if you are OK with the limitations of the community edition. Veeam is a big fan of supporting the 3-2-1-1-0 rule and is hands down my favorite backup & recovery software to use.
That said, there are other products out there that can also be used for the 3-2-1-1-0 rule and you don’t need my hand holding to follow it. But any business with any valuable digital assets (which is nearly everyone these days) should be following it. By far the biggest hurdle is honestly just having the understanding for the need of it and being committed to following through with it.
Final note, I would also like to echo what was stated by others… Backups are a “recovery tool”, not a “preventative tool”, so while backups are very important… make sure you take other reasonable security measures. The average person doesn’t know where to start when it comes to this. I recommend following NIST 800-53 because it’s FREE and is a constantly evolving set of standards that our government follows.
Be diligent. Stay safe. It’s a crazy world out there! 🙂
Wow! This is good stuff. Thanks Craig
That’s it! You got it down Craig! Hopefully an optic backup is re-writable, otherwise you can’t remove the original attack package if it can be found. I always like those DAT tape drives, because they were fast, and easy to swap out in the order you describe. However it isn’t probably for a large enterprise. I’d heard the resolution was improved to try to match previous drives, but I quit tracking that science years ago. Powervault LTO is the only brand I can think of off the top of my head.
Craig, that is an excellent description (the 3-2-1-0) of what I have preached for a long time. I practice the second 1, but have never described it as clearly – starkly, really – as you do here. Thanks.
My focus of the post was on preventing ransomware, not recovering from it after you have been hit. Backups and recovery are a whole different conversation, most companies just backup to the same machine or another system sitting on the network or next to it even. Another example of incompetence and laziness.
Agreed. We always have a back up plan in place for our clients, but overall the plan is to prevent the attack. Now this can never be 100% full proof, but we have put many layers in place in order to attempt to stop them.
If an attack does happen they are almost never the same and having the best back up available is recommended. The old school I save a backup every Friday just doesn’t protect you anymore.
1) Staff need to remember to actually do this weekly. (I have seen countless companies tell me they do a backup weekly and hasn’t been run for a month or even longer.)
2) You cannot verify it was not corrupted unless you actually run a test restore. Most companies never do this until they have been attacked. A corrupted backup doesn’t do anyone any good except the ransomware attacker.
3) Attackers most of the time have been in your system longer than you know. They will turn the data being backed up but leave the backups running. This means you think you are getting a backup but it is actually just a false positive.
Your goal as a company should be to prevent, remove, and protect against ransomware. A back up is your last line of defense but it should be your ” Hail Mary”
“When when you are compromised by ransomware you can just revert to your offsite backup in the worse case scenario.” <<< Only problem is you forgot about that Golden Ticket they wrote so they will be back in no time via the RAT they installed on other systems that they pivoted to and bam your entire network is ransomed this time. Backups only go so far, ask Sony about that. Not saying they are not important but its also extremely important to use best practices and prevent it in the first place.
Agreed. No excuse not to be at least 98% How much would that cost? For a small business, maybe few thousand (the hard part is finding a good provider).
Then a proper backup to cover the other 2% But local storage (with swaps off-site); the cloud is no good for a proper cold metal recovery that all systems should get.
Cloud is good for latest version of data, as a backup to the backup. The data just before encryption can be picked from the backup after all the OS’s are running clean. Just have be sure not to restore a RAT…
There’s a risk management decision involved. Implementing a broad list of technical countermeasures is one option. That is often costly and complicated, as others have pointed out. Another option is the school of fish defense. There are so many small to mid sized orgs, and really very few get hit by ransomware, management might make a risk decision to take their chances on doing less or even nothing at all. Just like with any other risk decision like how much insurance to carry, how much to spend on locks for the doors and windows, etc. But, it is taking a risk, and sometimes you lose. That’s when we get a story like this.
That ProPublica article that you posted was real informative , I read it a few days ago.
Of course they pay the ransom. It sure costs them less than paying for a good IT team to secure the data in the first place.
Darn. Now, even my teeth aren’t safe anymore!
feel bad for the employee who shared via chat that they are paying the ransom. probably sacked
Automatic, versioned, “pulled” backups that the client machines have no direct access to.
Don’t use any network storage protocol for backups like cifs/samba, dfs, nfs, etc.
“Pulled” is critical. Never use pushed backups.
Filesystem snapshots should be the first line of defense.
ZFS server storage (like FreeNAS or XigmaNAS) is even better, as those snapshots are not targeted by malware, and would be extremely difficult for malware to manipulate in any case.
Backups with something like Tarsnap, SpiderOak or Borgbackup to offisite storage: All store encrypted differential blocks — suddenly see a lot of new blocks going off to storage indicates a lot of changed files. All three allow you to roll back file systems very quickly to a previous point in time. Since they only store differential blocks, storing years of hourly snapshots is very cheap.
For Small and Medium Business, server storage with XigmaNAS is especially robust — the server runs FreeBSD, not Windows, and the entire server config is an xml file that allows you to wipe and reinstall your server environment in minutes. It also runs an Active Directory server in SAMBA.
TurnKey Linux also does something like this, but not as “turn-key” as XigmaNAS.
One minor typo, you have:
…PerCSoft and an affected dental office indicating the cloud provider was planing to pay the ransom
You mean “planning” not “planing” both real words, so the spell catch won’t flag it.
I love your site,
I wonder what will be the next announcement from PaySoft and/or DDS?
“Dear PaySoft Customers, we are working hand-in-hand with DDS to fully assure all of DDS customer data “Encrypted. Integrated, Simplified by ransom thieves. ”
“We strive to pay all ransom thieves softly and quietly. Unfortunately, our customers including DDS and their clients will find their next bill to be very hard and expensive to pay. Send us your data and your money now!”
Brian, have you heard of any ransomware on Macs?
I haven’t yet except for a bootlegged version of some expensive program like Photoshop that was on a torrent site years ago.
Several of my offices use only or mostly Macs now. Windows seems like Russian roulette.
It’s not about “does it exist”, but the user-base. I think that’s an extremely common misconception–especially with people that say, “Linux/MacOS is more secure because there’s no malware for it!” Couldn’t be further from the truth. Malware authors aren’t targeting MacOS because, why would they spend time to write malware and attack an OS that only has ~10% – 15% of the market share, when they can attack one that has ~75% – 80%? It’s not about an OS being inherently more secure, it’s really security through obscurity. If the roles were reversed and Macs had 80% market share with Windows having 15%, then we would be asking the reverse question.
Percsoft owner provided me the tools to bring a friends company back online. They were compromised and delivered the ransomware through backdoors left open by their own remote software connect wise. They pushed a patch to hundreds of companies that locked their data and did their absolute best to help get everyone back up. The very thing you are arguing about had been done, patching and security software. All of that goes out the window when the hackers go after the king of the castle that has your backdoor access. I hope the company recovers, that could happen to any MSP.
Please stop these people I have all the information you need
Dentists are yet another industry that has no legitimate business need to have Internet connected computers, going online and suffering for it. Whoever sold them on cloud usage should be imprisoned in the Phantom Zone.
Umm…no, you don’t understand dental offices at all. Many offices use cloud-based practice management software, many insurance providers only allow for online estimations and claims, all health care providers require offsite backups as per HIPAA, they all communicate with other offices through (hopefully encrypted) email, many labs require online referral forms, the list goes on and on. I was a periodontist for 10 years and have been a dental MSP for 20 years, few if any modern dental offices can manage without Internet connected computers.
I understand perfectly. Some dentists who do what the insurance companies tell them to do and run their clients’ data into clouds. These aren’t dentists, these are whores pimped out by corporations. A real dentist treats patients and would never compromise client data like this, not even for a reimbursement check to supplement the payment collected from the client.
Fyi, there are mechanisms for paper claims for every insurance company. One doesn’t need to be online to get third party payment.
Further, HIPAA does not require a specific mechanism of protecting data online or even encryption. Claiming that HIPAA compliance protects patients is nonsense. Don’t believe me? Read it, don’t just pontificate about it.
Recently while at my ophthalmologist, the doctor logged onto the computer in the examination room then left the room. I realized that the screen was a web browser. I leaned over and looked at the url. When I realized that they were using a cloud provider for their ERP my blood ran cold.
The angle of cyber insurers and cyber extortionists incentivizing one another is an intriguing one. Does anyone know of regulations or precedents that would make an insurance payout illegal if it’s expected to be used to pay off criminals? It sure feels like it ought to be.
Waiting for any regulation has gone long in the tooth.
really good article, its long and have lot of info
A few items.
1. I thought it was a backup system for dental offices. So how does that affect an office meeting payroll? Must have missed something.
2. It seems to be common (not sure how common) that even if you pay the ransom, at least some files will not be decrypted properly.
3. Companies don’t want to deal with security so they outsource it. But then they don’t want to spend the time, or have the knowledge to properly vet the companies they outsource their data to.
And companies simply are too cheap to hire the proper people to safeguard their systems. Sure a determine state may still hack into a system but it would be much less common and the easy security flaws would be patched and networks better managed.
At this point it is almost easier to stop pretending to be secure and just let everyone have all the data. I can’t count all of the systems that have been hacked that have my data, whether it is credit card companies, OPM, sites that were given my data via other companies, etc.
Hey Brian, we discovered this DDS Safe company used the same simple password across ALL their clients to access the Cloudberry account AND encrypt the backups online.
When we notified them we had recovered the password using a free utility from Nirsoft (BulletPassView), they instantly reset it for everyone, effectively breaking all cloud backups, and making it impossible to recover the existing cloud backups. They have NOT provided the cloud backups to anyone but only used the decryptor. Things were still corrupted so I ended up using the cloud backup that we downloaded before we told them about knowing the password to bring things back up.
To make it worse, the username was @ddssafe.com which means as long as you can Google the phone number, you could have accessed ANYONES information through the Cloudberry console, and even restored it because the encryption password was the same. Abysmal security practices. Horrible backup methods (only 2 folders, so everything else on the system was gone), and obviously little to no protection such as 2FA/MFA.
The password they used for everything? dental123
Sounds about right.
What is the name of the private fb group for IT professionals serving the dental industry?
We need local cyber criminal forces at local sheriffs offices like drug test forces. When we catch a hacker – give him/she/the state actor/ the max sentence or a strike back. Time for offense and more jail time. We got a new war on cyber and time to fight.
One of my customers got hit.
Stuck up my middle finger to the ransom demand and wiped the server, restored the secure back up image, back up and running in about 8 hours (took so long because we decided to use the opportunity to upgrade the old server).
Out of curiosity I checked on the demand: $4000 in BTC and if not paid in 6 days then it was $8000 in BTC (USD)
You are really lucky the old attack package wasn’t sitting in the backup files. They can sleep for months.
I got sample of fresh infection past weekend September 7th, 2019
Hybrid analisys link:
The ransomware attacks are now happening on a frequent basis going from one data storage infrastructure to another which makes me wonder, maybe some of the backup infrastructure providers are behind all of this. They are playing dirty to keep the business going.
Veeam are currently offering a free 10-socket license of Veeam (nWorks) Management Pack for new Veeam MP users who are using System Center Operations Manager 2012. This fantastic offer allows customers to officially license and get maintenance support for up to 5 (2 socket) ESX hosts. This is a substantial offer for any small/medium sized business.
Ransomware are real threat for your data. So you must have to do precautionary measures to be on the safer end.
Ransomware has been the most trending news recently. Many major organizations have been hit by a ransomware attacks, disrupting their everyday operations. These attacks have become so common that it comes as no shock that these are increasing day by day.
Criminal ransomware gangs are well financed (thanks to all those bitcoin ransoms) and employ increasingly sophisticated tactics. Only low-level crooks are interested in encrypting PCs one-by-one: the big gangs seek backdoors into corporate networks and then explore until they are ready to cause maximum chaos (and a big payday) by encrypting as many devices as possible in one go.
hi brian are this ransomwares ransom getting honored tho? like do they keep their endo f the bargin? i have a small business which heavily relies on internet service and am afraid this will happen to us one day. if ytou payransom they really return the files to you? and if so, what other measures should i do after files are restored? do ransomewares affect hardware? or will they be gone after a fresh format?? thanks brian
Great post! Cases like these help companies to prepare better data security tools and solutions.