Panerabread.com, the Web site for the American chain of bakery-cafe fast casual restaurants by the same name, leaked millions of customer records — including names, email and physical addresses, birthdays and the last four digits of the customer’s credit card number — for at least eight months before it was yanked offline earlier today, KrebsOnSecurity has learned.
The data available in plain text from Panera’s site appeared to include records for any customer who has signed up for an account to order food online via panerabread.com. The St. Louis-based company, which has more than 2,100 retail locations in the United States and Canada, allows customers to order food online for pickup in stores or for delivery.
Redacted records from Panera’s site, which let anyone search by a variety of customer attributes, including phone number, email address, physical address or loyalty account number. In this example, the phone number was a main line at an office building where many different employees apparently registered to order food online.
KrebsOnSecurity learned about the breach earlier today after being contacted by security researcher Dylan Houlihan, who said he initially notified Panera about customer data leaking from its Web site back on August 2, 2017.
A long message thread that Houlihan shared between himself and Panera indicates that Mike Gustavison, Panera’s director of information security, initially dismissed Houlihan’s report as a likely scam. A week later, however, those messages suggest that the company had validated Houlihan’s findings and was working on a fix.
“Thank you for the information we are working on a resolution,” Gustavison wrote.
Panera was alerted about the data leakage in early August 2017, and said it was fixing the problem then.
Fast forward to early this afternoon — exactly eight months to the day after Houlihan first reported the problem — and data shared by Houlihan indicated the site was still leaking customer records in plain text. Worse still, the records could be indexed and crawled by automated tools with very little effort.
For example, some of the customer records include unique identifiers that increment by one for each new record, making it potentially simple for someone to scrape all available customer accounts. The format of the database also lets anyone search for customers via a variety of data points, including by phone number.
“Panera Bread uses sequential integers for account IDs, which means that if your goal is to gather as much information as you can instead about someone, you can simply increment through the accounts and collect as much as you’d like, up to and including the entire database,” Houlihan said.
Asked whether he saw any indication that Panera ever addressed the issue he reported in August 2017 until today, Houlihan said no.
“No, the flaw never disappeared,” he said. “I checked on it every month or so because I was pissed.”
Shortly after KrebsOnSecurity spoke briefly with Panera’s chief information officer John Meister by phone today, the company briefly took the Web site offline. As of this publication, the site is back online but the data referenced above no longer appears to be reachable.
Panera took its site down today after being notified by KrebsOnSecurity.
Another data point exposed in these records included the customer’s Panera loyalty card number, which could potentially be abused by scammers to spend prepaid accounts or to otherwise siphon value from Panera customer loyalty accounts.
It is not clear yet exactly how many Panera customer records may have been exposed by the company’s leaky Web site, but incremental customer numbers indexed by the site suggest that number may be higher than seven million. It’s also unclear whether any Panera customer account passwords may have been impacted.
In a written statement, Panera said it had fixed the problem within less than two hours of being notified by KrebsOnSecurity. But Panera did not explain why it appears to have taken the company eight months to fix the issue after initially acknowledging it privately with Houlihan.
“Panera takes data security very seriously and this issue is resolved,” the statement reads. “Following reports today of a potential problem on our website, we suspended the functionality to repair the issue. Our investigation is continuing, but there is no evidence of payment card information nor a large number of records being accessed or retrieved.”
Update, 8:40 p.m. ET: Almost minutes after this story was published, Panera gave a statement to Fox News downplaying the severity of this breach, stating that only 10,000 customer records were exposed. Almost in an instant, multiple sources — especially @holdsecurity — pointed out that Panera had basically “fixed” the problem by requiring people to log in to a valid user account at panerabread.com in order to view the exposed customer records (as opposed to letting just anyone with the right link access the records).
Subsequent links shared by Hold Security indicate that this data breach may be far larger than the 7 million customer records initially reported as exposed in this story. The vulnerabilities also appear to have extended to Panera’s commercial division which serves countless catering companies. At last count, the number of customer records exposed in this breach appears to exceed 37 million. Thank you to Panera for pointing out the shortcomings of our research. As of this update, the entire Web site panerabread.com is offline.
For anyone interested in my response to Panera’s apparent end-run around my reporting, see my tweets.
Oh lovely.
They only start caring when they look like ignorant fools and it starts hitting the bottom line. Good IT and security costs money they do not want to spend.
Yes and no. This is really basic security stuff any solid developer should have implemented.
I’m developing an API right now and I do have some security knowledge (enough to know I don’t know anything) and I’m trying to figure out a good way to protect a public API. I would counter the pitchfork bearing crowd should come with not just shame, but pointers on HOW to fix this issue. Without details about what their API actually does, its hard to even suggest anything, But there are paths to make public API’s more secure, but perhaps not perfection.
I also have a notion that in the platonic sense, if there were a way to reduce bad actors abusing things, that might be a possible approach to at least think about.
I am not an expert but I think most publicly available API’s require you to at least sign up as a user and have a key before you can begin accessing the API (I am thinking of Google API’s for example)
Alright, advice: Passwords must be hashed and cannot be stored in plain text or any reversible fashion. Users should only have access to their own information and that which is publicly accessible. You should have a check at the start of a page which says “Does the currently logged in user have permission to see the data I’m about to display?” For example, if information is requested about user 20, but user 13 is logged in, you should report a 404 not found.
You should never trust input from the user and always clean is up into something that is safe: Escape any character sequences that have special meaning when used (For example, if you’re going to display it in HTML, strip out HTML or escape less-than, greater than. If you’re going to pass it into SQL, escape quotes. If you’re putting it into a CSV, escape the quotes. Etc.) — note that many languages do that for you now, know your tools.
Publicly accessible identifiers should be non-incrementing and unguessable (think Youtube’s “v” parameter). You should avoid mass-assignment of data from the user and only update the data that you expect to be updated.
Basically, go read up on OWASP and look up security on Pluralsight or other tutorial sites. “Hack yourself first” is geared toward teaching developers how hackable their code is and gives you ideas on how to fix it (since usually the “how” is domain specific).
@Karsten Rage:
Minimum you should start looking at are the following:
1. Enforce strong passwords for end users – not necessarily random gibberish that’s hard to read but strong. If you’re not clear on that and want an easy example, google xkcd strong passwords.
2. Make sure you can’t log on with just a username and make sure shared usernames aren’t allowed.
3. Make sure you’re not vulnerable to SQL injection – look up “OWASP” SQL Injection.
4. LOCK THE ACCOUNTS after a number of failed login attempts.
5. Any merchants (not customers) who you share API data with – also make sure that they need an additional token – don’t share tokens among multiple accounts.
That’s a good start – also check out OWASP top ten and also things like the PCI recommendations e.g. requirement for secure coding. PCI isn’t “security” per se but it’s security *lite* and is easy to understand.
“LOCK THE ACCOUNTS after a number of failed login attempts.”
NO! BAD! DO NOT DO THIS! This naturally leads to Denial Of Service attacks!
Let’s say I have a beef with someone who I know the username of. Perhaps usernames are publicly visible, perhaps your company uses first-initial-last-name and I want to mess around with the CEO, whose name is publicly visible.
All I have to do is write a script that tries the password ‘a’ 300 times every hour and the person I have a beef with is locked out of their account constantly.
DO. NOT. LOCK. VALID. ACCOUNTS.
Use other anti-hacking, anti-brute force techniques such as sliding wait times (delay bad logins more and more up to a few seconds to make brute forcing infeasible while allowing legit users to log in with minimal inconvenience), banning malicious acting IP-addresses (NOT ACCOUNTS) for a time, and taking other necessary anti-DOS and anti-DDOS measures.
This is the second worst “security” anti-pattern I commonly see, the first being the preventing pasting in password fields. It is security theater that *intentionally* creates security flaws *by design* in the name of security. Please, don’t do it.
Sonething is seriously wrong with Panera’s IT. They also double charge customers sporadically and it has happened time and time again. How many guests have not caught these bogus charges. There’s somethong really fishy with this company
2018 is going to be a lot for Security Experts. These basic security problems are not to ignore and more will pop up. Website developers have big challenge ahead.
Robin.
Website developers? How about incompetent CIOs which there seems to be plenty of the last few months *cough* equifax?
No, it’s still website developers. CIO won’t have to do any of the work, just decree that it be done or do nothing. Both take about the same effort.
Interesting that the same IT Security Director who dropped the ball also worked for Equifax for a while. Coincidence?
Due diligence is only one part of the equation moving forward. Vulnerabilities will continue to be found in even the most secure of applications, and when they are, the way that an organization handles disclosure by security professionals speaks volumes about their security posture.
This pisses me off. They knew for 8 months and sat on their asses and did nothing. For a fix that took 2 hours. Then they have the nerve to downplay the severity. I mean, I get it. Being breached seriously hurts a company’s reputation, so they’re trying to mitigate the damage, but when people find out how much they’re lying about the number of records leaked, that just makes them look so much worse than they would had they have just been honest. They don’t seem to take this too seriously and appear to want to just bury their heads in the sand rather than face the music.
Well, I’d say that they didn’t even fix it. If all they did was require a valid user login to access the info. All someone would have to do to circumvent this ‘fix’ is create a user account. This would take next to no effort.
So, not only did they sit on the info for 8 months, but they didn’t even really address the issue after claiming they had fixed it…
For a “fix” that took two hours, you mean.
If they knew about this for 8 months and did nothing about it they should receive a heavy fine for their deliberate abuse of the publics info. How do they continue to get away with this stuff. How about all the multiple and bogus charges as well! Could they show any more lack of respect and unethical business practices
I’m not surprised. Whomever is running Panera needs to step down. To add insult to injury, the quality and customer service is on the decline at at least 3 of there locations in Georgua: Lithonia Ga Alpharetta Ga and Conyers GA. They don’t seem to know procedures and the kitchens are terribly disorganized. Someone needs to randomly visit these locations and get it together. I love Panera but it hasn’t been loving me back with wrong orders, missing items and skimpy portions of food. I ordered the Sonoma chicken salad and they mixed up 2 other chicken salads together with it. I think they had ran out of the Sonoma but didn’t want tell me. I’ve gone there long enough to know the difference. I asked for the Tulip shaped cookie and the guy had no idea what I was talking about.
I work at a Panera and you are right about the disorganization and decline in quality. My managers are horrible. Our trainers aren’t trained properly, then they train new associates, then hey become trainers, and it’s a viscous cycle of decline. We have no accountability for doing things wrong. My managers are all passive aggressive. The company is crap. When the higher ups come we out on a dog and pony show, which they fall for every time. Surprise visits are a much needed idea, and some undercover employees as well. I’m not surprised by the website being low quality. The employee scheduling website sucks as well.
Data breach? Psssh! Tell me more about these cookie problems! Nom, nom, nom!
Why is this story dated April 18, 2002?
You are reading it incorrectly. It’s dated 02 April 2018
to his defense he did say he is confused lol
This entry was posted on Monday, April 2nd, 2018 at 5:37 pm and is filed under Data Breaches. You can follow any comments to this entry through the RSS 2.0 feed. You can
I had a similar bout of confusion my first time on Krebs’ site, so I understand your pain.
Even if I’d format it differently if making it myself, it follows a simple logic: ascending order when considering the unit of time (day, month, year).
Most of the rest of the world follows d/m/y
Which is ironic since yyyymmdd works perfect for numerical sorting, while both ddmmyyyy and mmddyyyy fail.
YES! Thank you for pointing this out. SeymourB, you da real MVP.
Crazy… Why would any IT individual ignore help? If someone gifted me a get out of jail free card, I would jump on it and resolve the issue as quickly as possible.
I can’t prove it, but I would guess that the issue existed prior to May 8, 2017, at which time I received a Panera-competitor-related solicitation from FocusVision.com on an email alias specific to my Panera Rewards account. I engaged FocusVision asking how they got my email address, but after initial contact, I stopped receiving FocusVision correspondence at said alias and they never followed-up. I also attempted to engage Panera to let them know that their customer database had been compromised, but they blew me off.
Or it could be that Focus Vision is a market research firm that runs surveys on behalf of its clients and Panera happens to be one of them.
Except that FocusVision doesn’t (or wasn’t at the time) do/ing market research on Panera’s behalf (this was confirmed by mmoreno at focusvision at the time).
If Focus Vision were working for Panera as, as you suggest, a “market research firm that runs surveys on behalf of its clients” then they would be extremely unethical to send solicitations for a competitor of their client to their client’s customers. Add to this if they refused to engage the legitimate request for “where did you get my info?” and their business starts to smell really bad.
How many times does this madness need to play out? For the busy execs at Panera who take data security very seriously, burn this six word rhyme into your brains: Care and share to be prepared. It’s not rocket science. Here are two posts you should read:
http://dgregscott.com/care-and-share-to-be-prepared-part-1-caring/
and
http://dgregscott.com/care-and-share-to-be-prepared-part-2-sharing
– Greg Scott
Is this tweet from one of the Panera devs? https://twitter.com/king_ry_ry/status/981140201652342784?s=21
Ok now hold on. The cards that we keep on file are normally only their Panera bread cards that people get with a membership, which they sign up with their name and email so they can get coupons and news on new deals and seasonal foods.
Also: https://twitter.com/king_ry_ry/status/981141857169264641?s=21
We keep their phone number so that if they forget their card they can still access their Panera account when checking out.
The problem with these tweets suggesting it was just a loyalty card number is that the guy who discovered this problem found his own information online and identified the “cardNumber” field as “Last four digits of saved credit card number” in his report. I think he would know if the number was for his CC or for his Panera card.
Mike Gustavision came to Panera from…you guess it…Equifax.
Thought you were kidding at first…but your not, he’s from Equifax
Guess that he’ll likely not be around much longer as the fall guy here (deserved or not…but if the 3rd time somewhere else isn’t the ‘charm’ for him, then he really needs to embark upon a new career path.
Panera is taking a deserves hit. They treat their employees like dirt. Working them all except managers just under the requirements for any benefits.
They claim to be eco conscious and recycle all used goods. Untrue. Once receptacles are emptied the bags all go in one dumpster anyway.
The food is overpriced and not unlike Subway in the way it is pre-sliced, prepackaged, and prepped from the company they order from.
It’s a good front. They take a little extra time on the dining experience, serving the food at your table, creating somewhat more restaurant style menus, and of course charging accordingly. It is a middle class fast food chain. But salad and frozen soups should not cost as much as they charge. And prepackaged meats are packaged meats no matter how you look at it.
Way to go Panera!!!
Wow! That is skeezy. What a false front they put on. How is that even legal!
It’s all true. Everything is prepackaged and frozen, then thawed for use. I tell everyone I know to avoid this place. Nobody uses utensils or portions things right. It’s a joke.
Wow… Great job Panera in retaining the public trust!
The magnetic strip on the credit card I used to buy a bagel at my local Panera store was copied and used at a gas station in Detroit to purchase $100 on fuel without my authorization. Panera kiosks have not been updated to read my chip so you must slide the magnetic strip from your credit card to complete your purchase through the Panera kiosks. I just ordered a replacement card with a new account number. I called Panera customer service multiple times to resolve other store kiosk problems over the last few years but there was no resolution from the Panera IT department.
What kind of remedial action they are offering to millions of customers? Our private info is out there.
“BOaTI”
Bend Over and Take It
Just a guess (based on So-You’ve- Been-Breached Best Practices v1.1): free one year of credit scrutiny.
Well, NOW I’m sure glad I didn’t add my CC info to the app when it tried to force me to. Is there anywhere we can look to see if our Panera account was one of the ones breached?
I noticed that http://www.panerabread.com is still accessible, but panerabread.com is not.
Not a surprise, they were one of the last to adopt chip vs swipe (and only recently) of the big chains I visit, and their rewards and quick order kiosks are still swipe dependent. Years behind on technology and security. Too much expansion, too little intelligent oversight.
Jersey Mike’s and Whataburger appear to be the same. In my town at Jersey Mike’s the chip slots are taped over.
I’m no longer breaking bread with these guys.
this company is toast
Does anyone know the time frame of the breach? I just added my card to the app yesterday morning and made a purchase before this surfaced.. Should I get a new card?
It depends on whether or not someone abused it.
Technically, your information was available to be taken.
Whether it was, and whether Panera will properly inform you if it was are open questions. Whether you should trust their denials? … Well…
“Thank you to Panera for pointing out the shortcomings of our research”
Omg I’m crying I can’t stop laughing
Mike Gustavison
Director of Information Security
Panera Bread
Mike has 15 years of experience in information security working in financial and consumer services industries. He has developed and built security organizations and programs that align with corporate strategies as a business enabler. At Panera, Mike and his team have created a layered defense strategy from an internal and external perspective. Secure zone and segmentation deployment was a critical program deployed by Mike and his team to meet internal security strategies by ensuring limited and restricted access to critical systems and assurance of proper visibility. Leveraging Akamai (Edge and WAF products) was a key element for Panera’s external defense strategy to enable visibility and proactive response to current risks in the wild. This allows for quick reaction, priceless metrics, and significant insight into activities that are occurring against digital properties.
https://www.eiseverywhere.com/ereg/popups/speakerdetails.php?eventid=126042&language=eng&speakerid=339242&
According to Mr. Gustavison formerly worked at Equifax as Sr. Director of Security Operations from January 2009 – June 2013 (according to his LinkedIn public profile).
According to the following Forbes article Equifax experienced a major security breach that *started* in April of 2013 while under Mr. Gustavison’s watch.
https://www.forbes.com/sites/thomasbrewster/2017/09/08/equifax-data-breach-history/#11796266677c
All this chat about Panera’s security expertise exposes a common error that CISO’s often ignore… data management. You can only secure what you know you have. Was this database properly identified by all lines of defense? Was it deemed a collaboration site in error? How in God’s-Green-Earth can a database this large fail to be properly secured?!? I’m smelling that Panera’s data flow diagrams are nonexistent thus much data are not secured.
Well it looks like I’m going to attempt to delete my account with them today. From the looks of things their security is a bad joke that won’t be getting fixed any time soon. At best they’ll patch this hole and get breached again in another week or two (if it hasn’t happened already). I barely use their rewards program anyway.
It’s funny the article mentions the rewards ID being leaked. With the phone number already in the breach you could just have them look up your ID at checkout anyway. One less step needed, I guess.
Is the data available somewhere, like on haveibeenpwned? I’ve used the Panera site but I think as a guest rather than having created an account. How do I know if my data was compromised?
HIBP (HaveIBeenPwned.com) doesn’t yet have data from this breach. I just checked.
haveibeenpwned probably won’t get it, at least not for quite some time. It would require somebody to specifically go out exploit the vulnerability to grab the entire database, for the express purpose of sending it to Troy to load in haveibeenpwned. Troy would probably frown on that. He would also definitely not go do it himself.
I guarantee you it has already been scraped. Troy will see in a week or 2.
Confirming that www. panerabread.com is up and available at 11:44 EDT, Aprile 3, 2018. Encouraging to see their crack PR department is on top of this too – last post:
January 28, 2018 St. Louis
Panera Bread Preemptively Recalls All 2 oz. and 8 oz. Cream Cheese Products
Not Cheesy Poofs!!! LOL…
Well, I just tried to change my password. http://www.panderabread.com let me log in with the existing password, so they didn’t invalidate the old set, as of April 3rd, 11:40am Eastern. On trying to save the new password, there’s just a spinning circle, possibly the .NET version of the hourglass.
Looking at what they store for site users, they keep birthday (month and day only), email, phone, name, “MyPanera Member Card” number. (Likely more if you’ve paid online; I have not.)
JERRY! You typo’d the link (panDera~ ..vs.. panera~) and it leads to a spammy ad site portal. Guess that is another place their IT and/or PR departments fell down on the job (monitoring for and challenging these “unintentional-typo” domains is part of the job).
FYI: This is called “typosquatting” and Panera’s legal team could have sued these creeps for $100K if they had had their head in the game. https://www.upcounsel.com/typosquatting
If I order food online as a guest and never login, is my account affected?
My credit card (which I had only used at one retailer for 5 months prior to this) was compromised within 24 hours of using the Panera App on my Note 8. My credit card was stored on the app.
I used the app to do a pick up order on March 18, and received a fraud alert text from Citibank on March 19. I even called Panera that night to inquire if they had a breach, and the lady who spoke to me on the phone seemed scared $hit-less and clueless.
This is way worse than they are letting on.
I think you might be overthinking it. I would think it’s unlikely that the two events you described are related to each other and not at all related to this disclosure.
It isn’t overthinking when the two events are so close together, and it’s one of two possibilities. I actually went back and checked, and the Panera app was the only other place I had used the card since my chip card was issued.
I’d also like to say from personal experience, the only people I’ve actually heard accuse someone of overthinking are sociopaths, in part because it’s easy to dismiss everything with hardly any effort, and easy to remember if they get called on the carpet again later.
Thank you for the insightful and detailed account, Brian.
Hopefully the aftermath won’t be too devastating for their customers. Ignoring the obvious concerns of sharing credit card numbers, users may recycle the same access information — usernames (and often passwords) — across many sites. This could make the data leak exponentially worse if the credentials are tied to bank, investment, credit card, and retirement accounts. In many cases, thinking of your email address as a second password can help protect you from these dangers.
Here’s an article about the practice:
https://medium.com/@bulcclub/the-new-paradigm-against-hackers-email-addresses-as-passwords-cfe63ea7f6de
1:53pm EDT: panerabread.com is back online! And at least one of the flaws described is still working. Names, email/username and 4 last digits of credit cards still exposed.
Which flaw is still working?