Big Data – big privacy concerns

Tom Breur

25 Augustus 2016

Big Data are here to stay. With ever more applications of analytics, companies will need to address their data strategy. As the number and size of data sources keep growing, so does the corporate responsibility to manage them in a prudent and diligent way. The more sources you tap into, the more different ways you are putting your reputation on the line, too. But it not just reputation, in particular in Europe there is also a considerable body of legislation you need to honor.

North America has long had a tradition of commercial resale of data. With the rise of Big Data, the richness of data sources keeps growing. The more you match and merge disparate data types, the more you can discern about individual’s behavior. Government driven data protection appears to be lagging behind technological developments.

Ever since Judge Bork’s video rental records became an object of interest (, there is awareness that data can reveal private details. Unfortunately for the tabloids, his interests weren’t nearly as juicy as hoped… It is only a matter of time until the next celebrity hits the front page, and it’s anybody’s guess how colorful their behavior might turn out to be.

When I was doing analytic work for a credit card company, and wanted attention for the sensitivity of our data, at some point I flagged a segment of credit card users with a rather remarkable usage profile. These customers were all cardholders who used their second card almost exclusively (>90%) for transaction in “shady bars.” This, coincidentally, is a merchant category with disproportionately high interchange fees, so these transactions were easy to single out. No names were made public, but my “business case” for tighter data security immediately got the attention from senior management that I had asked for…

There are two realistic boundaries to what you can and cannot do with your data. Either you choose the “high road” and adhere to some self-chosen, moral standard that exceeds what is required by law, or, you choose to do whatever can be made to fit within the legal boundaries. In many jurisdictions you can get away with quite a lot. The downside of this latter approach is mostly “publicity risk”: the day you hit the headlines with an unfavorable story, you know it was a business risk you knowingly and willingly took.

For NoSQL systems, security (preventing unauthorized access) and deliberately truncating (sensitive) data are two areas that have not yet received the attention they deserve, imho. See also this article: As more and more business critical applications with rely on NoSQL solutions, this topic will trickle up to the boardroom, is my guess. Until then, Big Data security appears the proverbial “hot potato” nobody wants on his plate.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s