We’ve all been there; one minute we’re thinking about a pair of shoes and the next, we’re bombarded with online ads for the very same product.
While Big Data has opened doors to a whole new field of tools at the disposal of companies using consumer data, gaining access to this cache is a tricky terrain. Data – be it personal, transactional, biometric, or even images and videos – has increasingly come to be known as the ‘oil of the digital era.’ Its value in recent times has skyrocketed as most businesses compete for ownership. And yet, it remains an unexplored area where the rules and regulations are striving to catch up with the galloping advance of companies using it.
This is where web scraping comes in. The term broadly refers to the business of gaining access to mounds of personal data through the process of ‘scraping’ and then using it for the purposes of market research, comparative product analysis, lead generation, and content aggregation.
While web scraping can be performed directly by the company using the data or by a third party specializing in scraping, it’s the end result which is contentious, depending on which side of the aisle the complainant is coming from. In general, scraping publicly available data is considered fair game. But there are issues when it comes to the terms of service proclaimed by websites, leading to suits for breach of contract, personal data (think Cambridge Analytica), intellectual property (there are already suits over using copyrighted material as training sets), and confidential information. This rapidly evolving terrain provides a fascinating conundrum, raising new issues which are discussed in detail here.