Is web scraping legal? Public data is only the first test

A price monitor and a private-profile scraper are not the same legal problem

A script that reads public product prices and a bot that signs into thousands of accounts to collect private profiles can both be called web scraping. Legally, they are very different.

There is no single rule that makes scraping legal or illegal. The answer usually turns on several separate questions:

Question	What changes the risk
How did you get the data?	A page open to anyone is different from an area protected by a login, subscription or other access control.
What did you agree to?	Account terms, API agreements and other contracts can create obligations separate from computer-access law.
What are you copying?	Raw facts are treated differently from articles, photographs, code and other protected expression.
Does it describe people?	Publicly visible personal information can still be regulated by privacy and data-protection law.
What will you do with it?	Internal analysis, resale, republication, profiling and AI training can raise different legal issues even when the collection method is identical.

That is why “the page was public” is useful information, but not a complete legal answer.

This is a legal explainer, not legal advice. Scraping law changes by jurisdiction and by the facts of the project. Large-scale commercial collection, personal data, access controls or a cease-and-desist letter are reasons to get advice for the specific operation.

In the US, the CFAA is mainly an access question

The federal law most often associated with scraping is the Computer Fraud and Abuse Act, 18 U.S.C. § 1030. Among other things, it covers intentionally accessing a computer “without authorization” or exceeding authorized access and obtaining information.

The Supreme Court narrowed one side of that language in Van Buren v. United States in 2021. A police officer had permission to use a law-enforcement database but used it for an improper purpose. The Court held that he did not “exceed authorized access” merely because his reason for using information he was otherwise allowed to reach violated policy. The Court framed the issue as a gates-up-or-down inquiry: did the person enter a part of the system their access did not extend to? The opinion distinguished access restrictions from rules about why already-accessible information may be used.

That matters to scraping because it makes a terms violation and a hacking-style access violation harder to collapse into the same thing.

The Justice Department’s current CFAA charging policy follows the same general distinction. It says prosecutors should not treat a mere contractual or terms-of-service violation as automatically withdrawing technical authorization. That policy guides federal prosecutors; it is not a private safe harbour and does not erase civil claims under other law.

hiQ v. LinkedIn did not make “public scraping” universally legal

The case most often reduced to a one-line scraping rule is hiQ Labs v. LinkedIn.

hiQ collected information from LinkedIn profiles that users had made visible to anyone with a web browser. After LinkedIn sent a cease-and-desist letter and tried to block the scraping, hiQ sought an injunction.

In 2022, the US Court of Appeals for the Ninth Circuit said that, on the preliminary-injunction record, hiQ had raised serious questions about whether the CFAA’s “without authorization” language applied to those publicly available pages. The court reasoned that a public website has no access gate to lift or lower in the first place. It also stressed the opposite case: areas protected by passwords or similar access requirements can still be subject to the CFAA. The Ninth Circuit opinion is powerful authority on public, logged-out access in that circuit.

But the details matter.

First, this was a Ninth Circuit case, not a Supreme Court ruling that every scraping project in the United States is lawful.

Second, the appellate decision was about the CFAA issue at the preliminary-injunction stage. It did not declare that hiQ was free from every other legal obligation.

The later district-court fight makes that distinction unusually clear. In November 2022, the court found that hiQ’s own scraping and use of scraped data violated LinkedIn’s User Agreement, although factual disputes over waiver and estoppel prevented LinkedIn from obtaining summary judgment on that part of its contract claim. The court separately granted LinkedIn summary judgment over conduct by hiQ contractors who logged into LinkedIn. The district-court order came from the same dispute that is so often cited as “scraping public data is legal.”

So hiQ is better read as a warning against one oversized conclusion:

Winning the computer-access question does not automatically win the contract, copyright, privacy or reuse questions.

Contract rules depend on what you actually agreed to

Website terms matter, but not in the simplistic form “the terms ban scraping, therefore scraping is illegal.”

Contract claims depend on whether a contract was formed, what its language actually covers, who is bound by it and what law applies. Being logged in can be especially important because creating or using an account commonly involves assent to platform terms.

A 2024 case between Meta and data provider Bright Data shows how fact-specific that analysis can become. Meta argued that Bright Data breached Facebook and Instagram terms by scraping their data. The federal district court found no evidence that Bright Data had scraped non-public data while logged in. It also concluded that the terms before it did not prohibit Bright Data’s logged-out scraping of public data, and granted Bright Data summary judgment on Meta’s breach-of-contract claim. The order repeatedly separated public, logged-out collection from authenticated access.

The same order is useful for another reason. Meta pointed to Bright Data’s ability to get around CAPTCHAs and other anti-bot mechanisms. The court said defeating a mechanism that blocks automated collection of public information was not, on that record, equivalent to scraping non-public data behind a login.

That does not make bypassing every technical restriction legally harmless. It means the legal significance of a technical barrier depends on what the barrier is protecting, the claim being brought and the facts around the access.

robots.txt is not a permission slip or a criminal statute

A robots.txt file tells automated crawlers which parts of a site the operator asks them to crawl or avoid.

The technical standard is unusually direct on the legal-sounding question. RFC 9309, which standardizes the Robots Exclusion Protocol, says its rules are not a form of access authorization.

So a robots rule should not be treated as if it were a court order, and an empty robots file should not be treated as a licence to do anything with the data.

It can still matter operationally. A site owner may use it to communicate crawler preferences. Search engines and responsible crawler operators may honour it. Ignoring it can contribute to a broader dispute about conduct, load or expectations. But the protocol itself does not answer whether a scrape is lawful.

Copyright protects expression, not every fact on a page

Being allowed to reach a page does not mean you own what is displayed there.

In the United States, the Copyright Office states that individual plain facts are not protected by copyright, while original authorship in a database’s selection or arrangement can be protected. Copyright can also exist in the underlying material inside a database — text, photographs, artwork and other creative content. Its database guidance draws that line explicitly.

That makes a scraper collecting a list of current product prices a different copyright problem from one copying complete news articles and photographs.

The distinction is not “structured data good, text bad.” A database can contain protected material, and an original compilation can have its own copyright even when the individual facts inside it do not. Whether a particular reproduction is infringing can also turn on licences, exceptions and doctrines such as fair use.

Outside the United States, the rights can be different again. European Union law adds a separate sui generis database right that can let a qualifying database maker prevent extraction or re-utilization of all or a substantial part of a protected database. A US copyright analysis should not be pasted onto an EU project and treated as complete.

Public personal data can still be regulated personal data

Privacy law is where the phrase public data becomes especially dangerous shorthand.

In May 2026, Canadian federal and provincial privacy regulators published their joint findings into OpenAI’s handling of personal information. They examined, among other things, personal information collected from publicly accessible websites for model development. The regulators stressed that the legal category of “publicly available” information under Canadian privacy statutes is narrower than the ordinary idea of information being visible on the public internet. They specifically said information collected from public social-media or professional profiles does not fall within PIPEDA’s publicly-available exception merely because the profiles can be viewed online. The findings are a current example of public accessibility and privacy permission being treated as different questions.

The European Union takes the same problem seriously under the GDPR. Article 6 requires a lawful basis for processing personal data; Article 9 adds stricter rules for special categories such as health data, biometrics used for identification, political opinions and religious beliefs.

The European Data Protection Board made web scraping an explicit 2026 priority in its new guidance for generative AI. Its July 2026 guidance says the GDPR applies when scraping includes personal-data operations such as collection, storage, organisation or retrieval, and highlights purpose limitation, transparency, data minimisation and the treatment of special-category data.

As of 9 October 2026, those EDPB guidelines are still in public consultation until 30 October 2026. They are useful evidence of the regulator’s current position, not a final court ruling.

For a scraper, the practical consequence is simple: a name, profile photo, location, work history or other information about a person does not stop being personal data just because a browser can see it.

For more on why an account’s visible profile and the identity behind it are different layers, see our guide to portable online identity.

The safest starting point is boring

No checklist can turn a complicated project into guaranteed legal compliance, but the lower-risk end of the spectrum is fairly recognizable.

A scraper is easier to analyze when it:

• reads pages that any visitor can access without credentials;
• collects factual or otherwise clearly reusable information rather than copying expressive works wholesale;
• does not collect sensitive personal information;
• does not depend on an account or contract that expressly restricts the activity;
• operates at a rate that does not disrupt the service;
• records where the data came from and why it is being collected;
• has a defined retention and downstream-use policy instead of collecting everything “just in case.”

Risk climbs quickly when a project needs someone else’s account, reaches behind a login or paywall, collects sensitive profiles at scale, republishes creative content, ignores an enforceable agreement, or keeps operating after a specific legal demand without reviewing it.

A useful pre-scrape question: If the project description is simply “it is public, so we can take it,” the legal analysis is not finished.

So, is web scraping legal?

Web scraping can be legal. It is not automatically legal.

For a public, logged-out page in the United States, cases such as hiQ v. LinkedIn make the federal anti-hacking question substantially less threatening than it is for authenticated or otherwise restricted areas. That is an important rule. It is not the whole rulebook.

The real analysis continues after access:

• Did you agree to terms that cover this conduct?
• Are you copying facts or protected expression?
• Are you processing personal or sensitive data?
• Do database or data-protection rights apply in the relevant country?
• What are you doing with the material after collection?

A browser being able to see something tells you where the data sits.

It does not, by itself, tell you every right you have over it.