Automated data extraction from digital platforms, public portals, and discussion boards is routine in studies of public opinion, consumer habits, and network structures. The ease of writing Python or R scrapers, however, does not remove privacy rules or ethical research standards.
In Brazil, the General Data Protection Law (Law 13,709/2018, known as the LGPD) and National Health Council resolutions (CNS 466/2012 and 510/2016) govern research involving personal records. Researchers must design collection and storage routines that prevent privacy breaches and regulatory issues.
1. Public availability and privacy limits
A frequent assumption is that public online posts can be used without privacy limits. Under Article 7, paragraph 4 of the LGPD, processing personal data made manifestly public still requires honoring the original purpose, good faith, and public interest behind its publication.
Storing real names or account photos in analytical databases can constitute a privacy violation if the records are shared outside their original context or used to single out individuals.
2. Pseudonymization and anonymization
Privacy standards differentiate between two technical steps:
- Pseudonymization (Article 13, paragraph 4 of the LGPD): records can no longer identify an individual directly without extra information, such as when user handles or profile IDs are replaced with salted cryptographic hashes (like SHA-256). This keeps network connections intact for analysis while preventing direct identification by anyone without the lookup key.
- Anonymization (Article 12 of the LGPD): the data cannot identify the individual by any reasonable technical means available during processing. Anonymized records cease to be classified as personal data under the law.
3. Research ethics review guidelines
For academic projects evaluated through Brazil’s CEP/Conep system, CNS Resolution 510/2016 outlines exemptions from formal review:
- Opinion polls with anonymous participants.
- Studies relying on information strictly in the public domain.
- Work using open databases that were already anonymized at the source.
Formal ethical review remains required when scraping involves monitoring closed groups, compiling individual tracking profiles, or handling sensitive categories such as religious beliefs, political opinions, health status, or sexual orientation (Article 5, item II of the LGPD).
Practical routine
Treating data governance as part of methodology rather than an administrative hurdle protects research from being dismissed or challenged later. Documenting extraction rules, retention limits, and cleanup routines is just as important to empirical credibility as the statistical analysis that follows.
For compliant workflows and automated extraction pipelines, explore our digital data collection services.