Connect with us

Hi, what are you looking for?

THE BIZNOB – Global Business & Financial News – A Business Journal – Focus On Business Leaders, Technology – Enterpeneurship – Finance – Economy – Politics & LifestyleTHE BIZNOB – Global Business & Financial News – A Business Journal – Focus On Business Leaders, Technology – Enterpeneurship – Finance – Economy – Politics & Lifestyle



Reddit should upgrade web standard to stop automated scraping

Listen to the article now

image credit: google play

Popular social media network Reddit has taken preemptive measures to preserve its material. A web standard used by the platform to restrict automated data scraping from its website will be updated on Tuesday. Reports that AI companies were circumventing the regulation to obtain material for their systems prompted this move.
Reddit’s action is noteworthy, particularly because AI corporations have been accused of harming publishers. These companies have been accused of plagiarizing publisher material to construct AI-generated summaries without acknowledgment or consent, undermining their hard work and rights.
Reddit proposed updating the Robots Exclusion Protocol, or “robots.txt,” a widely used standard for determining whether portions of a site may be scanned.
The business also claimed it would continue rate-limiting, which limits requests from one organization. It will prevent unknown bots and crawlers from scraping its website for raw data.
Recently, publishers have relied on robots.txt to prevent tech firms from exploiting their material for free to train AI algorithms and provide summaries for search queries.
Last week, content licensing company TollBit wrote to publishers that many AI businesses were scraping publisher sites by evading the web standard.
Following a Wired research, which opens a new tab, AI search company Perplexity likely overcame robots.txt web crawler blocking.
AI Weekly: Nvidia sets another record while Apple stumbles in Europe.
In June, Forbes accused opens new tab Perplexity of copying its investigative reports for generative AI systems without attribution.
On Tuesday, Reddit stated scholars and groups like the Internet Archive may still utilize its information for non-commercial purposes.

Comment Template

You May Also Like


  Ukrainian troops describe Russian tactics they see everyday with a brutal word. They call them “meat assaults”—waves of Russian troops attacking their defensive...


Family members prayed and honored Patricia Portillo and Brayan Godoy. On Saturday, people gathered outside a closed Las Colinas Chick-fil-A to memorialize two employees...


South Dallas AT&T customers lose service following copper cable theft. Due to copper wire theft, AT&T customers in South Dallas are experiencing lengthy service...


Elon Musk said on Wednesday on social media platform X that Dell Technologies  and Super Micro Computer would supply server racks for his artificial...

Notice: The Biznob uses cookies to provide necessary website functionality, improve your experience and analyze our traffic. By using our website, you agree to our Privacy Policy and our Cookie Policy.