Web lists-archives.com

Bug#875869: ITP: robot-detection -- Detect web crawlers from HTTP UserAgent




Package: wnpp
Severity: wishlist
Owner: Jonas Meurer <jonas@xxxxxxxxxxxxxxx>

* Package name    : robot-detection
  Version         : 0.4.0
  Upstream Author : Rory McCann <rory@xxxxxxxxxxxxxxx>
* URL             : https://github.com/rory/robot-detection
* License         : GPL-3
  Programming Lang: Python
  Description     : Detect web crawlers from HTTP UserAgent

This Python module detects if a given HTTP User Agent is a web crawler.
It uses the list of registered robots from the robotstxt.org.

* Packaging as dependency for mailman3
* Intended to be maintained inside DPMT