The www/anubis port
anubis-1.22.0v0 – proof-of-work proxy to protect web resources from scrapers (cvsweb github mirror)
Description
Anubis acts as middleware between a reverse proxy and backend web server.
It assesses whether a connection is likely to be from a scraper bot and,
if this seems that there's a chance of this, it issues a SHA-256 proof-
of-work challenge before allowing the connection to proceed.
As of 1.14.x, Anubis decides to present a challenge using this logic:
User-Agent contains "Mozilla"
Request path is not in /.well-known, /robots.txt, or /favicon.ico
Request path is not obviously an RSS feed (ends with .rss, .xml, or .atom)
This should ensure that git clients, RSS readers, and other low-harm
clients can get through without issue, but high-risk clients such as
browsers and AI scraper bots impersonating browsers will get blocked.
When a challenge is passed, a signed JSON Web Token (JWT) is provided
as a cookie, allowing future requests to pass without triggering the
challenge.
Using Anubis will likely result in your website not being indexed by
some search engines. This is considered a feature, not a bug.
WWW: https://anubis.techaro.lol/
Readme
+-------------------------------------------------------------------------------
| Running ${PKGSTEM} on OpenBSD
+-------------------------------------------------------------------------------
While Anubis is intended to be run in a Docker container, it can also be
manually configured and is able to run on OpenBSD. It must be "sandwiched"
between a front-end web server (for example, nginx in reverse proxy mode)
and a back-end.
Example configuration with nginx
================================
Anubis
------
Bind and target ports should be specified in ${SYSCONFDIR}/anubis.env.
Here's a contrived example:
export BIND=127.0.0.1:8923
export DIFFICULTY=3
export METRICS_BIND=:9090
export SERVE_ROBOTS_TXT=0
export TARGET=https://www.openbsd.org:443
Anubis listens for incoming HTTP connections on 127.0.0.1:8923, uses a
medium difficulty proof-of-work, and forwards connections to a backend
running at https://www.openbsd.org:443. It provides Prometheus-style
metrics at http://:9090/metrics (not bound to a specific
address here, but that is allowed).
Typically TARGET would be pointed at a backend server which is either
hosted locally, or hosted remotely but firewalled off so that bot traffic
can't reach it directly.
Front-end nginx
---------------
server {
listen 443 ssl;
listen [::]:443 ssl;
http2 on;
server_name somehost.example.org;
ssl_certificate /etc/ssl/server.crt;
ssl_certificate_key /etc/ssl/private/server.key;
ssl_protocols TLSv1.3 TLSv1.2;
ssl_ciphers 'TLSv1.3:TLSv1.2+AEAD+ECDHE';
location / {
proxy_pass http://127.0.0.1:8923;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host www.openbsd.org;
proxy_http_version 1.1;
}
}
Maintainer
The OpenBSD ports mailing-list
Only for arches
aarch64 amd64 arm i386 riscv64