Hardening Koha OPAC Against Automated Scraping

A Best-Practice Case Study Using Apache, mod_rewrite, Fail2Ban, and HTTPS Enforcement

Publicly accessible Koha OPACs are increasingly targeted by automated bots attempting to harvest bibliographic data using complex search queries. These bots exploit advanced CCL searches and repeatedly hit OPAC endpoints to enumerate records. This case study documents a real-world instance of Koha OPAC abuse and presents a layered mitigation strategy using Apache rewrite rules, Fail2Ban intrusion prevention, and mandatory HTTPS enforcement. The approach is non-intrusive, avoids Koha core modifications, and is suitable for production environments.

Recently, my Apache access logs revealed a classic case of automated harvesting attempts against a Koha OPAC. The attacker wasn’t subtle, which is almost admirable.

Background and Problem Statement

Koha OPAC is designed for discovery by human users. However, server access logs revealed a pattern of automated requests attempting to systematically download catalog records.

A repeated pattern appeared in the Apache access logs with following characteristics:

• Targeted endpoint:
  /cgi-bin/koha/opac-search.pl
• Use of Massive CCL queries (q=ccl=)
• Extremely long query strings
• Excessive repetition of search fields:
  • au: (author)
  • su-to: (subject)
  • itype: (item type)
• Requests made at high frequency
• Long, chained Boolean expressions
• Unicode terms in Punjabi and English
• Parameters like count=50, format=rss, and sort_by=acqdate_dsc

Example (shortened for sanity):

GET /cgi-bin/koha/opac-search.pl?
q=ccl=au:"Kapoor, Narinder Singh"
and su-to:Punjabi
and su-to:ਨਰਿੰਦਰ ਸਿੰਘ
and su-to:Punjabi literature
...
&count=50&format=rss

This is not how humans search. This is how a script enumerates records systematically, attempting to extract metadata in bulk. This behavior was consistent with a scraping bot, not a legitimate OPAC user.

Impact on Koha OPAC

Unchecked, this activity can:

• Degrade OPAC performance
• Put unnecessary load on the OPAC
• Exhaust search backend resources
• Enable unauthorized bulk harvesting of bibliographic metadata
• Degrade performance for legitimate users
• Bypass intended UI-based access
• Become a stepping stone to larger DoS-style attacks

Koha does not provide built-in bot detection or rate limiting for OPAC searches. Therefore, mitigation must occur at the web server level.

Apache Rewrite Rules to Block Abusive Searches

Instead of blocking the OPAC entirely or breaking legitimate searches, we apply surgical filtering using mod_rewrite.

Blocking CCL Queries with Excessive Author or Subject Terms

RewriteEngine On

RewriteCond %{REQUEST_URI} ^/cgi-bin/koha/opac-search\.pl$ [NC]
RewriteCond %{QUERY_STRING} q=ccl=.*(au:|su-to:).* [NC]
RewriteCond %{QUERY_STRING} (au:|su-to:){5,} [NC]
RewriteRule ^.*$ - [F]

Humans do not construct searches like this. Scripts do. This Blocks OPAC searches that repeatedly use au: or su-to: Five or more times is a strong indicator of automated harvesting.

Blocking Excessively Long Query Strings

This is for bots that change patterns but stay noisy:

RewriteCond %{REQUEST_URI} ^/cgi-bin/koha/opac-search\.pl$ [NC]
RewriteCond %{QUERY_STRING} ^.{1000,}$
RewriteRule ^.*$ - [F]

This blocks requests with query strings longer than 1000 characters, which is generous for real users and restrictive for scrapers.

Fail2Ban Integration for Koha OPAC Abuse

Apache rewrite rules stop the request, but Fail2Ban stops the attacker.

Create a Koha OPAC Filter

Create a new filter file:

/etc/fail2ban/filter.d/koha-opac.conf

[Definition]
failregex = <HOST> - .* "(GET|POST) /cgi-bin/koha/opac-search\.pl.*q=ccl=.*" 403
ignoreregex =

This filter watches for repeated 403 responses caused by blocked CCL scraping attempts.

Create a Jail for Koha OPAC

Edit or create:

/etc/fail2ban/jail.d/koha-opac.conf

[koha-opac]
enabled  = true
port     = http,https
filter   = koha-opac
logpath  = /var/log/apache2/access.log
maxretry = 5
findtime = 600
bantime  = 86400

Behavior

• Blocks IPs after 5 abusive attempts
• Ban duration: 24 hours
• Automatically protects against repeated scraping attempts

Restart Fail2Ban:

sudo systemctl restart fail2ban

Enforcing HTTPS for Koha OPAC

Allowing OPAC traffic over HTTP exposes search activity, cookies, and sessions. HTTPS should be mandatory.

Redirect All HTTP Traffic to HTTPS

In your Apache virtual host for port 80:

<VirtualHost *:80>
    ServerName opac.example.edu
    ServerAlias www.opac.example.edu

    RewriteEngine On
    RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</VirtualHost>

Ensure OPAC Is Served Only Over HTTPS

Your HTTPS virtual host (:443) should serve Koha OPAC exclusively.

Optional hardening:

Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains"

This enforces HTTPS in browsers permanently.

Best-Practice Summary for Koha Administrators

Layer	Protection
Apache mod_rewrite	Blocks abusive search patterns
Fail2Ban	Automatically bans scraper IPs
HTTPS enforcement	Secures OPAC traffic
Log monitoring	Early detection of abuse

This layered approach ensures:

• No Koha core file modification
• Minimal impact on legitimate users
• Strong protection against automated harvesting

Conclusion

Koha is powerful. Apache is flexible. Bots are relentless.
As Koha OPACs become more visible, automated scraping attempts are no longer hypothetical—they are routine. This case study demonstrates that with careful log analysis and server-level controls, libraries can effectively defend their catalogs.

Apache rewrite rules stop malicious requests.
Fail2Ban removes persistent offenders.
HTTPS ensures secure access.

Together, they form a practical, scalable, and maintainable best-practice model for securing Koha OPAC installations.