Author: Tim Düsterhus (TimWolla)
Committer: Derick Rethans (derickr)
Date: 2026-09-24T09:05:21+01:00
Commit: Allow search engines to crawl the archives · php/web-news@597a2b8 · GitHub
Raw diff: https://github.com/php/web-news/commit/597a2b84e2394ffab48e3a522596c9f964798961.diff
Allow search engines to crawl the archives
see php/infrastructure#40
Changed paths:
M robots.txt
Diff:
diff --git a/robots.txt b/robots.txt
index 8936d2c..d08e401 100644
--- a/robots.txt
+++ b/robots.txt
@@ -1,6 +1,2 @@
User-agent: *
-Disallow: /
-
-User-agent: bingbot
Crawl-delay: 5
-Disallow: