WordPress problem fix
A wrong robots.txt can disappear your site from search overnight. We restore proper rules and audit the cause within hours.
If Search Console says pages are blocked by robots.txt, or your robots.txt suddenly shows Disallow rules you did not write, traffic loss is hours away. Common causes are a security or SEO plugin overwriting the file, a leftover staging Disallow: / line after migration, or a malicious file injected into the site root. The fix is to confirm what is serving the file (physical file vs WordPress virtual robots), restore the right directives, re submit in Search Console, and harden the source so it does not get rewritten.
If any of these match, you are on the right page.
Search Console flags pages blocked by robots.txt
Organic traffic dropped after a migration or plugin install
robots.txt shows Disallow: / for the whole site
Rules appear in robots.txt that you never added
Either a physical robots.txt in the site root with a Disallow: / left over from staging, a plugin generating a virtual robots.txt with wrong rules, or a malicious file injected during a hack.
If a physical robots.txt exists in the site root, WordPress serves that. If not, WordPress generates one on the fly, and many SEO and security plugins inject rules into that virtual file.
Yes. Googlebot respects robots.txt almost immediately. A wrong Disallow can cut crawling within a day and rankings within a week if not caught.
The real method, in the order it works.
Open yoursite.com/robots.txt in a private window to see what Google sees.
Check the site root over SFTP for a physical robots.txt and remove or correct it.
Open every SEO and security plugin and check its robots.txt settings.
Restore correct directives (allow normal crawling, block only admin and search query parameters).
Submit robots.txt in Search Console and request validation.
Real fix, from our work
A client migrated to a new host and traffic collapsed within a week. robots.txt was the cause: a Disallow: / from the staging host had been copied across in the migration archive. I deleted the physical file, let WordPress serve its default virtual robots through Yoast, re submitted in Search Console, and pages were crawled again within 48 hours. Traffic recovered over the following two weeks.
Written by Ali Yasin Jatoi
Founder of WebCare Studios. Ali has worked with WordPress for more than 10 years, including managing a fleet of 150+ sites with WP-CLI automation for updates, security cleanup, and malware removal. He has hands on experience across major hosts including Cloudways, A2 Hosting, Hostinger, and Bluehost.
Site down, hacked, or broken checkout gets a senior engineer within 4 hours. No ticket queues, no bots.
Flat quote up front. If we cannot get you back online, you do not pay. Risk sits with us, not you.
We work on a snapshot first and never touch your live database until the fix is verified safe.
We run a fleet of WordPress sites every day. The errors you are seeing are ones we have closed hundreds of times.
Most modern WordPress installs do not need to. Yoast and Rank Math handle this automatically. Manual rules often cause more problems than they solve.
Google caches robots.txt for around 24 hours. After fixing, request a validation in Search Console and recrawl. The status updates within a day or two.
Possibly. If you did not create the file and no plugin owns it, treat it as a compromise: scan for malware, rotate credentials, and audit recent file changes.
Two fields. Email and your URL. A senior WordPress engineer reads it within minutes and replies on email and WhatsApp with what is wrong and what we will do next.