REMINDER
Reappearence of "disallow" status in robots.txt
-
Hello,
We’ve got a problem with the code editor.
We allowed the indexation of our site in the dashboard (checkbox unchecked). But the robots.txt file still forbade the indexation. We’ve modified that with the code editor and saved the changes. We also erased our blog sitemap. But the day after all of this is back. How can it be possible? Why the robots.txt file shows “disallow” status? Why the changes are not effective? Changes must be saved by another way?Is it because we published our site between the two occurences? I read that robots.txt files are re-generated on publish here :
Re: How are search engines supposed to find the blog pages?
Thank you for your time.
-
Hi Stéphane,
I'm curious about it.
Can you check your Project Configuration -> Indexed pages and see what is there?
Click the wrench icon on the left of pagebuilder, then "Indexed Pages"
-
Yep. Here what i've got. Pages are indexed except for two. All is good here.
-
It seems your project is on a free offer without domain name, you can't enable indexing on a free offer.
-
@Wassim Hello! The project is shared between two accounts (my boss and mine) and we've got a paid plan with my boss account wich houses the project if i remember correctly.
-
I assumed it was the project on your account, if not send us the project id on support@appdrag.com
-
@Stéphane-Anciaux said in Reappearence of "disallow" status in robots.txt:
We allowed the indexation of our site in the dashboard (checkbox unchecked). But the robots.txt file still forbade the indexation. We’ve modified that with the code editor and saved the changes. We also erased our blog sitemap. But the day after all of this is back. How can it be possible? Why the robots.txt file shows “disallow” status? Why the changes are not effective? Changes must be saved by another way?
Hi Stéphane,
Thank you for your email, the disallow you had were on deleted pages so you didn't see them in the list. It had no effects as those pages didn't exist anymore. I have removed them from the non indexed pages, you will only see brouillon and test-nathan in the robots.txt>disallow.
Have a nice day! -
@Wassim Perfect! Thank you Wassim.